Question posted in
Some documentation to get you started can be found here.

Apache Log grok pattern

Can anybody please help with the grok pattern for below example of logs?

85.85.85.85 webmail.company.com "CN=First Last/O=Company/C=CZ" [14/Dec/2020:05:58:18 +0100] "GET /mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=s_ReadViewEntries&PresetFields=DBQuotaInfo;1,FolderName;($Inbox),UnreadCountInfo;1,SearchSort;DateD,s_UsingHttps;1,noPI;1&TZType=UTC&Start=1&Count=23&resortdescending=6 HTTP/1.1" 200 2054 "https://webmail.company.com/mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=l_ScriptFrame&l=en&gz&CR&MX&TSF=20170318T181650,92Z&TSX=20180206T185427,18Z&EFF=%2FiNotes%2FForms9_x&charset=UTF-8&charset=UTF-8&KIC&ua=safari&pt" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" 125 INOTES_LOGIN_ID=First%20Last; Shimmer=SI_TLM:20210209T072811%2C40Z&ST_Counter:3&LAO:mail&SAB:1&CS_TLM:20210209T072831%2C15Z&V_TLM:20210210T080147%2C82Z&DMS:5&ui:X&MOTLM:20210129T113159%2C00Z&DBQS:1503571%2C%207168000%2C%206963200%2C%200%2C%201503571&SPRKL:1&KOSCZ:GTB&FISD:1; INOTES_LOGIN_ID=First%20Last; DWAShared=0; DWAMode=0; INOTES_LOGIN_ID=First%20Last; DWAShared=0; DWAMode=0; LtpaToken2=FpoGJJz33bYLI+CtWy6OlIgoTJouNGEiduvxvQbcN8HRI7K6LThCsb1Dl8CzN72Zi05RGOUmQRMiOQcTk1norKHi6SbkEGI6GlXzjSIweBRSc8c+XPyAwA44PKPbu3WzrPfR0+uoC0sgTPvochvQ/VfPL/sSaqUFoRswRwyI+UeaOwTs/DvKiWLCpiKrVkFk3SmDjrxPBHb/WiL5nDkpp8Dsjjxnlo4vpx7BdOoVNai1jybvHkW28KXxkb21o8SSpmU7ZFdHyZFjDWCYuuCVOx7asV/q4a3lWdxlPfWdPcUguHML+xDmsrMPm6fTUSKeKIKdQEPr6VDmitBi7Z5URIlkRrUyslkTcc28y6fQir3Y20Hc9TmOvwaBlG/ehnpv; LtpaToken=0x4JJ4oWKojdqoz08Ng+MRUkkJq2vYGLGN9lp8HL8FxbD+xnivE7qzCzf92Q6x5OAPOBFRNgxd3Qg225zLwnJFWO0lGeIweH8VDgyWOMImNe6E9z9HBnQAN43vQ2uwtpv3X5E5DN0oLIPKLxAkqsHUDJqJ0SE6NZ6UnfLoR82JyjZVC/s6QEov5DNdpAY/o2Gxh0vWmE+wuQGuCh4mVCIP9KU/dbX4F0Ld9JEExzIpkdzKELibU2Akov0Krv0eWADSV++m/5ECLpaf6N6/VzkZEkt5XoOoL6OD/6ni4zojvo3O+X9Bn7Mdk2MnsQ1AccIohj5eN8Oi81QbD0a9b7jw==; ShimmerS=ET:20210210T114045%2c00Z&R:0&AT:M" "D:/Lotus/Domino/Data/mail/User.nsf"

What I would need is Client IP (85.85.85.85), VirtualHostname (webmail / webmail.company.com) , User (part after CN=, First Last), Time (14/Dec/2020:05:58:18), URL (GET /mail/User.nsf/iNotes/Proxy/?OpenDocument&Form=s_ReadViewEntries&PresetFields=DBQuotaInfo;1,FolderName; … ) and the Device Info ( "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" )

I know it should start with below, however I can’t get anyhow the User name to proceed with [%{HTTPDATE:timestamp}] and possible next would be "(?:%{WORD:verb} %{NOTSPACE:request} and not sure how to get the Device info.
Any help would be appreciated!

%{IPORHOST:clientip} %{WORD:VirtualHost} ???

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Solved:

    %{IPORHOST:clientip} %{IPORHOST:destination.domain} "CN=%{DATA:username}" [%{HTTPDATE:apache.access.time}] "(?:%{WORD:http.request.method} %{DATA:url.original} HTTP/%{NUMBER:http.version}|-)?" %{NUMBER:http.response.status_code:long} (?:%{NUMBER:http.response.body.bytes:long}|-) ("%{DATA:http.request.referrer}") ("%{DATA:user_agent.original}")


  2. 0

    Since you have customized your log format, you have to build your own grok to match the log. You can use https://grokdebug.herokuapp.com/ to debug the pattern you’re going to use and you can copy some patterns from https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search