Nginx Question posted in
The official documentation can be found here.

Nginx – How to match optional group in regular expression

I want a regular expression for using in fluentd for parsing nginx error logs.

The sample row is:

2024/04/15 09:06:29 [error] 3443790#3443790: *176070165 limiting requests, excess: 2.957 by zone "RequestLimitForCommonApi", client: 77.81.151.129, server: test.com, request: "POST /capi/session/forgot HTTP/1.1", host: "test.com", referrer: "https://test.com/"

I’m using the following format for matching log parameters:

format1 /^(?<time>d{4}/d{2}/d{2} d{2}:d{2}:d{2}) [(?<log_level>w+)] (?<pid>d+).(?<tid>d+): (?<error>.*), (?<client>.*), (?<server>.*), (?<request>.*), (?<host>.*), (?<referrer>.*)/

But some log rows have ‘uptime’ parameter and some of them don’t have.

Now what regular expression should I use to also match the ‘uptime’ parameter value if exists?

Sample log row with uptime:

2024/04/15 02:01:32 [error] 3443790#3443790: *172976982 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 86.55.16.251, server: test.com, request: "POST /api/test HTTP/1.1", upstream: "http://127.0.0.1:30110/api/test", host: "test.com", referrer: "https://test.com/"
Tags:

2

Answers


  1. 0

    in this way, you can modify your code.

    /^(?<timestamp>d{4}/d{2}/d{2} d{2}:d{2}:d{2}) [(?<log_level>w+)] (?<pid>d+)#(?<tid>d+): *(?<connection>d+) (?<message>.+?)(?:, uptime: (?<uptime>d+.d+))?, client: (?<client_ip>d+.d+.d+.d+), server: (?<server>S+), request: "(?<request_method>w+) (?<request>[^"]+) HTTP/(?<http_version>d.d)", host: "(?<host>[^"]+)", referrer: "(?<referrer>[^"]+)"

    try this code. if there is uptime it will fill it and if there isn’t, it won’t .

    Login or Signup to reply.
  2. 0

    you can try this code to match client an request for example and if there was another phrase like "test" between them, it could be ignored.

    client: (?[^,]+),( test: )?((?[^,]+),)? request: (?[^,]+)

    as you noticed, "test:" and which comes after it is optional.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search