Question 98033 in Nginx Question posted in
The official documentation can be found here.

Remove a part of a log in Loki – Nginx

I have installed Grafana, Loki, Promtail and Prometheus with the grafana/loki-stack.

I also have Nginx set up with the Nginx helm chart.

Promtail is ingesting logs fine into Loki, but I want to customise the way my logs look. Specifically I want to remove a part of the log because it creates errors when trying to parse it with either logfmt or json (Error: LogfmtParserErr and Error: JsonParserErr respectively).

The logs look like this:

2022-02-21T13:41:53.155640208Z stdout F timestamp=2022-02-21T13:41:53+00:00 http_request_method=POST http_response_status_code=200 http_response_time=0.001 http_version=HTTP/2.0 http_request_body_bytes=0 http_request_bytes=63

and I want to remove the part where it says stdout F so the log will look like this:

2022-02-21T13:41:53.155640208Z timestamp=2022-02-21T13:41:53+00:00 http_request_method=POST http_response_status_code=200 http_response_time=0.001 http_version=HTTP/2.0 http_request_body_bytes=0 http_request_bytes=63

I have figured out that on the ingestion side it could be something with Promtail, but ist it also possible to make a LogQL query in Loki to just replace that string? And how would one set up the Promtail configuration for the wanted behaviour?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Promtail should be configured to replace the string with the replace stage.

    Here is a sample config that removes the stdout F part of the log for all logs coming from the namespace ingress.

    promtail:
  enabled: true
  pipelineStages:
  - docker: {}
  - match:
      selector: '{namespace="ingress"}'
      stages:
      - replace:
          expression: "(stdout F)"
          replace: ""

    Specifically this example works for the grafana/loki-stack chart.


  2. 0

    According to the docs a better approach is to enable the cri pipeline stage instead of the docker one. Assuming that you are ingesting logs from a recent kubernetes installation that uses CRI.

    https://grafana.com/docs/loki/latest/clients/promtail/configuration/#cri

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search