Question 18569 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Grafana – Loki data source receiving 400 Unauthorized error – Amazon Web Sevices

EKS 1.23, Prometheus-stack with Grafana v7.3.5

Installed Loki v2.7.3 and now trying to connect Grafana to Loki.

I added Loki datasource from Grafana and configured the loki-gateway service url:

enter image description here

Then I get the following error:

Loki: Bad Request. 400. Authentication to data source failed

From loki-gateway pod’s logs:

10.7.60.60 - - [01/Mar/2023:14:18:53 +0000]  401 "GET /loki/api/v1/label?start=1677679732842000000 HTTP/1.1" 10 "-" "Grafana/7.3.5" "10.7.127.117, 10.0.60.21, 10.0.60.21"

NOTE: loki-gateway basicAuth is disabled by default, hence no need to add user & password to Loki data source in Grafana.

Tried:

  1. Enable loki-gateway basicAuth and pass the credentials through Grafana Loki Data Source.

  2. Skip tls verification

Both ended up with the same error.

Tags:

2

Answers


  1. 0

    If you are using loki helm chart to deploy, check if auth_enabled value is set to true (it’s true by default), set it false and upgrade.

    Login or Signup to reply.
  2. 0

    I got the same promblem, then i check grafana log

    kubectl logs -n grafana-loki-dev loki-grafana-66dcf659b4-8qk22 grafana -f

    it shows

    enter image description here

    You can specify the orgid by adding an HTTP Header ‘X-Scope-OrgID’ in the Data Source configuration on the Grafana panel. Here, you can set the value of the Header to the orgid you want. This way, when Grafana proxies access to Loki’s API, it will have the correct Header and thus successfully authenticate and obtain access.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search