I am using ELK stack to visualize our application logs on Kibana.
However, we would improve the dashboard by adding nginx access log.
By this way, our devs could access to a full transaction logs, when the request arrives on nginx to the application logs.
My question is: is it a good practice to show the full logs on Kibana for developers by this way? Or we should split them into two differents dashboard, one for applications logs, one for nginx logs ?
Thank you
2
Answers
Probably developers want to see all logs for a single business transaction in one place.
Best practice is to use Distributed Tracing, so that all logs of a single business transaction can be grouped together in a single UI for the developers.
E.g. use Opentelemetry to instrument applications, there you have a Trace ID that gets attached to all application logs. Then you make your nginx opentelemetry aware via a plugin.
Then usually developer flow is such that they need to solve an error that gets reported via e.g. Sentry, there you have a link to a Trace e.g. in Grafana/Datadog etc and from the Trace you see all logs aggregated.
We do separate dashboards for application logs and nginx logs.
Yes the developers need full logs as it will be very helpful for them to troubleshoot any issues.
As a bonus, we have a consolidated dashboard which is designed based on the application which is a combination of (application log and nginx log).