I’m trying to use the Kafka Exporter packaged by Bitnami, https://github.com/bitnami/bitnami-docker-kafka-exporter, together with the Bitnami image for Kafka, https://github.com/bitnami/bitnami-docker-kafka. I’m trying to run the following docker-compose.yml
:
version: '2'
networks:
app-tier:
driver: bridge
services:
zookeeper:
image: 'bitnami/zookeeper:latest'
environment:
- 'ALLOW_ANONYMOUS_LOGIN=yes'
networks:
- app-tier
kafka:
image: 'bitnami/kafka:latest'
environment:
- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
- ALLOW_PLAINTEXT_LISTENER=yes
networks:
- app-tier
kafka-exporter:
image: bitnami/kafka-exporter:latest
ports:
- "9308:9308"
command:
- --kafka.server=kafka:9092
However, if I run this with docker-compose up
, I get the following error:
bitnami-docker-kafka-kafka-exporter-1 | F0103 17:44:12.545739 1 kafka_exporter.go:865] Error Init Kafka Client: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)
I’ve tried to use the answer to How to pass arguments to entrypoint in docker-compose.yml to specify a command
for the kafka-exporter
service which – assuming the entrypoint is defined in exec form – should append additional flags to the invocation of the Docker Exporter binary. However, it seems that either the value of kafka:9092
is not right for the value of the kafka.server
flag, or the flag is not getting picked up, or perhaps there is some kind of race condition where the exporter fails and exits before Kafka is up and running. Any ideas on how to get this example to work?
2
Answers
It would appear that this is just caused by a race condition with the Kafka Exporter trying to connect to Kafka before it has started up. If I just run
docker-compose up
and allow the Kafka Exporter to fail, and then separately run thedanielqsh/kafka-exporter
container, it works:and on http://localhost:9308 I can see metrics:
Update
A more reliable way to do this is to use a wrapper script in order to perform an application-specific health check as described in https://docs.docker.com/compose/startup-order/. With the following directory structure,
the following
Dockerfile
,and the following
run.sh
,and the following
docker-compose.yml
,Upon running
docker-compose build && docker-compose up
, I can see from the logs that after ~2 seconds (on the third attempt) the Kafka Exporter starts successfully:Its bit difficult to run all the required dependencies for kafka-exporter.
Myself did few simple steps as following below.
Step 1
Step 2
Step 3
Step 4
If you note the above command i used "host.docker.internal" which will helps kafka-exporter to listen on my machines localhost.