Question 85561 in Docker Question posted in
The official documentation can be found here.

Cannot connect docker app to kafka docker container

I am trying to connect my Kafka container and my docker image all running locally
and don’t want to create a container with Kafka and zookeeper images.

When I am running the app without dockerizing it, it is working fine.

After running the dockerize app the docker logs is showing

could not read message dial tcp 172.21.0.3:9092: i/o timeout

in Kafka logs, it is showing

kafka-zookeeper-1  | 2021-12-07 06:17:15,755 [myid:1] - WARN  [NIOWorkerThread-7:ZooKeeperServer@1411] - Connection request from old client /172.18.0.1:56350; will be dropped if server is in r-o mode

this is the docker compose for kafka

version: "3"
services:
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    ports:
      - '2181:2181'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: 'bitnami/kafka:latest'
    ports:
      - '9092:9092'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    depends_on:
      - zookeeper
    volumes:
      - /Users/myuser/docker/volumes/kafka:/var/lib/kafka/data

Can anyone help, what am I doing wrong?

Tags:

2

Answers


  1. 0

    When I am running the app without dockerizing it, it is working fine.

    Basically, you want your app to connect to Kafka in both cases right?

    • when you’re running your app locally (on the host machine)
    • and when you’re running it as a dockerized application.

    But you’re only advertising one listener for 127.0.0.1:9092 (the host machine) so even if a dockerized client (your app) can access Kafka container, it will still fail to establish a connection because of a misleading listener configuration.

    For example, I can use this for advertising two different listeners for two different networks (docker network and localhost on the host machine):

    KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092

    And when I run my dockerized application, it can connect to Kafka via broker:29092 and, similarly, when I run the app on the host machine, it can connect to Kafka via localhost:9092.

    This post gives a more detailed explanation of how Kafka advertised listeners work and how should we configure them. It basically says:

    You need to set advertised.listeners (or KAFKA_ADVERTISED_LISTENERS if
    you’re using Docker images) to the external address (host/IP) so that
    clients can correctly connect to it. Otherwise they’ll try to connect
    to the internal host address–and if that’s not reachable then problems
    ensue.

    Login or Signup to reply.
  2. 0

    Here is your answer,

    • You must use host ip address. Do not use 127.0.0.1 or container ip address
    • Because, the brokers are advertising its ADVERTISED_LISTENERS to their clients, and clients try to connect to this addresses.
    • So, the ip addresses in the ADVERTISED_LISTENERS must accessible from outside container or other containers.
    • Refer following example
    ...
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://<hostip>:9092
...
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search