Question 188625 in Docker Question posted in
The official documentation can be found here.

Docker – None of my applications can see my kafka even though they are running on the same network?

my docker-compose.yml file is:

version: '3'
services:
  kafka1:
    image: confluentinc/cp-kafka:7.3.2
    volumes:
      - ./init-scripts/kafka:/docker-entrypoint-initdb.d
    ports:
      - "9092:9092"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zoo1:2181
      KAFKA_LISTENERS: INTERNAL://:9092,EXTERNAL://:9093
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka1:9092,EXTERNAL://localhost:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    depends_on:
      - zoo1
    networks:
      - mynetwork

  zoo1:
    image: confluentinc/cp-zookeeper:7.3.2
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
    networks:
      - mynetwork

  cassandra:
    image: cassandra:latest
    volumes:
      - ./init-scripts/cassandra:/docker-entrypoint-initdb.d
    ports:
      - "9042:9042"

  flask-app:
    build: .
    volumes:
      - .:/app
    ports:
      - "5000:5000"
    depends_on:
      - kafka1
      - cassandra

networks:
  mynetwork:
    driver: bridge

I have a flask server running on this but it gives me NoBrokersAvaliable error. Even though my kafka and app is running on the same network, they can’t communicate with each other.

I have another container named esp32_simulator which has a docker-compose file like this:

version: '3'
services:

  esp32_simulator:
    build: .
    volumes:
      - ./:/app
    networks:
      - mynetwork

networks:
  mynetwork:
    external:
      name: bitirmetam_mynetwork

But when I run this app with kafka1:9092 it says NoBrokersAvaliable too. I tried localhost and everything but still gives errors.

Tags:

2

Answers


  1. 0

    on the same network

    But they aren’t. Your flask app isn’t using mynetwork

    Using localhost in your flask app also isn’t the address for the kafka broker

    Also unclear where your kafka process starts since you didn’t show the code, but the Confluent container takes about 20 seconds to fully start up, and depends_on will not wait for this. You’ll need to add an explicit sleep/port check/retry-loop somewhere in your Python code before creating a Kafka client

    Login or Signup to reply.
  2. 0

    One issue i see is that you are exposing post 9092 which is the internal config, but you should expose port 9093 instead.

    What happens when you connect to a kafka broker is that the application connects to a kafka broker, the broker looks internally which connection it should use (in your case INTERNAL or EXTERNAL). Since you connect via 9092, it will use INTERNAL.

    The broker then looks into "AdvertisedListener" list for internal and returns that value, which is

    http://KAFKA:9092

    but this url doesnt work obv if you connect from outside the docker network.

    Again, just change the exposed port from 9092:9092 to 9093:9093 and it should work

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search