Question 85553 in Docker Question posted in
The official documentation can be found here.

How to run mulitple Kafka brokers using docker-compose?

I am trying to run multiple Kafka brokers with docker-compose. I am going to use it both in Docker network and on local machine, that’s why I am using different ports for brokers:

version: '3'

services:
  zookeeper_local_dev:
    image: wurstmeister/zookeeper
    container_name: zookeeper_local_dev
    ports:
      - "2181:2181"
  kafka_local_dev_1:
    image: wurstmeister/kafka
    hostname: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ADVERTISED_HOST_NAME: kafka
      KAFKA_ZOOKEEPER_CONNECT: zookeeper_local_dev:2181
      KAFKA_LISTENERS: INTERNAL://kafka:9092
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
    depends_on:
      - zookeeper_local_dev
  kafka_local_dev_2:
    image: wurstmeister/kafka
    hostname: kafka
    ports:
      - "9093:9093"
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ADVERTISED_PORT: 9093
      KAFKA_ADVERTISED_HOST_NAME: kafka2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper_local_dev:2181
      KAFKA_LISTENERS: INTERNAL://kafka2:9093
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka2:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
    depends_on:
      - zookeeper_local_dev

I am getting an error for the second broker when it’s started:

org.apache.kafka.common.KafkaException: Socket server failed to bind to kafka2:9093: Unresolved address.

Could anybody assist?

Tags:

2

Answers


  1. 0

    Try this …

    • the point is hostname
    ...
  kafka_local_dev_2:
    image: wurstmeister/kafka
    hostname: kafka2
    ports:
      - "9093:9093"
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ADVERTISED_PORT: 9093
      KAFKA_ADVERTISED_HOST_NAME: kafka2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper_local_dev:2181
      KAFKA_LISTENERS: INTERNAL://kafka2:9093
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka2:9093
...
    Login or Signup to reply.
  2. 0

    You should not change the internal port of kafka broker. It will remain 9092 which is exposed by the kafka container.
    Try below

    kafka_local_dev_2:
    image: wurstmeister/kafka
    hostname: kafka
    ports:
      - "9093:9092"
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ADVERTISED_PORT: 9093
      KAFKA_ADVERTISED_HOST_NAME: kafka2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper_local_dev:2181
      KAFKA_LISTENERS: INTERNAL://kafka2:9092
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://localhost:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
    depends_on:
      - zookeeper_local_dev
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search