skip to Main Content

I’m trying to use Debezium with Kafka connect, I followed this tutorial, and everything connected just fine. However, the problem is that I cannot access Kafka from outside of docker containers anymore.

I use these commands to start containers:

docker run -it --rm --name zookeeper -p 2181:2181 -p 2888:2888 -p 3888:3888 debezium/zookeeper:2.0.0.Beta1

docker run -it --rm --name kafka -p 9092:9092 --link zookeeper:zookeeper debezium/kafka:2.0.0.Beta1

docker run -it --rm --name connect -p 8083:8083 -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my_connect_configs -e OFFSET_STORAGE_TOPIC=my_connect_offsets --link kafka:kafka debezium/connect:2.0.0.Beta1

I tried to set KAFKA_ADVERTISED_LISTENERS to PLAINTEXT://127.0.0.1:9092 which allowed me to connect to Kafka from the outside of the container but I could not connect from connect container to kafka container anymore. How can I achieve both?

2

Answers


  1. I think it’s not a Kafka issue, but a docker network one. It’s probably accessible via docker network or you need to expose it. https://docs.docker.com/network/network-tutorial-standalone/

    Login or Signup to reply.
  2. with this you can access the kafka container from your host on the port 9092

      zookeeper:
        image: confluentinc/cp-zookeeper:7.2.0
        ports:
          - "2181:2181"
        environment:
          ZOOKEEPER_CLIENT_PORT: 2181
          ZOOKEEPER_TICK_TIME: 2000
    
      kafka-broker:
        image: confluentinc/cp-kafka:7.2.0
        depends_on:
          - zookeeper
        ports:
          - "9092:9092"
        environment:
          KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
          KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:29092,OUTSIDE://0.0.0.0:9092
          KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-broker:29092,OUTSIDE://localhost:9092
          KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,OUTSIDE:PLAINTEXT
          KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search