Question posted in
The official documentation can be found here.

Kafka + Java app. in Docker on local computer

I have Java Spring Boot microservice app using Postgresql and Kafka on MacOS. There are 2 modules – filmservice and eventservice. Filmservice got kafka producer, eventservice got kafka consumer.
When I try to start it in Docker container – application modules can’t connect to Kafka server.

filmservice(producer) got this error:

2024-09-05 14:12:10 filminator-filmservice  | 2024-09-05T11:12:10.978Z  INFO 1 --- [           main] o.a.k.c.c.internals.LegacyKafkaConsumer  : [Consumer clientId=consumer-my-group-id-1, groupId=my-group-id] Subscribed to topic(s): my-topic
2024-09-05 14:12:10 filminator-filmservice  | 2024-09-05T11:12:10.990Z  INFO 1 --- [           main] c.z.f.filmservice.FilminatorFilmService  : Started FilminatorFilmService in 1.974 seconds (process running for 2.223)
2024-09-05 14:12:11 filminator-filmservice  | 2024-09-05T11:12:11.102Z  INFO 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-my-group-id-1, groupId=my-group-id] Node -1 disconnected.
2024-09-05 14:12:11 filminator-filmservice  | 2024-09-05T11:12:11.103Z  WARN 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-my-group-id-1, groupId=my-group-id] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Node may not be available.
2024-09-05 14:12:11 filminator-filmservice  | 2024-09-05T11:12:11.103Z  WARN 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-my-group-id-1, groupId=my-group-id] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected

eventservice(consumer) got this error:

2024-09-05 14:14:58 filminator-eventservice  | 2024-09-05T11:14:58.843Z  WARN 1 --- [ntainer#1-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-my-group-id-2, groupId=my-group-id] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Node may not be available.

I try to stop application in container and start in from IDE – app successfully connected to container Kafka server.

docker-compose

services:

  zookeeper:
    image: 'bitnami/zookeeper:latest'
    ports:
      - '2182:2182'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes

  kafka:
    image: 'bitnami/kafka:latest'
    ports:
      - '9092:9092'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_LISTENERS=PLAINTEXT://:9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    depends_on:
      - zookeeper

  userservice:
    build: eventservice
    image: eventservice-image
    container_name: filminator-eventservice
    ports:
      - "8082:8082"
    depends_on:
      - kafka
      - filmservice
      - postgres_db
    environment:
      - SPRING_DATASOURCE_URL=jdbc:postgresql://postgres_db:5432/filminator

  filmservice:
      build: filmservice
      image: filmservice-image
      container_name: filminator-filmservice
      ports:
        - "8081:8081"
      depends_on:
        - kafka
        - postgres_db
      environment:
        - SPRING_DATASOURCE_URL=jdbc:postgresql://postgres_db:5432/filminator


  postgres_db:
      image: postgres:13.7-alpine
      volumes:
        - /var/lib/postgresql/data/
      container_name: db_postgres
      ports:
        - "6541:5432"
      environment:
        - POSTGRES_DB=filminator
        - POSTGRES_USER=postgres
        - POSTGRES_PASSWORD=iamroot

Dockerfile:

FROM openjdk:19
COPY target/*.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

application.properties

filmservice:

server.port=8081
spring.datasource.driver-class-name=org.postgresql.Driver
spring.sql.init.mode=always
spring.datasource.url=jdbc:postgresql://localhost:5432/filminator
spring.datasource.username=postgres
spring.datasource.password=iamroot

spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=my-group-id

eventservice:

server.port=8082

other eventservice application.properties identical to filmservice

Is there way to launch my app and Kafka server in one docker container?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I solved the problem.

    In docker-compose file I change zookeeper and kafka services settings:

     zookeeper:
  image: confluentinc/cp-zookeeper:latest
  environment:
    ZOOKEEPER_CLIENT_PORT: 2181
    ZOOKEEPER_TICK_TIME: 2000
  ports:
    - 22181:2181

kafka:
  image: confluentinc/cp-kafka:latest
  depends_on:
    - zookeeper
  ports:
    - 29092:29092
  environment:
    KAFKA_BROKER_ID: 1
    KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
    KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
    KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
    KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

    in application.properties change bootstrap-servers from localhost to kafka : spring.kafka.bootstrap-servers=kafka:9092

    In my consumers/producer config classes change from localhost to kafka :

    configProps.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "kafka:9092");

    Now it works.


  2. 0

    You need to add a host to your docker-compose file, see this example

      kafka:
    image: 'bitnami/kafka:latest'
    hostname: kafka
    ports:
      - '9092:9092'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_LISTENERS=PLAINTEXT://:9092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    depends_on:
      - zookeeper

    Then you can connect inside the docker network like this

    spring.kafka.bootstrap-servers=kafka:9092

    I think the hostname defaults to the container name.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search