skip to Main Content

I have two docker containers, one running Kafka and one running python with the Kafka client. I am attempting to set the container with the python client as a consumer for the container running Kafka. both containers are on the same network. but then i am running my code to connect to the container i get the same error code:
your %3|1725885842.079|FAIL|rdkafka#consumer-1| [thrd:localhost:172/bootstrap]: localhost:172/bootstrap: Connect to ipv4#127.0.0.1:172 failed: Connection refused (after 0ms in state CONNECT)text
below i will attach my docker network code and my python code:

docker inspect my-network
[
    {
        "Name": "my-network",
        "Id": "c920653a8c4d9e068887bce98dab2f5bf2f554e112a2cbd337d3a7feefc45b5b",
        "Created": "2024-09-06T14:39:28.375456878Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.20.0.0/16",
                    "Gateway": "172.20.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "02bf3754f2b27276db07762cf6237872dfa36bcd85d4e1feb7d7d6f13ff97a82": {
                "Name": "broker",
                "EndpointID": "13d97b89c9c4980c7c71b25e8aba4fc94d9a60b344720fca44fb66adc40af15b",        
                "MacAddress": "02:42:ac:14:00:03",
                "IPv4Address": "172.20.0.3/16",
                "IPv6Address": ""
            },
            "8fb0fab1c6c8f97033739a5936661e04df02de25c6a77cc9bb4c8259b21095a7": {
                "Name": "agitated_shirley",
                "EndpointID": "9c14de0df28233489e56b5354bd7621239eca948b5c4c7adabf3fccf6a98779d",        
                "MacAddress": "02:42:ac:14:00:02",
                "IPv4Address": "172.20.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]
from typing import Union
from fastapi import FastAPI
from pydantic import BaseModel
from confluent_kafka import Consumer

conf = {'bootstrap.servers': 'localhost:172.20.0.3', 'group.id':'main-group','auto.offset.reset':'smallest'}

consumer = Consumer(conf)


running = True

i have tried creating new networks and containers from scratch. what am i doing wrong?

3

Answers


  1. First settings in your config

    'bootstrap.servers': 'localhost:172.20.0.3',
    

    should look like host:port, something like

    'bootstrap.servers': 'localhost:9092'
    

    (replace 9092 with your kafka port if you have another)

    Login or Signup to reply.
  2. bootstrap.servers likely expects the address in the format host:[port]. Your error message tells you that it tries to connect to 127.0.0.1:172 (or localhost:172).

    Change it to {'bootstrap.servers': '172.20.0.3'} or use the DNS name in the network {'bootstrap.servers': 'broker'}.

    Login or Signup to reply.
  3. The localhost refers to the container itself, if you are running in same network.

    So change the localhost to container name

    conf = {'bootstrap.servers': 'broker:9092', 'group.id':'main-group', 'auto.offset.reset':'smallest'}
    

    Change the kafka listening properties from server.properties file

    advertised.listeners=PLAINTEXT://broker:9092
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search