I have a Kafka Consumer service @KafkaListener( topics = "topic1", groupId = "cluster1", containerFactory = "KafkaListenerContainerFactory") public void consume(Message message) { logger.info(String.format("Message recieved -> %s", message.getMsg())); Long id = message.getId() RepoDetail repoDetail = testRepo.findByID(id); logger.info( String.format( "Message -> %s", repoDetail.getMessage()));…
I am running a docker file to execute a debezium service as entrypoint. It works fine and executes debezium service when I use the exact file path as parameter to Entrypoint command in my docker file command in docker file…
I have a docker-compose.yml file: version: '2' services: zookeeper: image: confluentinc/cp-zookeeper:latest environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 ports: - 2181:2181 kafka: image: confluentinc/cp-kafka:latest depends_on: - zookeeper ports: - 9092:9092 environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR:…
I installed PHP 7.4 and RdKafka by using PECL however the module didn't start, here is the output of php -v PHP Warning: PHP Startup: rdkafka: Unable to initialize module Module compiled with module API=20210902 PHP compiled with module API=20190902…
I'm trying to connect apache kafka 2.x to eclipse ditto in locally with default configurations. I'm following this https://www.eclipse.org/ditto/connectivity-protocol-bindings-kafka2.html, so I tried to create a new connection with a POST request, as follows: curl -X POST -i -u devops:foobar -H…
We have several collections in Mongo based on n tenants and want the kafka connector to only watch for specific collections. Below is my mongosource.properties file where I have added the pipeline filter to listen only to specific collections.It works…
I am using the confluent-kafka Python client in my project. I'm trying to create a Docker image with this client. I am facing the following error:- #11 8.015 [pipenv.exceptions.InstallError]: In file included from /tmp/pip-install-so_whhii/confluent-kafka_9d9553bf46cf489bb25fcb2ac7698747/src/confluent_kafka/src/Admin.c:17: #11 8.015 [pipenv.exceptions.InstallError]: /tmp/pip-install-so_whhii/confluent-kafka_9d9553bf46cf489bb25fcb2ac7698747/src/confluent_kafka/src/confluent_kafka.h:23:10: fatal error:…
I'm using two microservices in order to communicate with each other using Kafka. On one microservice there is the listener, and on the other microservice, the sender. The problem is that whenever producer sends a message, consumer will get 20…
I have two containers one is kafka container open port 9092:9092 and another is Fastapi container. If I don't dockerize Fastapi, I can make rest api request to fastapi to kafka. It sends message to kafka via fastapi. But when…
I have a kafka topic which receives messages like Kafka key: serverId value: server host name I just need the latest ones for each serverId so it's a very small segment size and log compaction is enabled. So what I…