I'm trying to connect apache kafka 2.x to eclipse ditto in locally with default configurations. I'm following this https://www.eclipse.org/ditto/connectivity-protocol-bindings-kafka2.html, so I tried to create a new connection with a POST request, as follows: curl -X POST -i -u devops:foobar -H…
We have several collections in Mongo based on n tenants and want the kafka connector to only watch for specific collections. Below is my mongosource.properties file where I have added the pipeline filter to listen only to specific collections.It works…
I am using the confluent-kafka Python client in my project. I'm trying to create a Docker image with this client. I am facing the following error:- #11 8.015 [pipenv.exceptions.InstallError]: In file included from /tmp/pip-install-so_whhii/confluent-kafka_9d9553bf46cf489bb25fcb2ac7698747/src/confluent_kafka/src/Admin.c:17: #11 8.015 [pipenv.exceptions.InstallError]: /tmp/pip-install-so_whhii/confluent-kafka_9d9553bf46cf489bb25fcb2ac7698747/src/confluent_kafka/src/confluent_kafka.h:23:10: fatal error:…
I'm using two microservices in order to communicate with each other using Kafka. On one microservice there is the listener, and on the other microservice, the sender. The problem is that whenever producer sends a message, consumer will get 20…
I have two containers one is kafka container open port 9092:9092 and another is Fastapi container. If I don't dockerize Fastapi, I can make rest api request to fastapi to kafka. It sends message to kafka via fastapi. But when…
I have a kafka topic which receives messages like Kafka key: serverId value: server host name I just need the latest ones for each serverId so it's a very small segment size and log compaction is enabled. So what I…
I'm using the Wurstmeister Docker Image to try to run a very basic single broker Kafka server just for testing my application. It's a simple docker-compose.yml: version: '2' services: zookeeper: image: wurstmeister/zookeeper ports: - '2181:2181' kafka: image: wurstmeister/kafka ports: -…
It's my first Kafka program. From a kafka_2.13-3.1.0 instance, I created a Kafka topic poids_garmin_brut and filled it with this csv: kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic poids_garmin_brut kafka-console-producer.sh --broker-list localhost:9092 --topic poids_garmin_brut < "Poids(1).csv" Durée,Poids,Variation,IMC,Masse grasse,Masse…
I'm following https://github.com/PacktPublishing/Apache-Kafka-Series---Kafka-Connect-Hands-on-Learning and I've below docker-compose file and using Mac. version: '2' services: # this is our kafka cluster. kafka-cluster: image: landoop/fast-data-dev:cp3.3.0 environment: ADV_HOST: localhost # Change to 192.168.99.100 if using Docker Toolbox RUNTESTS: 0 # Disable Running tests…
I want to determine the exact version of Kafka inside confluentinc/cp-kafka:6.2.2 image. On this site, https://docs.confluent.io/platform/current/installation/versions-interoperability.html#cp-and-apache-ak-compatibility I have read that inside image with tag 6.2.x kafka version 2.8.x is being used but I want to determine the exact version of…