I have a scenario where I have a topic_source which will have all generated messages by another application, these json/messages might be duplicated, so i need to deduplicate the messages `based on "window" size , say for every 10 sec…
I have an application (call it smscb-router) as shown in the diagram. It reads data from a legacy system (sms). Based on the content (callback type), I have to put into corresponding outgoing topic (such as billing-n-cdr, dr-cdr, ...) I…
I have a KStreams application running inside a Docker container which uses a persistent key-value store. My runtime environment is Docker 1.13.1 on RHEL 7. I have configured state.dir with a value of /tmp/kafka-streams (which is the default). When I…
It's my first Kafka program. From a kafka_2.13-3.1.0 instance, I created a Kafka topic poids_garmin_brut and filled it with this csv: kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic poids_garmin_brut kafka-console-producer.sh --broker-list localhost:9092 --topic poids_garmin_brut < "Poids(1).csv" Durée,Poids,Variation,IMC,Masse grasse,Masse…
is there any way, in Kafka, to produce a message once several related messages have been consumed ? (without having to manually control it at the application code...) The use case would be to pick a huge file, split it…
As I got right from book, Kafka Streams state store it is a memory key/value storage to store data to Kafka or after filtering. I am confused by some theoretical questions. What is differenct Kafka streams state from another memory…
When there are updates done to an underlying topic of a GlobalKTable, what is the logic for all instances of KStream apps to get the latest data? Below are my follow up questions: Would the GlobalKTable be locked at record…
I have a stream of events i need to match against a ktable / changelog topic but the matching is done by pattern matching on a property of the ktable entries. so i cannot join the streams based on a…