Redis – Kafka Streams join unrelated streams
I have a stream of events i need to match against a ktable / changelog topic but the matching is done by pattern matching on a property of the ktable entries. so i cannot join the streams based on a…
Kafka MirrorMaker 2.0 duplicate each messages – Debian
I am trying to replicate Kafka cluster with MirrorMaker 2.0. I am using following mm2.properties: name = mirror-site1-site2 topics = .* connector.class = org.apache.kafka.connect.mirror.MirrorSourceConnector tasks.max = 1 plugin.path=/usr/share/java/kafka/plugin clusters = site1, site2 # for demo, source and target clusters are…
kafka sarama lib how to know how many cosumers in a cosumergroup – Shopify
I read the consumer group example in here(provided by Shopify/sarama), and I wonder that how would I know how many consumers are in a consumer group, are there any ways I could control the number of consumers? Thanks a lot
Kafka producer/consumer are opening too many file descriptor – CentOS
We have a 3 node Kafka cluster deployment with 5 topics and 6 partitions per topic. we have configured the replication factor =3 , we are seeing very strange problem that number of file descriptor have been crossed the ulimit…
KAFKA and SSL : java.lang.OutOfMemoryError: Java heap space when using kafka-topics command on KAFKA SSL cluster – CentOS
this is my first post on Stackoverflow, i hope i didnt choose the wrong section. Context : Kafka HEAP size is configured on following file : /etc/systemd/system/kafka.service With following parameter : Environment="KAFKA_HEAP_OPTS=-Xms6g -Xmx6g" OS is "CentOS Linux release 7.7.1908". Kafka…
Apache Kafka: bootstrap-server is not a recognized option – CentOS
I'm trying to set up Apache Kafka to communicate between two virtual machines running CentOS on the same network. I originally set up a Kafka producer and consumer on one machine and everything was running smoothly. I then set up…
When spark streaming data from Kafka it shows connection timeout exception in Horton works – Twitter API
I want to stream data using spark from Kafka topic in Horton works. I have started the zookeeper and Kafka server. Then I have successfully built a maven project in Eclipse with all the streaming dependencies. Given below is the…
When spark streaming data from Kafka it shows connection timeout exception in Horton works – Twitter API
I want to stream data using spark from Kafka topic in Horton works. I have started the zookeeper and Kafka server. Then I have successfully built a maven project in Eclipse with all the streaming dependencies. Given below is the…
When spark streaming data from Kafka it shows connection timeout exception in Horton works – Twitter API
I want to stream data using spark from Kafka topic in Horton works. I have started the zookeeper and Kafka server. Then I have successfully built a maven project in Eclipse with all the streaming dependencies. Given below is the…
What technology should i use for streaming tweets and analysis? – Twitter API
I need to stream live tweets from twitter API and then analyse them. I should use kafka to get tweets or spark streaming directly or both ?