I have an EC2 machine with mysql DB (not containerized). Now I want to stream CDC events to Confluent-managed Kafka. I am using this docker compose file to start the connect platform on the same host. services: connect: image: confluentinc/cp-kafka-connect:latest…
I am trying to integrate Spring Cloud Stream Schema Registry with MongoDB as the Schema Registry backend. I just want to know with the current implementation will this be feasible and what kinds of changes will be required to achieve…
I'm building a Jira clone using a microservice architecture with the following services: User Service: Manages user-related operations. Task Service: Handles task-related operations. Auth Service: Manages authentication. I'm struggling with model separation and data access across microservices. For instance, when…
For broker 1 bin/kafka-configs.sh --bootstrap-server b-1.kaffka.6r40l0.c4.kafka.eu-central-1.amazonaws.com:9094 --entity-type brokers --entity-name 1 --alter --command-config client.properties --add-config advertised.listeners=[INTERNAL://b-1.kaffka.6r40l0.c4.kafka.eu-central-1.amazonaws.com:9096,EXTERNAL://MSK-kafka-Loadbalancer-25831a7b958a5031.elb.eu-central-1.amazonaws.com:9094] For broker 2 bin/kafka-configs.sh --bootstrap-server b-2.kaffka.6r40l0.c4.kafka.eu-central-1.amazonaws.com:9094 --entity-type brokers --entity-name 2 --alter --command-config client.properties --add-config advertised.listeners=[INTERNAL://b-2.kaffka.6r40l0.c4.kafka.eu-central-1.amazonaws.com:9096,EXTERNAL://MSK-kafka-Loadbalancer-25831a7b958a5031.elb.eu-central-1.amazonaws.com:9095], all works fine here bin/kafka-topics.sh --list --bootstrap-server b-1.kaffka.6r40l0.c4.kafka.eu-central-1.amazonaws.com:9096 --command-config client.…
I have two docker containers, one running Kafka and one running python with the Kafka client. I am attempting to set the container with the python client as a consumer for the container running Kafka. both containers are on the…
I have Java Spring Boot microservice app using Postgresql and Kafka on MacOS. There are 2 modules - filmservice and eventservice. Filmservice got kafka producer, eventservice got kafka consumer. When I try to start it in Docker container - application…
I have successfully created a mariadb database connection using debezium and kafka When I tried to stream the topic using pyspark this is the output that I get ------------------------------------------- Batch: 0 ------------------------------------------- +------+--------------------------------------------------------------------------------------------------------------------------+ |key |value | +------+--------------------------------------------------------------------------------------------------------------------------+ ||MaxDoe1.4.2.Finalnmysqlmariadbbtruebasecampemployees mysql-bin.000032�r�ȯݭd |…
We currently have an "UnwritableMetadataException" error starting Kafka in our test environment. Unfortunately, I don't know whether it is due to an incorrect configuration or something similar. Kafka is running in a Docker Container, but I don't think that's the…
I am writing my first Kafka based project and experience some issues with integration. Kafka and Kafka Connect. Here is my docker-compose.yaml version: '3.3' services: database: image: 'postgres:16.2-alpine' restart: always shm_size: 128mb ports: - 5432:5432 env_file: - ./database/.env volumes: -…
Hello guys i wanna ask some question about CDC mysql to kafka with azure database for mysql. I already following this tutorial: https://techcommunity.microsoft.com/t5/azure-database-for-mysql-blog/cdc-in-azure-database-for-mysql-flexible-server-using-kafka/ba-p/2780943 but got stuck while creating kafka conncetor on this part, and the error: {"error_code":400,"message":"Connector configuration is invalid…