I'm following this guide https://kafka.apache.org/quickstart to run kafka locally using docker. On step 2 I run docker pull apache/kafka:3.7.0 docker run -p 9092:9092 apache/kafka:3.7.0 Then, on step 3, I login into the container using Docker Desktop and run bin/kafka-topics.sh --create…
I'm trying to set up Change Data Capture (CDC) from a PostgreSQL database to Kafka using Debezium. I have successfully configured Debezium for CDC from a MySQL database, but I'm encountering issues when attempting to connect to a PostgreSQL database.…
I have a redpanda container which I want to use as a sink using vector when I am running this nomad job it is giving this error Feb 27 08:57:01.931 ERROR rdkafka::client: librdkafka: Global error: Resolve (Local: Host resolution failure):…
I have a dockerized spring-boot web service that needs to produce to a Kafka topic. I'm running Kafka with Zookeeper locally on Ubuntu. My web service is running in Docker on the same machine. I have confirmed with netstat -tlpn…
i want to send this simple pyspark dataframe to kafka and i done everything but i always get an error. i try it with simple python producer script and it works and the pyspark read stream work my only problem…
I have a JDBC sink connector retrieving data from a Postgres source connector, and I need to copy the data from my "parametros" source table to my "parametros_sistema" target table. I'm using the org.apache.kafka.connect.transforms.ReplaceField$Value transformer to target the specific columns…
I am trying to connect kafka and aws lambda on a different hosting provider. When I connect via AWS EC2, it works fine because they are in the same VPC, but when I try to connect kafka and lambda on…
I have an event schema stored in JSON format in Azure Event Hubs Schema Registry. I need to read that schema in my C# code, without hard coding the schema in the code. Schema needs to be read into a…
I'm having authentication issues when I try to connect a producer following the instruction of the helm chart installation: I'm installing this kafka helm chart: https://artifacthub.io/packages/helm/bitnami/kafka?modal=install I can perfectly install it with helm install, and this is the output from…
I have a working jdbc sink connector configuration for postgres using username and password configuration: { "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector", "connection.url": "jdbc:postgresql://localhost:5432/postgres", "connection.user": "postgres-user", "connection.password": "postgres-pw", "topics.regex": "my-topic", "transforms": "unwrap", "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState", "transforms.unwrap.drop.tombstones": "false", "key.converter": "org.apache.kafka.connect.json.JsonConverter", "key.converter.schemas.enable": "false", "value.converter": "org.apache.kafka.connect.json.JsonConverter", "value.converter.schemas.enable": "false",…