I'm having authentication issues when I try to connect a producer following the instruction of the helm chart installation: I'm installing this kafka helm chart: https://artifacthub.io/packages/helm/bitnami/kafka?modal=install I can perfectly install it with helm install, and this is the output from…
I have a working jdbc sink connector configuration for postgres using username and password configuration: { "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector", "connection.url": "jdbc:postgresql://localhost:5432/postgres", "connection.user": "postgres-user", "connection.password": "postgres-pw", "topics.regex": "my-topic", "transforms": "unwrap", "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState", "transforms.unwrap.drop.tombstones": "false", "key.converter": "org.apache.kafka.connect.json.JsonConverter", "key.converter.schemas.enable": "false", "value.converter": "org.apache.kafka.connect.json.JsonConverter", "value.converter.schemas.enable": "false",…
Hi I have been stuck in a problem since two weeks, I am trying to upload a CSV in kafka partition using a CSV source connector. First, I tried on windows 11 using first.row.as.header:true, Source connector was created but it's…
I'm a newbie to Kafka and started experimenting. I'm using this tutorial https://medium.com/high-alpha/data-stream-processing-for-newbies-with-kafka-ksql-and-postgres-c30309cfaaf8 I ran the following docker-compose.yml version: '2' services: postgres: image: 'debezium/postgres:11' environment: POSTGRES_PASSWORD: postgres command: postgres -c config_file=/home/config/postgresql.conf ports: - '5432:5432' volumes: - 'C:postgres-kafkaconfig:/home/config' - 'C:postgres-kafkaconfigpg_hba.conf:/home/config/pg_hba.conf' -…
I have a stack which consists of 3 Kafka instances a producer and a consumer and a schema registry. The producer sends a topic "Person" all the time and the consumer just receives it. I have created an AVRO schema…
I'm learning Kafka via Conduktor. I ran this command on their get-started page: curl -L https://releases.conduktor.io/console -o docker-compose.yml && docker compose up -d --wait && echo "Conduktor started on http://localhost:8080" Now when I visit the console and view the connection,…
I have the following project: https://github.com/ably/kafka-connect-ably Running the dockerfile locally works perfectly well. I have tried a few methods to get it working in k8s... I have tried Kompose. This created the correct .yaml files, as well as a persistent…
I'm building a Go application that reads messages from a Kafka topic and writes them to a PostgreSQL database. I've set up a loop that reads messages from Kafka using a kafka.Reader and inserts them into the database using a…
My Kafka cluster is IAM auth enabled. I am successfully able to produce and consume messages from topic test-topic2 by assuming the correct IAM role if the policy is as follows; { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowKafkaTopicWrite", "Effect":…
I am trying to sink data from Kafka to Postgres and the following is what I have configured - My zookeeper, Kafka, Schema registry & Kafka Connect is working fine Even i have configured a Mongo Sink Connector which works…