skip to Main Content

I am attempting to setup confluent kafka v5.4 and running the prometheus JMX exporter. I have found this blog for how to get this setup https://alex.dzyoba.com/blog/jmx-exporter/ . Kafka is setup and runs just fine but the endpoint on port 8080 returns nothing. I’ve tried just about everything for how I call the javaagent in the systemd script but nothing seems to work.

Description=Confluent Kafka Broker
After=network.target network-online.target remote-fs.target  zookeeper.service

[Service]
Type=forking
User=confluent
Group=confluent
Environment="KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false  -Dcom.sun.management.jmxremote.ssl=false -javaagent=/opt/prometheus/jmx_prometheus_javaagent.jar=8080:/opt/prometheus/config.yaml"
Environment=LOG_DIR=/var/log/confluent
ExecStart=/opt/confluent/confluent-5.4.0/bin/kafka-server-start -daemon /opt/confluent/confluent-5.4.0/etc/kafka/server.properties
ExecStop=/opt/confluent/confluent-5.4.0/bin/kafka-server-stop
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

Any ideas on how to call that java agent in the systemd script to get it to work correctly? I have tried multiple options for calling the OPTS with none of them working. I’ve tried putting the -javaagent command in KAFKA_OPTS. Feel like I’ve tried just about every option. Kafka logs don’t give any clues and I’m not sure of where else to look at logs for this type of issue.

OS Centos 7 JMX exporter 0.12.0 Java openJDK 11

Logs that I have found are not telling me anything as to why it’s not running. Maybe I’m looking at the wrong logs.

Edit:

conflue+ 11578 47.4 13.8 8679808 536764 ?      Sl   11:59   0:35 java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -Djava.awt.headless=true -Xlog:gc*:file=/var/log/confluent/kafkaServer-gc.log:time,tags:filecount=10,filesize=102400 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -javaagent=/opt/prometheus/jmx_prometheus_javaagent.jar=8080:/opt/prometheus/config.yaml -Dkafka.logs.dir=/var/log/confluent -Dlog4j.configuration=file:/etc/kafka/log4j.properties -cp /opt/confluent/confluent-5.4.0/bin/../ce-broker-plugins/build/libs/*:/opt/confluent/confluent-5.4.0/bin/../ce-broker-plugins/build/dependant-libs/*:/opt/confluent/confluent-5.4.0/bin/../ce-auth-providers/build/libs/*:/opt/confluent/confluent-5.4.0/bin/../ce-auth-providers/build/dependant-libs/*:/opt/confluent/confluent-5.4.0/bin/../ce-rest-server/build/libs/*:/opt/confluent/confluent-5.4.0/bin/../ce-rest-server/build/dependant-libs/*:/opt/confluent/confluent-5.4.0/bin/../ce-audit/build/libs/*:/opt/confluent/confluent-5.4.0/bin/../ce-audit/build/dependant-libs/*:/opt/confluent/confluent-5.4.0/bin/../share/java/kafka/*:/opt/confluent/confluent-5.4.0/bin/../share/java/confluent-metadata-service/*:/opt/confluent/confluent-5.4.0/bin/../share/java/rest-utils/*:/opt/confluent/confluent-5.4.0/bin/../share/java/confluent-common/*:/opt/confluent/confluent-5.4.0/bin/../share/java/confluent-security/schema-validator/*:/opt/confluent/confluent-5.4.0/bin/../support-metrics-client/build/dependant-libs-2.12.10/*:/opt/confluent/confluent-5.4.0/bin/../support-metrics-client/build/libs/*:/usr/share/java/support-metrics-client/*:/opt/confluent/confluent-5.4.0/bin/../support-metrics-fullcollector/build/dependant-libs-2.12.10/*:/opt/confluent/confluent-5.4.0/bin/../support-metrics-fullcollector/build/libs/*:/usr/share/java/support-metrics-fullcollector/* io.confluent.support.metrics.SupportedKafka /opt/confluent/confluent-5.4.0/etc/kafka/server.properties

2

Answers


  1. Chosen as BEST ANSWER

    Figured this out ExecStart=/opt/confluent/confluent-5.4.0/bin/kafka-server-start -daemon /opt/confluent/confluent-5.4.0/etc/kafka/server.properties

    was not correct

    ExecStart=/opt/confluent/confluent-5.4.0/bin/kafka-server-start -daemon /etc/kafka/server.properties

    was correct, even though the 2 are symlinked.


  2. Make sure you run systemctl daemon-reload each time you edit a service file.


    Also, I would suggest not using LOG_DIR, removing the RollingFileAppender from the log4j.properties, then letting journalctl handle all your logging from SystemD

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search