skip to Main Content

I tried to follow this guide to configure prometheus with synapse metrics: https://github.com/matrix-org/synapse/blob/master/docs/metrics-howto.md

I’m having difficulties though. Here’s how i set it up:

$ sudo ufw allow 9090

$ sudo nano /etc/matrix-synapse/homeserver.yaml

# in listeners: list
  - type: metrics
    port: 9000
    bind_addresses:
      - '0.0.0.0'

## Metrics ###

# Enable collection and rendering of performance metrics
#
enable_metrics: true

Restarted Synapse, installed Docker.

Create ‘/etc/prometheus/prometheus.yml’ edit it like so:

$ sudo nano /etc/prometheus/prometheus.yml

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']

  - job_name: "synapse"
    metrics_path: "/_synapse/metrics"
    scheme: "https"
    static_configs:
      - targets: ["localhost:9000"]

Attempt to start Prometheus:

$ docker run -p 9090:9090 -v /etc/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml –name prometheus prom/prometheus

level=info ts=2020-05-06T03:42:50.799Z caller=main.go:298 msg="no time or size retention was set so using the default time retention" duration=15d
level=info ts=2020-05-06T03:42:50.799Z caller=main.go:333 msg="Starting Prometheus" version="(version=2.17.2, branch=HEAD, revision=18254838fbe25dcc732c950ae05f78ed4db1292c)"
level=info ts=2020-05-06T03:42:50.799Z caller=main.go:334 build_context="(go=go1.13.10, user=root@9cb154c268a2, date=20200420-08:27:08)"
level=info ts=2020-05-06T03:42:50.800Z caller=main.go:335 host_details="(Linux 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1+deb10u1 (2020-04-27) x86_64 0cf4549b1dcd (none))"
level=info ts=2020-05-06T03:42:50.800Z caller=main.go:336 fd_limits="(soft=1048576, hard=1048576)"
level=info ts=2020-05-06T03:42:50.800Z caller=main.go:337 vm_limits="(soft=unlimited, hard=unlimited)"
level=info ts=2020-05-06T03:42:50.802Z caller=main.go:667 msg="Starting TSDB ..."
level=info ts=2020-05-06T03:42:50.802Z caller=web.go:515 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2020-05-06T03:42:50.806Z caller=head.go:575 component=tsdb msg="replaying WAL, this may take awhile"
level=info ts=2020-05-06T03:42:50.806Z caller=head.go:624 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0
level=info ts=2020-05-06T03:42:50.807Z caller=head.go:627 component=tsdb msg="WAL replay completed" duration=403.999µs
level=info ts=2020-05-06T03:42:50.808Z caller=main.go:683 fs_type=9123683e
level=info ts=2020-05-06T03:42:50.808Z caller=main.go:684 msg="TSDB started"
level=info ts=2020-05-06T03:42:50.808Z caller=main.go:788 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
level=info ts=2020-05-06T03:42:50.809Z caller=main.go:816 msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml
level=info ts=2020-05-06T03:42:50.810Z caller=main.go:635 msg="Server is ready to receive web requests."

But it just hangs taking the console away from me. 😛 I can detach it while running it at least.

I could see Prometheus at 192.168.1.171:9090 but it was not recording/showing synapse metrics. :/

Downloaded synapse-v2.rules from here: https://github.com/matrix-org/synapse/tree/master/contrib/prometheus

Edit: Thanks i edited out the https section, and pointed it to the new rules file, here is my prometheus.yml:

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  - "/etc/prometheus/synapse-v2.rules"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['localhost:9090']

  - job_name: "synapse"
    metrics_path: "/_synapse/metrics"
    static_configs:
      - targets: ["localhost:9000"]

Then i restarted the Prometheus docker but it still isn’t presenting the synapse metrics in the web dropdown.

There are prometheus statistics but no synapse ones.

3

Answers


  1. Chosen as BEST ANSWER

    Tried downloading the compiled package and running that instead, worked well.


  2. scheme: "https"

    Synapse’s metrics listener doesn’t use tls. Remove this.

    Login or Signup to reply.
  3. static_configs:
      - targets: ["localhost:9000"]
    

    $ docker run -p 9090:9090 -v /etc/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml –name prometheus prom/prometheus

    localhost here is the Docker container’s localhost. Only things within the Docker container will be reachable in there.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search