Nginx - Keycloak clustering

pooja
August 30, 2023
300 views
0 votes
2 Answers

For High availability and performance improvement, we wanted to setup keycloak in multiple instances.
As i had referred blogs and got to know the configuration has to be done in below files:

In conf/keycloak.conf

# Basic settings for running in production. Change accordingly before deploying the server.

# Database

# The database vendor.
db=mariadb

# The username of the database user.
db-username=<DB_USERNAME>

# The password of the database user.
db-password=<DB_PASSWORD>

# The full database JDBC URL. If not provided, a default URL is set based on the selected database vendor.
db-url=jdbc:mariadb://<DB_HOST>:3306/keycloak

# HTTP

# The file path to a server certificate or certificate chain in PEM format.
https-certificate-file=${kc.home.dir}/conf/fullchain.pem

# The file path to a private key in PEM format.
https-certificate-key-file=${kc.home.dir}/conf/privkey.pem

# The proxy address forwarding mode if the server is behind a reverse proxy.
proxy=edge

#The name of the header holding the client certificate
spi-x509cert-lookup-nginx-ssl-client-cert=SSL_CLIENT_CERT

# The prefix of the headers holding additional certificates in the chain
spi-x509cert-lookup-nginx-ssl-cert-chain-prefix=CERT_CHAIN

# The maximum length of the certificate chain.
spi-x509cert-lookup-nginx-certificate-chain-length=10

# Hostname for the Keycloak server.
hostname=<ip1>

# Hostname for the Keycloak server admin console.
#hostname-admin=<keycloak-management-hostname>

cache=ispn
cache-stack=tcp

In conf/cache-ispn-tcp-ping.xml

<jgroups>
<stacks><stack name="my-stack" extends="tcp">
<TCP bind_addr="${jgroups.bind.address:0.0.0.0.tcp.address:0.0.0.0}"
        bind_port="${jgroups.bind.port,jgroups.tcp.port:7800}"
        enable_diagnostics="true"
        thread_naming_pattern="pl"
        send_buf_size="640k"
        sock_conn_timeout="300"
        bundler_type="transfer-queue"



        thread_pool.min_threads="${jgroups.thread_pool.min_threads:0}"
        thread_pool.max_threads="${jgroups.thread_pool.max_threads:200}"
        thread_pool.keep_alive_time="60000"



        thread_dumps_threshold="${jgroups.thread_dumps_threshold:10000}"
   />
<TCPPING initial_hosts="${jgroups.tcpping.initial_hosts:<private_ip_1>[7800],<private_ip_2>[7800]}" port_range="2" />
</stack></stacks>
</jgroups>

Same configurations are done in the other instance.

Command used to start keycloak

nohup ./bin/kc.sh start --cache-config-file=cache-ispn-tcp-ping.xml &

Logs:

2023-07-27 06:42:38,262 INFO  [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000128: Infinispan version: Infinispan 'Triskaidekaphobia' 13.0.10.Final
2023-07-27 06:42:38,452 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000078: Starting JGroups channel `ISPN`
2023-07-27 06:42:38,452 INFO  [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000088: Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
2023-07-27 06:42:38,627 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
2023-07-27 06:42:38,628 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 20.00MB, but the OS only allocated 212.99KB
2023-07-27 06:42:38,628 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
2023-07-27 06:42:38,629 WARN  [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 25.00MB, but the OS only allocated 212.99KB
2023-07-27 06:42:40,656 INFO  [org.jgroups.protocols.pbcast.GMS] (keycloak-cache-init) ip-4182: no members discovered after 2006 ms: creating cluster as coordinator

Output of command netstat -tlpn

<private_ip_1>:7800

Unable to find the other instance added to cluster and also bind port 7800 to 0.0.0.0. Changes made to file "cache-ispn-tcp-ping.xml" are not reflecting.

Tags: keycloak

Answers

- sbrajchuk
- August 21, 2023 at 12:04 pm
- 0 votes
0
This solution pertains to Keycloak version 20.0.5, and there’s a possibility that it might be applicable in your situation. Within your conf/keycloak.conf file, you should comment out the line:
```
#cache-stack=tcp
```
Furthermore, in your conf/cache-ispn-tcp-ping.xml file, you should replace the line:
```
<TCP bind_addr="${jgroups.bind.address:0.0.0.0.tcp.address:0.0.0.0}"
```
with:
```
<TCP bind_addr="${jgroups.bind.address,jgroups.tcp.address:GLOBAL}"
```
Include the following:
```
stack.combine="REPLACE" stack.position="MPING"
```
within the line:
```
<TCPPING initial_hosts="${jgroups.tcpping.initial_hosts:<private_ip_1>[7800],<private_ip_2>[7800]}" port_range="2" />
```
in order to achieve:
```
<TCPPING initial_hosts="${jgroups.tcpping.initial_hosts:<private_ip_1>[7800],<private_ip_2>[7800]}" port_range="2" stack.combine="REPLACE" stack.position="MPING"/>
```
In this context, I believe it would be more suitable to use public (accessible via the Internet) IP addresses. Thus, you may need to substitute private_ip_1 and private_ip_2 with their corresponding public counterparts.

Additionally, you should add the line:
```
stack="my-stack"
```
to the transport configuration of your cache container, resulting in:
```
<transport lock-timeout="60000" stack="my-stack"/>
```
Login or Signup to reply.

<cache-container name="keycloak">
    <!-- <transport lock-timeout="60000" /> -->

    <transport cluster="my-cluster"
        lock-timeout="60000"
        stack="my-stack"
        node-name="superman" />


    <local-cache name="realms" simple-cache="true">
        <encoding>
            <key media-type="application/x-java-object" />
            <value media-type="application/x-java-object" />
        </encoding>
        <memory max-count="10000" />
    </local-cache>
    <local-cache name="users" simple-cache="true">
        <encoding>
            <key media-type="application/x-java-object" />
            <value media-type="application/x-java-object" />
        </encoding>
        <memory max-count="10000" />
    </local-cache>
    <distributed-cache name="sessions" owners="2">
        <expiration lifespan="-1" />
    </distributed-cache>
    <distributed-cache name="authenticationSessions" owners="2">
        <expiration lifespan="-1" />
    </distributed-cache>
    <distributed-cache name="offlineSessions" owners="2">
        <expiration lifespan="-1" />
    </distributed-cache>
    <distributed-cache name="clientSessions" owners="2">
        <expiration lifespan="-1" />
    </distributed-cache>
    <distributed-cache name="offlineClientSessions" owners="2">
        <expiration lifespan="-1" />
    </distributed-cache>
    <distributed-cache name="loginFailures" owners="2">
        <expiration lifespan="-1" />
    </distributed-cache>
    <local-cache name="authorization" simple-cache="true">
        <encoding>
            <key media-type="application/x-java-object" />
            <value media-type="application/x-java-object" />
        </encoding>
        <memory max-count="10000" />
    </local-cache>
    <replicated-cache name="work">
        <expiration lifespan="-1" />
    </replicated-cache>
    <local-cache name="keys" simple-cache="true">
        <encoding>
            <key media-type="application/x-java-object" />
            <value media-type="application/x-java-object" />
        </encoding>
        <expiration max-idle="3600000" />
        <memory max-count="1000" />
    </local-cache>
    <distributed-cache name="actionTokens" owners="2">
        <encoding>
            <key media-type="application/x-java-object" />
            <value media-type="application/x-java-object" />
        </encoding>
        <expiration max-idle="-1" lifespan="-1" interval="300000" />
        <memory max-count="-1" />
    </distributed-cache>
</cache-container>

this is my config file placed in a conf/cache-ispn-tcp-ping.xml

With change in the nodename for the second host. I Established the connection between the two and formed a cluster.

Please signup or login to give your own answer.

Click here to cancel reply.

Nginx – Keycloak clustering

Answers