I want to run three docker containers(grafana, questdb and clickhousedb) inside a single pod (We are trying to do run some tests with the databases).
If I were to run them using the docker command, I’d do the following:
docker run -d -p 9000:9000 -p 8812:8812 -p 9009:9009 -v questdb:/var/lib/questdb questdb/questdb
docker run -d -p 3000:3000 grafana/grafana
docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144
-p 18123:8123 -p 19000:9000 -p 19009:9009
-v $(realpath ./ch_data):/var/lib/clickhouse
-v $(realpath ./ch_logs):/var/log/clickhouse-server
clickhouse/clickhouse-server
Using Kubernetes, I have the following config:
apiVersion: v1
kind: Service
metadata:
name: test-service
spec:
selector:
app: test
ports:
- protocol: 'TCP'
name: grafana
port: 3000
targetPort: 3000
- protocol: 'TCP'
name: qdbhttp
port: 9000
targetPort: 9000
- protocol: 'TCP'
name: qdba
port: 9009
targetPort: 9009
- protocol: 'TCP'
name: qdbb
port: 8812
targetPort: 8812
- protocol: 'TCP'
name: ch
port: 8123
targetPort: 8123
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-deployment
labels:
app: test
spec:
replicas: 3
selector:
matchLabels:
app: test
template:
metadata:
labels:
app: test
spec:
containers:
- name: qdb
image: questdb/questdb
ports:
- containerPort: 9000
- containerPort: 8812
- containerPort: 9009
- name: grafana
image: grafana/grafana
ports:
- containerPort: 3000
- name: clickhousedb
image: clickhouse/clickhouse-server
ports:
- containerPort: 9009
- containerPort: 8123
- containerPort: 9000
There are two problems:
- The ports for
questdb
andclickhouse
overlap (i.e. 9000 and 9009). With docker command, I can map them to different ports using--p 19000:9000 -p 19009:9009
. How do I do this in kubernetes config file? - Using the command
minikube service test-app
, I get a url with a port. However, if I have 3 replicas, how do I access each pod separately?
2
Answers
First question is answered here:
In addition, I could not find the reason for putting all the containers in a pod. You could just create separate deployments.
Second, you could use a
headless
service in front of your deployment. Thesrv
query will return all the pods’ addresses behind the service.You should not want to do this. In k8s a pod is the smallest unit, so you want to run one "service" in one pod that would be the main container in the pod.
Although you can run multiple containers in one pod it is not recommended to run more than one "main" container but you can run "sidecar" containers but the purpose for those is to complement the main container (logging, monitoring, proxy etc). Main and sidecar containers are not differentiated technically at all by k8s they are just logically different but normal containers otherwise.
Except init containers, they are a special kind, you can read about them here: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/ but this is out of the scope of this question.
For example you have a legacy app that runs in the main container but the logging of that app can not be configured or modified at all, it can only write files to some log directory, so you can make a sidecar log shipper container for it so the software that is running on the log shipper container will read the log files and send them to an appropriate log store database or wherever you want them to be sent. So you will not need to modify the legacy app at all to make it run in k8s. https://kubernetes.io/docs/concepts/cluster-administration/logging/#streaming-sidecar-container
So what you want here is to run a k8s deployment for grafana and 2 statefulsets for the databases (or if you don’t want to persist data and just want to do some tests you can run them in deployments too) give them
kind: Service
service objects so they can communicate with each other through their corresponding services.