I want to run Kubernetes v1.13.5
unit-tests inside a CentOS docker container running on a macOS host. I get errors trying to run kubeadm
as it doesn’t like having swap enabled.
1) I tried kubeadm init --fail-swap-on=false
but get Error: unknown flag: --fail-swap-on
. Maybe only supported in older k8s but docs don’t seem clear.
2) I tried
kubeadm init --config /etc/kubernetes/kubeadm-config.yaml --ignore-preflight-errors=all --skip-token-print
with /etc/kubernetes/kubeadm-config.yaml
like
---
apiVersion: kubeadm.k8s.io/v1alpha3
controlPlaneEndpoint: ""
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
networking:
podSubnet: "10.244.0.0/16"
serviceSubnet: "10.96.0.0/12"
kubernetesVersion: "v1.13.5"
failSwapOn: false
but that gives … error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster", ... SNIP ... error unmarshaling JSON: while decoding JSON: json: unknown field "failSwapOn""
3) I tried switching off swap inside my container but I get …
docker run -i -t centos
[root@2ed611b32f1a /]# swapoff -a
swapoff: Not superuser.
or it when privileged
mode it ran but didn’t affect setup
docker run --privileged -i -t centos
[root@94f9a0e5e46a /]# swapoff -a
[root@94f9a0e5e46a /]# free -h
total used free shared buff/cache available
Mem: 12Gi 371Mi 11Gi 1.0Mi 1.1Gi 12Gi
Swap: 2.0Gi 0B 2.0Gi
4) I tried flags like docker run --memory 256M --memory-swap 256M
but these are not honoured inside the container despite https://docs.docker.com/config/containers/resource_constraints/ saying …
PREVENT A CONTAINER FROM USING SWAP
If –memory and –memory-swap are set to the same value, this prevents
containers from using any swap. This is because –memory-swap is the
amount of combined memory and swap that can be used, while –memory is
only the amount of physical memory that can be used.
5) I tried in docker desktop v2.1.0.3 on my laptop adjusting the swap setting via the preferences in its UI, but it won’t let me decrease swap below 512.0 MiB.
Can you advise a way to get k8s running inside CentOS docker container running on my macOS host ?
3
Answers
My 6th (and finally successful) attempt was this. See last 4 lines below.
kubeadm init --config /etc/kubernetes/kubeadm-config.yaml --ignore-preflight-errors=all --skip-token-print
wherekubeadm-config.yaml
looks like ...I think the problem may be your
apiVersion
for kubeadm! You have an old deprecated version in there which isv1alpha3
It is highly recommended here that you migrate your old configuration to use the newer api which isv1beta1
If you decided to enable swap, then I’ll guess your workloads probably needs it.
I’ll just add a smaller disclaimer for readers which are choosing this option as the default solution regardless of there specific workloads.
It seems the Kubelet is not designed to work properly with swap enabled – K8S is very clear about this topic as you can see in the Kubeadm installation:
I would recommend reading about Evicting end-user Pods and the relevant features that K8S provides to prioritize memory of pods:
1 ) The 3 qos classes – Make sure that your high priority workloads are running with the
Guaranteed
(or at leastBurstable
) class.2 ) Pod Priority and Preemption.