I have a new install of K8s master and node both on ubuntu-18. The master is using weave for CNI and all pods are running:
$ sudo kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-6d4b75cb6d-29qg5 1/1 Running 0 31m
kube-system coredns-6d4b75cb6d-kxxc8 1/1 Running 0 31m
kube-system etcd-ubuntu-18-extssd 1/1 Running 2 31m
kube-system kube-apiserver-ubuntu-18-extssd 1/1 Running 2 31m
kube-system kube-controller-manager-ubuntu-18-extssd 1/1 Running 2 31m
kube-system kube-proxy-nvqjl 1/1 Running 0 31m
kube-system kube-scheduler-ubuntu-18-extssd 1/1 Running 2 31m
kube-system weave-net-th4kv 2/2 Running 0 31m
When I execute the kubeadm join
command on the node I get the following error:
sudo kubeadm join 192.168.0.12:6443 --token ikk2kd.177ij0f6n211sonl --discovery-token-ca-cert-hash sha256:8717baa3c634321438065f40395751430b4fb55f43668fac69489136335721dc
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: E0724 16:24:41.009234 8391 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2022-07-24T16:24:41-06:00" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
The only problem showing up in journalctl -r -u kubelet
is:
kubelet.service: Main process exited, code=exited, status=1/FAILURE
...
Error: failed to load kubelet config file, error: failed to load Kubelet config file /var/lib/kubelet/config.yaml
That is from several minutes before the join
failed when kubelet was trying to start. I would expect that config.yaml file to be missing until the node joined a cluster.
The preflight error message says
[ERROR CRI]: container runtime is not running: output: E0724 16:32:41.120653 10509 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
What is this trying to tell me?
====Edit=====
I am running CrashPlan on the worker node that is failing, but I have fs.inotify.max_user_watches=1048576
in /etc/sysctl.conf.
This node worked before both with on-prem master and with GKE with kubernetes 1.20.
3
Answers
Check the error message. The container runtime is down
run the below command to check the runtime used in the cluster
see the last column in the picture below
Ensure the container runtime is running, In this case it is containerd
[ERROR CRI]: container runtime is not running: output: E0724 16:32:41.120653 10509 remote_runtime.go:925] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"…………..
rm /etc/containerd/config.toml…
systemctl restart containerd…now u can run kubeadm init command
Obtain the same error when I attempted to join another master node.
The error was corrected in accordance with the instructions of this article.