I want to configure native Kubernetes cluster using Terraform script. I tried this Terraform script:
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
version = "2.13.1"
}
kubectl = {
source = "gavinbunney/kubectl"
version = "1.14.0"
}
helm = {
source = "hashicorp/helm"
version = "2.6.0"
}
}
}
provider "kubectl" {
# run kubectl cluster-info to get expoint and port
host = "https://192.168.1.139:6443/"
token = "eyJhbGciOiJSUzI1NiIsImt....."
insecure = "true"
}
provider "kubernetes" {
# run kubectl cluster-info to get expoint and port
host = "https://192.168.1.139:6443/"
token = "eyJhbGciOiJSUzI1NiIsImt....."
insecure = "true"
}
resource "kubernetes_namespace" "example" {
metadata {
annotations = {
name = "example-annotation"
}
labels = {
mylabel = "label-value"
}
name = "terraform-example-namespace"
}
}
ref: https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs
https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs
I tried to create a user from this tutorial: https://killercoda.com/kimwuestkamp/scenario/k8s1.24-serviceaccount-secret-changes
kubectl create sa cicd
kubectl get sa,secret
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: cicd
spec:
serviceAccount: cicd
containers:
- image: nginx
name: cicd
EOF
kubectl exec cicd -- cat /run/secrets/kubernetes.io/serviceaccount/token && echo
kubectl exec cicd cat /run/secrets/kubernetes.io/serviceaccount/token && echo
kubectl create token cicd
kubectl create token cicd --duration=999999h
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
name: cicd
annotations:
kubernetes.io/service-account.name: "cicd"
EOF
kubectl get sa,secret
kubectl describe secret cicd
kubectl describe sa cicd
kubectl get sa cicd -oyaml
kubectl get sa,secret
When I run the Terraform script I get error:
kubernetes_namespace.example: Creating...
╷
│ Error: namespaces is forbidden: User "system:serviceaccount:default:cicd" cannot create resource "namespaces" in API group "" at the cluster scope
│
│ with kubernetes_namespace.example,
│ on main.tf line 36, in resource "kubernetes_namespace" "example":
│ 36: resource "kubernetes_namespace" "example" {
Can you advise what user configuration I’m missing?
Can you advise what is the proper way to implement this script and provision HELM chart into native Kubernetes.
2
Answers
The service account
cicd
in namespacedefault
is lacked of permissions. You can first assigncluster-admin
permissions to ensure your pipeline is functioning, then trim the permissions gradually according to your use case. Apply the following spec before your pipeline starts:You need implement RBAC (Role Back Access Control) by attaching
ClusterRole
andClusterRoleBinding
with yourServiceAccount
.SA
only contains token for authentication with the k8s api-server. Authorization is enabled via RBAC Here’s an example:Check out the official k8s document.