Kubernetes Newbie here.
I was following a youtube tutorial from Marc Lamberti on how to install airflow using Kubernetes locally – using Kind. I was able to create the k8s cluster using below commands:
kind create cluster --name airflow-cluster --config kind-cluster.yaml
helm repo add apache-airflow https://airflow.apache.org
helm repo update
kubectl create namespace airflow
helm install airflow apache-airflow/airflow --namespace airflow --debug
# Port forward
kubectl port-forward svc/airflow-webserver 8080:8080 -n airflow --context kind-airflow-cluster
The next part was to upgrade the generate values.yaml using below command
helm show values apache-airflow/airflow > values.yaml
The next step was to modify this values.yaml and run a helm upgrade, however, whether I modify the values.yaml file or not, I am getting the below error:
helm upgrade --install airflow apache-airflow/airflow -n airflow -f values.yaml --debug
history.go:56: [debug] getting history for release airflow
upgrade.go:144: [debug] preparing upgrade for airflow
Error: UPGRADE FAILED: execution error at (airflow/charts/postgresql/templates/secrets.yaml:20:15):
PASSWORDS ERROR: The secret "airflow-postgresql" does not contain the key "password"
helm.go:84: [debug] execution error at (airflow/charts/postgresql/templates/secrets.yaml:20:15):
PASSWORDS ERROR: The secret "airflow-postgresql" does not contain the key "password"
UPGRADE FAILED
main.newUpgradeCmd.func2
helm.sh/helm/v3/cmd/helm/upgrade.go:203
github.com/spf13/cobra.(*Command).execute
github.com/spf13/[email protected]/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/[email protected]/command.go:1044
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/[email protected]/command.go:968
main.main
helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
runtime/proc.go:250
runtime.goexit
runtime/asm_arm64.s:1172
On examining the secret airflow-postgresql, I found that it had postgres-password in data, but not password, so I added it.
$ k describe secret airflow-postgresql
Name: airflow-postgresql
Namespace: airflow
Labels: app.kubernetes.io/instance=airflow
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=postgresql
helm.sh/chart=postgresql-12.1.9
Annotations: meta.helm.sh/release-name: airflow
meta.helm.sh/release-namespace: airflow
Type: Opaque
Data
====
postgres-password: 8 bytes
How I added the password –
export POSTGRES_PASS=$(kubectl get secret airflow-postgresql -o jsonpath="{.data.postgres-password}" | base64 --decode)
k create secret generic airflow-postgresql --from-literal=password=$POSTGRES_PASS --dry-run=client -o yaml | k apply -f -
I added the password data to the secret and re-tried. This time I did not have the same error, but the postgres pod went into CrashLoop Backoff and I can see in the logs of the pod that the pod is shutting down.
│ postgresql 23:36:12.33 INFO ==> ** Starting PostgreSQL ** ││ 2023-04-04 23:36:12.348 GMT [1] LOG: pgaudit extension initialized │
│ 2023-04-04 23:36:12.349 GMT [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 ││ 2023-04-04 23:36:12.349 GMT [1] LOG: listening on IPv6 address "::", port 5432 │
│ 2023-04-04 23:36:12.350 GMT [1] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432" ││ 2023-04-04 23:36:12.356 GMT [95] LOG: database system was shut down at 2023-04-04 23:36:12 GMT │
│ 2023-04-04 23:36:12.360 GMT [1] LOG: database system is ready to accept connections ││ 2023-04-04 23:37:42.055 GMT [1] LOG: received smart shutdown request │
│ 2023-04-04 23:37:42.066 GMT [1] LOG: background worker "logical replication launcher" (PID 101) exited with exit c ││ 2023-04-04 23:37:42.066 GMT [96] LOG: shutting down │
│ 2023-04-04 23:37:42.076 GMT [1] LOG: database system is shut down ││ Stream closed EOF for airflow/airflow-postgresql-0 (postgresql) │
Due to the above, other pods – scheduler, webserver are not able to connect to the DB and are in CrashLoopBackoff state as well. (I could see in their logs that there were last trying to connect to the DB)
I am not sure what I need to do to be able to use values.yaml to upgrade the helm chart to a new version.
Many thanks for reading so far. Any guidance would be really appreciated
Thank you!
2
Answers
According to the official manual https://helm.sh/docs/helm/helm_show_values/,
helm show valuesonly show the original values based on its originalchart.Since you are not specifying the chart version, your
installcommand might use different version frominstall --upgradecommand.The right way to upgrade your installation is get
helm releasevalues from your installation andcompareit with thevalues.yamlfrom newchart versionand adjust the new requiredvalue.Some step might be helpful:
Step1: Get the chart version you’ve installed:
helm -n airflow list, you’ll see something as following(my releasetooljetfor example):Step2: Fetch the chart values of your previous installed helm release:
helm show values apache-airflow/airflow --version=0.7.7 > values-old.yaml(remember to change0.7.7to yourhelm releaseversion as showed instep1)Step3: Fetch the latest chart vaues:
helm show values apache-airflow/airflow > values-new.yamlStep4: Compared
values-old.yamlandvalues-new.yamlto find out whatvaluesare newly required from new chart. Now you can add missing values invalues-old.yamland using it in yourhelm upgrade --install airflow apache/airflow -f values-old.yamlcommand.WARN: If you’ve changed the chart values, you’d better to merge
values.yamlfrom yourhelm release, you can get its effectivevaluesfrom command:helm -n airflow get values airflow > values-effect.yaml. Now your final values should contains values merged fromvalues-old.yaml,values-effect.yamlandmissing new required values.Thank you. I had the same issue like you. and I also fix the problem to change the version from 1.8.0 to 1.7.0. 🙂