skip to Main Content

I am currently working on a deployment script to backup my cluster resources and data. To test the script, I created a MySQL deployment and added data to the database to test if the resources and data are moved to the new cluster.

This is my Velero script;

kube_config_path=$1
clustername=$2
namespace="mysql"

sudo velero install --provider aws 
    --plugins velero/velero-plugin-for-aws:v1.10.0 
    --secret-file /home/velero-cred 
    --bucket $bucketName 
    --prefix $clustername 
    --backup-location-config region=eu-central-1,s3Url=https://s3.eu-central-1.amazonaws.com 
    --snapshot-location-config region=eu-central-1 
    --use-node-agent 
    --features=EnableCSI 
    --default-volumes-to-fs-backup=true 
    --default-snapshot-move-data=true 
    --kubeconfig $kube_config_path

sleep 40
if [ -z "$namespace" ]; then
    velero backup create crd-backup --include-resources=CustomResourceDefinition --kubeconfig=$kube_config_path
    velero backup create full-cluster-backup --snapshot-volumes=true --snapshot-move-data=true --kubeconfig=$kube_config_path
else
    velero backup create crd-backup --include-resources=CustomResourceDefinition --kubeconfig=$kube_config_path
    velero backup create $namespace-resources-backup --include-namespaces $namespace --snapshot-volumes=true --snapshot-move-data=true --kubeconfig=$kube_config_path
fi

For the MySQL deployment I created an NFS storage class in the Cluster using this installation.

This is my MySQL deployment manifest;

apiVersion: v1
kind: Namespace
metadata:
  creationTimestamp: null
  name: mysql
spec: {}
status: {}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
  namespace: mysql
spec:
  storageClassName: nfs
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
---
apiVersion: v1
kind: Secret
metadata:
  namespace: mysql
  name: mysql-secret
type: kubernetes.io/basic-auth
stringData:
  password: YourPassword$@
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
  namespace: mysql
spec:
  selector:
    matchLabels:
      app: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - image: mysql:latest
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: password
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

---
apiVersion: v1
kind: Service
metadata:
  namespace: mysql
  name: mysql
spec:
  ports:
  - port: 3306
  selector:
    app: mysql

After running the backup script, this is what I am getting

velero backup get --kubeconfig /home/kubeconfigs/velero-test-kubeconfig.yaml
NAME                     STATUS       ERRORS   WARNINGS   CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
crd-backup               Completed    0        0          2025-01-18 01:54:42 +0000 UTC   29d       default            <none>
mysql-resources-backup   InProgress   0        0          2025-01-18 01:54:43 +0000 UTC   29d       default            <none>

When I describe the mysql-resources-backup this is what I am getting;

Name:         mysql-resources-backup
Namespace:    velero
Labels:       velero.io/storage-location=default
Annotations:  velero.io/resource-timeout=10m0s
              velero.io/source-cluster-k8s-gitversion=v1.31.1
              velero.io/source-cluster-k8s-major-version=1
              velero.io/source-cluster-k8s-minor-version=31

Phase:  InProgress


Namespaces:
  Included:  mysql
  Excluded:  <none>

Resources:
  Included:        *
  Excluded:        <none>
  Cluster-scoped:  auto

Label selector:  <none>

Or label selector:  <none>

Storage Location:  default

Velero-Native Snapshot PVs:  true
Snapshot Move Data:          true
Data Mover:                  velero

TTL:  720h0m0s

CSISnapshotTimeout:    10m0s
ItemOperationTimeout:  4h0m0s

Hooks:  <none>

Backup Format Version:  1.1.0

Started:    2025-01-18 01:54:43 +0000 UTC
Completed:  <n/a>

Expiration:  2025-02-17 01:54:43 +0000 UTC

Estimated total items to be backed up:  22
Items backed up so far:                 0

Resource List:  <backup resource list not found>

Backup Volumes:
  Velero-Native Snapshots: <none included>

  CSI Snapshots: <none included or not detectable>

  Pod Volume Backups - kopia:
    New:
      mysql/mysql-c5bd89b69-bgwkt: mysql-persistent-storage

Please I need guide on what I am doing wrong and how I can resolve this.

2

Answers


  1. Chosen as BEST ANSWER

    I was moving resources from a DigitalOcean cluster to a Microk8s cluster, I decided to check the logs of the velero node agent and I found this error; An error occurred: unexpected directory structure for host-pods volume, ensure that the host-pods volume corresponds to the pods subdirectory of the kubelet root directory then I remembered I added this line of code after my installation;

    /snap/bin/microk8s kubectl --kubeconfig="$kube_config_path" -n velero patch daemonset.apps/node-agent --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/volumes/0/hostPath/path", "value":"/var/snap/microk8s/common/var/lib/kubelet/pods"}]'
    

    This was interfering with the original hostpath value var/lib/kubelet/pods needed for normal Kubernetes cluster. I added a condition as to when the patch is to be done and it started working properly.


  2. Pls describe your error properly. Your error was not properly stated here. You only showed what you see when you describe the backup and not seeing any error in the description.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search