skip to Main Content

we want to use Ceph but we want to use Docker and Kubernetes to
deploy new instances of Ceph quickly.

I tried to use the default ceph docker hub: ceph/daemon-base. But I didn’t work.

I tried to use the ceph-container. Seems like it doesn’t work.

This is my last deployment file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ceph3-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ceph3
  template:
    metadata:
      labels:
        app: ceph3
    spec:
      containers:
        - name: ceph
          image: ceph/daemon-base:v3.0.5-stable-3.0-luminous-centos-7
          resources:
            limits:
              memory: 512Mi
              cpu: "500m"
            requests:
              memory: 256Mi
              cpu: "250m"
          volumeMounts:
            - mountPath: /etc/ceph
              name: etc-ceph
            - mountPath: /var/lib/ceph
              name: lib-ceph
      volumes:
        - name: etc-ceph
          hostPath:
            path: /etc/ceph
        - name: lib-ceph
          hostPath:
            path: /var/lib/ceph

Does someone already install a ceph instance on Kubernetes?

I tried to follow the tutorial here

But pods not working:

pod/ceph-mds-7b49574f48-vhvtl         0/1     Pending            0          81s
pod/ceph-mon-75c49c4fd5-2cq2r         0/1     CrashLoopBackOff   3          81s
pod/ceph-mon-75c49c4fd5-6nprj         0/1     Pending            0          81s
pod/ceph-mon-75c49c4fd5-7vrp8         0/1     Pending            0          81s
pod/ceph-mon-check-5df985478b-d87rs   1/1     Running            0          81s

2

Answers


  1. Error from server (Forbidden): pods is forbidden: User “system:serviceaccount:ceph:default” cannot list resource “pods” in API group “” in the namespace “ceph”.

    This error means Your default user doesn’t have an access to resources in default namespace, look into documentation how to deal with rbac and how to grant user accesses. There is the same error on github.

    Install Ceph

    To install ceph I would recommend You to use helm

    Install helm

    There are few ways to install helm, You can find them there.

    Personally i am using helm github releases to download latest version. When answering this question latest release was 2.15.0. That’s how You can install it.

    1.Download helm (If you want to install current version of helm, check above helm github releases link, there might be a new version of it. Everything you need to do then is change v2.15.0 in wget and tar command to new, for example v2.16.0)

    wget https://get.helm.sh/helm-v2.15.0-linux-amd64.tar.gz
    

    2.Unpack it

    tar -zxvf helm-v2.15.0-linux-amd64.tar.gz
    

    3.Find the helm binary in the unpacked directory, and move it to its desired destination

    mv linux-amd64/helm /usr/local/bin/helm
    

    4.Install tiller

    The easiest way to install tiller into the cluster is simply to run helm init. This will validate that helm’s local environment is set up correctly (and set it up if necessary). Then it will connect to whatever cluster kubectl connects to by default (kubectl config view). Once it connects, it will install tiller into the kube-system namespace.

    And then follow official documentation to install Ceph.

    Login or Signup to reply.
  2. The common practice for deploying stateful systems on Kubernetes is to use an Operator to manage and codify the lifecycle management of the application. Rook is an operator that provides Ceph lifecycle management on Kubernetes clusters.

    Documentation for using Rook to deploy Ceph clusters can be found at https://rook.io/docs/rook/v1.1/ceph-storage.html

    For a basic introduction, you can use the Rook Storage Quickstart guide

    The core Ceph team is highly involved in working on Rook and with the Rook community, and Rook is widely deployed within the Kubernetes community for distributed storage applications, and the Ceph Days event now has added Rook] explicitly to become Ceph + Rook Days

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search