skip to Main Content

I’m trying to deploy a Minio instance (not the operator) on a kubernetes server (Microk8s) using the helm chart provided in https://github.com/minio/minio/tree/master/helm/minio.

To match with my test server, I did some modifications :

  • The number of server is equal to 1 instead of 16
  • The requested memory is equal to 1Gb instead of 16Gb
  • I used the command –persistence.enabled=false to simplify the problem as much as possible

The complete installation command is (the chart folder is present directly in the server):

microk8s helm3 install --namespace kube-system --generate-name --set persistence.enabled=false minio

The modified values.yaml content is the following :

## Provide a name in place of minio for `app:` labels
##
nameOverride: ""

## Provide a name to substitute for the full names of resources
##
fullnameOverride: ""

## set kubernetes cluster domain where minio is running
##
clusterDomain: cluster.local

## Set default image, imageTag, and imagePullPolicy. mode is used to indicate the
##
image:
  repository: quay.io/minio/minio
  tag: RELEASE.2021-11-24T23-19-33Z
  pullPolicy: IfNotPresent

imagePullSecrets: []
# - name: "image-pull-secret"

## Set default image, imageTag, and imagePullPolicy for the `mc` (the minio
## client used to create a default bucket).
##
mcImage:
  repository: quay.io/minio/mc
  tag: RELEASE.2021-11-16T20-37-36Z
  pullPolicy: IfNotPresent

## minio mode, i.e. standalone or distributed or gateway (nas)
## Distributed MinIO ref: https://docs.minio.io/docs/distributed-minio-quickstart-guide
## NAS Gateway MinIO ref: https://docs.min.io/docs/minio-gateway-for-nas.html
mode: distributed

## NOTE: currently only "nas" gateway is supported.

## Additional labels to include with deployment or statefulset
additionalLabels: []

## Additional annotations to include with deployment or statefulset
additionalAnnotations: []

## Additional arguments to pass to minio binary
extraArgs: []

## Port number for MinIO S3 API Access
minioAPIPort: "9000"

## Port number for MinIO Browser COnsole Access
minioConsolePort: "9001"

## Update strategy for Deployments
DeploymentUpdate:
  type: RollingUpdate
  maxUnavailable: 0
  maxSurge: 100%

## Update strategy for StatefulSets
StatefulSetUpdate:
  updateStrategy: RollingUpdate

## Pod priority settings
## ref: https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
##
priorityClassName: ""

## Set default rootUser, rootPassword
## AccessKey and secretKey is generated when not set
## Distributed MinIO ref: https://docs.minio.io/docs/distributed-minio-quickstart-guide
##
rootUser: ""
rootPassword: ""

## Use existing Secret that store following variables:
##
## | Chart var             | .data.<key> in Secret    |
## |:----------------------|:-------------------------|
## | rootUser              | rootUser                 |
## | rootPassword          | rootPassword             |
##
## All mentioned variables will be ignored in values file.
## .data.rootUser and .data.rootPassword are mandatory,
## others depend on enabled status of corresponding sections.
existingSecret: ""

## Directory on the MinIO pof
certsPath: "/etc/minio/certs/"
configPathmc: "/etc/minio/mc/"

## Path where PV would be mounted on the MinIO Pod
mountPath: "/export"
## Override the root directory which the minio server should serve from.
## If left empty, it defaults to the value of {{ .Values.mountPath }}
## If defined, it must be a sub-directory of the path specified in {{ .Values.mountPath }}
##
bucketRoot: ""

# Number of drives attached to a node
drivesPerNode: 1
# Number of MinIO containers running
replicas: 1
# Number of expanded MinIO clusters
pools: 1

# Deploy if 'mode == gateway' - 4 replicas.
gateway:
  type: "nas" # currently only "nas" is supported.
  replicas: 4

## TLS Settings for MinIO
tls:
  enabled: false
  ## Create a secret with private.key and public.crt files and pass that here. Ref: https://github.com/minio/minio/tree/master/docs/tls/kubernetes#2-create-kubernetes-secret
  certSecret: ""
  publicCrt: public.crt
  privateKey: private.key

## Trusted Certificates Settings for MinIO. Ref: https://docs.minio.io/docs/how-to-secure-access-to-minio-server-with-tls#install-certificates-from-third-party-cas
## Bundle multiple trusted certificates into one secret and pass that here. Ref: https://github.com/minio/minio/tree/master/docs/tls/kubernetes#2-create-kubernetes-secret
## When using self-signed certificates, remember to include MinIO's own certificate in the bundle with key public.crt.
## If certSecret is left empty and tls is enabled, this chart installs the public certificate from .Values.tls.certSecret.
trustedCertsSecret: ""

## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistence:
  enabled: true
  annotations: {}

  ## A manually managed Persistent Volume and Claim
  ## Requires persistence.enabled: true
  ## If defined, PVC must be created manually before volume will be bound
  existingClaim: ""

  ## minio data Persistent Volume Storage Class
  ## If defined, storageClassName: <storageClass>
  ## If set to "-", storageClassName: "", which disables dynamic provisioning
  ## If undefined (the default) or set to null, no storageClassName spec is
  ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
  ##   GKE, AWS & OpenStack)
  ##
  ## Storage class of PV to bind. By default it looks for standard storage class.
  ## If the PV uses a different storage class, specify that here.
  storageClass: "standard"
  VolumeName: ""
  accessMode: ReadWriteOnce
  size: 80Gi

  ## If subPath is set mount a sub folder of a volume instead of the root of the volume.
  ## This is especially handy for volume plugins that don't natively support sub mounting (like glusterfs).
  ##
  subPath: ""

## Expose the MinIO service to be accessed from outside the cluster (LoadBalancer service).
## or access it from within the cluster (ClusterIP service). Set the service type and the port to serve it.
## ref: http://kubernetes.io/docs/user-guide/services/
##
service:
  type: ClusterIP
  clusterIP: ~
  ## Make sure to match it to minioAPIPort
  port: "9000"
  nodePort: 32000

## Configure Ingress based on the documentation here: https://kubernetes.io/docs/concepts/services-networking/ingress/
##

ingress:
  enabled: false
  # ingressClassName: ""
  labels: {}
    # node-role.kubernetes.io/ingress: platform

  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
    # kubernetes.io/ingress.allow-http: "false"
    # kubernetes.io/ingress.global-static-ip-name: ""
    # nginx.ingress.kubernetes.io/secure-backends: "true"
    # nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    # nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
  path: /
  hosts:
    - chart-example.local
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

consoleService:
  type: ClusterIP
  clusterIP: ~
  ## Make sure to match it to minioConsolePort
  port: "9001"
  nodePort: 32001

consoleIngress:
  enabled: false
  # ingressClassName: ""
  labels: {}
    # node-role.kubernetes.io/ingress: platform

  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
    # kubernetes.io/ingress.allow-http: "false"
    # kubernetes.io/ingress.global-static-ip-name: ""
    # nginx.ingress.kubernetes.io/secure-backends: "true"
    # nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    # nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
  path: /
  hosts:
    - chart-example.local
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

## Node labels for pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector: {}
tolerations: []
affinity: {}

## Add stateful containers to have security context, if enabled MinIO will run as this
## user and group NOTE: securityContext is only enabled if persistence.enabled=true
securityContext:
  enabled: true
  runAsUser: 1000
  runAsGroup: 1000
  fsGroup: 1000

# Additational pod annotations
podAnnotations: {}

# Additional pod labels
podLabels: {}

## Configure resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
resources:
  requests:
    memory: 4Gi

## List of users to be created after minio install
##
users:
  ## Username, password and policy to be assigned to the user
  ## Default policies are [readonly|readwrite|writeonly|consoleAdmin|diagnostics]
  ## Add new policies as explained here https://docs.min.io/docs/minio-multi-user-quickstart-guide.html
  ## NOTE: this will fail if LDAP is enabled in your MinIO deployment
  ## make sure to disable this if you are using LDAP.
  - accessKey: console
    secretKey: console123
    policy: consoleAdmin


## Additional Annotations for the Kubernetes Job makeUserJob
makeUserJob:
  podAnnotations:
  annotations:
  securityContext:
    enabled: false
    runAsUser: 1000
    runAsGroup: 1000
    fsGroup: 1000
  resources:
    requests:
      memory: 128Mi
  nodeSelector: {}
  tolerations: []
  affinity: {}

## List of buckets to be created after minio install
##
buckets:
  #   # Name of the bucket
  # - name: bucket1
  #   # Policy to be set on the
  #   # bucket [none|download|upload|public]
  #   policy: none
  #   # Purge if bucket exists already
  #   purge: false
  #   # set versioning for
  #   # bucket [true|false]
  #   versioning: false
  # - name: bucket2
  #   policy: none
  #   purge: false
  #   versioning: true

## Additional Annotations for the Kubernetes Job makeBucketJob
makeBucketJob:
  podAnnotations:
  annotations:
  securityContext:
    enabled: false
    runAsUser: 1000
    runAsGroup: 1000
    fsGroup: 1000
  resources:
    requests:
      memory: 128Mi
  nodeSelector: {}
  tolerations: []
  affinity: {}

## Use this field to add environment variables relevant to MinIO server. These fields will be passed on to MinIO container(s)
## when Chart is deployed
environment:
  ## Please refer for comprehensive list https://docs.min.io/minio/baremetal/reference/minio-server/minio-server.html
  ## MINIO_SUBNET_LICENSE: "License key obtained from https://subnet.min.io"
  ## MINIO_BROWSER: "off"

networkPolicy:
  enabled: false
  allowExternal: true

## PodDisruptionBudget settings
## ref: https://kubernetes.io/docs/concepts/workloads/pods/disruptions/
##
podDisruptionBudget:
  enabled: false
  maxUnavailable: 1

## Specify the service account to use for the Minio pods. If 'create' is set to 'false'
## and 'name' is left unspecified, the account 'default' will be used.
serviceAccount:
  create: true
  ## The name of the service account to use. If 'create' is 'true', a service account with that name
  ## will be created.
  name: "minio-sa"

metrics:
  serviceMonitor:
    enabled: false
    additionalLabels: {}
    public: true
    # namespace: monitoring
    # interval: 30s
    # scrapeTimeout: 10s

The generated pod appears in error in the k8s dashboard and when I check the logs, I obtain:

ERROR Invalid command line arguments: Incorrect number of endpoints provided [http://minio-1638458317-{0...0}.minio-1638458317-svc.kube-system.svc.cluster.local/export]
      > Please provide correct combination of local/remote paths
      HINT:
        For more information, please refer to https://docs.min.io/docs/minio-erasure-code-quickstart-guide

I have absolutely no idea of what I can do to solve that issue. Could anyone help me on this?

Thanks in advance,

Rémi

2

Answers


  1. Try this:

    helm install --namespace minio --create-namespace --set rootUser=admin,rootPassword=admin123 --set persistence.enabled=true --set persistence.size=2Gi --set resources.requests.cpu=1 --set resources.limits.cpu=2 --set resources.requests.memory=1Gi --set resources.limits.memory=2Gi --set mode=distributed,replicas=4 --generate-name minio/minio

    Make sure you have 4 or more pod replicas running, I had the same issue where the format of the IP address specified while starting the minIO server goes with incorrect argument http://minio-1640891988-{0..0}.minio-1640891988-svc.minio.svc.cluster.local/export or works only with contiguous IP addresses.

    Refer: [https://github.com/minio/minio/issues/6629]

    Login or Signup to reply.
  2. It’s basically complaining about an incoherent number of IPs provided.
    Try to run an even number of replicas of minio. It could also help to set mode to distributed

    helm upgrade --install -n your-name-space --create-namespace  --set mode=distributed,replicas=4 minio minio/minio -f .local/minio-values.yaml
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search