I’m trying to deploy a Minio instance (not the operator) on a kubernetes server (Microk8s) using the helm chart provided in https://github.com/minio/minio/tree/master/helm/minio.
To match with my test server, I did some modifications :
- The number of server is equal to 1 instead of 16
- The requested memory is equal to 1Gb instead of 16Gb
- I used the command –persistence.enabled=false to simplify the problem as much as possible
The complete installation command is (the chart folder is present directly in the server):
microk8s helm3 install --namespace kube-system --generate-name --set persistence.enabled=false minio
The modified values.yaml content is the following :
## Provide a name in place of minio for `app:` labels
##
nameOverride: ""
## Provide a name to substitute for the full names of resources
##
fullnameOverride: ""
## set kubernetes cluster domain where minio is running
##
clusterDomain: cluster.local
## Set default image, imageTag, and imagePullPolicy. mode is used to indicate the
##
image:
repository: quay.io/minio/minio
tag: RELEASE.2021-11-24T23-19-33Z
pullPolicy: IfNotPresent
imagePullSecrets: []
# - name: "image-pull-secret"
## Set default image, imageTag, and imagePullPolicy for the `mc` (the minio
## client used to create a default bucket).
##
mcImage:
repository: quay.io/minio/mc
tag: RELEASE.2021-11-16T20-37-36Z
pullPolicy: IfNotPresent
## minio mode, i.e. standalone or distributed or gateway (nas)
## Distributed MinIO ref: https://docs.minio.io/docs/distributed-minio-quickstart-guide
## NAS Gateway MinIO ref: https://docs.min.io/docs/minio-gateway-for-nas.html
mode: distributed
## NOTE: currently only "nas" gateway is supported.
## Additional labels to include with deployment or statefulset
additionalLabels: []
## Additional annotations to include with deployment or statefulset
additionalAnnotations: []
## Additional arguments to pass to minio binary
extraArgs: []
## Port number for MinIO S3 API Access
minioAPIPort: "9000"
## Port number for MinIO Browser COnsole Access
minioConsolePort: "9001"
## Update strategy for Deployments
DeploymentUpdate:
type: RollingUpdate
maxUnavailable: 0
maxSurge: 100%
## Update strategy for StatefulSets
StatefulSetUpdate:
updateStrategy: RollingUpdate
## Pod priority settings
## ref: https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
##
priorityClassName: ""
## Set default rootUser, rootPassword
## AccessKey and secretKey is generated when not set
## Distributed MinIO ref: https://docs.minio.io/docs/distributed-minio-quickstart-guide
##
rootUser: ""
rootPassword: ""
## Use existing Secret that store following variables:
##
## | Chart var | .data.<key> in Secret |
## |:----------------------|:-------------------------|
## | rootUser | rootUser |
## | rootPassword | rootPassword |
##
## All mentioned variables will be ignored in values file.
## .data.rootUser and .data.rootPassword are mandatory,
## others depend on enabled status of corresponding sections.
existingSecret: ""
## Directory on the MinIO pof
certsPath: "/etc/minio/certs/"
configPathmc: "/etc/minio/mc/"
## Path where PV would be mounted on the MinIO Pod
mountPath: "/export"
## Override the root directory which the minio server should serve from.
## If left empty, it defaults to the value of {{ .Values.mountPath }}
## If defined, it must be a sub-directory of the path specified in {{ .Values.mountPath }}
##
bucketRoot: ""
# Number of drives attached to a node
drivesPerNode: 1
# Number of MinIO containers running
replicas: 1
# Number of expanded MinIO clusters
pools: 1
# Deploy if 'mode == gateway' - 4 replicas.
gateway:
type: "nas" # currently only "nas" is supported.
replicas: 4
## TLS Settings for MinIO
tls:
enabled: false
## Create a secret with private.key and public.crt files and pass that here. Ref: https://github.com/minio/minio/tree/master/docs/tls/kubernetes#2-create-kubernetes-secret
certSecret: ""
publicCrt: public.crt
privateKey: private.key
## Trusted Certificates Settings for MinIO. Ref: https://docs.minio.io/docs/how-to-secure-access-to-minio-server-with-tls#install-certificates-from-third-party-cas
## Bundle multiple trusted certificates into one secret and pass that here. Ref: https://github.com/minio/minio/tree/master/docs/tls/kubernetes#2-create-kubernetes-secret
## When using self-signed certificates, remember to include MinIO's own certificate in the bundle with key public.crt.
## If certSecret is left empty and tls is enabled, this chart installs the public certificate from .Values.tls.certSecret.
trustedCertsSecret: ""
## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistence:
enabled: true
annotations: {}
## A manually managed Persistent Volume and Claim
## Requires persistence.enabled: true
## If defined, PVC must be created manually before volume will be bound
existingClaim: ""
## minio data Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
## Storage class of PV to bind. By default it looks for standard storage class.
## If the PV uses a different storage class, specify that here.
storageClass: "standard"
VolumeName: ""
accessMode: ReadWriteOnce
size: 80Gi
## If subPath is set mount a sub folder of a volume instead of the root of the volume.
## This is especially handy for volume plugins that don't natively support sub mounting (like glusterfs).
##
subPath: ""
## Expose the MinIO service to be accessed from outside the cluster (LoadBalancer service).
## or access it from within the cluster (ClusterIP service). Set the service type and the port to serve it.
## ref: http://kubernetes.io/docs/user-guide/services/
##
service:
type: ClusterIP
clusterIP: ~
## Make sure to match it to minioAPIPort
port: "9000"
nodePort: 32000
## Configure Ingress based on the documentation here: https://kubernetes.io/docs/concepts/services-networking/ingress/
##
ingress:
enabled: false
# ingressClassName: ""
labels: {}
# node-role.kubernetes.io/ingress: platform
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
# kubernetes.io/ingress.allow-http: "false"
# kubernetes.io/ingress.global-static-ip-name: ""
# nginx.ingress.kubernetes.io/secure-backends: "true"
# nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
# nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
path: /
hosts:
- chart-example.local
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
consoleService:
type: ClusterIP
clusterIP: ~
## Make sure to match it to minioConsolePort
port: "9001"
nodePort: 32001
consoleIngress:
enabled: false
# ingressClassName: ""
labels: {}
# node-role.kubernetes.io/ingress: platform
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
# kubernetes.io/ingress.allow-http: "false"
# kubernetes.io/ingress.global-static-ip-name: ""
# nginx.ingress.kubernetes.io/secure-backends: "true"
# nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
# nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
path: /
hosts:
- chart-example.local
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
## Node labels for pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector: {}
tolerations: []
affinity: {}
## Add stateful containers to have security context, if enabled MinIO will run as this
## user and group NOTE: securityContext is only enabled if persistence.enabled=true
securityContext:
enabled: true
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
# Additational pod annotations
podAnnotations: {}
# Additional pod labels
podLabels: {}
## Configure resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
resources:
requests:
memory: 4Gi
## List of users to be created after minio install
##
users:
## Username, password and policy to be assigned to the user
## Default policies are [readonly|readwrite|writeonly|consoleAdmin|diagnostics]
## Add new policies as explained here https://docs.min.io/docs/minio-multi-user-quickstart-guide.html
## NOTE: this will fail if LDAP is enabled in your MinIO deployment
## make sure to disable this if you are using LDAP.
- accessKey: console
secretKey: console123
policy: consoleAdmin
## Additional Annotations for the Kubernetes Job makeUserJob
makeUserJob:
podAnnotations:
annotations:
securityContext:
enabled: false
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
resources:
requests:
memory: 128Mi
nodeSelector: {}
tolerations: []
affinity: {}
## List of buckets to be created after minio install
##
buckets:
# # Name of the bucket
# - name: bucket1
# # Policy to be set on the
# # bucket [none|download|upload|public]
# policy: none
# # Purge if bucket exists already
# purge: false
# # set versioning for
# # bucket [true|false]
# versioning: false
# - name: bucket2
# policy: none
# purge: false
# versioning: true
## Additional Annotations for the Kubernetes Job makeBucketJob
makeBucketJob:
podAnnotations:
annotations:
securityContext:
enabled: false
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
resources:
requests:
memory: 128Mi
nodeSelector: {}
tolerations: []
affinity: {}
## Use this field to add environment variables relevant to MinIO server. These fields will be passed on to MinIO container(s)
## when Chart is deployed
environment:
## Please refer for comprehensive list https://docs.min.io/minio/baremetal/reference/minio-server/minio-server.html
## MINIO_SUBNET_LICENSE: "License key obtained from https://subnet.min.io"
## MINIO_BROWSER: "off"
networkPolicy:
enabled: false
allowExternal: true
## PodDisruptionBudget settings
## ref: https://kubernetes.io/docs/concepts/workloads/pods/disruptions/
##
podDisruptionBudget:
enabled: false
maxUnavailable: 1
## Specify the service account to use for the Minio pods. If 'create' is set to 'false'
## and 'name' is left unspecified, the account 'default' will be used.
serviceAccount:
create: true
## The name of the service account to use. If 'create' is 'true', a service account with that name
## will be created.
name: "minio-sa"
metrics:
serviceMonitor:
enabled: false
additionalLabels: {}
public: true
# namespace: monitoring
# interval: 30s
# scrapeTimeout: 10s
The generated pod appears in error in the k8s dashboard and when I check the logs, I obtain:
ERROR Invalid command line arguments: Incorrect number of endpoints provided [http://minio-1638458317-{0...0}.minio-1638458317-svc.kube-system.svc.cluster.local/export]
> Please provide correct combination of local/remote paths
HINT:
For more information, please refer to https://docs.min.io/docs/minio-erasure-code-quickstart-guide
I have absolutely no idea of what I can do to solve that issue. Could anyone help me on this?
Thanks in advance,
Rémi
2
Answers
Try this:
helm install --namespace minio --create-namespace --set rootUser=admin,rootPassword=admin123 --set persistence.enabled=true --set persistence.size=2Gi --set resources.requests.cpu=1 --set resources.limits.cpu=2 --set resources.requests.memory=1Gi --set resources.limits.memory=2Gi --set mode=distributed,replicas=4 --generate-name minio/minio
Make sure you have 4 or more pod replicas running, I had the same issue where the format of the IP address specified while starting the minIO server goes with incorrect argument http://minio-1640891988-{0..0}.minio-1640891988-svc.minio.svc.cluster.local/export or works only with contiguous IP addresses.
Refer: [https://github.com/minio/minio/issues/6629]
It’s basically complaining about an incoherent number of IPs provided.
Try to run an even number of replicas of minio. It could also help to set mode to
distributed