Question 81457 in Docker Question posted in
The official documentation can be found here.

GCP instance group doesn't start containers – Docker

I have an instance template that is supposed to run my app in a container running on Google Cloud’s Container-Optimized OS. When I create a single VM from this template it runs just fine, but when I use it to create an instance group the containers don’t start.

According to the logs the machine didn’t even try to start them.

I tried to compare the output from gcloud compute instances describe <instance-name> for the instance that works OK against one of the instances in the MIG, but other than some differences in the network interfaces and some that are due to the fact that one instance is managed by an instance group and the other one isn’t I don’t see anything unusual.

I also noticed that when I SSH to the instance that works, I get this message:

  ########################[ Welcome ]########################
  #  You have logged in to the guest OS.                    #
  #  To access your containers use 'docker attach' command  #
  ###########################################################

but when I SSH to one of the instances in the MIG, I don’t see it.

Is there a problem with using the container-optimized OS in an instance group?

My instance template is defined as follows:

creationTimestamp: '2022-11-09T03:25:29.896-08:00'
description: ''
id: '757769630202081478'
kind: compute#instanceTemplate
name: server-using-docker-hub-1
properties:
  canIpForward: false
  confidentialInstanceConfig:
    enableConfidentialCompute: false
  description: ''
  disks:
  - autoDelete: true
    boot: true
    deviceName: server-using-docker-hub
    index: 0
    initializeParams:
      diskSizeGb: '10'
      diskType: pd-balanced
      sourceImage: projects/cos-cloud/global/images/cos-stable-101-17162-40-20
    kind: compute#attachedDisk
    mode: READ_WRITE
    type: PERSISTENT
  keyRevocationActionType: NONE
  labels:
    container-vm: cos-stable-101-17162-40-20
  machineType: e2-micro
  metadata:
    fingerprint: 76mZ3i--POo=
    items:
    - key: gce-container-declaration
      value: |-
        spec:
          containers:
          - name: server-using-docker-hub-1
            image: docker.io/rinbar/kwik-e-mart
            env:
            - name: AWS_ACCESS_KEY_ID
              value: <redacted>
            - name: AWS_SECRET_ACCESS_KEY
              value: <redacted>
            - name: SECRET_FOR_SESSION
              value: <redacted>
            - name: SECRET_FOR_USER
              value: <redacted>
            - name: MONGODBURL
              value: mongodb+srv://<redacted>@cluster0.<redacted>.mongodb.net/kwik-e-mart
            - name: DEBUG
              value: server:*
            - name: PORT
              value: '80'
            stdin: false
            tty: false
          restartPolicy: Always
        # This container declaration format is not public API and may change without notice. Please
        # use gcloud command-line tool or Google Cloud Console to run Containers on Google Compute Engine.
    kind: compute#metadata
  networkInterfaces:
  - kind: compute#networkInterface
    name: nic0
    network: https://www.googleapis.com/compute/v1/projects/rons-project-364411/global/networks/default
    stackType: IPV4_ONLY
    subnetwork: https://www.googleapis.com/compute/v1/projects/rons-project-364411/regions/me-west1/subnetworks/default
  reservationAffinity:
    consumeReservationType: ANY_RESERVATION
  scheduling:
    automaticRestart: true
    onHostMaintenance: MIGRATE
    preemptible: false
    provisioningModel: STANDARD
  serviceAccounts:
  - email: [email protected]
    scopes:
    - https://www.googleapis.com/auth/devstorage.read_only
    - https://www.googleapis.com/auth/logging.write
    - https://www.googleapis.com/auth/monitoring.write
    - https://www.googleapis.com/auth/servicecontrol
    - https://www.googleapis.com/auth/service.management.readonly
    - https://www.googleapis.com/auth/trace.append
  shieldedInstanceConfig:
    enableIntegrityMonitoring: true
    enableSecureBoot: false
    enableVtpm: true
  tags:
    items:
    - http-server
selfLink: https://www.googleapis.com/compute/v1/projects/rons-project-364411/global/instanceTemplates/server-using-docker-hub-1
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Since the instances in the group have no external IP addresses, you need to enable Private Google Access or Cloud NAT to allow the instances to pull the container image from Container Registry / Artifact Registry / Docker Hub / any other container registry.


  2. 0

    I’m unable to replicate your issue; it worked for me.

    I wonder whether your issue is container registry permissions? I don’t use MIGs but assume a MIG runs as a service account and that perhaps yours doesn’t have appropriate permission (to access the container registry)?

    The one caveat is that gcloud compute instance-templates create-with-container is confusing and I am unable to resolve how to use --create-disk and --disk flags. I ended up using the Console to create the template. The Console’s tool to generate the equivalent gcloud command is also incorrect (submitted feedback).

    Q="74331370"
PROJECT="$(whoami)-$(date +%y%m%d)-${Q}"
ZONE="us-west1-c"

TEMPLATE="tmpl"
GROUP="group"
IMAGE="gcr.io/kuar-demo/kuard-amd64:blue"

SIZE="2"
MIN=${SIZE}
MAX=${MIN}

# This command is confusing
# Ultimately I used the console to save time
gcloud compute instance-templates create-with-container ${TEMPLATE} 
--project=${PROJECT} 
--machine-type=f1-micro 
--tags=http-server 
--container-image=${IMAGE} 
--create-disk=image-project=cos-cloud,image-family=cos-stable,mode=rw,size=10,type=pd-balanced 
--disk=auto-delete=yes,boot=yes,device-name=${TEMPLATE}

gcloud beta compute instance-groups managed create ${GROUP} 
--project=${PROJECT} 
--base-instance-name=${GROUP} 
--size=${SIZE} 
--template=${TEMPLATE} 
--zone=${ZONE} 
--list-managed-instances-results=PAGELESS

gcloud beta compute instance-groups managed set-autoscaling ${GROUP} 
--project=${PROJECT} 
--zone=${ZONE} 
--min-num-replicas=${MIN} 
--max-num-replicas=${MAX} 
--mode=off

INSTANCES=$(
  gcloud compute instance-groups managed list-instances ${GROUP} 
  --project=${PROJECT} 
  --zone=${ZONE} 
  --format="value(instance)")

for INSTANCE in ${INSTANCES}
  do
    gcloud compute ssh ${INSTANCE} 
    --project=${PROJECT} 
    --zone=${ZONE} 
    --command="docker container ls"
done

    Yields (edited for clarity):

    CONTAINER ID   IMAGE     COMMAND   CREATED        STATUS        NAMES
dd902f2d5e29   ${IMAGE}  "/kuard"  4 minutes ago  Up 4 minutes  klt-tmpl-rqhp

CONTAINER ID   IMAGE     COMMAND   CREATED        STATUS        NAMES
0182f3e7f3dc   ${IMAGE}  "/kuard"  4 minutes ago  Up 4 minutes  klt-tmpl-azxs
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search