skip to Main Content

I have a batch process (packaged in a docker image) that needs to run once a day somewhere in the cloud. Google Compute Engine (GCE) seems like a simple and easy to use option for this with it’s container-optimized OS VM. This works really well, except that I cannot find an easy way to automatically shutdown the VM after the docker container finishes. You can specify a startup script but GCE seems to execute it before the container is started, so doing docker wait there does not work. I don’t care that it’s a docker-optimized VM. One of the other basic Linux OSs (like Debian) is fine as long as it can easily be setup with docker.

Is there an easy way of automatically shutting a Linux VM down after the docker container finishes?

2

Answers


  1. Chosen as BEST ANSWER

    I was able to avoid shutting down the VM from within the docker container by simply deploying a regular Debian VM on Compute Engine. Steps:

    First, create a Debian VM on Compute Engine and pass a startup script that installs docker. Startup script:

    #!/bin/bash
    # Install docker. Taken from the docker documentation.
    # Passed into the gcloud command that creates the VM instance.
    apt-get -y remove docker docker-engine docker.io containerd runc
    apt-get update
    curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
    echo 
      "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian 
      $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
    apt-get update
    apt-get -y install docker-ce docker-ce-cli containerd.io
    echo Done installing docker.
    

    Second, replace that startup script with one that pulls and runs the docker image:

    #!/bin/bash
    ...
    # Get authorization to pull from artifact registry
    gcloud -q auth configure-docker us-east1-docker.pkg.dev
    # Pull the image and run it, then shutdown.
    docker pull $image
    docker rm $container
    docker run -v $vol_ini:$app_ini -v $vol_log:$app_log --name $container $image
    shutdown now
    

    Now this VM can be scheduled to run the docker image on a daily basis and automatically shut itself down.


  2. As requested in the question comments, Python code to shutdown a Compute Engine instance.

    Note: The Google Compute Engine Metadata server can provide the REPLACE variables in the script. Also, you do not need to wait for the results to be STOPPED, just verify that the script did not fail. You can test this program locally as well.

    See this answer for tips on COS container credentials. The program below uses ADC (Application Default Credentials).

    from pprint import pprint
    import time
    
    from googleapiclient import discovery
    from oauth2client.client import GoogleCredentials
    
    credentials = GoogleCredentials.get_application_default()
    
    service = discovery.build('compute', 'v1', credentials=credentials)
    
    # Project ID for this request.
    project = 'REPLACE_WITH_PROJECT_ID'
    
    # The name of the zone for this request.
    zone = 'REPLACE_WITH_COMPUTE_ENGINE_ZONE'
    
    # Name of the instance resource to start.
    instance = 'REPLACE_WITH_COMPUTE_ENGINE_INSTANCE_NAME'
    
    request = service.instances().stop(project=project, zone=zone, instance=instance)
    response = request.execute()
    
    pprint(response)
    
    print('Waiting for operation to finish...')
    print('Name:', response['name'])
    
    while True:
        result = service.zoneOperations().get(
            project=project,
            zone=zone,
            operation=response['name']).execute()
    
        print('status:', result['status'])
    
        if result['status'] == 'DONE':
            print("done.")
            break;
    
        if 'error' in result:
            raise Exception(result['error'])
    
        time.sleep(1)
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search