Question 266840 in Ubuntu Question posted in
The official Ubuntu documentation can be found here.

Ubuntu – How to get docker on Azure virtual machine scaleset build agents

I have been loading a docker image from tar file(from a build artifact), tagging it and pushing to an Azure container registry.

This works fine from "regular" build agents. Now we isolated everything in a vnet, and so Im trying to use build agent running on a virtual machine scaleset in the vnet(so I dont need to open anything in the vnet for outside access(devops)).

For Agent pool=Azure pipelines, I get the option of choosing the agent image, and when choosing "ubuntu-22.04", the agent comes with docker installed, and all the predefined docker commands(load, push) works:

Azure devops agent pool

When choosing the vmss agent pool, I do not get the option of choosing the image:

Azure devops vmss agent pool

Im assuming it just uses the image specified when creating the vmss, like this:

az vmss create --name  my-vmss --image Ubuntu2204 --vm-sku Standard_D2_v3 --storage-sku StandardSSD_LRS --disable-overprovision --authentication-type SSH --generate-ssh-keys --upgrade-policy-mode manual --single-placement-group false --platform-fault-domain-count 1 --load-balancer "" --resource-group my-rg  --vnet-name my-vnet --subnet Build

This image(Ubuntu2204) is missing docker, and I cant for the life of me manage to install it in user mode(rootless) in the pipelinescript.

This is what I tried to install rootless docker:

sudo apt update -y
sudo apt install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update -y
sudo apt-get install -y uidmap
apt-cache policy docker-ce
sudo apt install -y docker-ce

dockerd-rootless-setuptool.sh install

export PATH=/usr/bin:$PATH
export XDG_RUNTIME_DIR=/home/AzDevOps/.docker/run
export DOCKER_HOST=unix:///home/AzDevOps/.docker/run/docker.sock

dockerd-rootless.sh

Its just hanging on the last line…

Any tips for how I can get the image used by the "regular" build agents, or get the rootless docker install running or a simpler way of doing this?
(by "this", I mean deploy docker image to container registry placed inside a vnet)

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    If anybody else struggles with this, I got it working by manually installing docker.io in a command line step:

    sudo apt-get -y update
sudo apt install -y docker.io
sudo systemctl start docker
sudo chmod 777 /var/run/docker.sock

    After this I can run docker commands like I can on a regular Azure build agent.

    Still not sure why the AzDevOps user could not connect to docker instance, simply by being member of the docker group, but I suspect it needed a reboot as suggested here: Docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock

    But since vmss build agents are setup from scratch for every job, the chmod hack was the only solution I found...


  2. 0

    The operation to configure one VMSS resource as a self-hosted agent pool in Azure DevOps will add the Microsoft.Azure.DevOps.Pipelines.Agent extension into the VMSS resource, which executes the configuration script to create a local user named AzDevOps.

    enter image description here

    For most scenarios, the configuration script then immediately starts the agent to run as the local user AzDevOps. The agent goes Online and is ready to run pipeline jobs.

    In addition, taking the current version of configuration script for example, it already includes the command below to set the local user AzDevOps to use docker Cli. So all we need to do is to install docker on VMSS instances.

    sudo usermod -a -G docker AzDevOps

    At the moment that the VMSS agent is just configured, we cannot run sudo command in pipeline script, as it will get stuck.

    Instead, you may try to connect to one of your VMSS instance via SSH and use the command below to install docker capability for this VM.

    sudo apt install docker.io

    I ran ssh azureuser@<VMInstancePublicIP> with password of azureuser to connect one of my VM instance azvmss-li000001.

    enter image description here

    After that the pipeline job running as AzDevOps on the agent machine azvmss-li000001was able to run docker commands.

    enter image description here

    As a kind reminder, this is only a workaround that you can try inside one single instance. For newly scaled out VM instances, it will fail since they have never installed docker.

    For this, you may consider using custom script or generating custom VM images to install docker for all your VMSS instances, before configuring them as self-hosted agents of pipelines. See more details in Azure Virtual Machine Scale Set agents – Azure Pipelines | Microsoft Learn

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search