Question posted in 
I’m currently exploring the transition from Microsoft-hosted agents to Azure Scale Sets for our CI/CD pipelines in Azure DevOps. However, I’m trying to wrap my head around the actual benefits of this move, especially considering we have a fixed limit on the number of parallel jobs we can run at any given time (e.g., 10 parallel jobs).
Given this constraint, would switching to Azure Scale Sets truly offer significant advantages? My understanding is that even with scale sets, we would still be bound by our parallel job limit, so I’m curious about how scale sets might improve our CI/CD process efficiency, cost management, or resource utilization.
Thank you
2
Answers
There are two substantial advantages for a VM Scale Set as Azure DevOps pipeline agents, both deriving from the fact that they are VMs running in your Azure environment: (1) connectivity to resources using Private Endpoints and (2) the ability to use a User-Assigned Managed Identity assigned to the scale set as the authentication backing the service connection between Azure DevOps and Azure.
In terms of management complexity, management complexity is much higher by definition because you have your own compute resources. In practice, I have not had to do any management after setting these up – especially since I have the configuration from the Azure DevOps side such that agents are destroyed and re-created after pipeline runs, with a minimal number always available. I also used ephemeral disks, while the cost in my case would not be significant, why pay for things that one does not need? There will presumably be some management effort when the Scale Set needs to be switched to a new version of Linux but one way or another that will not be significant.
Nice discussion before. I would like to add more details below, hope it helps.
Firstly, scale set agent is one of
self-hostedagents, it doesn’t have parallel job limit as Microsoft-hosted. Foronescale set agent pool, you can have maximum1000virtual machines, but you can have many scale set agent pools in your organization.For
Microsoft-hostedagent, new organization begins with a cap of25 parallel jobs. If your needs exceed this limit, you need to reach out to support to request an increase of the job number for your DevOps organization. You can check the doc Configure and pay for parallel jobs for more details.With scale set agent pool, you can have more flexibility on the agents, there are more specifics below(benefit):
When you need more memory, CPU, storage,VM type than what Microsoft-hosted agents can offer.
When you need specific software installed or preinstalled, OS type on the agent which Microsoft-hosted agent cannot offer.
When you have specific network required on the agent, eg: when deploying to a private Azure resource within a private VNET, or restrict network connectivity of agent machines to approved sites only.
If your jobs exceed the Microsoft-hosted agent timeout.
When you want to run consecutive jobs on an agent to benefit from incremental source and package caches.
When you would like to debug locally when error occurs, or run the agent interactively.
Note that scale set agents support Windows or Linux agents, but not Mac agents.
There also some disadvantages with scale set agent pool. The maintenance cost of using Microsoft-hosted agents is almost zero, but you need to maintain the Azure scale set by yourself:
Although it’s supported to managed how the scale set agent works in agent pool configuration, it could still happen some issue occurs on scale set which caused the pool cannot be used, like the connection could be lost, and the scale set cannot be scale up.
You also need to prepare, update the image on scale set by yourself.
It could take more time(scale up, install exetions…etc) for scale set agent to be ready for DevOps jobs.
From consumption, they are charged in different, you can check and compare with below. You can choose the better type according to your real situation.
Scale set Pricing
DevOps Microsoft-hosted