We are using the Azure DevOps server 2019 and we created agent pool and configure two agents in single server. Now my problem is when am trying to run the two pipelines on this pool parallel, anyone of them cancelling after few minutes with error: The operation was cancelled without any logs. I don’t know why it is happening. Single job is running successfully but on parallel jobs, anyone pipeline failing with above error. Is this cause because of running two agents in single server, building on single msbuild. Please someone help me on this. Attaching few pipeline screenshots. Thank you.
Please find the below system.debug logs of two different pipelines failed with same error:
##[debug]Re-evaluate condition on job cancellation for step: 'Build solution **/*.sln'.
##[debug]Leaving Get-SolutionFiles.
##[debug]Leaving C:agentsw1_tasksMSBuild_c6c4c611-aa2e-4a33-b606-5eaba21968241.166.2MSBuild.ps1.
##[error]The operation was canceled.
##[debug]System.OperationCanceledException: The operation was canceled. at System.Threading.CancellationToken.ThrowOperationCanceledException() at Microsoft.VisualStudio.Services.Agent.Util.ProcessInvoker.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.ProcessInvokerWrapper.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.DefaultStepHost.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Boolean inheritConsoleHandler, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.PowerShell3Handler.RunAsync() at Microsoft.VisualStudio.Services.Agent.Worker.TaskRunner.RunAsync() at Microsoft.VisualStudio.Services.Agent.Worker.StepsRunner.RunStepAsync(IStep step, CancellationToken jobCancellationToken)
Finishing: Build solution **/*.sln
##[debug] C:agentsw1106sClientAppnode_modulesfakerliblocalesde_CHname (directory)
##[error]The operation was canceled.
##[debug]System.OperationCanceledException: The operation was canceled. at System.Threading.CancellationToken.ThrowOperationCanceledException() at Microsoft.VisualStudio.Services.Agent.Util.ProcessInvoker.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.ProcessInvokerWrapper.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.DefaultStepHost.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Boolean inheritConsoleHandler, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.NodeHandler.RunAsync() at Microsoft.VisualStudio.Services.Agent.Worker.TaskRunner.RunAsync() at Microsoft.VisualStudio.Services.Agent.Worker.StepsRunner.RunStepAsync(IStep step, CancellationToken jobCancellationToken)
Finishing: NuGet restore
2
Answers
In my investigation I found, When you download specific version pipeline agent from https://github.com/microsoft/azure-pipelines-agent/releases, instead of Azure Devops portal and install, it has a mechanism to downgrade the agent to older version. Use below steps to disable the downgrade.
For more info follow these links: https://crushonme.github.io/2021/09/07/How-To-Disable-Agent-Downgrade-In-AzureDevOps/#how-to-disable-agent-downgrade
https://github.com/microsoft/azure-pipelines-agent/issues/2589
Its working perfect for me.
Based on the simple error message , the cause of this issue maybe that the running time of the agent job has reached the maximum. please check the timeout in
For agent job: Options -> Build job -> Build job timeout in minutes
If it doesn’t work, for further solving the issue, please define
System.Debug
and set ittrue
. After setting it, you should see more detailed logs about the error :”The operation was cancelled “.In addition, there is a note : Although multiple agents can be installed per machine, we strongly suggest to only install one agent per machine. Installing two or more agents may adversely affect performance and the result of your pipelines.
Please refer to doc :azure pipeline agents.
You can use parallel jobs to run pipelines parallelly. Please refer to doc: Configure and pay for parallel jobs