When deploying a Logic App (standard) and the resource doesn’t exist, approx every third deployment will fail with a "kudu service unavailable 503" error.
The deployment is done in two steps:
Infrastructure
ARM template is deployed using the AzureResourceManagerTemplateDeployment@3 task.
The deployment is successful, however:
- the App Service overview page displays "error" instead of the runtime version
- kudu becomes unresponsive
- the Log Stream is unavailable
- In the activity log in the azure portal, there is a Validate action which fails with a message: “The gateway did not receive a response from ‘Microsoft.Web’ within the specified time period: 504"
Stopping/starting the app or changing the App Service Plan fixes this, and the ASP CPU/memory usage is near 0%, which seems to point to an unhandled runtime exception.
In the pipeline, if also deploying the application using AzureFunctionApp@1:
- Within a few minutes Azure DevOps displays “No message found for this issue”
- It may take up to 45 mins to deploy, or it may fail with the output below:
Starting: Deploying Standard Logic App
Task : Azure Functions Deploy
Description : Update a function app with .NET, Python, JavaScript, PowerShell, Java based web applications
Version : 2.238.1
Author : Microsoft Corporation
Help : https://aka.ms/azurefunctiontroubleshooting
Got service connection details for Azure App Service:'logicapp-dev'
NOTE: Function app is VNet integrated.
Deleting App Service Application settings. Data: ["WEBSITE_RUN_FROM_ZIP","WEBSITE_RUN_FROM_PACKAGE"]
App Service Application settings are already present.
Validating deployment package for functions app before Zip Deploy
Package deployment using ZIP Deploy initiated.
##[error]Failed to deploy web package to App Service.
##[warning]Can't find loc string for key: KuduStackTraceURL
##[error]KuduStackTraceURL https://$logicapp-dev:***@logicapp-dev.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace
##[error]Error: Error: Failed to deploy web package to App Service. Service Unavailable (CODE: 503)
##[warning]Error: Failed to update deployment history. Error: Service Unavailable (CODE: 503)
App Service Application URL: https://logicapp-dev.azurewebsites.net
Finishing: Deploying Standard Logic App
Following the link to the kudu trace logs provides the following:
{
"Message": "An error has occurred.",
"ExceptionMessage": "Illegal characters in path.",
"ExceptionType": "System.ArgumentException",
"StackTrace": " at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)rn
at System.IO.Path.Combine(String path1, String path2)rn
at Kudu.Services.Infrastructure.VfsControllerBase.GetOriginalLocalFilePath()
in C:\__w\1\s\Kudu.Services\Infrastructure\VfsControllerBase.cs:line 327rn
at Kudu.Services.Infrastructure.VfsControllerBase.GetLocalFilePath()
in C:\__w\1\s\Kudu.Services\Infrastructure\VfsControllerBase.cs:line 316rn
at Kudu.Services.Infrastructure.VfsControllerBase.GetItem()
in C:\__w\1\s\Kudu.Services\Infrastructure\VfsControllerBase.cs:line 52rn
at lambda_method(Closure , Object , Object[] )rn
at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor
...continued...
}
I’ve spent about 5 days ruling out the following:
- Application itself (workflows, connections, parameters, host.json etc)
- API connections
- App Service Plan (tried deleting + scaling up)
- Application runtime (tried reverting from dotnet to Node)
- ARM template (new logic app ARM template created in Azure Portal also fails)
- Pipeline task (tried using AzureFunctionApp@2)
- Logic App name (tried renaming the logic app and fileshare, initially succeeded in DEV/SIT then same error in UAT)
- Storage Account Firewall (tried enabling public access)
- Not deploying file share manually (deploying a Standard Logic App also provisions one automatically unless WEBSITE_SKIP_CONTENTSHARE_VALIDATION is set to 1)
- Deleting deployments folder via Kudu (some users reported the residual files being there related to the connection between DevOps and Azure)
Notes:
- Logic App is VNet integrated using a storage account in a separate resource group.
- Storage account is configured with private endpoints and to allow access to Trusted Azure Services.
- App Settings have WEBSITE_CONTENTAZUREFILECONNECTIONSTRING, WEBSITE_CONTENTOVERVNET and WEBSITE_SKIP_CONTENTSHARE_VALIDATION configured (storage File Share is deployed first)
- App Insights show no failures from the logic app to the storage account file share
Has anyone come across this kudu 503 error before when deploying a Logic App (standard) or Function App?
2
Answers
I am able to deploy the Logic App ARM Template with ARM Template Deployment task and Logic App project (Function task too).
My ARM Template:-
Release pipeline:-
Output:-
My logic app FunctionAppDeployment task:-
Yaml code:-
Output:-
To rule out the possibility of a host ID collision (which can occur in Standard Logic Apps as well as in Function Apps), try adding the AzureFunctionsWebHost__hostid app setting with a unique value.