I am attempting to deploy an Azure function which has been implemented in Python using the v2 programming model. I’ve tested it locally using Python 3.6-3.9 and it works just fine.
I’m deploying to Azure using Azure Pipelines with the out-of-the-box Python Pipeline deployment processed (modified slightly to use a script task to upgrade pip and then install requirements).
However, the function has yet to execute so much as once.
Two things I have noticed:
- The settings that are in my hosts file don’t appear to be the ones that are being used by Azure
- The function appears to be looking in ‘/home/site/wwwroot/.python_packages/lib/site-packages’ for packages, but I’m using the RUN_FROM_PACKAGE option so this doesn’t seem to make any sense (plus, when I check in /home/site/wwwroot/, there isn’t any .python_packages file).
I’ve asked Genie who has found the following issues:
Issue 1
No job functions found. Try making your job classes and methods public. If you're using binding extensions (e.g. Azure Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), builder.AddTimers(), etc.).
Issue 2
Worker failed to index functions
Result: Failure Exception: ImportError: cannot import name '_imaging' from 'PIL' (/home/site/wwwroot/.python_packages/lib/site-packages/PIL/init.py). Troubleshooting Guide: https://aka.ms/functions-modulenotfound Stack: File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/dispatcher.py', line 338, in _handle__functions_metadata_request fx_metadata_results = self.index_functions(function_path) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/dispatcher.py', line 607, in index_functions indexed_functions = loader.index_function_app(function_path) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/utils/wrappers.py', line 48, in call raise extend_exception_message(e, message) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/utils/wrappers.py', line 44, in call return func(*args, **kwargs) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/loader.py', line 151, in index_function_app imported_module = importlib.import_module(module_name) File '/usr/local/lib/python3.10/importlib/init.py', line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File '', line 1050, in _gcd_import File '', line 1027, in _find_and_load File '', line 1006, in _find_and_load_unlocked File '', line 688, in _load_unlocked File '', line 883, in exec_module File '', line 241, in _call_with_frames_removed File '/home/site/wwwroot/function_app.py', line 2, in from core.data.training_image_filestore import TrainingImageFilestore File '/home/site/wwwroot/core/data/training_image_filestore.py', line 1, in from PIL import Image File '/home/site/wwwroot/.python_packages/lib/site-packages/PIL/Image.py', line 103, in from . import _imaging as core
Logging from the log stream
2023-06-26T19:17:33Z [Information] Host Status: {
"id": "func-xxxxxx",
"state": "Running",
"version": "4.21.3.3",
"versionDetails": "4.21.3+2e42e3beb40b89d4f5d3dd962f3a5d420d376d71",
"platformVersion": "",
"instanceId": "9A4EF22A-638234024389436903",
"computerName": "",
"processUptime": 1400880,
"functionAppContentEditingState": "NotAllowed",
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "4.5.0"
}
}
2023-06-26T19:17:40Z [Information] Host lock lease acquired by instance ID '000000000000000000000000CB538C6F'.
2023-06-26T19:18:18Z [Verbose] Received request to drain the host
2023-06-26T19:18:18Z [Information] DrainMode mode enabled
2023-06-26T19:18:18Z [Information] Calling StopAsync on the registered listeners
2023-06-26T19:18:18Z [Information] Call to StopAsync complete, registered listeners are now stopped
2023-06-26T19:18:32Z [Information] Host lock lease acquired by instance ID '00000000000000000000000086BA5408'.
2023-06-26T19:18:34Z [Information] Loading functions metadata
2023-06-26T19:18:34Z [Information] Reading functions metadata
2023-06-26T19:18:34Z [Information] 1 functions found
2023-06-26T19:18:34Z [Information] 0 functions loaded
2023-06-26T19:18:35Z [Information] Loading functions metadata
2023-06-26T19:18:35Z [Information] Reading functions metadata
2023-06-26T19:18:35Z [Information] 1 functions found
2023-06-26T19:18:35Z [Information] 0 functions loaded
2023-06-26T19:18:36Z [Information] Host Status: {
"id": "func-xxxxxx",
"state": "Running",
"version": "4.21.3.3",
"versionDetails": "4.21.3+2e42e3beb40b89d4f5d3dd962f3a5d420d376d71",
"platformVersion": "",
"instanceId": "9A4EF22A-638234036774982929",
"computerName": "",
"processUptime": 223550,
"functionAppContentEditingState": "NotAllowed",
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "4.5.0"
}
}
function_app.py
from core.data.training_image_filestore import TrainingImageFilestore
from core.services.dataset_service import DatasetService
from core.services.model_service import ModelService
from settings.key_vault_settings import KeyVaultSettings
import datetime
import logging
import azure.functions as func
import os
app = func.FunctionApp()
@app.function_name(name='func-xxxxxx')
@app.schedule(schedule="0 0 0 * * *", arg_name="myTimer", run_on_startup=True,
use_monitor=False)
def funcgcimgorientation(myTimer: func.TimerRequest) -> None:
utc_timestamp = datetime.datetime.utcnow().replace(
tzinfo=datetime.timezone.utc).isoformat()
if myTimer.past_due:
logging.info('The timer is past due!')
logging.info('Python timer trigger function ran at %s', utc_timestamp)
settings = KeyVaultSettings(os.getenv('KEY_VAULT_URL'))
base_path = settings['BaseStoragePath']
logging.info(base_path)
logging.info(base_path + settings['OriginalImagesSource'])
logging.info(base_path + settings['RotatedImagesSource'])
training_image_repo = TrainingImageFilestore(
base_path + settings['OriginalImagesSource'],
base_path + 'training_images/rotated/'
)
dataset_service = DatasetService(training_image_repo)
model_service = ModelService(training_image_repo)
#dataset_service.generate_training_images(angles=[0, 90, 180, 270])
dataset = dataset_service.get_dataset()
model_service.train_model(dataset, model_type=settings['ModelType'])
host.json
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
}
}
},
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[4.*, 5.0.0)"
}
}
Deployment Pipeline YAML
# Python Function App to Linux on Azure
# Build a Python function app and deploy it to Azure as a Linux function app.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/python
trigger:
- main
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: '4ae24131-0b22-421c-8e3e-6d766e891ece'
# Function app name
functionAppName: 'func-xxxxxx'
# Agent VM image name
vmImageName: 'ubuntu-latest'
# Working Directory
workingDirectory: '$(System.DefaultWorkingDirectory)'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- bash: |
if [ -f extensions.csproj ]
then
dotnet build extensions.csproj --runtime ubuntu.16.04-x64 --output ./bin
fi
workingDirectory: $(workingDirectory)
displayName: 'Build extensions'
- task: UsePythonVersion@0
displayName: 'Use Python 3.9'
inputs:
versionSpec: 3.9 # Functions V2 supports Python 3.6 as of today
- bash: |
python -m pip install --upgrade pip
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
workingDirectory: $(workingDirectory)
displayName: 'Install application dependencies'
- task: ArchiveFiles@2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(workingDirectory)'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'development'
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureFunctionApp@1
displayName: 'Azure functions app deploy'
inputs:
azureSubscription: '$(azureSubscription)'
appType: functionAppLinux
appName: $(functionAppName)
package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
2
Answers
SiddeshDesai's answer above pointed me in the right direction.
One of the workarounds suggested on the following GitHub post was to downgrade from v2 to v1 if importing custom Python modules: https://github.com/Azure/azure-functions-python-worker/issues/1262
I did this, and this resolved the issue.
I tried deploying a simple default Timer Trigger Python V2 Function via azure Devops in my function app and timer Trigger was visible. Refer below:-
I created one Azure Python Function app with version 3.10 and added the below setting for Python V2 trigger to get deployed successfully in Azure.
Setting reference
My function_app.py
My Azure Devops YAML pipeline:-
Timer Trigger with Python V2 got deployed successfully:-
If the above method does not work. As, you are importing a custom module in your Code, Its a known issue with Python V2 programming model. Add your comment, In this Github issue raised by me on the same error.