skip to Main Content

I am attempting to deploy an Azure function which has been implemented in Python using the v2 programming model. I’ve tested it locally using Python 3.6-3.9 and it works just fine.

I’m deploying to Azure using Azure Pipelines with the out-of-the-box Python Pipeline deployment processed (modified slightly to use a script task to upgrade pip and then install requirements).

However, the function has yet to execute so much as once.

Two things I have noticed:

  1. The settings that are in my hosts file don’t appear to be the ones that are being used by Azure
  2. The function appears to be looking in ‘/home/site/wwwroot/.python_packages/lib/site-packages’ for packages, but I’m using the RUN_FROM_PACKAGE option so this doesn’t seem to make any sense (plus, when I check in /home/site/wwwroot/, there isn’t any .python_packages file).

I’ve asked Genie who has found the following issues:

Issue 1

    No job functions found. Try making your job classes and methods public. If you're using binding extensions (e.g. Azure Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), builder.AddTimers(), etc.).

Issue 2

Worker failed to index functions
Result: Failure Exception: ImportError: cannot import name '_imaging' from 'PIL' (/home/site/wwwroot/.python_packages/lib/site-packages/PIL/init.py). Troubleshooting Guide: https://aka.ms/functions-modulenotfound Stack: File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/dispatcher.py', line 338, in _handle__functions_metadata_request fx_metadata_results = self.index_functions(function_path) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/dispatcher.py', line 607, in index_functions indexed_functions = loader.index_function_app(function_path) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/utils/wrappers.py', line 48, in call raise extend_exception_message(e, message) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/utils/wrappers.py', line 44, in call return func(*args, **kwargs) File '/azure-functions-host/workers/python/3.10/LINUX/X64/azure_functions_worker/loader.py', line 151, in index_function_app imported_module = importlib.import_module(module_name) File '/usr/local/lib/python3.10/importlib/init.py', line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File '', line 1050, in _gcd_import File '', line 1027, in _find_and_load File '', line 1006, in _find_and_load_unlocked File '', line 688, in _load_unlocked File '', line 883, in exec_module File '', line 241, in _call_with_frames_removed File '/home/site/wwwroot/function_app.py', line 2, in from core.data.training_image_filestore import TrainingImageFilestore File '/home/site/wwwroot/core/data/training_image_filestore.py', line 1, in from PIL import Image File '/home/site/wwwroot/.python_packages/lib/site-packages/PIL/Image.py', line 103, in from . import _imaging as core

Logging from the log stream

2023-06-26T19:17:33Z   [Information]   Host Status: {
  "id": "func-xxxxxx",
  "state": "Running",
  "version": "4.21.3.3",
  "versionDetails": "4.21.3+2e42e3beb40b89d4f5d3dd962f3a5d420d376d71",
  "platformVersion": "",
  "instanceId": "9A4EF22A-638234024389436903",
  "computerName": "",
  "processUptime": 1400880,
  "functionAppContentEditingState": "NotAllowed",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "4.5.0"
  }
}
2023-06-26T19:17:40Z   [Information]   Host lock lease acquired by instance ID '000000000000000000000000CB538C6F'.
2023-06-26T19:18:18Z   [Verbose]   Received request to drain the host
2023-06-26T19:18:18Z   [Information]   DrainMode mode enabled
2023-06-26T19:18:18Z   [Information]   Calling StopAsync on the registered listeners
2023-06-26T19:18:18Z   [Information]   Call to StopAsync complete, registered listeners are now stopped
2023-06-26T19:18:32Z   [Information]   Host lock lease acquired by instance ID '00000000000000000000000086BA5408'.
2023-06-26T19:18:34Z   [Information]   Loading functions metadata
2023-06-26T19:18:34Z   [Information]   Reading functions metadata
2023-06-26T19:18:34Z   [Information]   1 functions found
2023-06-26T19:18:34Z   [Information]   0 functions loaded
2023-06-26T19:18:35Z   [Information]   Loading functions metadata
2023-06-26T19:18:35Z   [Information]   Reading functions metadata
2023-06-26T19:18:35Z   [Information]   1 functions found
2023-06-26T19:18:35Z   [Information]   0 functions loaded
2023-06-26T19:18:36Z   [Information]   Host Status: {
  "id": "func-xxxxxx",
  "state": "Running",
  "version": "4.21.3.3",
  "versionDetails": "4.21.3+2e42e3beb40b89d4f5d3dd962f3a5d420d376d71",
  "platformVersion": "",
  "instanceId": "9A4EF22A-638234036774982929",
  "computerName": "",
  "processUptime": 223550,
  "functionAppContentEditingState": "NotAllowed",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "4.5.0"
  }
}

function_app.py

from core.data.training_image_filestore import TrainingImageFilestore
from core.services.dataset_service import DatasetService
from core.services.model_service import ModelService
from settings.key_vault_settings import KeyVaultSettings

import datetime
import logging
import azure.functions as func

import os

app = func.FunctionApp()

@app.function_name(name='func-xxxxxx')
@app.schedule(schedule="0 0 0 * * *", arg_name="myTimer", run_on_startup=True,
              use_monitor=False) 
def funcgcimgorientation(myTimer: func.TimerRequest) -> None:
    utc_timestamp = datetime.datetime.utcnow().replace(
        tzinfo=datetime.timezone.utc).isoformat()

    if myTimer.past_due:
        logging.info('The timer is past due!')

    logging.info('Python timer trigger function ran at %s', utc_timestamp)

    settings = KeyVaultSettings(os.getenv('KEY_VAULT_URL'))
    base_path = settings['BaseStoragePath']
    logging.info(base_path)
    logging.info(base_path + settings['OriginalImagesSource'])
    logging.info(base_path + settings['RotatedImagesSource'])
    training_image_repo = TrainingImageFilestore(
        base_path + settings['OriginalImagesSource'],
        base_path + 'training_images/rotated/'
        )
    dataset_service = DatasetService(training_image_repo)
    model_service = ModelService(training_image_repo)

    #dataset_service.generate_training_images(angles=[0, 90, 180, 270])
    dataset = dataset_service.get_dataset()
    model_service.train_model(dataset, model_type=settings['ModelType'])

host.json

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[4.*, 5.0.0)"
  }
}

Deployment Pipeline YAML

# Python Function App to Linux on Azure
# Build a Python function app and deploy it to Azure as a Linux function app.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/python

trigger:
- main

variables:
  # Azure Resource Manager connection created during pipeline creation
  azureSubscription: '4ae24131-0b22-421c-8e3e-6d766e891ece'

  # Function app name
  functionAppName: 'func-xxxxxx'

  # Agent VM image name
  vmImageName: 'ubuntu-latest'

  # Working Directory
  workingDirectory: '$(System.DefaultWorkingDirectory)'

stages:
- stage: Build
  displayName: Build stage

  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)

    steps:
    - bash: |
        if [ -f extensions.csproj ]
        then
            dotnet build extensions.csproj --runtime ubuntu.16.04-x64 --output ./bin
        fi
      workingDirectory: $(workingDirectory)
      displayName: 'Build extensions'

    - task: UsePythonVersion@0
      displayName: 'Use Python 3.9'
      inputs:
        versionSpec: 3.9 # Functions V2 supports Python 3.6 as of today

    - bash: |
        python -m pip install --upgrade pip
        pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
      workingDirectory: $(workingDirectory)
      displayName: 'Install application dependencies'

    - task: ArchiveFiles@2
      displayName: 'Archive files'
      inputs:
        rootFolderOrFile: '$(workingDirectory)'
        includeRootFolder: false
        archiveType: zip
        archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
        replaceExistingArchive: true

    - publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
      artifact: drop

- stage: Deploy
  displayName: Deploy stage
  dependsOn: Build
  condition: succeeded()

  jobs:
  - deployment: Deploy
    displayName: Deploy
    environment: 'development'
    pool:
      vmImage: $(vmImageName)

    strategy:
      runOnce:
        deploy:

          steps:
          - task: AzureFunctionApp@1
            displayName: 'Azure functions app deploy'
            inputs:
              azureSubscription: '$(azureSubscription)'
              appType: functionAppLinux
              appName: $(functionAppName)
              package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'

2

Answers


  1. Chosen as BEST ANSWER

    SiddeshDesai's answer above pointed me in the right direction.

    One of the workarounds suggested on the following GitHub post was to downgrade from v2 to v1 if importing custom Python modules: https://github.com/Azure/azure-functions-python-worker/issues/1262

    I did this, and this resolved the issue.


  2. I tried deploying a simple default Timer Trigger Python V2 Function via azure Devops in my function app and timer Trigger was visible. Refer below:-

    I created one Azure Python Function app with version 3.10 and added the below setting for Python V2 trigger to get deployed successfully in Azure.

    Setting reference

    AzureWebJobsFeatureFlags
    EnableWorkerIndexing
    

    enter image description here

    My function_app.py

    import datetime
    import logging
    import azure.functions as func
    
    app = func.FunctionApp()
    
    @app.schedule(schedule="0 * * * * *", arg_name="myTimer", run_on_startup=True,
                  use_monitor=False) 
    def TimerTrigger(myTimer: func.TimerRequest) -> None:
        utc_timestamp = datetime.datetime.utcnow().replace(
            tzinfo=datetime.timezone.utc).isoformat()
    
        if myTimer.past_due:
            logging.info('The timer is past due!')
    
        logging.info('Python timer trigger function ran at %s', utc_timestamp)
    
    {
      "version": "2.0",
      "logging": {
        "applicationInsights": {
          "samplingSettings": {
            "isEnabled": true,
            "excludedTypes": "Request"
          }
        }
      },
      "extensionBundle": {
        "id": "Microsoft.Azure.Functions.ExtensionBundle",
        "version": "[4.*, 5.0.0)"
      }
    }
    

    My Azure Devops YAML pipeline:-

    trigger:
    - master
    
    variables:
      azureSubscription: 'xxxxx-xxxx767b85bd948'
    
      # Function app name
      functionAppName: 'siliuconfunc654'
    
      # Agent VM image name
      vmImageName: 'ubuntu-latest'
    
      # Working Directory
      workingDirectory: '$(System.DefaultWorkingDirectory)'
    
    stages:
    - stage: Build
      displayName: Build stage
    
      jobs:
      - job: Build
        displayName: Build
        pool:
          vmImage: $(vmImageName)
    
        steps:
        - bash: |
            if [ -f extensions.csproj ]
            then
                dotnet build extensions.csproj --runtime ubuntu.16.04-x64 --output ./bin
            fi
          workingDirectory: $(workingDirectory)
          displayName: 'Build extensions'
    
        - task: UsePythonVersion@0
          displayName: 'Use Python 3.6'
          inputs:
            versionSpec: 3.10 
    
        - bash: |
            pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
          workingDirectory: $(workingDirectory)
          displayName: 'Install application dependencies'
    
        - task: ArchiveFiles@2
          displayName: 'Archive files'
          inputs:
            rootFolderOrFile: '$(workingDirectory)'
            includeRootFolder: false
            archiveType: zip
            archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
            replaceExistingArchive: true
    
        - publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
          artifact: drop
    
    - stage: Deploy
      displayName: Deploy stage
      dependsOn: Build
      condition: succeeded()
    
      jobs:
      - deployment: Deploy
        displayName: Deploy
        environment: 'development'
        pool:
          vmImage: $(vmImageName)
    
        strategy:
          runOnce:
            deploy:
    
              steps:
              - task: AzureFunctionApp@1
                displayName: 'Azure functions app deploy'
                inputs:
                  azureSubscription: '$(azureSubscription)'
                  appType: functionAppLinux
                  appName: $(functionAppName)
                  package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
    

    enter image description here

    Timer Trigger with Python V2 got deployed successfully:-

    enter image description here

    If the above method does not work. As, you are importing a custom module in your Code, Its a known issue with Python V2 programming model. Add your comment, In this Github issue raised by me on the same error.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search