Azure Question posted in
The official documentation to get you started can be found here.

Get retry count inside Azure Durable Function activity

I’m using an Azure Durable Function v4 (isolated) orchestrator that starts a couple of activities. There is a retry-policy that will retry a failed activity a certain amount of time. Inside the activity, I would like to get the number of retries that have been performed for that activity. Is this possible? If yes, how would I go about it?

This small class corresponds to my actual setup:

using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.DurableTask;
using Microsoft.DurableTask.Client;
using Microsoft.Extensions.Logging;

namespace Functions;

public class DemoDurableFunction
{
  [Function(nameof(StartOrchestrator))]
  public async Task<object> StartOrchestrator(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "start")] HttpRequestData req,
    [DurableClient] DurableTaskClient client)
  {
    var instanceId = await client.ScheduleNewOrchestrationInstanceAsync(nameof(RunOrchestrator), input: "");
    var response = client.CreateCheckStatusResponse(req, instanceId);

    return response;
  }

  [Function(nameof(RunOrchestrator))]
  public async Task RunOrchestrator([OrchestrationTrigger] TaskOrchestrationContext context)
  {
    var replaySafeLogger = context.CreateReplaySafeLogger(nameof(RunOrchestrator));
    var options = TaskOptions.FromRetryPolicy(new RetryPolicy(maxNumberOfAttempts: 2, firstRetryInterval: TimeSpan.FromSeconds(1)));

    var tasks = new List<Task<string>>
    {
      context.CallActivityAsync<string>(nameof(RunActivity), input: "Alice", options: options),
      context.CallActivityAsync<string>(nameof(RunActivity), input: "Bob", options: options),
      context.CallActivityAsync<string>(nameof(RunActivity), input: "Chuck", options: options)
    };

    try
    {
      await Task.WhenAll(tasks);
    }
    catch (Exception e)
    {
      replaySafeLogger.LogError("Something horrible happened: {Message}", e.Message);
    }

    foreach (var task in tasks)
    {
      if (task.IsFaulted)
      {
        replaySafeLogger.LogInformation(">>> Activity failed: {Message}", task.Exception?.Message);
      }
      else
      {
        replaySafeLogger.LogInformation(">>> Activity result: {Result}", task.Result);
      }
    }
  }

  [Function(nameof(RunActivity))]
  public async Task<string> RunActivity([ActivityTrigger] string name, FunctionContext executionContext)
  {
    // TODO: log the number of retries that have been performed for this specific activity (not any activity of the 3).

    await Task.Delay(500);

    if (name.StartsWith(""C"))
    {
      throw new Exception($"No C-names allowed {executionContext.RetryContext}");
    }

    return $"DONE: {name}";
  }
}

where the Program is simply:

using Microsoft.Extensions.Hosting;

var host = new HostBuilder()
  .ConfigureFunctionsWorkerDefaults()
  .ConfigureLogging(_ => {})
  .Build();

host.Run();

The FunctionContext inside the activity has a RetryContext property, but that is always null.

Everything works as expected: 2 tasks are successfully processed, and the "Chuck" fails and retried a couple of times.

Tags:

2

Answers


  1. 0

    Get retry count inside Azure Durable Function activity

    The Activity functions are unaware of how many times they are called or retried and there are no built-in function. There is no way to get the retry count from Function context from Activity function. To get retry count Alternatively, you can use a global variable which is initialized with 0 and then incremented in activity function like below code (Modified your code a bit) :

    using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.DurableTask;
using Microsoft.DurableTask.Client;
using Microsoft.Extensions.Logging;

namespace FunctionApp116
{
    public static class Function1
    {
        static int rithwik_count = 0;
        [Function(nameof(StartOrchestrator))]
        public static async Task<object> StartOrchestrator([HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "start")] HttpRequestData req,[DurableClient] DurableTaskClient client)
        {
            var instanceId = await client.ScheduleNewOrchestrationInstanceAsync(nameof(RunOrchestrator), input: "");
            var response = client.CreateCheckStatusResponse(req, instanceId);

            return response;
        }
        [Function(nameof(RunOrchestrator))]
        public static async Task RunOrchestrator([OrchestrationTrigger] TaskOrchestrationContext context)
        {
            var replaySafeLogger = context.CreateReplaySafeLogger(nameof(RunOrchestrator));
            var options = TaskOptions.FromRetryPolicy(new RetryPolicy(maxNumberOfAttempts: 2, firstRetryInterval: TimeSpan.FromSeconds(1)));
  
            var tasks = new List<Task<string>>
            {
              context.CallActivityAsync<string>(nameof(RunActivity), input: "Rithwik", options: options),
              context.CallActivityAsync<string>(nameof(RunActivity), input: "Bojja", options: options),
              context.CallActivityAsync<string>(nameof(RunActivity), input: "Chotu", options: options)
            };

            try
            {
                await Task.WhenAll(tasks);
            }
            catch (Exception e)
            {
                replaySafeLogger.LogError("Something horrible happened: {Message}", e.Message);
            }

            foreach (var task in tasks)
            {
                if (task.IsFaulted)
                {
                    replaySafeLogger.LogInformation(">>> Activity failed: {Message}", task.Exception?.Message);
                }
                else
                {
                    replaySafeLogger.LogInformation(">>> Activity result: {Result}", task.Result);
                }
            }
            Console.WriteLine($"Hello Rithwik Bojja, Number of retries are : {rithwik_count}");
        }

        [Function(nameof(RunActivity))]
        public static async Task<string> RunActivity([ActivityTrigger] string name, FunctionContext executionContext)
        {   
            await Task.Delay(500);
            if (name.StartsWith("C"))
            {
                rithwik_count++;
                throw new Exception($"No C-names allowed {executionContext.RetryContext}");
            }
            return $"Hi Man it is DONE: {name}";
        }
    }
}

    Output:

    enter image description here

    Login or Signup to reply.
  2. 0

    Sure, here’s a revised response with a more casual tone:

    Hey there!

    So you’re dealing with Azure Durable Functions and wanna track retries for your activities, right? Well, the bad news is, as you’ve found out, the RetryContext in FunctionContext isn’t always reliable. But don’t sweat it, there’s a sneaky little way around this.

    Here’s the deal: you can sort of DIY your retry tracker within the activity. It’s a bit of manual work, but hey, it gets the job done. You’ll need to tweak your RunActivity method a bit. Here’s how you could do it:

    [Function(nameof(RunActivity))]
public async Task<string> RunActivity([ActivityTrigger] string name, FunctionContext executionContext)
{
    int retryCount = 0;

    // Let's see if we already got a retry count hanging around
    if (executionContext.CustomStatus != null && int.TryParse(executionContext.CustomStatus.ToString(), out int existingRetryCount))
    {
        retryCount = existingRetryCount;
    }

    try
    {
        await Task.Delay(500); // Just simulating some work here

        if (name.StartsWith("C"))
        {
            // Uh-oh, we don't like "C" names, let's bail!
            throw new Exception("No C-names allowed!");
        }

        return $"DONE: {name}";
    }
    catch
    {
        // Oops, let's bump up that retry count and remember it
        retryCount++;
        executionContext.CustomStatus = retryCount;
        throw; // And throw again to keep the retry party going
    }
}

    What this does is it keeps an eye on how many times you’ve retried. Each time your activity throws a fit (I mean, an exception), you bump up your retry counter. This count gets stored in CustomStatus, so it’s there the next time the activity runs.

    Just a heads up, this isn’t the most elegant solution out there, but when you’re in a pinch, it gets the job done. And always remember to keep your error handling sharp, especially with all the moving parts in Durable Functions.

    Hope this gives you what you need! Hit me up if you get stuck or have more questions.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search