Azure ethernal orchestration - Cancel timer

Question

I'm working with the Azure Ethernal Orchestrations, with a tasks pool. I have many tasks to execute, with some priority. I have a service which give me the top priority task. But sometimes I need to wait until a specific date to execute the next task.

So I've made a kind of watchdog using eternal orchestrations.

My problem is when I need to cancel the orchestrations, I need to cancel the Timer too. But when I'm doing it, the function is still in a Running State and my OutputStatus doen't change.

Here my orchetration function :

    [FunctionName(nameof(RunTaskWatchdog))]
    public async Task RunTaskWatchdog(
        [OrchestrationTrigger] IDurableOrchestrationContext context)
    {
        var poolName = context.GetInput<string>();
        TaskStatus output = new()
        {
            PoolName = poolName,
            Message = "Getting next task"
        };
        context.SetCustomStatus(output);

        // Retrieve the next task to execute
        var task = await context.CallActivityAsync<TaskExecution>(nameof(Activities.GetNextResearchForExecution), poolName);
        if (task == null)
        {
            output.Message = "Unable to get a task";
            context.SetCustomStatus(output);
            return;
        }
        if (string.IsNullOrEmpty(task.TaskName))
        {
            output.Message = task.Error;
            context.SetCustomStatus(output);
            return;
        }

        output.Message = "Got task";
        output.CurrentTask = task;
        if (task.NextExecution.HasValue)
        {
            // If the next task can't be executed before a date, sleep till the good date
            output.Message = "Need to wait to respect task delay";
            output.NextExcecution = task.NextExecution.Value;
            context.SetCustomStatus(output);

            try
            {
                await context.CreateTimer(task.NextExecution.Value, tokens[poolName].Token);
            }
            catch (OperationCanceledException)
            {
                output.Message = "Operation cancelled while sleeping";
                output.NextExcecution = null;
                context.SetCustomStatus(output);

                return;

                // After this line, the CustomStatus has not changed, and the orchestration is still in running state
            }
        }
        else
        {
            context.SetCustomStatus(output);
        }

        var executionSuccess = await context.CallActivityAsync<bool>(nameof(Activities.ExecuteResearchActivity), task.TaskName);

        var doneSuccess = await context.CallActivityAsync<bool>(nameof(Activities.SetExecutingTaskDone), poolName);

        DateTime nextTask = context.CurrentUtcDateTime.AddSeconds(10);
        output.Message = "Waiting next execution";
        output.NextExcecution = nextTask;
        output.CurrentTask = null;
        context.SetCustomStatus(output);

        try
        {
            await context.CreateTimer(nextTask, tokens[poolName].Token);
        }
        catch (OperationCanceledException)
        {
            output.Message = "Operation cancelled";
            output.NextExcecution = null;
            context.SetCustomStatus(output);
            return;
        }

        context.ContinueAsNew(poolName);
    }

As you can see, in the catch section, I enter in this in debug, but I don't see any changes when I retrieve the status with Postman (http://localhost:7071/runtime/webhooks/durabletask/instances)

I've made a sample code available here : https://github.com/Rizov74/TestDurableOrchestrator

Answer 1

Ok I've found a way by using the external events :

instead of

        try
        {
            await context.CreateTimer(task.NextExecution.Value, tokens[poolName].Token);
        }
        catch (OperationCanceledException)
        {
            output.Message = "Operation cancelled while sleeping";
            output.NextExcecution = null;
            context.SetCustomStatus(output);

            return;

            // After this line, the CustomStatus has not changed, and the orchestration is still in running state
        }

I use :

            using (var cts = new CancellationTokenSource())
            {
                Task sleepingTask = context.CreateTimer(task.NextExecution.Value, output, cts.Token);
                Task timeoutTask = context.WaitForExternalEvent("stop");

                Task winner = await Task.WhenAny(sleepingTask, timeoutTask);
                if (winner == sleepingTask)
                {
                    // Can continue
                }
                else
                {
                    // Cancel timer token
                    cts.Cancel();
                    if (!context.IsReplaying)
                        Console.WriteLine($"{context.InstanceId}: wait cancelled {poolName} : {task.TaskName}");

                    output.Message = "Operation cancelled while sleeping";
                    output.NextExcecution = null;
                    context.SetCustomStatus(output);

                    // If the watchdog has been cancelled, we need to telle the pool the task is cancelled
                    var ok = await context.CallActivityAsync<bool>(nameof(Activities.CancelExecutingTask), poolName);

                    if (!context.IsReplaying)
                        Console.WriteLine($"{context.InstanceId}: task correctly canceled {poolName} : {task.TaskName}");

                    // Stop the watchdog
                    return;
                }
            }

I have updated my repo on github, you can take a look if you want an complex example of durable orchestrator