Polly weird behavior (infinite loop?)

Question

I'm using CircuitBreaker with Polly the first time and I'm facing a problem:

My program executes everything without throwing an Error. As soon as I get to this codeline, the program Executes it and nothing happens. I cant debug any further. The program is executing this method for ever without a timeout:

var cB = Policy.Handle<Exception>()
                .CircuitBreakerAsync(3, TimeSpan.FromMinutes(1),
                (ex, t) =>
                {
                    Console.WriteLine("Circuit broken!");
                },
                () =>
                {
                    Console.WriteLine("Circuit Reset!");
                });

HttpClient client = new HttpClient();

var response = await cB.ExecuteAsync((ct) => // <--- Executing this line of code lasts forever
            {
                var request = new HttpRequestMessage(HttpMethod.Get, new Uri(
                    endPoint
                ));
                return client.SendAsync(request, ct);
            }, System.Threading.CancellationToken.None);

This is a .NET Framework 4.8 Project.

I created a new Project. Also a .NET Framework 4.8 Project and copy-pasted the code in the picture into the new project. And: It runs fine!!

So I suggest, there is a configuration? or an adjustment? in the Project I'm working in, which isn't allowing Polly to work correctly.

But what can it be? I get no feedback by e.g. an error...

Does somebody know what can interact with Polly like that?

Answer 1

The observed behaviour was that the application did not make any progress and get stuck at the await ExecuteAsync line. Without knowing what the ExecuteAsync does people can suspect:

• either we have an infinite loop
• or we have a deadlock

Since the Circuit Breaker implementation (so called Engine) does not perform any kind of things that can cause infinite loop that's why we can exclude that as a possible root cause.

cancellationToken.ThrowIfCancellationRequested();
breakerController.OnActionPreExecute();

try
{
    TResult result = action(context, cancellationToken);
    if (shouldHandleResultPredicates.AnyMatch(result))
    {
        breakerController.OnActionFailure(new DelegateResult<TResult>(result), context);
    }
    else
    {
        breakerController.OnActionSuccess(context);
    }

    return result;
}
catch (Exception ex)
{
    Exception handledException = shouldHandleExceptionPredicates.FirstMatchOrDefault(ex);
    if (handledException == null)
    {
        throw;
    }

    breakerController.OnActionFailure(new DelegateResult<TResult>(handledException), context);

    if (handledException != ex)
    {
        ExceptionDispatchInfo.Capture(handledException).Throw();
    }
    throw;
}

So the deadlock has remained as our primary suspect. Back in the old days the ASP.NET MVC was using SynchronizationContext. As Stephen Clearly describe it in detail it was used to continue the operation on the same thread as the invoker of the async method.

In other words the awaiter and the awaitee are waiting for each other. The caller thread waits for the ExecuteAsync to finish and the ExecuteAsync waits for the caller thread to continue.

With the ConfigureAwait(false) you are basically stating that please do not capture the SynchronizationContext, so the ExecuteAsync can continued on any thread it should not be the caller thread.

In case of ASP.NET Core we don't have to worry about this because the SynchronizationContext.Current is null :)