'Why my asp.net methods is hanging under load?
I have a legacy asp.net webapi application (.net 4.7.2) to debug from a performance perspective. A specific endpoint is doiing a lot of work and returning multiple informations by analysing a lists of flags filled by the caller. The first problem that I spotted is that the legacy application is not using any async code when calling the database or other internals apis. I had then run a stress test and find that the application is not using all the CPU and seeems to queue the requests during the load test. In the enpoint of the api, there is a bunch of tasks that are run parralely and all on theyre own background thread (task.run is used for each tasks) :
//Entry point of the api :
var taskList = GetListOfTask(requestInfo, userInfo);
Task.WhenAll(taskList).Wait();
//Computing the list of tasks to run in parallel :
private List<Task> GetListOfTask(RequestInfoDTO requestInfo, UserInfo userInfo)
{
List<Task> taskList = new List<Task>();
if (requestInfo.FindUserDetailCriteria){
//GetUserDetailInfo is a method doing a search in current database on a background thread
taskList.Add(GetUserDetailInfo(requestInfo, userInfo));
}
if (requestInfo.TestBlackList){
//GetFromBLService is a method calling another REST service synchronously on a background thread
taskList.Add(GetFromBLService(requestInfo, userInfo ));
}
if (requestInfo.ReturnPendingActionToDo){
//GetPendingActionToDo is a method doing a BIG recursive search in current database on a background thread
taskList.Add(GetPendingActionToDo(requestInfo, userInfo));
}
//others task looking in service db or making a call with a background thread
...
return taskList;
}
And here an example of one of this task (they are all calling either synchronously another REST service API, either making synchronously a bunch of calls in the database) :
private Task GetUserDetailInfo(RequestInfoDTO requestInfo, UserInfo userInfo)
{
var currentMethod = System.Reflection.MethodBase.GetCurrentMethod();
Context current = Context.Current;
HttpContext httpContext = HttpContext.Current;
return Task.Run(() =>
{
Context.Use(current);
HttpContext.Current = httpContext;
try
{
var userRepository = new UserRepository();
userInfo.DetailInfo = userRepository.FindByUserId(requestInfo.UserId).Details;
}
catch (Exception ex)
{
Ilog.Error(ex, currentMethod);
}
});
}
Under load stress test, I see that nearly 20% of the threads are all wainting with the same kind of stacktrace :
Thread 63
Current frame: ntdll!NtWaitForMultipleObjects+0xc
ChildEBP RetAddr Caller, Callee
37bee12c 74a79d73 KERNELBASE!WaitForMultipleObjectsEx+0x133, calling ntdll!NtWaitForMultipleObjects
37bee174 74a79dc9 KERNELBASE!WaitForMultipleObjectsEx+0x189, calling ntdll!RtlActivateActivationContextUnsafeFast
37bee1b8 6cac31f0 clr!SafeHandle::Release+0xde, calling clr!SafeHandle::RunReleaseMethod
37bee1f8 6cac30c6 clr!SafeHandle::Dispose+0x3f
37bee23c 273768c4 (MethodDesc 25484774 +0x5c UserManagement.GetPendingActionToDo(UserManagement.RequestInfoDTO, UserManagement.UserInfo)), calling (MethodDesc 6b3aff60 +0 System.Reflection.MethodBase.GetCurrentMethod())
37bee28c 273768c4 (MethodDesc 25484774 +0x5c UserManagement.GetPendingActionToDo(UserManagement.RequestInfoDTO, UserManagement.UserInfo)), calling (MethodDesc 6b3aff60 +0 System.Reflection.MethodBase.GetCurrentMethod())
37bee2c0 6cb353ee clr!WaitForMultipleObjectsEx_SO_TOLERANT+0x3c, calling KERNELBASE!WaitForMultipleObjectsEx
37bee310 6cb35134 clr!Thread::DoAppropriateWaitWorker+0x237, calling clr!WaitForMultipleObjectsEx_SO_TOLERANT
37bee39c 6cb35225 clr!Thread::DoAppropriateWait+0x64, calling clr!Thread::DoAppropriateWaitWorker
37bee408 6cb35394 clr!CLREventBase::WaitEx+0x128, calling clr!Thread::DoAppropriateWait
37bee454 6cbc8efb clr!CLREventBase::Wait+0x1a, calling clr!CLREventBase::WaitEx
37bee468 6cbb62d0 clr!Thread::Block+0x25, calling clr!CLREventBase::Wait
37bee47c 6cbb640a clr!SyncBlock::Wait+0x189, calling clr!Thread::Block
37bee4e4 6cbccc6d clr!ObjHeader::GetSyncBlock+0x1ff, calling clr!CrstBase::Leave
37bee52c 6cbb64ea clr!ObjectNative::WaitTimeout+0xcb, calling clr!SyncBlock::Wait
37bee594 6cbb6486 clr!ObjectNative::WaitTimeout+0x36, calling clr!LazyMachStateCaptureState
37bee5a0 6b6c1cdb (MethodDesc 6b2f906c +0x5b System.Threading.CancellationToken.Register(System.Action`1<System.Object>, System.Object, Boolean, Boolean)), calling clr!JIT_ByRefWriteBarrier
37bee5d0 6b6b6ff7 (MethodDesc 6b2f8758 +0x17 System.Threading.Monitor.Wait(System.Object, Int32, Boolean)), calling clr!ObjectNative::WaitTimeout
37bee5e0 6b6a885c (MethodDesc 6b2f8770 +0xc System.Threading.Monitor.Wait(System.Object, Int32)), calling (MethodDesc 6b2f8758 +0 System.Threading.Monitor.Wait(System.Object, Int32, Boolean))
37bee5e8 6b6c2da6 (MethodDesc 6b3c693c +0x226 System.Threading.ManualResetEventSlim.Wait(Int32, System.Threading.CancellationToken)), calling (MethodDesc 6b2f8770 +0 System.Threading.Monitor.Wait(System.Object, Int32))
37bee638 6b6c74dc (MethodDesc 6b3c6048 +0x9c System.Threading.Tasks.Task.SpinThenBlockingWait(Int32, System.Threading.CancellationToken)), calling (MethodDesc 6b3c693c +0 System.Threading.ManualResetEventSlim.Wait(Int32, System.Threading.CancellationToken))
37bee678 6b76914d (MethodDesc 6b3c603c +0x155 System.Threading.Tasks.Task.InternalWait(Int32, System.Threading.CancellationToken)), calling (MethodDesc 6b3c6048 +0 System.Threading.Tasks.Task.SpinThenBlockingWait(Int32, System.Threading.CancellationToken))
37bee6dc 6b6c7395 (MethodDesc 6b3c6024 +0x35 System.Threading.Tasks.Task.Wait(Int32, System.Threading.CancellationToken)), calling (MethodDesc 6b3c603c +0 System.Threading.Tasks.Task.InternalWait(Int32, System.Threading.CancellationToken))
37bee6ec 6b6c72f8 (MethodDesc 6b3c5ff4 +0x10 System.Threading.Tasks.Task.Wait()), calling (MethodDesc 6b3c6024 +0 System.Threading.Tasks.Task.Wait(Int32, System.Threading.CancellationToken))
37bee6f4 26f18775 (MethodDesc 254847b0 +0x4a5 UserManagement.GetAllUserInfo(UserManagement.RequestInfoDTO)), calling (MethodDesc 6b3c5ff4 +0 System.Threading.Tasks.Task.Wait())
37bee798 26f13b5e (MethodDesc 267744ec +0xf6 UserManagement.UserWebServices.Controllers.UserWebApiController.GetAllUserInfo(UserManagement.RequestInfoDTO)), calling (MethodDesc 254847b0 +0 UserManagement.GetAllUserInfo(UserManagement.RequestInfoDTO))
37bee8a0 26173279 (MethodDesc 267e77cc +0x69 DynamicClass.lambda_method(System.Runtime.CompilerServices.Closure, System.Object, System.Object[])), calling (MethodDesc 267744ec +0 UserManagement.UserWebServices.Controllers.UserWebApiController.GetAllUserInfo(UserManagement.RequestInfoDTO))
37bee8ac 268268f5 (MethodDesc 267e30bc +0x1d System.Web.Http.Controllers.ReflectedHttpActionDescriptor+ActionExecutor+<>c__DisplayClass6_2.<GetExecutor>b__2(System.Object, System.Object[]))
37bee8c8 26826a6f (MethodDesc 267759e0 +0xf System.Web.Http.Controllers.ReflectedHttpActionDescriptor+ActionExecutor.Execute(System.Object, System.Object[]))
37bee8d0 268287f1 (MethodDesc 267730dc +0x81 System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(System.Web.Http.Controllers.HttpControllerContext, System.Collections.Generic.IDictionary`2<System.String,System.Object>, System.Threading.CancellationToken)), calling (MethodDesc 267759e0 +0 System.Web.Http.Controllers.ReflectedHttpActionDescriptor+ActionExecutor.Execute(System.Object, System.Object[]))
37bee8fc 26828400 (MethodDesc 267e44b4 +0x78 System.Web.Http.Controllers.ApiControllerActionInvoker+<InvokeActionAsyncCore>d__1.MoveNext())
37bee938 26827c66 (MethodDesc 267e4598 +0x3e System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Controllers.ApiControllerActionInvoker+<InvokeActionAsyncCore>d__1, System.Web.Http]](<InvokeActionAsyncCore>d__1 ByRef)), calling (MethodDesc 267e44b4 +0 System.Web.Http.Controllers.ApiControllerActionInvoker+<InvokeActionAsyncCore>d__1.MoveNext())
37bee97c 6cbbdba2 clr!MethodTable::CanCastToInterfaceNoGC+0x40, calling clr!MethodTable::CanCastToNonVariantInterface
37bee98c 26827bfc (MethodDesc 2014ea2c +0x7c System.Web.Http.Controllers.ApiControllerActionInvoker.InvokeActionAsyncCore(System.Web.Http.Controllers.HttpActionContext, System.Threading.CancellationToken)), calling (MethodDesc 267e4598 +0 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Controllers.ApiControllerActionInvoker+<InvokeActionAsyncCore>d__1, System.Web.Http]](<InvokeActionAsyncCore>d__1 ByRef))
37bee9e0 26827a4d (MethodDesc 2014ea24 +0x15 System.Web.Http.Controllers.ApiControllerActionInvoker.InvokeActionAsync(System.Web.Http.Controllers.HttpActionContext, System.Threading.CancellationToken)), calling (MethodDesc 2014ea2c +0 System.Web.Http.Controllers.ApiControllerActionInvoker.InvokeActionAsyncCore(System.Web.Http.Controllers.HttpActionContext, System.Threading.CancellationToken))
37bee9f0 268276ca (MethodDesc 267d7530 +0x22 System.Web.Http.Controllers.ActionFilterResult+<>c.<ExecuteAsync>b__5_0(ActionInvoker)), calling 065fb43e
37bee9fc 26827690 (MethodDesc 267d7c6c +0x18 System.Web.Http.Controllers.ActionFilterResult+<>c__DisplayClass7_0`1[[System.Web.Http.Controllers.ActionFilterResult+ActionInvoker, System.Web.Http]].<InvokeActionWithActionFilters>b__0())
37beea0c 26826f8d (MethodDesc 267e2fc0 +0x7d System.Web.Http.Filters.ActionFilterAttribute+<CallOnActionExecutedAsync>d__6.MoveNext())
37beea10 6b6e44a6 (MethodDesc 6b3c1d3c +0xb6 System.Threading.ExecutionContextSwitcher.Undo()), calling (MethodDesc 6b3afbf0 +0 System.Threading.ExecutionContext.OnAsyncLocalContextChanged(System.Threading.ExecutionContext, System.Threading.ExecutionContext))
37beea4c 26826c1e (MethodDesc 267e31e4 +0x3e System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Filters.ActionFilterAttribute+<CallOnActionExecutedAsync>d__6, System.Web.Http]](<CallOnActionExecutedAsync>d__6 ByRef)), calling (MethodDesc 267e2fc0 +0 System.Web.Http.Filters.ActionFilterAttribute+<CallOnActionExecutedAsync>d__6.MoveNext())
37beeaa0 26826bb5 (MethodDesc 246a15e4 +0x85 System.Web.Http.Filters.ActionFilterAttribute.CallOnActionExecutedAsync(System.Web.Http.Controllers.HttpActionContext, System.Threading.CancellationToken, System.Func`1<System.Threading.Tasks.Task`1<System.Net.Http.HttpResponseMessage>>)), calling (MethodDesc 267e31e4 +0 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Filters.ActionFilterAttribute+<CallOnActionExecutedAsync>d__6, System.Web.Http]](<CallOnActionExecutedAsync>d__6 ByRef))
37beeb04 26826813 (MethodDesc 267e2ccc +0xcb System.Web.Http.Filters.ActionFilterAttribute+<ExecuteActionFilterAsyncCore>d__5.MoveNext()), calling (MethodDesc 246a15e4 +0 System.Web.Http.Filters.ActionFilterAttribute.CallOnActionExecutedAsync(System.Web.Http.Controllers.HttpActionContext, System.Threading.CancellationToken, System.Func`1<System.Threading.Tasks.Task`1<System.Net.Http.HttpResponseMessage>>))
37beeb3c 268264ee (MethodDesc 267e2db0 +0x3e System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Filters.ActionFilterAttribute+<ExecuteActionFilterAsyncCore>d__5, System.Web.Http]](<ExecuteActionFilterAsyncCore>d__5 ByRef)), calling (MethodDesc 267e2ccc +0 System.Web.Http.Filters.ActionFilterAttribute+<ExecuteActionFilterAsyncCore>d__5.MoveNext())
37beeb90 26826485 (MethodDesc 246a15d8 +0x85 System.Web.Http.Filters.ActionFilterAttribute.ExecuteActionFilterAsyncCore(System.Web.Http.Controllers.HttpActionContext, System.Threading.CancellationToken, System.Func`1<System.Threading.Tasks.Task`1<System.Net.Http.HttpResponseMessage>>)), calling (MethodDesc 267e2db0 +0 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Filters.ActionFilterAttribute+<ExecuteActionFilterAsyncCore>d__5, System.Web.Http]](<ExecuteActionFilterAsyncCore>d__5 ByRef))
37beebe4 2682633b (MethodDesc 246a15c8 +0x1b System.Web.Http.Filters.ActionFilterAttribute.System.Web.Http.Filters.IActionFilter.ExecuteActionFilterAsync(System.Web.Http.Controllers.HttpActionContext, System.Threading.CancellationToken, System.Func`1<System.Threading.Tasks.Task`1<System.Net.Http.HttpResponseMessage>>)), calling (MethodDesc 246a15d8 +0 System.Web.Http.Filters.ActionFilterAttribute.ExecuteActionFilterAsyncCore(System.Web.Http.Controllers.HttpActionContext, System.Threading.CancellationToken, System.Func`1<System.Threading.Tasks.Task`1<System.Net.Http.HttpResponseMessage>>))
37beebf8 2682627b (MethodDesc 267e2bf0 +0x23 System.Web.Http.Controllers.ActionFilterResult+<>c__DisplayClass6_1.<InvokeActionWithActionFilters>b__1()), calling 065fb42a
37beec10 267caddb (MethodDesc 267d7394 +0x1e3 System.Web.Http.Controllers.ActionFilterResult+<ExecuteAsync>d__5.MoveNext())
37beec14 6b6af63b (MethodDesc 6b3c5cb0 +0x3b System.Threading.Tasks.Task.NewId()), calling (JitHelp: CORINFO_HELP_GETSHARED_GCSTATIC_BASE)
37beec54 267cb21e (MethodDesc 267d7f38 +0x3e System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Controllers.ActionFilterResult+<ExecuteAsync>d__5, System.Web.Http]](<ExecuteAsync>d__5 ByRef)), calling (MethodDesc 267d7394 +0 System.Web.Http.Controllers.ActionFilterResult+<ExecuteAsync>d__5.MoveNext())
37beeca8 267caf2c (MethodDesc 26797460 +0x7c System.Web.Http.Controllers.ActionFilterResult.ExecuteAsync(System.Threading.CancellationToken)), calling (MethodDesc 267d7f38 +0 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Controllers.ActionFilterResult+<ExecuteAsync>d__5, System.Web.Http]](<ExecuteAsync>d__5 ByRef))
37beecf4 267c96fe (MethodDesc 267d5584 +0x256 System.Web.Http.Controllers.AuthenticationFilterResult+<ExecuteAsync>d__5.MoveNext()), calling 065fb02e
37beed2c 267c8bfe (MethodDesc 267d5718 +0x3e System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Controllers.AuthenticationFilterResult+<ExecuteAsync>d__5, System.Web.Http]](<ExecuteAsync>d__5 ByRef)), calling (MethodDesc 267d5584 +0 System.Web.Http.Controllers.AuthenticationFilterResult+<ExecuteAsync>d__5.MoveNext())
37beed80 267c8b94 (MethodDesc 267976c4 +0x7c System.Web.Http.Controllers.AuthenticationFilterResult.ExecuteAsync(System.Threading.CancellationToken)), calling (MethodDesc 267d5718 +0 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Controllers.AuthenticationFilterResult+<ExecuteAsync>d__5, System.Web.Http]](<ExecuteAsync>d__5 ByRef))
37beede0 267c84b2 (MethodDesc 267d4b80 +0x4a System.Web.Http.Controllers.ExceptionFilterResult+<ExecuteAsync>d__6.MoveNext()), calling 065faff2
37beee1c 267c7fbe (MethodDesc 267d4f7c +0x3e System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Controllers.ExceptionFilterResult+<ExecuteAsync>d__6, System.Web.Http]](<ExecuteAsync>d__6 ByRef)), calling (MethodDesc 267d4b80 +0 System.Web.Http.Controllers.ExceptionFilterResult+<ExecuteAsync>d__6.MoveNext())
37beee70 267c7f54 (MethodDesc 267975fc +0x7c System.Web.Http.Controllers.ExceptionFilterResult.ExecuteAsync(System.Threading.CancellationToken)), calling (MethodDesc 267d4f7c +0 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Controllers.ExceptionFilterResult+<ExecuteAsync>d__6, System.Web.Http]](<ExecuteAsync>d__6 ByRef))
37beeed8 2675e023 (MethodDesc 26773e10 +0x1db System.Web.Http.ApiController.ExecuteAsync(System.Web.Http.Controllers.HttpControllerContext, System.Threading.CancellationToken)), calling 065fafde
37beef08 267c0ccb (MethodDesc 267995cc +0x173 System.Web.Http.Dispatcher.HttpControllerDispatcher+<SendAsync>d__15.MoveNext()), calling 26780d8a
37beef48 267c377e (MethodDesc 26799ecc +0x3e System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Dispatcher.HttpControllerDispatcher+<SendAsync>d__15, System.Web.Http]](<SendAsync>d__15 ByRef)), calling (MethodDesc 267995cc +0 System.Web.Http.Dispatcher.HttpControllerDispatcher+<SendAsync>d__15.MoveNext())
37beef8c 6b6e946e (MethodDesc 6b3a6994 +0xe System.Collections.Generic.Dictionary`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].TryGetValue(System.__Canon, System.__Canon ByRef)), calling (MethodDesc 6b3a6954 +0 System.Collections.Generic.Dictionary`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].FindEntry(System.__Canon))
37beef9c 267c1077 (MethodDesc 26797870 +0x7f System.Web.Http.Dispatcher.HttpControllerDispatcher.SendAsync(System.Net.Http.HttpRequestMessage, System.Threading.CancellationToken)), calling (MethodDesc 26799ecc +0 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.Dispatcher.HttpControllerDispatcher+<SendAsync>d__15, System.Web.Http]](<SendAsync>d__15 ByRef))
37beeff8 2745149c (MethodDesc 25b18bfc +0x64 System.Net.Http.HttpMessageInvoker.SendAsync(System.Net.Http.HttpRequestMessage, System.Threading.CancellationToken))
37bef010 267c18f1 (MethodDesc 2679a154 +0x1a9 System.Web.Http.Dispatcher.HttpRoutingDispatcher.SendAsync(System.Net.Http.HttpRequestMessage, System.Threading.CancellationToken))
37bef02c 278e2137 (MethodDesc 202144c0 +0x27 System.Net.Http.DelegatingHandler.SendAsync(System.Net.Http.HttpRequestMessage, System.Threading.CancellationToken))
37bef040 267c2a79 (MethodDesc 2679a2d0 +0x141 System.Web.Http.HttpServer+<SendAsync>d__24.MoveNext()), calling (MethodDesc 202144c0 +0 System.Net.Http.DelegatingHandler.SendAsync(System.Net.Http.HttpRequestMessage, System.Threading.CancellationToken))
37bef084 267c36fe (MethodDesc 2679a9ec +0x3e System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.HttpServer+<SendAsync>d__24, System.Web.Http]](<SendAsync>d__24 ByRef)), calling (MethodDesc 2679a2d0 +0 System.Web.Http.HttpServer+<SendAsync>d__24.MoveNext())
37bef0d8 267c2fc7 (MethodDesc 04c0cde8 +0x7f System.Web.Http.HttpServer.SendAsync(System.Net.Http.HttpRequestMessage, System.Threading.CancellationToken)), calling (MethodDesc 2679a9ec +0 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[System.Web.Http.HttpServer+<SendAsync>d__24, System.Web.Http]](<SendAsync>d__24 ByRef))
37bef134 2745149c (MethodDesc 25b18bfc +0x64 System.Net.Http.HttpMessageInvoker.SendAsync(System.Net.Http.HttpRequestMessage, System.Threading.CancellationToken))
37bef14c 267c343b (MethodDesc 2679a8d4 +0xc3 System.Web.Http.WebHost.HttpControllerHandler+<ProcessRequestAsyncCore>d__12.MoveNext())
37bef188 267c3033 (MethodDesc 2679a9ac +0x43 System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[System.Web.Http.WebHost.HttpControllerHandler+<ProcessRequestAsyncCore>d__12, System.Web.Http.WebHost]](<ProcessRequestAsyncCore>d__12 ByRef)), calling (MethodDesc 2679a8d4 +0 System.Web.Http.WebHost.HttpControllerHandler+<ProcessRequestAsyncCore>d__12.MoveNext())
37bef1dc 267c2f25 (MethodDesc 267983e8 +0x6d System.Web.Http.WebHost.HttpControllerHandler.ProcessRequestAsyncCore(System.Web.HttpContextBase)), calling (MethodDesc 2679a9ac +0 System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[System.Web.Http.WebHost.HttpControllerHandler+<ProcessRequestAsyncCore>d__12, System.Web.Http.WebHost]](<ProcessRequestAsyncCore>d__12 ByRef))
37bef230 267c2ea1 (MethodDesc 267983e0 +0x29 System.Web.Http.WebHost.HttpControllerHandler.ProcessRequestAsync(System.Web.HttpContext)), calling (MethodDesc 267983e8 +0 System.Web.Http.WebHost.HttpControllerHandler.ProcessRequestAsyncCore(System.Web.HttpContextBase))
37bef244 278bf883 (MethodDesc 278ab49c +0x13 System.Web.HttpTaskAsyncHandler+<>c__DisplayClass4_0.<System.Web.IHttpAsyncHandler.BeginProcessRequest>b__0())
37bef24c 278bf8c0 (MethodDesc 278aaf6c +0x28 System.Web.TaskAsyncHelper.BeginTask(System.Func`1<System.Threading.Tasks.Task>, System.AsyncCallback, System.Object))
37bef260 278bfa1e (MethodDesc 278a89a8 +0x4e System.Web.HttpTaskAsyncHandler.System.Web.IHttpAsyncHandler.BeginProcessRequest(System.Web.HttpContext, System.AsyncCallback, System.Object)), calling (MethodDesc 278aaf6c +0 System.Web.TaskAsyncHelper.BeginTask(System.Func`1<System.Threading.Tasks.Task>, System.AsyncCallback, System.Object))
37bef278 278bfc7b (MethodDesc 277cea4c +0x183 System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute())
37bef2bc 278b47cd (MethodDesc 20212e64 +0x85 System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep)), calling 273b1aaa
37bef2d0 278b4944 (MethodDesc 20212e80 +0xa4 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)), calling (MethodDesc 20212e64 +0 System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep))
37bef2d4 278b4d78 (MethodDesc 277cd1b0 +0xa8 System.Web.Util.SynchronizationHelper.TrySetCompletionContinuation(System.Action)), calling (MethodDesc 6b3de054 +0 System.Threading.Interlocked.Exchange[[System.__Canon, mscorlib]](System.__Canon ByRef, System.__Canon))
37bef30c 2789a3a2 (MethodDesc 277cd99c +0x5ca System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)), calling (MethodDesc 20212e80 +0 System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef))
37bef3f0 2789a6f6 (MethodDesc 20212f7c +0x66 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback))
37bef404 2789f7da (MethodDesc 1fed1498 +0x1ba System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext)), calling (MethodDesc 20212f7c +0 System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext, System.AsyncCallback))
37bef448 278b1879 (MethodDesc 25156c40 +0x3a9 System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)), calling (MethodDesc 1fed1498 +0 System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest, System.Web.HttpContext))
37bef540 27895db8 (MethodDesc 25156c34 +0x20 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr, Int32)), calling (MethodDesc 25156c40 +0 System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32))
37bef568 0664e17a 0664e17a
37bef57c 6caaed5c clr!UM2MThunk_WrapperHelper+0x10
37bef588 6cae8ee1 clr!UM2MThunk_Wrapper+0x76, calling clr!UM2MThunk_WrapperHelper
37bef5ac 6cae8e92 clr!UM2MThunk_Wrapper+0x22, calling clr!_alloca_probe
37bef5e8 6cae8e12 clr!Thread::DoADCallBack+0xbc
37bef64c 6ca2ab9a iiscore!W3_RESPONSE::GetHeaderChanges+0x102, calling msvcrt!memset
37bef658 6ca2ad32 iiscore!W3_RESPONSE::GetHeaderChanges+0x29a, calling iisutil!BUFFER::~BUFFER
37bef6b4 6cae8d7a clr!UM2MDoADCallBack+0x92, calling clr!Thread::DoADCallBack
37bef70c 0664e1d3 0664e1d3, calling clr!UM2MDoADCallBack
37bef740 6da3ab23 webengine4!W3_MGD_HANDLER::ProcessNotification+0x62
37bef764 6da3ab83 webengine4!ProcessNotificationCallback+0x33, calling webengine4!W3_MGD_HANDLER::ProcessNotification
37bef778 6cb362e2 clr!UnManagedPerAppDomainTPCount::DispatchWorkItem+0x1d6
37bef7c4 6cb3750b clr!ThreadpoolMgr::ExecuteWorkRequest+0x4f
37bef7dc 6cb373f9 clr!ThreadpoolMgr::WorkerThreadStart+0x3d3, calling clr!ThreadpoolMgr::ExecuteWorkRequest
37bef80c 6cab8cab clr!EEHeapFree+0x3b, calling kernel32!HeapFreeStub
37bef844 6cbc8d41 clr!Thread::intermediateThreadProc+0x55
37bef850 5dda21fa msvcr100!_initptd+0x9b, calling msvcr100!_SEH_epilog4
37bef87c 5dda21fa msvcr100!_initptd+0x9b, calling msvcr100!_SEH_epilog4
37bef880 5dda20c3 msvcr100!_CrtEndBoot+0x163, calling msvcr100!_initptd
37bef888 5dda20de msvcr100!_CrtEndBoot+0x198, calling msvcr100!_SEH_epilog4
37bef8b4 5dda20de msvcr100!_CrtEndBoot+0x198, calling msvcr100!_SEH_epilog4
37bef8b8 5dda2116 msvcr100!_CRTDLL_INIT+0x1b, calling msvcr100!_CrtEndBoot+0xc
37bef8cc 772ca93c ntdll!RtlDeactivateActivationContextUnsafeFast+0x9c, calling ntdll!__security_check_cookie
37bef8e8 772c816c ntdll!LdrpCallInitRoutine+0x55, calling ntdll!LdrxCallInitRoutine
37bef8f8 772c818a ntdll!LdrpCallInitRoutine+0x73, calling ntdll!_SEH_epilog4
37bef924 7730162c ntdll!NtSetEvent+0xc
37bef928 772d7759 ntdll!LdrpDropLastInProgressCount+0x38, calling ntdll!NtSetEvent
37bef938 77315856 ntdll!_SEH_epilog4_GS+0xa, calling ntdll!__security_check_cookie
37bef93c 772c822b ntdll!LdrpInitializeThread+0x58, calling ntdll!_SEH_epilog4_GS
37bef994 772ecd6b ntdll!_LdrpInitialize+0x84, calling ntdll!LdrpInitializeThread
37bef998 772c8457 ntdll!LdrpInitializeThread+0x284, calling ntdll!RtlDeactivateActivationContextUnsafeFast
37befa20 773030fc ntdll!NtTestAlert+0xc
37befa24 772ecd8b ntdll!_LdrpInitialize+0xa4, calling ntdll!_SEH_epilog4
37befa70 7730199c ntdll!NtContinue+0xc
37befa74 772ecca9 ntdll!LdrInitializeThunk+0x29, calling ntdll!NtContinue
37befcd4 6cbc8d27 clr!Thread::intermediateThreadProc+0x3b, calling clr!_alloca_probe_16
37befce8 77170419 kernel32!BaseThreadInitThunk+0x19
37befcf8 772f72fd ntdll!__RtlUserThreadStart+0x2f
37befd54 772f72cd ntdll!_RtlUserThreadStart+0x1b, calling ntdll!__RtlUserThreadStart
I think that the proper way to have the application to be more performant would be to make all the calls asynchronous. But this would require a lot of time to change and test it, so I am searching a way to make it more performant right now , before taking the time to change it competely with using async/await. By reading the stacktrace I understand that all the threads are waiting on the UserManagement.GetPendingActionToDo to finish . Am I right ? (I am new to windbg and the post mortem debugging). How could I rewrite the Task.WhenAll(taskList).Wait() in order to be more performant ?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|