InvalidCastException running integration tests using InMemoryDatabase, EF Core, and AutoMapper

Question

We have an API application with multiple working endpoints and integrations tests, but one set of our tests is failing with the following error (Only the integration test fails. The application itself works properly):

System.InvalidCastException : Unable to cast object of type 'System.Linq.Expressions.NewExpression' to type 'System.Linq.Expressions.MethodCallExpression'.

This error is produced when we add the following item to our mapping profile (see more code/explanation below):

.ForMember(x => x.MaterialKeys, d => d.MapFrom(a => a.ItemMaterial.Select(x => x.MaterialKey)))

We are using:

• Net Core 3.1
• Microsoft.EntityFrameworkCore 3.1.4 (same error when we tried upgrading to 5.0.2)
• AutoMapper 9.0.0
• Microsoft.EntityFrameworkCore.InMemory 3.1.4 (same error when we tried upgrading to 5.0.2)
• Visual Studio Community 2019 16.8.3

Here is a failing test, which fails at the line Assert.IsNotNull(response.Result.Data) (all code snippets are simplified versions):

public void MyIntegrationTest()
{
      // Arrange 
      var options = GetContextOptions();

      using var context = new MyContext(options);
      Setup2ItemsContext(context, out Item myItem);

      var handler = new GetItemHandler(context, Mapper);
      var request = new GetItemRequest { PageNumber = 1, PageSize = 5 };

      // Act
      var response = handler.HandleAsync(request);

      //Assert 
      Assert.IsNotNull(response.Result.Data);
}

private static void Setup2ItemsContext(MyContext context, out Item myItem)
{
      myItem = new Item
      {
          ItemKey = 1,
          ItemMaterial = new List<ItemMaterial>()
          { 
              new ItemMaterial 
              {
                  MaterialKey = 1,
                  ItemKey = 1
              }
          }
      };
      context.Add(myItem);

      var myItem2 = new Item
      {
          ItemKey = 2,
          ItemMaterial = new List<ItemMaterial>()
          { 
              new ItemMaterial 
              {
                  MaterialKey = 2,
                  ItemKey = 2
              }
          }
      };
      context.Add(myItem2);

      context.SaveChanges();
}

Here is the mapping:

public class MappingProfiles : Profile
{
    public MappingProfiles()
    {
        CreateMap<Item, ItemDto>()
           .ForMember(x => x.ItemKey, d => d.MapFrom(a => a.ItemKey))
           .ForMember(x => x.MaterialKeys, d => d.MapFrom(a => a.ItemMaterial.Select(x => x.MaterialKey)));
    }
}

Here is the ItemDto class:

public class ItemDto
{
    public int ItemKey { get; set; }
    public List<int> MaterialKeys { get; set; }
}

Here are the Item and ItemMaterial context classes:

public partial class Item
{
    public Item()
    {
        ItemMaterial = new HashSet<ItemMaterial >();
    }
    public int? ItemKey { get; set; }
    public virtual ICollection<ItemMaterial> ItemMaterial { get; set; }
}

public partial class ItemMaterial
{
    public ItemMaterial()
    {
        // irrelevant code here
    }
    public int ItemKey { get; set; }
    public int MaterialKey { get; set; }
    public virtual Item ItemKeyNavigation { get; set; }
}

The code above gives the InvalidCastExceptionError when we run the integration tests. However, the tests run without errors if we remove this line from the mapping: .ForMember(x => x.MaterialKeys, d => d.MapFrom(a => a.ItemMaterial.Select(x => x.MaterialKey)))

The tests also run without error if we change the mapping to instead include the line: .ForMember(x => x.MaterialKeys, d => d.MapFrom(a => a.ItemMaterial.Select(x => x.MaterialKey).Count())), and change MaterialKeys in ItemDto to be type int instead of List<int>.

Here is the stack trace:

  Message: 
    System.AggregateException : One or more errors occurred. (Unable to cast object of type 'System.Linq.Expressions.NewExpression' to type 'System.Linq.Expressions.MethodCallExpression'.)
      ----> System.InvalidCastException : Unable to cast object of type 'System.Linq.Expressions.NewExpression' to type 'System.Linq.Expressions.MethodCallExpression'.
  Stack Trace: 
    Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
    Task`1.GetResultCore(Boolean waitCompletionNotification)
    Task`1.get_Result()
    GetItemHandlerTests.MyIntegrationTest() line 177
    --InvalidCastException
    InMemoryQueryExpression.AddSubqueryProjection(ShapedQueryExpression shapedQueryExpression, Expression& innerShaper)
    InMemoryProjectionBindingExpressionVisitor.Visit(Expression expression)
    InMemoryProjectionBindingExpressionVisitor.VisitMemberAssignment(MemberAssignment memberAssignment)
    ExpressionVisitor.VisitMemberBinding(MemberBinding node)
    InMemoryProjectionBindingExpressionVisitor.VisitMemberInit(MemberInitExpression memberInitExpression)
    MemberInitExpression.Accept(ExpressionVisitor visitor)
    ExpressionVisitor.Visit(Expression node)
    InMemoryProjectionBindingExpressionVisitor.Visit(Expression expression)
    InMemoryProjectionBindingExpressionVisitor.Translate(InMemoryQueryExpression queryExpression, Expression expression)
    InMemoryQueryableMethodTranslatingExpressionVisitor.TranslateSelect(ShapedQueryExpression source, LambdaExpression selector)
    QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    InMemoryQueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
    MethodCallExpression.Accept(ExpressionVisitor visitor)
    ExpressionVisitor.Visit(Expression node)
    QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
    Database.CompileQuery[TResult](Expression query, Boolean async)
    QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
    <>c__DisplayClass12_0`1.<ExecuteAsync>b__0()
    CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
    QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken)
    EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken)
    EntityQueryable`1.GetAsyncEnumerator(CancellationToken cancellationToken)
    ConfiguredCancelableAsyncEnumerable`1.GetAsyncEnumerator()
    EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)
    ICollectionExtensions.ToPagedResult[T,TOut](IQueryable`1 source, IMapper mapper, Int32 pageNumber, Int32 pageSize, CancellationToken token) line 37
    GetItemHandler.GetItemQuery(GetItemRequest request) line 155
    GetItemHandler.HandleAsync(GetItemRequest request) line 94

EDIT: here is some additional (simplified) code

Here is the GetItemRequest class:

public class GetItemRequest : IRequest<PagedQueryResult<ItemDto>>
{
    public int ItemKey { get; set; }
    public int PageSize { get; set; }
    public int PageNumber { get; set; }
}

Here is the GetItemHandler class:

public class GetItemHandler : IRequestHandler<GetItemRequest,PagedQueryResult<ItemDto>>
{
    private readonly MyContext _context;
    private readonly IMapper _mapper;

    public GetItemHandler(MyContext context,
        IMapper mapper)
    {
        _context = context;
        _mapper = mapper;
    }

    public async Task<Response<PagedQueryResult<ItemDto>>> HandleAsync(GetItemRequest request)
    {
        var response = await GetItemQuery(request);
        return response.AsResponse();
    }

    private async Task<PagedQueryResult<ItemDto>> GetItemQuery(GetItemRequest request)
    {
        var items = _context.Set<Item>().AsQueryable();
        if (request.ItemKey != default)
            items = items.Where(x => x.ItemKey == request.ItemKey);

        items = items.OrderBy(x => x.ItemKey).AsQueryable();

        var response = await items.ToPagedResult<Item,ItemDto>(
            _mapper,
            request.PageNumber,
            request.PageSize);

        return response;
    }
}

Here is ToPagedResult():

public static async Task<PagedQueryResult<TOut>> ToPagedResult<T, TOut>(
    this IQueryable<T> source,
    IMapper mapper,
    int pageNumber,
    int pageSize,
    CancellationToken token = default)
{
    var totalItemCount = await source.CountAsync(token);
    var totalPageCount = (totalItemCount + pageSize - 1) / pageSize;
    var startIndex = (pageNumber - 1) * pageSize;

    var items = await source
        .Skip(startIndex)
        .Take(pageSize)
        .ProjectTo<TOut>(mapper.ConfigurationProvider)
        .ToListAsync(token);

    return new PagedQueryResult<TOut>
    {
        Items = items,
        PageCount = totalPageCount,
        PageNumber = pageNumber,
        PageSize = pageSize,
        TotalItemCount = totalItemCount
    };
}

EDIT2:

If you received the same exception System.InvalidCastException : Unable to cast object of type 'System.Linq.Expressions.NewExpression' to type 'System.Linq.Expressions.MethodCallExpression' but you are using .Count() after Select in your mapping, then according to @AntiqTech:

Please check this github discussion : https://github.com/dotnet/efcore/issues/17620 Exception is exactly what you get. If you're using Count() in a subquery , jgbpercy suggest to wrap it with Convert.ToInt32.

Answer 1

This is due to a bug in EF Core that should be fixed by version 6.0.0 (see thread here: https://github.com/dotnet/efcore/issues/24047)

Answer 2

I ran into this same NewExpression to MethodCallExpression cast exception while projecting a chain of .Any into a bool, something like this:

.Select(q => new AlreadyDefinedClass
{
   HasAnyThings = q.Parents.Any(p => p.Children.Any(c => c.Answers.Any(a => a.IsThing)))
}

This worked at runtime, but failed in unit tests using the In-Memory provider.

I found github issue 24047, which then linked to 24092, which then says:

Problem was that EF wasn't correctly compensating for nullability change of a non-nullable property projected from a left join (after expand). Workaround is to make that property nullable.

Changing AlreadyDefinedClass.HasAnyThings to bool? did make the unit tests pass, but caused runtime problems downstream. So instead, I moved the nested .Any statements up into an anonymous projection and casted it to nullable bool, and then coalesced it in the concrete projection, like so:

.Select(q => new
{
   HasAnyThings = (bool?)q.Parents.Any(p => p.Children.Any(c => c.Answers.Any(a => a.IsThing)))
}
.Select(q => new AlreadyDefinedClass
{
   HasAnyThings = q.HasAnyThings ?? false
}

This has no effect in the generated SQL other than the nullability compensation mentioned:

Hope that helps anyone coming across this work around it.