Using Prefetch with Batch slows down message receipt in Azure Service Bus

Question

I've set the PrefetchCount and am receiving a batch of messages - like this:

receiver.PrefetchCount = 1000;
var msgs = await receiver.ReceiveAsync(1000);

This is in a loop, because ReceiveAsync doesn't always retrieve the total number of messages that I ask for (which I understand is a know issue / by design). However, doing this actually takes longer (or at least a similar time) - than just:

var msgs = await receiver.ReceiveAsync(1000);

My understanding here was that the PrefetchCount would force a retrieval of 1000 messages in the background, and the batch receipt would either bring down the correct amount, or it would be faster, as the messages are already cached.

Why would using Prefetch not speed up the process?