Azure blob storage streaming performance issue

Question

My application till this day was working with local zip files, meaning I was using a direct return new FileStream() in the application and the local zip file that was located on the SDD/Network drive path (zip files can be hundreds of GB).

I configured the application to work with Azure Blob Storage, meaning each FileStream that was returned in now return as the Azure Blob SDK method:

GetBlobStreamAsync(ContainerName, BlobName).ConfigureAwait(false).GetAwaiter().GetResult()

I uploaded some zip files to a container in the blob storage and set the connection string in the application to work with that storage account.

The application was deployed and running on a virtual windows machine located in the same region of the Azure Storage Blob.

Note: This is a private cloud network.

When the app is streaming the zip file on Azure blob storage it seems that the performance has decreased by at least 8-9 times (problematic with hundreds of GB).

Speed comparison is between local C: drive on the same windows virtual machine that the application is running on an Azure Storage account which is located in the same region.

Note: NW Bandwidth - is 50 GB on the VM on azure

Solutions that I tried:

1. Azure blob Premium Performance storage - Didn’t improve performance
2. .Net Core - advantage of performance enhancements (we work with .Net framework so this is irrelevant).
3. Network File System (NFS) 3.0 performance considerations in Azure Blob storage - (Does not work with private cloud).
4. Hot, Cool, and Archive access tiers for blob data - The default is Hot so we already tried this scenario with no improvements.

Solutions I want to try:

1. Azure Files Share Storage as a cache solution
2. .Net Framework configuration - lists several quick configuration settings that you can use to make significant performance improvements

Question:

Does anyone have any suggestions on how can I optimize the streaming in front of the Azure Storage Blob?

Answer 1

Azure Files (share) or Storage Blob services are likely not the right services to be utilized for this scenario. There are two possible paths:

1. Break a single file into multiple files and leverage Storage Blob service that handles throughput better than Azure Files. Azure Files performs better with small(er) files which are typical to user documents (PDFs, Word, Excel, etc.)
2. Switch over to a more dedicated service that is designed specifically for large-size data transfer if breaking up a single file into multiple blobs is not an option.

The recommendation for each option will highly depend on the implementation details, requirements and constraints of the system.