Python Asyncio file.write after request.get(file) not working

Question

I'm using Asyncio and aiohttp to asynchronously get files from an endpoint. My status codes for the request are successful but when I try to write the files everything is always empty for some reason.

This is what my code looks like right now:

async def download_link(url:str,my_session:ClientSession, filename:Path):
    async with my_session.get(url, allow_redirects=True) as response:
        with filename.open(mode='wb') as f: #Line 3
            await f.write(response.content)

async def download_all(urls:list, filenames:list):
    my_conn = aiohttp.TCPConnector(limit=10)
    async with aiohttp.ClientSession(connector=my_conn) as session:
        tasks = []
        for item in zip(urls,file_names):
            task = asyncio.ensure_future(download_link(url=item[0],my_session=session, filename=item[1]))
            tasks.append(task)
        await asyncio.gather(*tasks,return_exceptions=True)

I've also tried to put async in front of the with on line 3, inside the download_link function. And I've also tried making the code that opens the file and writes into it a separate async function a such:

async def store_response(response, filename:Path):
    async with filename.open(model='wb') as f:
        f.write(response.content)

I know the files I'm fetching from do have data, when I use multi-threading I'm able to get data back. Anyone know why this is happening?

Answer 1

I have used this code to download files asynchronously with no problem and good speed.

import asyncio
import aiohttp
import aiofile

async def download_file(url: str):
    filename = url.split('?')[0].split('/')[-1]

    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            if not resp.ok:
                print(f"Invalid status code: {resp.status}")
            else:
                try:
                    async with aiofile.async_open(filename, "wb+") as afp:
                        async for chunk in resp.content.iter_chunked(1024 * 512):   # 500 KB
                            await afp.write(chunk)
                except asyncio.TimeoutError:
                    print(f"A timeout ocurred while downloading '{filename}'")

asyncio.run(download_file("https://www.python.org/static/img/python-logo.png"))