Handling asyncio CancelledError simultaneous

Question

Imagine the following python code to cancel a subtask.
It is not a real program, but should demonstrate the structure.

import asyncio

async def subtask():
    sleep = 99999
    while True:
        try:
            print("Very long task")
            await asyncio.sleep(sleep)
        except asyncio.CancelledError:
            print("Incorrectly caught error")
            sleep = 2  # make it visible that task is still running
            pass

async def handling1():
    for i in range(3):
        print("Start subtask")
        task = asyncio.create_task(subtask())
        await asyncio.sleep(1)
        task.cancel()
        print("Subtask canceled")

async def handling2():
    for i in range(3):
        print("Start subtask")
        task = asyncio.create_task(subtask())
        await asyncio.sleep(1)
        task.cancel()
        try:
            await task
        except asyncio.CancelledError:
            pass
        print("Subtask canceled")


asyncio.run(handling1())

Replace asyncio.run(handling1()) with asyncio.run(handling2()) to see the other handling.

handling1:

• When subtask it catching the CancelledError it will run forever and cause a memory leak.
  • It is not obvious in a larger project which task caused this.

handling2:

• It will be obvious that subtask was not canceled through the await task
• But it can happen that at the same time of calling await task, the task handling2 itself was canceled too.
  So handling2 will itself catch the CancelledError.
  • It is really rare, but it happend in my larger project. (hard to debug)

So is there another way to handle canceling tasks and wait for their end?
(subtask is not a real example, but should demonstrate a wrongly caught CancelledError)

Answer 1

The bug is of course in subtask, it should be cancelled when you call .cancel() on it.

Since you already know that, said it's incorrectly caught, and know that handling1 is the correct solution but does not accommodate for the bug, here is a fix to handling2:

task = asyncio.create_task(subtask())
await asyncio.sleep(1)
task.cancel()
try:
    # Shield will make sure 2nd cancellation from wait_for will not block.
    await asyncio.wait_for(asyncio.shield(task), 1)
except asyncio.TimeoutError:
    if task.cancelled():  # Slight chance of happening
        return # Task was cancelled correctly after exactly 1 second.
    print("Task was not cancelled after 1 second! bug bug bug")
except asyncio.CancelledError:
    if task.cancelled():
        return  # Task was cancelled correctly
    print("handling 2 was cancelled!")
    raise