Running separate infinite background thread in a python webapp (fastapi/flask/django)

Question

How can i launch an application which does below 2 things:

1. Expose rest endpoint via FastAPI.
2. Run a seperate thread infintely (rabbitmq consumer - pika) waiting for request.

Below is the code through which i am launching the fastAPI server. But when i try to run a thread, before execution of below line, it says coroutine was never awaited.

How can both be run parallely?

Answer 1

Maybe this is not the answer you are looking for. There is a library called celery that makes multithreading easy to manage in python.

Check it out: https://docs.celeryproject.org/en/stable/getting-started/introduction.html

Answer 2

import asyncio
from concurrent.futures import ThreadPoolExecutor
from fastapi import FastAPI
import uvicorn

app = FastAPI()


def run(corofn, *args):
    loop = asyncio.new_event_loop()
    try:
        coro = corofn(*args)
        asyncio.set_event_loop(loop)
        return loop.run_until_complete(coro)
    finally:
        loop.close()


async def sleep_forever():
    await asyncio.sleep(1000)


#
async def main():
    loop = asyncio.get_event_loop()
    executor = ThreadPoolExecutor(max_workers=2)
    futures = [loop.run_in_executor(executor, run, sleep_forever),
               loop.run_in_executor(executor, run, uvicorn.run, app)]
    print(await asyncio.gather(*futures))


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

Note: This may hinder your FastAPI performance. Better approach would be to use a Celery Task