'Using requests_html with arender() function gives RuntimeError: 'Event loop is closed'

I'm trying to scrape Amazon asynchronously with requests-html package. See the following code:

import datetime
from requests_html import AsyncHTMLSession
import asyncio

asins = ['B09GB6GSMM', 'B09GB7BCQH', 'B075D23KXX']

async def get_price_and_title_for_asin(s, asin):

    r = await s.get(f'https://www.amazon.de/dp/{asin}')

    await r.html.arender()

    try:
        price = r.html.find("#corePrice_feature_div > div > span > span")[0].text
    except:
        price = r.html.find(".priceToPay > span")[0].text

    title = r.html.find('#productTitle')[0].text
    date = datetime.datetime.now().replace(microsecond=0)
    print(asin, date, price)
    return [title, price]


async def main(asins):
    s = AsyncHTMLSession()
    coroutines = [get_price_and_title_for_asin(s, asin) for asin in asins]
    return await asyncio.gather(*coroutines)


asyncio.run(main(asins))

and I get the following Error:

B09GB6GSMM 2022-04-05 17:26:28 109,90€
B075D23KXX 2022-04-05 17:26:31 339,00€
B09GB7BCQH 2022-04-05 17:26:31 239,00€
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Users\Christian\anaconda3\lib\site-packages\pyppeteer\launcher.py", line 153, in _close_process
    self._loop.run_until_complete(self.killChrome())
  File "C:\Users\Christian\anaconda3\lib\asyncio\base_events.py", line 591, in run_until_complete
    self._check_closed()
  File "C:\Users\Christian\anaconda3\lib\asyncio\base_events.py", line 508, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
sys:1: RuntimeWarning: coroutine 'Launcher.killChrome' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Users\Christian\anaconda3\lib\site-packages\pyppeteer\launcher.py", line 153, in _close_process
    self._loop.run_until_complete(self.killChrome())
  File "C:\Users\Christian\anaconda3\lib\asyncio\base_events.py", line 591, in run_until_complete
    self._check_closed()
  File "C:\Users\Christian\anaconda3\lib\asyncio\base_events.py", line 508, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Users\Christian\anaconda3\lib\site-packages\pyppeteer\launcher.py", line 153, in _close_process
    self._loop.run_until_complete(self.killChrome())
  File "C:\Users\Christian\anaconda3\lib\asyncio\base_events.py", line 591, in run_until_complete
    self._check_closed()
  File "C:\Users\Christian\anaconda3\lib\asyncio\base_events.py", line 508, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

So the program is printing out the 3 lines as required, but then throws an Error. Does anyone know how to solve this? I checked several other posts on similar topics but nothing worked for me.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source