ModuleNotFoundError: No module named '...'

Question

I'm trying to install a few packages (beautifulsoup4 and requests) for a project.

I installed these packages by running these commands in the terminal on macOS:

pip3 install beautifulsoup4
pip3 install requests

both did install successfully.

Now if I open my project in PyCharm, I can import the modules by using:

from bs4 import BeautifulSoup
import requests

Both packages are imported successfully and can be used in my code. To be sure everything was installed correctly, I looked at the venv/lib/Python3.10 folder. beautifulsoup4-4.10.0.dist-info, bs4, requests and requests-2.27.1.dist-info are present.

However, when I run the CGI script (Python), I get the following error in the terminal:

    from bs4 import BeautifulSoup
ModuleNotFoundError: No module named 'bs4'

Even when I open a new terminal window and run the following commands:

python3
>>> from bs4 import BeautifulSoup

it runs fine without any errors. The same issue happens for the requests package.

I also ran pip3 install bs4 but that also didn't fix anything.

Answer 1

I found what caused the issue. bs4 and requests were both installed correctly but in my CGI script, I used the wrong Python path. I had to change #!/usr/bin/python3 to the path that which python3 gave me.

Answer 2

I would recommend to always install packages using python3 -m pip install <name_of_the_package>. In this way you can be sure that you are using the correct pip.

To make sure that the python you a using really has the packages installed, try python3 -m pip list.

If all is good, check if the python interpreter the pycharm is using in your project is the right one.

It sounds to me that in pycharm you're using a virtual environment, but when you are using python3 at the terminal, you a using the main installation.