How to find a Python package's dependencies

Question

How can you programmatically get a Python package's list of dependencies?

The standard setup.py has these documented, but I can't find an easy way to access it from either Python or the command line.

Ideally, I'm looking for something like:

$ pip install somepackage --only-list-deps
kombu>=3.0.8
billiard>=3.3.0.13
boto>=2.26

or:

>>> import package_deps
>>> package = package_deps.find('somepackage')
>>> print package.dependencies
['kombu>=3.0.8', 'billiard>=3.3.0.13', 'boto>=2.26']

Note, I'm not talking about importing a package and finding all referenced modules. While this might find most of the dependent packages, it wouldn't be able to find the minimum version number required. That's only stored in the setup.py.

Answer 1

In addition to the pip show [package name] command, there is pipdeptree.

Just do

$ pip install pipdeptree

then run

$ pipdeptree

and it will show you your dependencies in a tree form, e.g.,

flake8==2.5.0
  - mccabe [required: >=0.2.1,<0.4, installed: 0.3.1]
  - pep8 [required: !=1.6.0,>=1.5.7,!=1.6.1,!=1.6.2, installed: 1.5.7]
  - pyflakes [required: >=0.8.1,<1.1, installed: 1.0.0]
ipdb==0.8
  - ipython [required: >=0.10, installed: 1.1.0]

The project is located at https://github.com/naiquevin/pipdeptree, where you will also find usage information.

Answer 2

Try to use show command in pip, for example:

$ pip show tornado
---
Name: tornado
Version: 4.1
Location: *****
Requires: certifi, backports.ssl-match-hostname

Update (retrieve deps with specified version):

from pip._vendor import pkg_resources


_package_name = 'somepackage'
_package = pkg_resources.working_set.by_key[_package_name]

print([str(r) for r in _package.requires()])  # retrieve deps from setup.py

---

Output: ['kombu>=3.0.8', 
         'billiard>=3.3.0.13', 
         'boto>=2.26']

Answer 3

Brief summary about the methods that are found and tested on the Windows machine:

1. Parse the json file of PyPI: https://pypi.org/pypi/<package>/<version>/json (#7)

2. Check /site-packages/<package-version>.dist-info/METADATA (#13, pre-installation required)

3. pip install --no-install <package>: deprecated (#11)

4. pip install --download <package>: deprecated (#12)

5. pip show <package> (#1, #2, pre-installation required)

6. Write a script with import pip and pip._vendor.pkg_resources: deprecated (#6)

7. Write a script with import pkg_resources of setuptools package (#4, pre-installation required)

8. Check https://libraries.io/ (#5)

9. Use pipdeptree package (#3, #8, pre-installation required)

10. Use Johnnydep package (#10): Test hangs in many cases.

11. conda info [package_name]: deprecated (#9)

12. conda search [package_name] --info

    • Additional packages can be listed: vc2015_runtime, python_abi, libflang, and etc.
    • Note that this method can list different packages depending on the build and channel.
      e.g. conda search "Django==3.2" --info -c conda-forge

django 3.2 pyhd3eb1b0_0
-----------------------
file name   : django-3.2-pyhd3eb1b0_0.conda
...
timestamp   : 2021-04-06 20:19:41 UTC
dependencies:
  - asgiref
  - psycopg2
  - python
  - pytz
  - sqlparse

django 3.2 pyhd8ed1ab_0
-----------------------
file name   : django-3.2-pyhd8ed1ab_0.tar.bz2
...
timestamp   : 2021-04-07 21:15:25 UTC
dependencies:
  - asgiref >=3.3.2,<4
  - python >=3.6
  - pytz
  - sqlparse >=0.2.2

One more thing to note is that each method can provide a different result.

For example, requests/setup.py identifies chardet, idna, urllib3, and certifi are required. Moreover, extra packages pyOpenSSL, cryptography, socks, PySocks, win-inet-pton can be needed.

• Methods 1 and 2 agree with it.
• Method 8 just list all of them (pressing the explore dependencies button just hangs).
• Method 12 just list chardet, idna, urllib3, and certifi.
• Methods 5 and 7 does not list any dependency if requests is installed using pip in the Linux docker.
• Methods 5, 7, 9, and 10 list chardet, idna, urllib3, and certifi if requests is installed in the conda environment of the Windows machine.

Answer 4

Quite a few answers here show pip being imported for use in programs. The documentation for pip strongly advises against this usage of pip.

Instead of accessing pkg_resources via the pip import, you can actually just import pkg_resources directly and use the same logic (which is actually one of the suggested solutions in the pip docs linked for anyone wanting to see package meta information programmatically) .

import pkg_resources

_package_name = 'yourpackagename'
  
def get_dependencies_with_semver_string():
    package = pkg_resources.working_set.by_key[_package_name]
    return [str(r) for r in package.requires()]

If you're having some trouble finding out exactly what your package name is, the WorkingSet instance returned by pkg_resources.working_set implements __iter__ so you can print all of them and hopefully spot yours in there :)

i.e.

import pkg_resources

def print_all_in_working_set():
    ws = pkg_resources.working_set
    for package_metadata in ws:
        print(package_metadata)

This works with both python 2 and 3 (though you'll need to adjust the print statements for python2).

Answer 5

Use https://libraries.io/. It is a good place to explore dependencies before installing using pip.

Eg. Type google-cloud-storage and search, then you can find the page for the library (https://libraries.io/rubygems/google-cloud-storage). Select the version for which you want to explore the dependencies from the 'Releases' (default is the latest), Under 'Dependencies' you can find the dependency list and their supported versions.

Answer 6

The standard library offers importlib.metadata.requires() since version 3.8 and onwards:

In [1]: from importlib.metadata import requires

In [2]: requires('pytype')
Out[2]:
['attrs (>=21.2.0)',
 'importlab (>=0.6.1)',
 'libcst',
 'ninja (>=1.10.0.post2)',
 'pyyaml (>=3.11)',
 'six',
 'tabulate',
 'toml',
 'typed-ast (>=1.4.3)',
 'dataclasses ; python_version < "3.7"']

For older Python versions, there's the importlib-metadata available. Related section in Python docs: Distribution requirements.

If you need to parse the strings returned by requires(), I strongly suggest to use packaging library since its packaging.requirements module is a reference implementation of PEP 508. Example with a complex dependency specification string:

In [3]: from packaging.requirements import Requirement

In [4]: spec = 'requests [security,test] (>=2.21.0) ; implementation_name == "cpython"'

In [5]: r = Requirement(spec)

In [6]: r.name
Out[6]: 'requests'

In [7]: r.specifier
Out[7]: <SpecifierSet('>=2.21.0')>

In [8]: r.extras
Out[8]: {'security', 'test'}

In [9]: r.marker
Out[9]: <Marker('implementation_name == "cpython"')>

Answer 7

(THIS IS A LEGACY ANSWER AND SHOULD BE AVOIDED FOR MODERN PIP VERSIONS AND LEFT HERE FOR REFERENCE TO OLDER PIP VERSIONS ) Alex's answer is good (+1). In python:

pip._vendor.pkg_resources.working_set.by_key['twisted'].requires()

should return something like

[Requirement.parse('zope.interface>=3.6.0')]

where twisted is the name of the package, which you can find in the dictionary :

pip._vendor.pkg_resources.WorkingSet().entry_keys

to list them all:

dict = pip._vendor.pkg_resources.WorkingSet().entry_keys
for key in dict:
    for name in dict[key]:
        req =pip._vendor.pkg_resources.working_set.by_key[name].requires()
        print('pkg {} from {} requires {}'.format(name,
                                                  key,
                                                  req))

should give you lists like this:

pkg pyobjc-framework-syncservices from /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC requires [Requirement.parse('pyobjc-core>=2.5.1'), Requirement.parse('pyobjc-framework-Cocoa>=2.5.1'), Requirement.parse('pyobjc-framework-CoreData>=2.5.1')]

Answer 8

There is the nice tool johnnydep. The benefit of it is that the package to inspect does not have to be installed.

pip install johnnydep

Use it like:

johnnydep deepspeech-tflite

Answer 9

Try this according to this article in python:

import pip 
installed_packages = pip.get_installed_distributions()
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
     for i in installed_packages]) 
print(installed_packages_list)

It will show like:

['behave==1.2.4', 'enum34==1.0', 'flask==0.10.1', 'itsdangerous==0.24', 
 'jinja2==2.7.2', 'jsonschema==2.3.0', 'markupsafe==0.23', 'nose==1.3.3', 
 'parse-type==0.3.4', 'parse==1.6.4', 'prettytable==0.7.2', 'requests==2.3.0',
 'six==1.6.1', 'vioozer-metadata==0.1', 'vioozer-users-server==0.1', 
 'werkzeug==0.9.4']