'How to re-export everything from a star (*) import in a python __init__.py file

When I write a python library, I often end up with something like this:

/my_lib
   __init__.py
   module_a.py
   module_b.py
   ...

Now lets suppose that module_a and module_b are very big, and define a lot public and non-public methods. E.g.:

import ...
from ... import ...
... etc

__all__ = [
    "PublicClass",
    "public_fn",
    "PUBLIC_STATIC_VAR",
    "OtherPublicClass",
    ...
]

class _PrivateClass:
    ...

I put a lot of effort in curating a well thought out list of symbols to export in __all__. Now in my __init__.py I would like to do this (which does not work of course):

from my_lib.module_a import * as module_a
from my_lib.module_b import * as module_b
from third_party_lib import * as third_party
from other_lib import OtherA, OtherB

__all__ = [
    *module_a,
    *module_b,
    *third_party,
    "OtherA",
    "OtherB",
]

In essence, what I want is that, if someone imports my_lib, they can access all public symbols from module_a, module_b and third_party from the same namespace. I.e.:

import my_lib

obj1 = my_lib.pub_fn_from_module_a()
obj2 = my_lib.PubClassFromModuleB()
obj3 = my_lib.CONSTANT_FROM_THIRD_PARTY_LIB
obj4 = my_lib.OtherA()
obj5 = my_lib.OtherB()

Poor solutions:

1. Implicit __all__

I know I can do:

from my_lib.module_a import *
from my_lib.module_b import *
from third_party_lib import *
from other_lib import OtherA, OtherB

# __all__ is not defined in __init__.py

This will implicitly define __all__ and achieves the desired behavior, but it is not possible for static code analysis tools to distinguish this from dead code. For example pre-commit hooks or code formatting tools may delete these imports.

2. Re-type everything

I can also do:

from my_lib.module_a import (
    # <type all public symbols from module_a here>
)
from my_lib.module_b import (
    # <type all public symbols from module_b here>
)
from third_party_lib import (
    # <type all public symbols from third_party_lib here>
)
from other_lib import OtherA, OtherB

__all__ = [
    # <copy-paste the body of module_a.__all__ here>
    # <copy-paste the body of module_b.__all__ here>
    # <type all public symbols from third_party_lib here>
    "OtherA",
    "OtherB",
]

But now I basically have to retype the name of each symbol in module_a three times, once in module_a.__all__, once in the import of __init__.py and once in the __all__ definition in __init__.py. The same goes for module_b, I also need make sure al 3 remain in sync during development.

3. re-use __all__

This kind of works, but looks really awkward:

from my_lib.module_a import * as module_a
from my_lib.module_b import * as module_b
from my_lib import module_a, module_b
from third_party_lib import * as third_party
from other_lib import OtherA, OtherB

# only purpose of `_keep` is to fake the usage of a few symbols 
# such that the star imports are not flagged as dead code.
# It does nothing at runtime.
def _keep(x): pass

_keep(random_symbol_from_module_a)
_keep(random_symbol_from_module_b)
_keep(random_symbol_form_third_party)

__all__ = [
    *module_a.__all__,
    *module_b.__all__,
    *third_party_lib.__all__,
    "OtherA",
    "OtherB",
]

The star imports (from ... import *) are required because e.g. pycharm or other static code analysis tools can't infer what the *module_a.__all__ statement does. So without the star imports, static code analysis tools will report missing imports everywhere.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source