Given a Python *.py file what's the canonical way to determine the dotted import path for that file, programmatically?

Question

Suppose you get a pathlib.Path that points to a *.py file.

And also suppose that this is a resource you could import in another Python file, given the appropriate import path, because your sys.path allows for it.

How do you determine the dotted import path to use in import, from just the python file?

Unlike most of the import-related questions, this is NOT about the path from one Python file to another within a directory hierarchy, it is really more about the import path you could specify in the REPL from anywhere to import that module and that's affected by sys.path contents.

Example:

$test_366_importpath$ tree -I __pycache__
.
└── sub
    └── somemodule.py

somemodule.py

"some module"

class Foo:
    "Foo class"

If I start python at that location, because sys.path gets the current directory, this works:

from sub.somemodule import Foo

sub.somemodule is what I am interested in.

However, if the sys.path gets altered, then I can use a different import path.

import sys
sys.path.insert(0, "/Users/me/explore/test_366_importpath/sub")
from somemodule import Foo

(note: I wouldn't be doing this "for real", neither the sys.path.insert, nor varying the dotted path I'd use, see @CryptoFool's comment. This is just a convient way to show sys.path impact)

Question:

How do I determine, programmatically, that import sub.somemodule needs to be used as the dotted path? Or import somemodule given different sys.path conditions?

Raising an ImportError or ValueError or some other exceptions if the *.py file is not importable is perfectly OK.

I'm writing a helper using

pa_script = pathlib.Path("somemodule.py").absolute().resolve() and then looking at sys.path. Once I find that a given sys.path entry is the parent for the pa_script, I can use pa_script.relative_to(parent).

From there it's trivial to get the import path by removing the .py extension and replacing os.sep with ..

Then I can feed that dotted path to importlib. Or paste into my code editor.

It's a bit tricky but not particularly hard. Makes me wonder however if there isn't a builtin or canonical way however.

I can post my code, but really if there is a canonical way to do it, it would just give the wrong impression that's it's necessary to do these complicated steps.

Answer 1

Well here goes then, in case anyone needs something similar

(for Python 3.10+, but removing typehints should make it work down to much earlier 3.x versions)

from pathlib import Path
import sys
import os

def get_dotted_path(path_to_py: str | Path, paths: list[str] | None = None) -> str:
    """
    return a dotted-path import string from a Python filename

    if given, `paths` will be examined as if it was `sys.path` else
    `sys.path` is used to determine import points (this is to compute paths
    assuming a different sys.path context than the current one)
    )

    example:
    .../lib/python3.10/collections/__init__.py => "collections"
    .../lib/python3.10/collections/abc.py => "collections.abc"

    raises ImportError if the Python script is not in sys.path or paths
    """
    parent = None
    pa_target = Path(path_to_py)
    paths = paths or sys.path

    # get the full file path AND resolve if it's a symlink
    pa_script = pa_target.absolute().resolve().absolute()

    # consider pkg/subpk/__init__.py as pkg/subpk
    if pa_script.name == "__init__.py":
        pa_script = pa_script.parent

    for path in paths:
        pa_path = Path(path)
        if pa_path in pa_script.parents:
            parent = pa_path
            break
    else:
        newline = "\n"
        raise ImportError(
            f"{pa_script} nowhere in sys.path: {newline.join([''] + paths)}"
        )

    pa_relative = pa_script.relative_to(parent)
    res = str(pa_relative).removesuffix(".py").replace(os.sep, ".")
    return res