'Python Setuptools including data with a specific folder structure

I've done a bit of searching and can't find any examples that match my scenario.

I'm trying to create a x-lang module to return some data that we use internally.

My git repo would contain both the powershell and python version of the module, and each module would reference the static data.

I'm struggling to package it in a way that doesn't have a wierd folder structure and works from a python packaging point of view:

the folder structure looks like this (anonymised)

<Root of Repo>
│   .gitignore
│   MANIFEST.in
│   pyproject.toml
│   README.md
│   requirements.txt
│   setup.py
│
├───data
│       1.json
│       2.json
│       3.json
│       4.json
│
├───src
│   ├───powershell
│   │   │   readme.md
│   │   │   test.ps1
│   │   │   xlangTest.psd1
│   │   │
│   │   ├───classes
│   │   ├───private
│   │   │       get-something1.ps1
│   │   │
│   │   └───public
│   │           Get-DataExample.ps1
│   │
│   └───python
│       │   1.py
│       │   2.py
│       │   3.py
│       │   4.py
│       │   5.py
│       │   6.py
│       │   7.py
│       │   8.py
│       │   9.py
│       │   __init__.py
│       │
│       └───ff.ff.ff.egg-info
│               dependency_links.txt
│               PKG-INFO
│               SOURCES.txt
│               top_level.txt
│
├───tests
│   ├───powershell
│   │       Get-DataExample.tests.ps1
│   │       get-something1.tests.ps1
│   │
│   └───python

and my setup.py looks like this:

import setuptools

with open("README.md", "r", encoding="utf-8") as fh:
    long_description = fh.read()

setuptools.setup(
    name="ff.ff.ff",
    version="0.0.1",
    author="ffff",
    author_email="[email protected]",
    description="An xlang module test",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url="https://ffff/ffff",
    project_urls={
        "Bug Tracker": "https://ffff/ffff",
    },
    classifiers=[
        "Programming Language :: Python :: 3",
        "Operating System :: OS Independent",
    ],
    package_dir={"": "src"},
    # data_files={
    #     "data":['data/*.json'],
    # },
    # include_package_data = True,
    # package_data = {
    #     "data" : ["../data/*.json"]
    # },
    packages=setuptools.find_packages(where="src"),
    # install_requires=[
    #     'readContent',
    #     'json',
    # ],
    python_requires=">=3.6",
)

manifest.IN contains:

graft data

when I package it I end up with the following in the tar file within the tar.gz

C:.
│   ff.ff.ff-0.0.1.tar
│
└───ff.ff.ff-0.0.1
    │   @PaxHeader
    │
    └───ff.ff.ff-0.0.1
        │   MANIFEST.in
        │   PKG-INFO
        │   pyproject.toml
        │   README.md
        │   setup.cfg
        │   setup.py
        │
        ├───data
        │       1.json
        │       2.json
        │       3.json
        │       4.json
        │
        └───src
            ├───python
            │       1.py
            │       2.py
            │       3.py
            │       4.py
            │       5.py
            │       6.py
            │       7.py
            │       8.py
            │       9.py
            │       __init__.py
            │
            └───ff.ff.ff.egg-info
                    dependency_links.txt
                    PKG-INFO
                    SOURCES.txt
                    top_level.txt

I guess what I'm looking for is a sanity check, is this ok, or can I some how flatten the src/python in the distribution or even rename to the module name?

and I suppose the key point here is that I want the data to be common across the two modules. and ultimatley I want to do the same for test cases next.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source