How to create python namespace packages in Python 3?

Namespace packages are a special type of package introduced in Python 3.3 that allows you to split package contents across multiple directories. If you don't include at least an empty __init__.py file in your package, then your package becomes a namespace package.

In Python, a namespace package allows you to spread Python code among several projects. This is useful when you want to release related libraries as separate downloads.

Currently, there are three methods for developing namespace packages:

• Native namespace packages (PEP 420)
• pkgutil-style namespace packages
• pkg_resources-style namespace packages

Using Native Namespace Packages

Native namespace packages were introduced in Python 3.3. Their key characteristic is the absence of __init__.py in the namespace directory. Here's the directory structure −

project1/
    setup.py
    mynamespace/
        # No __init__.py here!
        subpackage_a/
            __init__.py
            module.py

project2/
    setup.py
    mynamespace/
        # No __init__.py here either!
        subpackage_b/
            __init__.py
            another_module.py

The code below shows how to configure your setup.py file for both projects −

Project 1 Setup

from setuptools import setup, find_namespace_packages

setup(
    name="project1",
    version="0.1",
    packages=find_namespace_packages(include=["mynamespace.*"]),
    # Or use packages=["mynamespace.subpackage_a"]
)

Project 2 Setup

from setuptools import setup, find_namespace_packages

setup(
    name="project2",
    version="0.1",
    packages=find_namespace_packages(include=["mynamespace.*"]),
    # Or use packages=["mynamespace.subpackage_b"]
)

After installing both packages, you can import either of the subpackages using the following code −

# Both imports work, even though they come from different distributions
from mynamespace.subpackage_a import module
from mynamespace.subpackage_b import another_module

print("Successfully imported from both namespace subpackages!")

Successfully imported from both namespace subpackages!

Using pkgutil-style Namespace Packages

The pkgutil-style is compatible with Python 2 and 3, making it suitable for projects that need to support both versions. Here's the directory structure −

project1/
    setup.py
    mynamespace/
        __init__.py  # Contains pkgutil code
        subpackage_a/
            __init__.py
            module.py

project2/
    setup.py
    mynamespace/
        __init__.py  # Contains pkgutil code
        subpackage_b/
            __init__.py
            another_module.py

For this approach, the __init__.py in the namespace directory should contain the following code as it modifies the package's __path__ variable to include all directories in sys.path that match the namespace package's name −

# mynamespace/__init__.py
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

Setup Configuration

Configure your setup.py file for both projects −

from setuptools import setup, find_packages

setup(
    name="project1",
    version="0.1",
    packages=find_packages(),
    # Or explicitly use packages=["mynamespace", "mynamespace.subpackage_a"]
)

Using pkg_resources-style Namespace Packages

This is the setuptools approach and is useful for projects that need to be zip-safe or work with other setuptools packages. The directory structure −

project1/
    setup.py
    mynamespace/
        __init__.py  # Contains pkg_resources code
        subpackage_a/
            __init__.py
            module.py

project2/
    setup.py
    mynamespace/
        __init__.py  # Contains pkg_resources code
        subpackage_b/
            __init__.py
            another_module.py

The __init__.py file for the namespace package needs to contain the following −

__import__('pkg_resources').declare_namespace(__name__)

Every distribution that uses the namespace package must include such an __init__.py. If any distribution does not, it will cause the namespace logic to fail and the other sub-packages will not be importable. Any additional code in the __init__.py will be inaccessible.

Every distribution must provide the namespace_packages argument to setup() in setup.py −

from setuptools import find_packages, setup

setup(
    name='mynamespace-subpackage-a',
    version="0.1",
    packages=find_packages(),
    namespace_packages=['mynamespace']
)

Comparison

Conclusion

Use native namespace packages for Python 3.3+ projects as they require no __init__.py files. For legacy compatibility, choose pkgutil-style, and for zip-safe requirements, use pkg_resources-style namespace packages.