Advanced Python Package Management

Photo by Vladimir Proskurovskiy on Unsplash

In my previous blog, I shared about the definition of python packages and modules and the way to organise python packages. In this blog, we are going to talk about the two types of python packages, and executing directories.

Two kinds of packages

Python defines two types of packages, regular packages and namespace packages.

• Regular packages are traditional packages as they existed in Python 3.2 and earlier. A regular package is typically implemented as a directory containing an __init__.py file. When a regular package is imported, this __init__.py file is implicitly executed, and the objects it defines are bound to names in the package’s namespace. Note that __init__.py is optional in Python 3.3+, but it is still recommend to have it, as “explicit is better than implicit”.
• Namespace packages allow you to split the sub-packages and modules within a single package across multiple, separate distribution packages . Namespace packages can be useful for a large collection of loosely-related packages. With namespace packages, there is no __init__.py file.

The namespace package discovery algorithm is:

• scan each directory in sys.path
• import standard package if found
• import standard module if found
• otherwise, all matching directories contribute to a namespace package

Let’s see a comparison of regular package and namespace package:

==============in a regular 
example_package
-__init__.py
-sub_dir1
--__init__.py
--sub1.py
-sub_dir2
--__init__.py
--sub2.py
==============in a namespace
path1:
example_package
-sub_dir1
--__init__.py
--sub1.py
path2:
example_package
-sub_dir2
--__init__.py
--sub2.py

Executable directories

We know that in a directory, we can convert it into a python package by adding __init__.py . In python, you can also execute a directory or even a zip file directly if it contains __main__.py .

$ python program_dir
$ python program.zip
$ python -m program_dir

Note that the -m switch, this is to denote if the directory is been executed as a directory or a module. But there are some differences:

python directory                | python -m directory
execute as a directory          | execute as a package
directory added to the sys.path | directory treated as a package
directory/__main__.py is not in | directory/__main__.py is a 
a package                       | submodule of the directory package

So with the -m switch, package semantics (including relative imports) are honoured, and the package directory itself is never added to the system path.

It is recommended to have the -m where possible to avoid problems with relative import.

That’s so much of it!

Happy Reading!