module_directories.Rmd

---
jupyter:
  jupytext:
    notebook_metadata_filter: all,-language_info
    split_at_heading: true
    text_representation:
      extension: .Rmd
      format_name: rmarkdown
      format_version: '1.2'
      jupytext_version: 1.10.3
  kernelspec:
    display_name: Python 3
    language: python
    name: python3
---

# Module directories

In the [modules introduction](on_modules.Rmd), we found that we could make a
Python module with a single file that has a `.py` file extension.  The `.py`
file has to be in directory on the [Python path](sys_path.Rmd).

This is the simplest form of a Python module — a single `.py` file.

These can be useful, but we often want to add many functions and attributes to our module.  This would make the `.py` file very large.

Python gives us another way of creating a module that we will call *module
directories*.  This allows us to define a module with a *directory* that can have many `.py` files.

It is easiest to see this by example.

**Note**: Here we use Python and the notebook commands to create the module.
This is just to show the process.  Normally you would use your terminal and
text editor to create the directories and files.

We will call our new module `mydirmod` (for My Directory Module).

## Creating the directory

It's a directory, so we first create the directory.

The directory may already exist from a prior run through of this notebook, so we start by deleting the directory, if it exists.

We use the [os.path](path_manipulation.Rmd) routines to check if the directory
exists, and the `rmtree` function from the `shutil` module to remove it, if it
does:

```{python}
import os
import os.path as op
import shutil
```

```{python}
# Remove mydirmod directory if it exists.
if op.isdir('mydirmod'):
    shutil.rmtree('mydirmod')
```

With that out of the way, we create the directory.

```{python}
os.mkdir('mydirmod')
```

Next we put a `.py` file into that directory, using the `%%file` notebook command:

```{python}
# %%file mydirmod/some_module.py
""" A sub-module in mydirmod
"""

def myfunc(a):
    return a * 10
```

Just to confirm, we show the files in `mydirmod`:

```{python}
os.listdir('mydirmod')
```

But, our work here is not yet done.  To make the `mydirmod` into a directory
module, we have to do one more step.


## Making `mydirmod` into a module

The key step to tell Python that `mydirmod` is a directory module, is to create
an `__init__.py` file inside the directory.  Notice the [double
underscores](dunders.Rmd), indicating that this filename is special for Python.
For the moment, let's create a file that has (virtually) nothing in it:

```{python}
# %%file mydirmod/__init__.py
""" An __init__.py file that only has a docstring
"""
```

Hey presto, we can import the module.

```{python}
import mydirmod
```

However, tab-completion reveals that `mymod` has nothing inside it.  In particular, it does not have the `some_module.py` file sub-module, nor does it have the `myfunc` function from that sub-module.

To import the sub-module, we need to do it explicitly, like this:

```{python}
import mydirmod.some_module

# Use myfunc from the sub-module.
print(mydirmod.some_module.myfunc(9))
```

Actually, there is another way to get to functions in sub-modules, and that is to import the sub-module in the `__init__.py` file.  Let's do that:

```{python}
# %%file mydirmod/__init__.py
""" An __init__.py file that only has a docstring
"""

# Notice the . at the beginning of .some_module.
from .some_module import myfunc
```

The `.` at the beginning of `.some_module` refers to the current directory,
meaning, the directory containing the `__init__.py` file.  It tells
`__init__.py` to import the `some_module.py` file in its directory.  **Note**:
This is called a [relative
import](https://docs.python.org/3/reference/import.html#package-relative-imports),
because `some_module.py` is in the directory `.` *relative to* the
`__init__.py` file.  These kinds of imports only work in module directories.


OK, let's try that:

```{python tags=c("raises-exception")}
import mydirmod

# Use myfunc from the sub-module.
print(mydirmod.myfunc(9))
```

Oh dear - it didn't work.  Why not?  Because we need to {ref}`reload` the
module.

```{python}
import importlib

importlib.reload(mydirmod)
```

```{python tags=c("raises-exception")}
# Use myfunc from the sub-module.
print(mydirmod.myfunc(9))
```

As your modules become more than slightly complicated, you will want to switch
from using single `.py` file module, to directory modules like this one.

## See also

[Modules](https://docs.python.org/3/tutorial/modules.html) in the standard
Python tutorial.