How long would it take a new joiner to reproduce this? A delivery manager asked me that about a churn notebook I had spent six weeks on, and I answered, confidently, about a day. It took her four. Nothing in that notebook was wrong. It simply carried nine unwritten assumptions about where files sat on disk, which cells had already been executed, and which packages happened to be installed in one particular environment on one particular laptop.

Cost of a notebook nobody else can run

Where the project stands: across Parts 4 to 20 we loaded the Telco churn data, engineered features from it, fitted linear and logistic models, tuned them, rebuilt one as a network and backtested another. All of that lives in notebook cells. This part changes none of the modelling and all of the plumbing, turning those cells into an installable package with a command line entry point, a test suite and versioned artifacts. Nothing after this point in the series assumes a notebook.

Notebooks are excellent at exploration and terrible at handover, and it is worth being precise about why rather than treating it as taste. Four failure modes account for nearly all of it. Execution order is invisible, so a notebook that runs top to bottom today may depend on a cell you deleted an hour ago. State is hidden, so a variable defined in cell 4 and mutated in cell 31 has no declared lifetime. Paths are relative to wherever the kernel was started, which is usually not where the reader will start theirs. And there is no dependency declaration at all, so the environment is whatever the author accumulated.

Packaging fixes each of those with a specific mechanism, and it helps to hold the mapping in mind while reading the rest of this part. Import order replaces execution order. Function arguments replace hidden state. Paths derived from the module file replace paths derived from the shell. A dependency table in pyproject.toml replaces whatever was lying around. None of these are advanced techniques and all four take an afternoon.

One objection comes up every time I propose this, and it deserves an answer rather than a dismissal: packaging feels like overhead on work that might get thrown away. My rule is a threshold rather than a principle. If the analysis will be run more than twice, or by more than one person, or on a schedule, it goes in a package. Everything else stays in a notebook and is allowed to be messy. Roughly a third of my exploratory work never crosses that line, and that is fine.

Layout that survives contact with a second person

Two layouts are in common use. Flat layout puts your package directory at the project root, next to pyproject.toml. Src layout puts it one level down, inside a folder named src. That distinction sounds cosmetic and is not, because Python adds the current directory to the import path. Under a flat layout, running Python from the project root imports your source folder whether or not the package is installed, which means your tests can pass against code that would fail entirely for anybody who installed the wheel. A src layout makes that impossible: nothing imports until you have installed it, so the thing you test is the thing you ship. Python packaging documentation recommends src for exactly this reason, and so does pytest.

Here is the shape of the churn project as I would hand it over. Six modules, one test file, one configuration file, and a data folder that is not tracked in version control.

Metadata goes in pyproject.toml, a single file that replaces setup.py, setup.cfg and most tool specific configuration. PEP 621 standardised the project table inside it, and setuptools reads that table directly. Three sections carry the weight: build-system says which tool builds your package, project describes it, and project.scripts creates command line entry points.

# pyproject.toml
# tested against: Python 3.10.12, setuptools 83.0.0, pandas 2.3.3,
# numpy 2.2.6, joblib 1.5.3, pytest 9.1.1

[build-system]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"

[project]
name = "telco-churn"
version = "0.3.0"
description = "Churn feature pipeline and model for the Telco dataset"
requires-python = ">=3.10"
dependencies = ["pandas>=2.2", "numpy>=2.0", "joblib>=1.4"]

[project.optional-dependencies]
dev = ["pytest>=8.0"]

[project.scripts]
churn = "telco_churn.cli:main"

[tool.setuptools.packages.find]
where = ["src"]

[tool.pytest.ini_options]
addopts = "--import-mode=importlib"
testpaths = ["tests"]

Two entries in there repay a closer look. Under project.scripts, the string maps a command name to a module and a function, so installing this package puts an executable called churn on the path that calls main in cli.py. That is how you give somebody a tool rather than a folder of scripts. And tool.setuptools.packages.find with where set to src is what tells the build backend to look one level down; leave it out with a src layout and you will build an empty wheel that installs cleanly and imports nothing.

Splitting modules by responsibility rather than by convenience is what makes the package readable later. My default split has held up across a decade of projects, and each module answers one question.

Configuration, paths and a working directory trap

Relative paths are the single most common reason a handed over project fails on the first attempt, and they fail in a way that looks like the recipient made a mistake. A notebook sitting in a notebooks folder reads its data with two dots and a slash, which resolves correctly from that folder and nowhere else. Move the same line into a module and it breaks the moment anybody runs it from the project root, which is where a package is normally run from.

$ cd notebooks && python nbstyle.py       # the line from the notebook
7043

$ cd .. && python nbstyle.py             # identical file, project root
Traceback (most recent call last):
  File "/tmp/nbstyle.py", line 2, in <module>
    df = pd.read_csv("../data/telco.csv")
  File "pandas/io/common.py", line 873, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '../data/telco.csv'

Fix it by deriving paths from the module file rather than from the shell. Every Python module knows its own location through the dunder file attribute, so a config module can resolve the project root once and everything else imports from there. Adding an environment variable override on top of that costs two lines and buys you the ability to point the same package at a different dataset without editing anything.

# src/telco_churn/config.py
from pathlib import Path
import os

PKG_ROOT = Path(__file__).resolve().parent
PROJECT_ROOT = PKG_ROOT.parents[1]
DATA_DIR = Path(os.environ.get('CHURN_DATA_DIR', PROJECT_ROOT / 'data'))
ARTIFACT_DIR = Path(os.environ.get('CHURN_ARTIFACT_DIR', PROJECT_ROOT / 'artifacts'))
RAW_FILE = DATA_DIR / 'telco.csv'
SEED = 7

Loading gets the same treatment. Missing data should produce a message that tells the reader what to do next, not a raw pandas traceback, and required columns should be checked once at the boundary rather than discovered halfway through feature engineering.

# src/telco_churn/data.py, load function only
COLS = ['customerID', 'tenure', 'MonthlyCharges', 'TotalCharges',
        'Contract', 'InternetService', 'Churn']

def load_raw(path=None):
    path = RAW_FILE if path is None else path
    if not path.exists():
        raise FileNotFoundError(
            f'{path} not found. Run: churn bootstrap --out {path}')
    df = pd.read_csv(path)
    missing = set(COLS) - set(df.columns)
    if missing:
        raise ValueError(f'missing columns: {sorted(missing)}')
    return df

$ cd / && churn train
Traceback (most recent call last):
  File "/tmp/churn/src/telco_churn/cli.py", line 19, in main
    df = data.load_raw()
  File "/tmp/churn/src/telco_churn/data.py", line 33, in load_raw
    raise FileNotFoundError(
FileNotFoundError: /tmp/churn/data/telco.csv not found. Run: churn bootstrap --out /tmp/churn/data/telco.csv

Tests worth writing for a data science package

Testing model quality is a trap, and it is where most data scientists start. An assertion that accuracy exceeds some threshold breaks every time the data shifts, which trains everybody to ignore the test suite. Test the assumptions instead: shapes, dtypes, column order, boundary values and the handling of the specific ugliness you know lives in your dataset. Model quality belongs in monitoring, which Part 25 covers, not in a unit test.

Telco churn supplies three concrete assumptions worth pinning. TotalCharges arrives as text with eleven blank strings in it, tenure can be zero which makes any per month average a division hazard, and feature column order must be stable or a saved model will silently score the wrong columns. Each becomes one short test.

# tests/test_features.py
import pytest
from telco_churn import data, features, model

@pytest.fixture(scope='module')
def sample():
    return data.make_sample(n=500, seed=1)

def test_no_missing_after_build(sample):
    X, y = features.build(sample)
    assert X.notna().all().all()
    assert set(y) <= {0, 1}

def test_blank_total_charges_survives(sample):
    df = sample.copy()
    df.loc[df.index[:3], 'TotalCharges'] = ' '
    X, _ = features.build(df)
    assert X['avg_charge'].iloc[:3].eq(0.0).all()

def test_zero_tenure_does_not_divide_by_zero(sample):
    df = sample.copy(); df.loc[df.index[0], 'tenure'] = 0
    X, _ = features.build(df)
    assert X['avg_charge'].iloc[0] == 0.0

def test_column_order_is_fixed(sample):
    X, _ = features.build(sample)
    assert list(X.columns) == features.FEATURES

def test_model_beats_coin_flip(sample):
    X, y = features.build(sample)
    m = model.fit(X, y)
    assert model.auc(y, m.predict_proba(X)) > 0.65

Run those before installing the package and you meet the src layout doing its job, which surprises people the first time.

$ python -m pytest -q
ImportError while importing test module '/tmp/churn/tests/test_features.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_features.py:2: in <module>
    from telco_churn import data, features, model
E   ModuleNotFoundError: No module named 'telco_churn'
=========================== short test summary info ============================
ERROR tests/test_features.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
1 error in 0.39s

Installing the package in editable mode fixes it. Editable means the interpreter is pointed at your source folder rather than a copy, so edits take effect without reinstalling, while imports still go through the real installed package metadata. That is the mode you develop in.

$ python -m pip install -e ".[dev]"
$ python -c "import telco_churn; print(telco_churn.__file__)"
/tmp/churn/src/telco_churn/__init__.py

$ python -m pytest -q --durations=5
.....                                                                    [100%]
============================= slowest 5 durations ==============================
0.01s call     tests/test_features.py::test_model_beats_coin_flip
(4 durations < 0.005s hidden.  Use -vv to show these durations.)
5 passed in 0.23s

Two details in that pyproject test configuration are worth defending. Setting import-mode to importlib is now the recommendation in pytest documentation, and it stops pytest from mutating the import path, which means your tests import the installed package rather than a folder that happened to be adjacent. It also removes the rule that every test file needs a globally unique name, which becomes irritating the moment you have two subdirectories both wanting a test_features file. Setting testpaths keeps a bare pytest invocation from wandering into your data folder.

Giving your package a command line

An entry point is what converts a package into a tool. Whoever installs telco-churn gets a churn command, and every parameter that used to be a variable somebody had to edit becomes a flag they can pass. Keeping the entry function thin matters: it parses arguments, calls into the modules, prints, and returns an exit code. No modelling logic belongs here, because nothing in this file is easy to test.

# src/telco_churn/cli.py, train branch only
def main(argv=None):
    p = argparse.ArgumentParser(prog='churn')
    p.add_argument('--version', action='version', version=__version__)
    sub = p.add_subparsers(dest='cmd', required=True)
    b = sub.add_parser('bootstrap'); b.add_argument('--out', default=str(RAW_FILE))
    t = sub.add_parser('train'); t.add_argument('--holdout', type=float, default=0.25)
    a = p.parse_args(argv)
    ...
    df = data.load_raw()
    X, y = features.build(df)
    n = int(len(X) * (1 - a.holdout))
    m = model.fit(X.iloc[:n], y[:n])
    tr = model.auc(y[:n], m.predict_proba(X.iloc[:n]))
    te = model.auc(y[n:], m.predict_proba(X.iloc[n:]))
    path = model.save(m)
    print(f'rows {len(X)} features {len(m.features)}')
    print(f'train auc {tr:.4f}  holdout auc {te:.4f}')
    print(f'saved {path}')
    return 0

One note on the estimator. Fitting here is roughly forty lines of gradient descent written directly in NumPy, so the package installs and runs with only pandas, NumPy and joblib as dependencies. Swapping in the LogisticRegression class from Part 10 touches one function and adds one dependency line; every structural decision in this part is unchanged by that choice, which is rather the point of separating the model module from everything else.

Now that holdout size is a flag rather than a hardcoded number, sweeping it becomes a shell loop instead of a code edit, and the result is a genuinely reproducible experiment because the seed lives in config.

Read that chart carefully, because it makes an argument for the packaging work rather than about model quality. Train AUC barely moves across the sweep, sitting between 0.8124 and 0.8168, while holdout AUC climbs 0.032 as the evaluation set grows. Nothing about the model changed. What changed is measurement noise from evaluating on 705 rows instead of 3,522. Anybody reading a single number from a notebook would have taken 0.7776 as a property of the model, and the only reason this is visible at all is that the sweep cost one shell loop. Cross validation, covered in Part 11, is the proper answer here; cheap reproducibility is what made the problem noticeable.

Artifacts, versions and what joblib will not tell you

Saving a fitted model with joblib takes one line, which is exactly the problem. A pickle file records object state and nothing about the code that produced it, so a model saved by version 0.3.0 of your package will load happily under 0.9.0 even if the feature list changed underneath. Predictions come back, no warning appears, and the numbers are wrong in a way that no test catches.

Guard against it by stamping the package version and the feature list into the object at fit time and refusing to load a mismatch. Five lines, and it converts a silent wrong answer into a loud stop.

# src/telco_churn/model.py, persistence only
def save(m, path=None):
    path = (ARTIFACT_DIR / 'model.joblib') if path is None else path
    path.parent.mkdir(parents=True, exist_ok=True)
    joblib.dump(m, path)
    return path

def load(path=None):
    path = (ARTIFACT_DIR / 'model.joblib') if path is None else path
    m = joblib.load(path)
    from . import __version__
    if m.version != __version__:
        raise RuntimeError(
            f'artifact built by telco-churn {m.version}, running {__version__}')
    return m

# artifact trained under 0.3.0, package bumped to 0.4.0, then:
Traceback (most recent call last):
  File "<string>", line 3, in <module>
  File "/tmp/churn/src/telco_churn/model.py", line 43, in load
    raise RuntimeError(
RuntimeError: artifact built by telco-churn 0.3.0, running 0.4.0

Library versions deserve the same suspicion. Scikit-learn does raise InconsistentVersionWarning when an estimator is unpickled under a different version than it was pickled with, which is more than joblib gives you by itself, but its own documentation is blunt that loading across versions is unsupported and may produce different results. Treat that warning as an error in any serving path. And never load a pickle you did not create, because unpickling executes code by construction.

Building a distributable wheel is the last step and takes one command. Inspecting what came out is the habit worth forming, because a wheel is a zip file and reading its contents catches packaging mistakes in seconds.

$ python -m pip wheel . --no-deps -w dist
$ python -c "import zipfile,glob;print('n'.join(sorted(zipfile.ZipFile(glob.glob('dist/*.whl')[0]).namelist())))"
telco_churn-0.3.0.dist-info/METADATA
telco_churn-0.3.0.dist-info/RECORD
telco_churn-0.3.0.dist-info/WHEEL
telco_churn-0.3.0.dist-info/entry_points.txt
telco_churn-0.3.0.dist-info/top_level.txt
telco_churn/__init__.py
telco_churn/cli.py
telco_churn/config.py
telco_churn/data.py
telco_churn/features.py
telco_churn/model.py

Packaging order I would follow on Monday

Do it in this sequence, because each step makes the next one cheaper. Create the src directory and move your code into it first, even before writing pyproject.toml, since that move is what surfaces every hidden import. Write pyproject.toml second with a name, a version, three or four dependencies and the packages find entry. Install editable third and fix the import errors that appear, which will be more than you expect and all of which are real. Pull hardcoded paths into a config module fourth. Write four or five assumption tests fifth. Add the command line entry point last, once you know what the tool actually needs to do.

On the choices where reasonable people disagree, here is what I would pick and what I would avoid. Take src layout over flat layout without hesitation; flat is the one I would avoid, and my eleven weeks of green builds shipping a broken wheel is the reason. Take setuptools as your build backend unless you have a specific reason not to, because it is the most widely documented and nothing in this part needs anything more. Take a plain pyproject.toml over a Makefile full of PYTHONPATH exports; that pattern papers over exactly the import problems a src layout is designed to expose. Avoid putting model quality thresholds in your unit tests, and avoid writing a README that explains how to set environment variables that your config module should be deriving on its own.

One measurable target is worth holding yourself to: a colleague with your repository URL and nothing else should reach a trained model in five commands and under ten minutes, most of which is dependency download. Everything in this part exists to make that true. Try it on your current project this week by cloning your own repository into a fresh directory, creating a clean virtual environment, and following your own README literally without using any knowledge you have in your head. Whatever breaks first is the highest value thing you will fix all month.

Part 22 takes this installable package and asks how it should be served, comparing batch scoring, real time endpoints and streaming, along with the latency and cost consequences of each.

References

• Writing your pyproject.toml, Python Packaging User Guide
• Configuring setuptools using pyproject.toml files, setuptools 83.0.0
• PEP 621, Storing project metadata in pyproject.toml
• Good Integration Practices, pytest documentation
• Model persistence, scikit-learn documentation

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts