A colleague sent me a notebook that scored 0.81 on his machine. Same notebook, same CSV, same seed, my laptop: 0.78. We lost most of a Thursday to it before finding the cause, which had nothing to do with the model and everything to do with the fact that neither of us could say which version of scikit-learn we were running.

That is the failure this part exists to prevent. Setting up Python sounds like a chore you do once and forget, and it is treated that way in most tutorials: install Anaconda, open a notebook, start typing. Then six months in you cannot rerun your own work, a package upgrade quietly changes a default, and the model you promised the business is not the model you can rebuild. Environment discipline is not administrative overhead. It is the thing that decides whether your results are findings or anecdotes.

Where broken environments come from

Python installs packages into a folder called site-packages that belongs to one specific interpreter. Install pandas and it lands next to whichever python binary ran the install command. Nothing about that is per project. If you install into your system interpreter, every project on the machine shares one pile of packages, and the pile can only hold one version of anything.

Trouble starts the day two projects disagree. Your churn work wants pandas 2.3.3 for the newer string dtype behaviour. A reporting script written eighteen months ago depends on pandas 1.5.3 because it uses an argument that was later removed. Upgrading fixes one and breaks the other, so you upgrade, then downgrade, then upgrade again, and eventually somebody keeps a text file of which packages to reinstall before running which script. I have seen that text file in three organisations. It never works.

A virtual environment solves this by giving each project its own site-packages folder and its own python link back to a base interpreter. Projects stop seeing each other. Deleting a project means deleting a folder, not unpicking a global install.

One virtual environment per project

Python ships a module called venv that creates these folders, and it has been in the standard library since 3.3, so there is nothing to install. Three commands get you a clean, isolated project.

mkdir churn-project && cd churn-project
python3 -m venv .venv
source .venv/bin/activate      # Windows: .venvScriptsactivate
python -m pip install pandas scikit-learn

Tested on Python 3.10.12. After activation your shell prompt is prefixed with (.venv), which is the only visual confirmation you get that the isolation is on.

Name the folder .venv, put it inside the project, and add it to .gitignore. Since Python 3.13 venv writes a .gitignore file into the environment for you, but do not rely on that if your team is on an older interpreter. Environments are rebuilt from a lockfile, never committed. A .venv with the scikit-learn and pandas stack in it measured 285 MB on disk in my test, which is not something you want in version control.

Activation is the step people skip, and skipping it is the source of the single most common beginner error in Python data work. Here is what that looks like when it happens.

$ python3 -c 'import sklearn; print(sklearn.__version__)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'sklearn'

Real traceback from my test machine, taken minutes after a successful install of scikit-learn.

Nothing is wrong with the install. Something is wrong with which interpreter is running. Install went to one python, the import ran under another, and the two do not share a site-packages folder. Diagnose it in one line rather than guessing.

import sys
print(sys.executable)

Output on my machine before activating: /usr/bin/python3. After activating the project environment: /tmp/churn-project/.venv/bin/python. If the path does not contain .venv, your isolation is not on and every install you do is landing somewhere else.

Run that snippet as the first cell of any notebook that behaves strangely. It has resolved more import problems for me than any amount of reinstalling. Prefer python -m pip install over a bare pip, too, because the -m form guarantees the install goes to the interpreter you just invoked rather than whichever pip happens to be first on your PATH.

Choosing between pip, conda and uv

Three tools cover almost every data science project, and they are not interchangeable. Standard venv plus pip is universal and needs no installation. conda handles packages that are not pure Python, such as GDAL or certain CUDA builds, because it manages compiled system libraries alongside Python ones. uv is a Rust implementation of the packaging workflow that also creates environments, resolves dependencies and writes a lockfile.

My verdict: use uv. Resolution of the churn stack took 16 milliseconds in my test against roughly a minute for pip on the same machine and the same network, and more importantly uv produces a uv.lock file that records every transitive dependency at an exact version. pip on its own has no lockfile. What I would avoid is a base conda environment that everything gets installed into, which is the default habit Anaconda encourages and is just the global install problem with a nicer installer in front of it. Reach for conda only when a dependency genuinely needs non Python system libraries, and even then, one named environment per project.

flowchart TD
  A[New project] --> B{Needs compiled system libraries}
  B -->|Yes| C[conda or mamba, one env per project]
  B -->|No| D{Can you install a new tool}
  D -->|Yes| E[uv init and uv add]
  D -->|No| F[python -m venv plus pip]
  E --> G[Commit uv.lock]
  F --> H[Commit pinned requirements.txt]
  G --> I[Anyone can rebuild the exact env]
  H --> I

Starting a project with uv takes two commands, and both write files you commit.

uv init churn-project
cd churn-project
uv add 'pandas==2.3.3' 'scikit-learn==1.7.2'

Actual output from uv 0.11.19: Initialized project `demo`, then Resolved 10 packages in 16ms and Installed 6 packages in 312ms, listing numpy==2.2.6, pandas==2.3.3, python-dateutil==2.9.0.post0, pytz==2026.2, six==1.17.0 and tzdata==2026.3.

Two files appear. pyproject.toml lists what you asked for, and uv.lock records what was actually resolved, including every transitive dependency you never named. Notice that asking for two packages pulled in six. Those four extras are exactly the ones that break a rebuild eighteen months later if nothing recorded them, which is the whole argument for a lockfile.

# pyproject.toml, as written by uv add
[project]
name = 'demo'
version = '0.1.0'
requires-python = '>=3.10'
dependencies = [
    'pandas==2.3.3',
]

Real file contents, with the TOML double quotes rendered here as single quotes for display. The matching uv.lock begins with version = 1, revision = 3, requires-python = >=3.10, then one block per resolved package.

Notebooks without the usual traps

Notebooks are genuinely good at exploration. You look at a column, plot it, change your mind, look again. For the kind of work covered in my Data Analyst Series post on exploratory data analysis, a notebook is the right tool and I use one every week.

Two properties make them dangerous anyway. First, execution order is whatever you clicked, not what the page shows, so a notebook can look correct top to bottom while depending on a cell you deleted twenty minutes ago. Second, a notebook holds state, so a variable defined in a cell you have since changed is still sitting in memory quietly propping up your results.

Three habits remove nearly all of this. Restart the kernel and run all cells before you believe any number you are going to tell someone else. Keep a notebook under roughly two hundred lines and move anything reused into a .py file that the notebook imports. Clear outputs before committing, because a committed notebook with outputs produces unreadable diffs and can leak customer data straight into your repository.

Register the kernel explicitly so a notebook can never silently attach to the wrong interpreter. One command, done once per project.

python -m pip install ipykernel
python -m ipykernel install --user --name churn --display-name 'Python (churn)'
jupyter lab

Run these with the project environment activated. A kernel named Python (churn) then appears in the JupyterLab launcher and points at that environment. JupyterLab 4.6.1 was current at the time of writing.

Getting this wrong throws the same ModuleNotFoundError shown earlier, except now it appears inside a notebook where it looks mysterious rather than in a terminal where the cause is obvious. Same diagnosis: put sys.executable in the first cell and read the path.

Pinning versions so a rerun means something

Writing pandas in a requirements file means the latest pandas at whatever moment the install runs. Two people installing a month apart get different software. Writing pandas==2.3.3 means one specific build, and that is the difference between a project that rebuilds and one that merely usually rebuilds.

If you are on plain pip, freeze the environment after it works.

python -m pip freeze > requirements.txt

# rebuilding elsewhere
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

Real freeze output from my test environment included numpy==2.2.6, pandas==2.3.3, scikit-learn==1.7.2, scipy==1.15.3, threadpoolctl==3.6.0, six==1.17.0, pytz==2026.2 and tzdata==2026.3. Ten packages from a two package request.

Freezing has a known weakness. It captures what you happen to have, including anything you installed while experimenting and no longer need, and it says nothing about which packages you actually asked for versus which arrived as dependencies. A lockfile keeps those two ideas separate, which is the practical reason I prefer uv. Either way, the file goes in version control and gets updated deliberately, not by accident.

Seeds and honest reproducibility

Plenty of steps in a modelling workflow use randomness. Splitting data into training and test sets picks rows at random. Random forests sample features at random. Neural networks start from random weights. Leave that randomness unseeded and every run gives a slightly different answer, which makes it impossible to tell a genuine improvement from noise.

A seed fixes the starting point of the random number generator so the same sequence comes out every time. Here is the effect, run for real.

import numpy as np
from sklearn.model_selection import train_test_split

X = np.arange(20).reshape(10, 2)
y = np.array([0, 1] * 5)

a, _, _, _ = train_test_split(X, y, test_size=0.3)
b, _, _, _ = train_test_split(X, y, test_size=0.3)
print('unseeded splits identical:', np.array_equal(a, b))

a, _, _, _ = train_test_split(X, y, test_size=0.3, random_state=42)
b, _, _, _ = train_test_split(X, y, test_size=0.3, random_state=42)
print('seeded splits identical:  ', np.array_equal(a, b))

print('rng(42) draw: ', np.random.default_rng(42).integers(0, 100, 5))
print('rng(42) again:', np.random.default_rng(42).integers(0, 100, 5))

Actual output on scikit-learn 1.7.2, pandas 2.3.3, NumPy 2.2.6: unseeded splits identical: False / seeded splits identical: True / rng(42) draw: [ 8 77 65 43 43] / rng(42) again: [ 8 77 65 43 43].

Note the modern NumPy form. np.random.seed sets a single global state that any library can quietly change underneath you, whereas np.random.default_rng(42) creates an isolated generator you pass around explicitly. Use the generator form for new code and reserve random_state for the scikit-learn functions that expect it.

One caution worth carrying into the modelling parts. A seed guarantees the same answer, not a good answer. If a model looks strong on random_state=42 and weak on 0, the number you should report is not the better one, it is the spread across several seeds. Reporting the flattering seed is a quiet form of cheating that Part 11 on evaluation will come back to.

flowchart LR
  A[Raw data file] --> B[Environment rebuilt from lockfile]
  B --> C[Seeded split]
  C --> D[Train and score]
  D --> E[Metrics written to a file]
  E --> F{Same as last run}
  F -->|Yes| G[Change was real]
  F -->|No| H[Environment or seed differs, investigate]

Project layout for the churn work

From Part 4 onward this series runs one project the whole way: predicting and reducing churn for a subscription business, using the public Telco Customer Churn dataset published by IBM, which holds 7,043 customers across 21 columns with a Churn column as the target. Part 4 loads it and puts NumPy and pandas to work on it. Right now there is no project yet, only the environment that will hold it, and that is deliberate. Build the container before you put anything in it.

Here is the layout I would create today, before writing any analysis code.

churn-project/
  .venv/                 # never committed
  .gitignore             # ignores .venv, data/raw, .ipynb_checkpoints
  pyproject.toml         # what you asked for
  uv.lock                # what was actually resolved
  data/
    raw/                 # the CSV as downloaded, never edited
    processed/           # anything your code produces
  notebooks/
    01-explore.ipynb
  src/
    churn/
      __init__.py
      data.py            # loading, shared by notebook and scripts
  README.md              # how to rebuild the env in two commands

Numbering notebooks keeps them in reading order. Keeping data/raw read only means a bad cleaning step is always recoverable.

One rule I enforce on myself: raw data is never edited in place. Load from data/raw, write to data/processed, and the moment you find you have destroyed something, you re-run rather than re-download. That habit pairs directly with the cleaning work covered in my analyst post on cleaning messy data, which Part 5 assumes you have read rather than repeating.

Setup I would standardise on

If I were setting a team standard tomorrow, it would be five lines long. Use uv, one environment per project, in a folder named .venv that is never committed. Pin every version and commit uv.lock. Register a named Jupyter kernel per project so a notebook cannot attach to the wrong interpreter. Seed everything, and report the spread across seeds rather than the best one. Keep raw data read only.

What I would drop is the habit of installing everything into one big Anaconda base environment. It works fine until the day two projects need different versions, and by then the untangling is a day of work rather than the two minutes it would have cost to isolate them at the start.

Part 4 picks up here and starts using the environment: NumPy and pandas past the basics, loading the Telco churn table, and the vectorisation, merge and memory behaviour that decides whether your data handling scales past a laptop. Before moving on, build the environment described above and run the sys.executable check inside a fresh notebook. If that path contains .venv, you are ready.

References

• venv, Creation of virtual environments, Python documentation
• uv documentation, Astral
• train_test_split, scikit-learn API reference
• Telco Customer Churn dataset, IBM on GitHub

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts