Skip to main content

From Notebook to Maintainable Project

A notebook is excellent for discovering a useful question. It is a weak production contract when results depend on cell order, objects left in memory, a file from a previous run, or an unstated working directory. The goal is not to ban notebooks. It is to move the accepted training path behind a command that starts from declared inputs and either publishes a complete result or fails clearly.

This learning path follows one local Study Sprint classifier. It predicts completion from prior score, practice minutes, and learning track. The dataset is small and synthetic so the engineering behavior—not model quality—is the lesson.

Prerequisites​

Be comfortable with building and using scikit-learn pipelines and train/test evaluation. Run commands from the repository root with the dependencies in requirements-content.txt available.

Learning goals​

  • separate exploration from the accepted training entry point.
  • identify hidden notebook state and replace it with files or arguments.
  • define success through observable artifacts and metrics.

A narrow project contract​

The maintained path has three immutable inputs for a run:

  1. fixtures/ml-engineering/study-sprint-model.csv is the data snapshot.
  2. examples/ml-engineering/training-config.json declares schema, features, holdout IDs, seed, and estimator parameters.
  3. examples/ml-engineering/train.py is the executable recipe, identified by a caller-supplied code version.

Its output directory contains model.json, metrics.json, predictions.csv, and run.json. The script does not read a notebook kernel, make a network request, infer a split from the clock, or silently reuse an old artifact.

Worked example​

Run the accepted command into a new directory:

.venv-content/bin/python examples/ml-engineering/train.py \
  --config examples/ml-engineering/training-config.json \
  --output-dir /tmp/study-sprint-run \
  --code-version ticket-39-fixture-v1

The documented result is:

trained 32 rows; evaluated 8 rows
accuracy=0.875000 log_loss=0.159471 roc_auc=1.000000
wrote model.json, metrics.json, predictions.csv, and run.json

These numbers are regression expectations for this teaching fixture, not evidence that the classifier will generalize to learners elsewhere. The public behavior is the command, exit status, output filenames, schemas, and metrics. Functions inside the script may change without changing that contract.

Repeatability, reproducibility, and determinism​

Terminology varies across fields, so this path uses explicit working definitions:

  • Repeatability: the same team reruns the recipe in the same declared environment and obtains the expected result.
  • Reproducibility: another person can reconstruct the result from the published data, code, configuration, environment description, and procedure.
  • Determinism: within a stated execution envelope, identical inputs select one output rather than a distribution of outputs.

A fixed seed helps, but it does not freeze floating-point reductions, CPU instructions, GPU kernels, operating systems, or library versions. This tiny CPU example is byte-repeatable in the verified environment. That local fact is not a universal promise across hardware and software stacks.

Common mistakes​

  • Exporting a final notebook cell while earlier hidden state still defines variables.
  • Putting data paths, target names, or thresholds in a developer's shell profile.
  • Treating a successful exit as proof that the model is useful.
  • Calling a seeded result reproducible without recording data, code, configuration, and library versions.
  • Mixing exploratory plots and the accepted training contract until neither has a clear owner.

Practice​

List every value the Study Sprint command reads before its first model fit. Classify each as data, configuration, code, environment, or accidental state. Then name the artifact that lets a reviewer verify each declared input after the run.

Next steps​

Reproducible Environments defines the execution envelope and explains what exact version pins can and cannot guarantee.

Primary sources​

Checked 2026-10-15: scikit-learn common pitfalls and recommended practices, scikit-learn pipelines and composite estimators, and the Python Packaging User Guide on packaging projects.