Data and Code Versioning

A path is a location, not an identity. training.csv can name different bytes tomorrow, and a branch name can move to a new commit. Reproducible model development gives each material input an immutable identifier and preserves the mapping from friendly names to those identifiers.

Prerequisites

Complete Configuration Without Hidden State. Know that Git commits identify snapshots of tracked content.

Learning goals

• distinguish a data location from a data version.
• connect code, configuration, and data identities in one Study Sprint run.
• understand what a content hash proves and what it does not.

Version the three inputs together

The teaching CSV and configuration are small enough to track in Git. Production datasets often are not. An object-store version ID, immutable table snapshot, lakehouse version, or data-versioning manifest can play the same role if access and retention are governed.

A SHA-256 digest detects byte changes. It does not explain who produced the data, whether labels are correct, whether rows were lawfully collected, or whether two differently encoded files have the same meaning. Pair content identity with ownership, schema, time coverage, and provenance.

Worked example

The Study Sprint run.json records code identity supplied by the caller plus hashes of the exact config and data bytes:

{
  "code_version": "ticket-39-fixture-v1",
  "inputs": {
    "config": {
      "sha256": "<64 hexadecimal characters>"
    },
    "data": {
      "path": "fixtures/ml-engineering/study-sprint-model.csv",
      "sha256": "<64 hexadecimal characters>"
    }
  }
}

In a Git-backed project, obtain a commit identifier at the orchestration boundary and pass it explicitly:

git rev-parse --verify HEAD

Do not let the trainer query whatever repository happens to surround it. An exported source archive, container build, or scheduled runner may have no .git directory. Passing code identity keeps that boundary visible and testable. A dirty working tree needs a policy: reject it for governed builds, or record a patch/tree digest rather than pretending HEAD describes uncommitted code.

Data evolution and leakage

An immutable snapshot can still be wrong. Check entity uniqueness, label definition, schema, temporal cutoffs, and train/test membership. If a label is corrected, create a new version and link it to the correction; do not mutate the old snapshot in place. If time defines availability, a hash alone cannot prevent future information from entering past features.

For Study Sprint, explicit student IDs prevent row reordering from changing the holdout. A real learner dataset would also need privacy controls, retention rules, consent or legal basis, and a design for deletions. Immutability is an audit technique, not permission to retain data forever.

Common mistakes

• Recording main or latest where an immutable revision is required.
• Hashing a mutable manifest but not the referenced partitions.
• Treating Git LFS, DVC, or an object store as automatic provenance.
• Reusing a test set repeatedly until it becomes a tuning set.
• Assuming identical bytes imply valid labels or equivalent semantics.

Practice

Imagine one Study Sprint label is corrected after training. Write the lineage edges for the old snapshot, corrected snapshot, old run, and replacement run. State which artifacts remain immutable and which friendly alias may move.

Next steps

Experiment Tracking turns one set of input identities, parameters, metrics, and artifacts into a queryable run record.

Primary sources

Checked 2026-10-15: Git revision parsing, Git object model in Pro Git, DVC data versioning documentation, and W3C PROV overview.