Containerizing Inference

A container image packages a filesystem, metadata, and process configuration. It can make an inference command easier to move and inspect, but it does not by itself make the build reproducible, the artifact trustworthy, or the runtime secure.

Prerequisites

Complete Building a Prediction API. Understand the Study Sprint model's trusted provenance and the local server's loopback-only teaching boundary.

Learning goals

• identify what an image captures and what remains outside it.
• separate image identity, model identity, configuration, and deployment identity.
• apply small, non-root, local-first packaging practices without requiring a registry.

Worked example

Choose whether the model is baked into the image or mounted/configured at runtime. Baking creates one immutable image-to-model pairing and larger rebuilds. Mounting lets one image serve reviewed model versions, but requires independent artifact integrity, compatibility, and access controls. In either design, record both image digest and model digest/version in deployment evidence.

A conceptual Study Sprint batch image can remain standard-library-only at inference time:

FROM python:3.13-slim
WORKDIR /app
RUN useradd --create-home --uid 10001 inference
COPY --chown=inference:inference examples/ml-engineering/predict.py /app/predict.py
USER inference
ENTRYPOINT ["python", "/app/predict.py"]

Build and exercise it locally; no registry account is involved:

docker build --tag study-sprint-inference:local -f Dockerfile .
docker run --rm --network none \
  --mount type=bind,src="$PWD/build/study-sprint",dst=/model,readonly \
  --mount type=bind,src="$PWD/fixtures/ml-engineering",dst=/input,readonly \
  --mount type=bind,src="$PWD/build",dst=/output \
  study-sprint-inference:local batch \
  --model /model/model.json \
  --input /input/inference-requests.jsonl \
  --output /output/predictions.jsonl

This is an illustrative Dockerfile, not an acceptance dependency: the repository's deterministic verifier executes the public CLI directly and does not require Docker or a network pull.

What the container does not solve

• A mutable base tag can resolve to different bytes; a digest pin trades surprise for an explicit update process.
• Package pins do not cover the kernel, CPU behavior, build tooling, or external data.
• A non-root user reduces privileges but is not a complete sandbox.
• An image scan finds known patterns and vulnerabilities; it cannot prove model behavior or absence of malicious logic.
• Secrets in build arguments or environment layers can persist; use appropriate runtime/build secret mechanisms.

Rebuild regularly to take security fixes, generate an SBOM/provenance where the environment warrants it, test the resulting image, and promote by immutable digest. Reproducibility and timely patching are managed policies, not opposing one-time choices.

API note

The repository service intentionally binds only to loopback. Publishing a container port usually requires listening on a container interface, which this teaching service refuses. Do not weaken that check for the exercise. A real containerized API needs a production server, explicit interface policy, authentication, TLS termination, resource limits, and network controls.

Common mistakes

• Calling an image secure because it runs as a container.
• Copying the repository, credentials, caches, and test data into the build context.
• Using latest as a deployment identity.
• Installing compilers and editors in the runtime image without need.
• Assuming the container fixes incompatible model and dependency versions.

Practice

Threat-model the conceptual image from base selection through local build, model mount, execution, and deletion. For each stage, name the trusted identity, verification, principal, writable path, secret exposure, and patch owner.

Next steps

Latency, Throughput, and Cost turns the packaged process into a workload model instead of publishing a misleading toy benchmark.

Primary sources

Checked 2026-10-15: Docker build best practices, Docker build secrets, Docker build attestations, and scikit-learn environment compatibility guidance.