Security, Privacy, and Supply-Chain Risk
An ML system joins ordinary software threats with data poisoning, model theft, unsafe serialization, privacy leakage, and manipulated evaluation. Security is a lifecycle property: provenance helps establish origin, but only validation, isolation, authorization, and monitoring limit what an artifact can do.
Prerequisites
Complete Model Cards and Operational Documentation and Model Artifacts and Safe Serialization.
Learning goals
- trace threats across source, dependencies, data, training, registry, deployment, and feedback.
- use least privilege, immutable identity, review, and isolation as layered controls.
- separate provenance, integrity, safety, privacy, and fitness claims.
Worked example
Apply a threat review to the Study Sprint capstone:
| Boundary | Example threat | Minimum local control |
|---|---|---|
| Source and CI | altered dependency or workflow | exact pins, reviewed lockfiles, action SHAs, read-only token |
| Data | poison row, leakage, unlawful retention | schema and split tests, digest, provenance, purpose and deletion rules |
| Training | hidden config or untrusted code | explicit CLI/config, clean environment, no secret in untrusted job |
| Artifact | executable deserialization | inert JSON, size/type/dimension validation, digest |
| Registry | unauthorized alias movement | immutable versions, separate approval, least privilege, audit record |
| Serving | malformed input or extraction | strict contract, bounds, authentication, rate limits, safe logs |
| Monitoring | sensitive logs or feedback poison | minimization, aggregation, access/retention policy, delayed-label audit |
The local capstone uses committed fixtures, CPU-only commands, inert JSON, loopback/file boundaries, and no secret, cloud, account, or network dependency. That narrow environment reduces exposure; it does not prove a production system secure.
Record hashes and SLSA-style provenance for origin and integrity. Verify attestations before use where available. Produce an SBOM or dependency inventory, monitor advisories, and rebuild from reviewed sources. Provenance cannot prove the data were consented, the model is unbiased, or the code has no vulnerability. Privacy review must cover collection purpose, minimization, access, retention, deletion, linkage, logs, model inversion/membership risk, and incident response.
Common mistakes
- Loading pickle, joblib, or cloudpickle from an untrusted registry.
- Exposing secrets to pull-request code or embedding them in images and artifacts.
- Logging raw features, labels, prompts, identifiers, or errors without retention review.
- Believing a checksum establishes safety or a license grants data consent.
- Letting the same identity build, approve, deploy, and erase audit evidence.
Practice
Choose three Study Sprint boundaries and write asset, actor, threat, control, residual risk, owner, and evidence. Include one privacy and one supply-chain scenario.
Next steps
Continue with Build vs Buy and Platform Costs to evaluate controls and ownership as part of total cost.
Primary sources
Checked 2026-10-15: NIST AI RMF 1.0, NIST Secure Software Development Framework 1.1, SLSA 1.2 provenance, OWASP Machine Learning Security Top Ten, and Python pickle warning.