
Software Supply Chain Security
From SolarWinds to xz, the recent security headlines have been supply chain. Master SLSA levels, SBOMs (CycloneDX/SPDX), Sigstore (cosign, Fulcio, Rekor), in-toto, build provenance, and the engineering practice that produces verifiable, signed, reproducible artifacts.
What you will learn
Almost every recent high-impact incident — SolarWinds, Codecov, npm event-stream, ua-parser-js, the xz-utils backdoor, MOVEit — was a supply-chain compromise. The attacker did not breach the victim. They tainted something the victim trusted and built upon. The defenses are no longer optional: producing a signed, traceable, attestable software artifact is becoming the baseline expectation, codified in EO 14028, NIST SSDF, and the EU CRA.
The Supply-Chain Threat Model
SLSA's threat model decomposes the chain into source → build → package → consumer, with eight distinct attack vectors at the joints. Memorize this picture; every recent incident maps onto exactly one of these arrows.
What dominates the data
- Dependency confusion / typosquatting — uploading
my-internal-pkgto npm hoping a CI accidentally fetches it. Endemic in npm, PyPI, RubyGems. - Compromised maintainer accounts — phished/MFA-bypassed, then a malicious release. event-stream, ua-parser-js, the entire 2023 PyPI wave.
- Build-system compromise — SolarWinds (Orion), Codecov (bash uploader), Reuters' MOVEit. The attacker writes themselves into the binary you trust.
- Source repo infiltration — slow social engineering of a maintainer position (xz-utils 2024).
- Registry poisoning / cache poisoning — modifying the package between upload and your fetch.
SLSA — Levels and What They Buy
SLSA (Supply-chain Levels for Software Artifacts) is the OpenSSF / Linux Foundation framework. Versions 1.0 (2023) and 1.1 (2024) define a build track with four levels (L0-L3) describing increasing rigor on how build integrity is established. Source and dependency tracks are in development.
| Level | Buys you | Concretely |
|---|---|---|
| L0 | Nothing. | No provenance produced. |
| L1 | Provenance exists. | The build emits a signed metadata document declaring source, builder, and inputs. Caught the SolarWinds class. |
| L2 | Provenance authenticated and from a hosted build platform. | Builder signs with a key only it can use; provenance is non-forgeable. Hosted CI (GitHub Actions, GitLab, Buildkite-managed). |
| L3 | Hardened build platform. | Build runs on isolated, hardened workers; non-falsifiable provenance; build script cannot tamper with provenance signing. |
The reusable workflow path
The fastest way to SLSA L3 today is the slsa-github-generator — a set of reusable GitHub Actions workflows that produce SLSA-L3 provenance for Go binaries, Node packages, container images, and Python wheels. The workflow runs on hosted runners with isolation and emits a Sigstore-signed in-toto attestation alongside your artifact.
SBOM — Software Bill of Materials
You cannot defend what you cannot enumerate. An SBOM lists everything that went into your artifact: direct and transitive dependencies, license, version, hash, optionally provenance. When Log4Shell hit, the teams who recovered fastest were those with SBOMs in their CI artifact registry — they could query "which of our 400 services include log4j-core 2.14.x" in seconds. The teams without SBOMs spent days running grep across container images.
The two formats
- SPDX — Linux Foundation, ISO/IEC 5962. Comprehensive, license-focused.
- CycloneDX — OWASP. Tighter for security use; first-class support for vulnerability and VEX.
Both are widely supported. Generate both if your tooling allows; consumers will ask for whichever the auditor prefers. syft (Anchore), cdxgen, trivy, and Microsoft's sbom-tool all produce them from images, repos, or filesystems.
# 1. Build SBOM from the image we just pushed syft ghcr.io/my-org/payments:1.4.0 -o cyclonedx-json > sbom.cdx.json # 2. Attach as an OCI artifact alongside the image cosign attach sbom \ --sbom sbom.cdx.json \ --type cyclonedx \ ghcr.io/my-org/payments:1.4.0 # 3. Sign the SBOM (keyless, OIDC identity from CI) cosign attest \ --predicate sbom.cdx.json \ --type cyclonedx \ ghcr.io/my-org/payments:1.4.0
VEX — telling the truth about vulnerabilities
An SBOM with raw CVE counts becomes wallpaper. VEX (Vulnerability Exploitability eXchange — NTIA / CycloneDX VEX / OpenVEX) lets you publish per-CVE statements: "in our build, CVE-2024-12345 in libfoo is not_affected because the vulnerable code path is not reached." Tools like Grype consume VEX to suppress the noise. Teams that ship VEX with their releases turn the scanner output back into a useful signal.
Sigstore — Keyless Signing for the Rest of Us
The historical reason almost no one signed software is that running PKI is hard. Sigstore (CNCF graduated, 2024) inverts the problem: no long-lived signing keys. Instead, your CI authenticates to Fulcio using its OIDC identity and gets a short-lived signing certificate (~10 minutes) that binds the signature to that identity. Every signature is logged in Rekor, a public transparency log. Verification looks up the identity, the issuer, and the Rekor entry.
Verification at admission
The full payoff lands at the cluster gate. Kyverno's verifyImages (Day 3) checks Sigstore signatures and rejects any pod referencing an image not signed by the expected OIDC subject — typically your release workflow. The check is a few lines; the payoff is enormous: a stolen registry credential cannot push a working malicious image, because the signing identity will be wrong.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata: { name: signed-and-attested }
spec:
validationFailureAction: Enforce
rules:
- name: keyless-signed-by-our-release-workflow
match: { resources: { kinds: ["Pod"] } }
verifyImages:
- imageReferences: ["ghcr.io/my-org/*"]
attestors:
- entries:
- keyless:
issuer: "https://token.actions.githubusercontent.com"
subject: "https://github.com/my-org/*/.github/workflows/release.yml@refs/heads/main"
attestations:
- type: "https://slsa.dev/provenance/v1"
attestors:
- entries:
- keyless:
issuer: "https://token.actions.githubusercontent.com"
subject: "https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@refs/tags/*"Two requirements: image is signed by our release workflow, and a SLSA provenance attestation exists, signed by the SLSA generator workflow. Forging both requires compromising GitHub's OIDC + the SLSA reusable workflow + Sigstore's transparency log. The attack surface is now "break the entire ecosystem", not "steal a registry token."
in-toto and Attestations
in-toto is the data format Sigstore wraps: a structured statement that something attests some predicate about some subject. The subjects are content-addressable (SHA-256 digests). The predicates are extensible — SLSA provenance, SPDX SBOM, vulnerability scan, code review approval. A modern release ships an attestation bundle alongside the artifact:
- Provenance (SLSA) — built by X workflow at commit Y from inputs Z.
- SBOM (CycloneDX/SPDX) — these are the components.
- Vulnerability scan (Trivy/Grype) — known CVEs at build time.
- VEX — exploitability statements.
- Code review (review-bot attestation) — approved by N reviewers.
- Test results — coverage, integration, fuzzing.
Verifiers like cosign verify-attestation + policy-controller let you write declarative policies: "the image must have an SLSA-L3 provenance, an SBOM, and zero CRITICAL CVEs without VEX." The result is a verifiable claim chain you can audit.
Build Reproducibility
The strongest provenance claim is reproducibility: rebuild from the same source and you get the bit-identical artifact. With reproducibility, anyone can independently verify a build did what its provenance says. Without it, you trust a builder.
Achieving reproducibility means controlling timestamps, file ordering, locale, randomness — the Reproducible Builds project documents techniques per language. Bazel and Nix are the build systems that get this right by construction; Buildah / ko for containers; Debian achieves it for the majority of its packages. For most teams, strict reproducibility is aspirational; prioritize SLSA L3 hosted-builder provenance first, work toward reproducibility for sensitive artifacts later.
Dependency Hygiene
The supply-chain is mostly someone else's source code. The hygiene rules are old and unglamorous:
- Pin dependencies — hash-pinned (npm
integrity, pip--require-hashes, Go's checksum DB, cargoCargo.lock+--frozen). - Vendor or mirror — a private package mirror (Artifactory, Nexus, or AWS CodeArtifact / GCP Artifact Registry) is the only audit point you control. Direct upstream is a third-party trust delegation.
- Restrict scope — each project pulls only the dependencies it declares; no transitive cross-project leaks.
workspace-level restrictions matter at scale. - Auto-update with review — Renovate / Dependabot create PRs; CI runs the full suite; humans approve. The point is not zero-day patching but visibility.
- Beware install-time scripts — npm's
postinstall, Python'ssetup.py, Cargo build scripts.npm ci --ignore-scripts,pip install --no-build-isolationwith reviewed wheels, and prebuilt-only dependencies eliminate huge classes of supply-chain payloads. - Watch the maintainer surface — projects with one maintainer, no recent activity, or recently transferred ownership are higher risk.
scorecard(OpenSSF) automates checks.
@my-org/internal-tool and lives only in your private registry, an attacker can publish internal-tool (no scope) on public npm. A misconfigured CI that searches public first will fetch the malicious one. The fix: always use scoped names, configure registry resolution to private-first or private-only for those scopes, and verify with hash-pinning. Microsoft, Apple, PayPal, Tesla and Yelp all paid security researchers for finding this in 2021.The Producer-Side Paved Road
Pulling everything together, a modern "green" pipeline looks like:
- Source — branch protection on default branch, signed commits (
git config commit.gpgsignor SSH signing), required reviews from CODEOWNERS, no force-push. - CI on a hosted runner — GitHub Actions, GitLab, Buildkite-managed, with an OIDC trust to your cloud and to Sigstore. No long-lived secrets in the runner.
- Hermetic build — Bazel, Nix, or a containerized build with sealed dependencies; offline once started.
- SLSA L3 provenance via
slsa-github-generatoror equivalent, signed by Sigstore. - SBOM generated by
syft, signed and attached as an OCI artifact. - Vulnerability scan (Trivy/Grype) with VEX overlays for known-non-exploitable.
- Push to a private registry, by digest only.
- Admission policy in target clusters: signed by the right OIDC subject, SLSA provenance present, no critical-without-VEX, image age within policy.
- Artifact retention long enough for audit (3-7 years for some regulated industries).
Show answer
repo:my-org/payments-service:ref:refs/heads/main and the workflow path .github/workflows/release.yml. Even if the attacker has a valid token, the workflow they ran is theirs, not yours; the resulting subject would be something like repo:attacker-org/evil:ref:refs/heads/main. Kyverno's subject regex on the verifyImages policy rejects it. This is the core promise of keyless signing — the identity is in the cert, and the cert binds to the actual workflow run.The Consumer-Side Picture
Most engineers are also consumers. Three rules cover most of it.
- Read SBOMs you receive. If a vendor ships an SBOM, ingest it into your asset inventory. When the next Log4Shell hits, your search is across all vendors.
- Verify signatures by default.
cosign verifyin your image-pull script.chainguard imagesship signed by default; expand outward. - Pin third-party Actions and CI plugins. Use full SHA, not a tag. The 2022 ESLint, 2023 cdxgen, and 2024 tj-actions/changed-files compromises all hit teams using floating tags.
- Sign every release with Sigstore (keyless).
- Attest provenance (SLSA), SBOM, scans.
- Scan + VEX to keep noise low.
- Pin by hash; never by floating tag.
- Mirror upstream registries; vendor when stable.
- SLSA — Specification v1.1slsa.dev
- Sigstore — How it workssigstore.dev
- in-toto — Attestation frameworkgithub.com
- CycloneDX — Specification overviewcyclonedx.org
- SPDX — Specificationsspdx.dev
- NIST — Secure Software Development Framework (SSDF)nist.gov
- OpenSSF Scorecard — Open-source project security checksgithub.com
- Reproducible Builds — Projectreproducible-builds.org
Finished reading?