You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Latest run informing this report: runs/2026-05-05T18-49-30Z —
280 evaluations across the 9-wrapper × 36-fixture matrix against
mikebom v0.1.0-alpha.14. This run uses scripts/run_with_sbomqs.py,
so per-pair sbomqs scoring is captured (Section 10). Per-tool totals
raw findings are in cross-tool-summary.md in this gist.
The prior alpha.13 baseline (runs/2026-05-04T02-50-56Z/) is
preserved on-disk for diff'ing; numerics in sections 1–9 are
unchanged within float-precision noise (only mikebom's row
moved, by 2 of 12,165 findings — see the alpha.13 → alpha.14
delta block below).
alpha.13 → alpha.14 mikebom-side delta (2026-05-05): the
smallest pin bump shipped under the protocol so far. Upstream
CHANGELOG declares: "No SBOM output shape change. CDX 1.6 /
SPDX 2.3 / SPDX 3.0.1 emissions are byte-identical to alpha.13
(all 27 byte-identity goldens unchanged)."
What alpha.14 actually ships (none of which changes
emission shape):
Granular network-enrichment skip flags (mikebom PR #136):
--no-clearly-defined, --no-deps-dev-graph,
--enrich-sources. The conformance wrapper does NOT use
these flags; runs continue with full enrichment for the
published benchmark.
mikebom sbom parity-check upgraded to real value-equality
(mikebom milestone 071, PR #138): pre-071 the CLI subcommand
checked only presence per universal-parity row, so a row
where CDX had ["go.sum"] and SPDX 2.3 had ["DRIFTED.sum"]
reported Parity gaps: 0 because both were non-empty.
Post-071 it applies real set-equality (SymmetricEqual),
subset (CdxSubsetOfSpdx), presence-parity (PresenceOnly),
CDX-non-empty (CdxOnly) invariants. No effect on this
conformance harness — we don't shell out to parity-check;
we run our own compare/annotations.py. Recorded for
cross-reference for external harnesses that DO shell out.
docs/reference/conformance-harness-guide.md (mikebom
milestone 071): a 7-section reference for external SBOM-
conformance harness authors. Covers (1) how mikebom:*
metadata is carried per format (CDX properties[], SPDX 2.3
SPDX 3 MikebomAnnotationCommentV1 envelope), (2) the 7
inherent format-spec asymmetries (catalog rows A12 / B4 /
C19 / C22 / C42 / D1 / E1) that should NOT be flagged as
cross-format-inequivalence findings, (3) canonical
directionality-aware comparison logic, (4) 8 false-positive
patterns to suppress (envelope decode, delimiter-joined
strings, PURL escape conventions, etc.), (5) mikebom-specific
quirks + known weaknesses, (6) Python decoder example + jq
recipes + recommended wiring. This conformance harness has
NOT yet adopted the guide — the 11,130 CFI count on mikebom
rows in this report reflects pre-guide harness behavior.
Adopting the guide is harness-side work tracked separately
and will produce a materially smaller CFI count on a future
re-run; that work is NOT in scope for this pin bump.
Independent verification on our 36-fixture set:
runs/2026-05-05T18-49-30Z/ (alpha.14) vs
runs/2026-05-04T02-50-56Z/ (alpha.13):
Non-mikebom rows (9 tools × 36 fixtures): 0
finding-count differences. As expected — alpha.14 only
changes mikebom.
mikebom rows: 35/36 fixtures show identical finding
counts AND identical per-kind histograms. One delta:
polyglot-builder-image 1471 → 1473 findings (+2
CROSS_FORMAT_INEQUIVALENCE). The +2 lands entirely on
mikebom:sbom-tier: the key's CFI count went from 1 → 3.
The 2 net-new findings show CDX vs SPDX 3 disagreement on
the sbom-tier value: one component reports
CDX="analyzed" / SPDX 3="source", another reports the
reverse. Whether this is alpha.13→alpha.14 emission drift on
a corner-case component or per-run scan-order
non-determinism that landed on different components this run
is unclear without a third run. Either way: tiny scope (2 of
1473 findings = 0.14% of polyglot's mikebom rows) and
isolated to one annotation key.
Reconciliation with the upstream "byte-identical" claim:
mikebom's CHANGELOG asserts byte-identity across its 27
internal byte-identity goldens. polyglot-builder-image is
our complex polyglot fixture and is almost certainly NOT in
those 27 goldens (it's bigger and more multi-ecosystem than
any of mikebom's internal byte-identity tests). So the
upstream claim is consistent with this result; the 2-finding
drift on polyglot is on a fixture that extends beyond
mikebom's golden set.
mikebom sbomqs mean: 7.622 → 7.627 (+0.005). Within
float-precision noise; the +2 findings on polyglot don't
move the aggregate score meaningfully. Other tools'
per-wrapper sbomqs averages are byte-identical across runs.
Verdict: pin bump verified safe. Conformance numbers are
essentially unchanged; the published benchmark narrative does
not need a substantive refresh for this bump alone. The next
refresh-worthy event is harness-side adoption of the
conformance-harness-guide (separate work).
alpha.9 → alpha.13 mikebom-side delta (2026-05-04): four
upstream releases shipping the largest set of SBOM-shape changes
since alpha.6. Cross-run comparisons across this bump WILL differ
materially on every fixture in every covered ecosystem. Headline
upstream changes:
alpha.10: filesystem-walker symlink-loop hang fixed
(mikebom #102/#110); Go source-tree synthetic main-module
component (metadata.component.purl flips from
pkg:generic/...@0.0.0 to pkg:golang/<module-path>@<version>);
default scan emits ALL lifecycle scopes (Dev/Build/Test no
longer silently dropped); the legacy mikebom:dev-dependency
annotation is REMOVED in favor of CDX scope: "excluded" /
SPDX *_DEPENDENCY_OF / SPDX 3 lifecycleScope;
--include-dev deprecated to a parse-and-warn shim.
alpha.11: Go transitive-edge resolver (4-step ladder over
go mod graph → $GOMODCACHE → proxy.golang.org → no-edges
fallthrough); Go main-module's dependsOn set shrinks to
direct (non-// indirect) requires only — for the simple-
module fixture, edges drop from 10 → 5; new
mikebom:graph-completeness document-level annotation
(complete / partial) + mikebom:graph-completeness-reason
free-text + per-component mikebom:orphan-reason (catalog
rows C44 + C45); Go main-module LICENSE detection (Layer 1).
alpha.12: cargo + npm main-module components (PURLs flip
to pkg:cargo/<name>@<version> and pkg:npm/<name>@<version>
/ pkg:npm/%40scope/name@version); mikebom:component-role: main-module (C40) extended to both ecosystems; cargo
workspace-root direct-dep edges; multi-main-module super-root
with plural DESCRIBES.
alpha.13 ("issue #104 closure release"): pip + gem + maven
main-modules (PURLs flip to pkg:pypi/<pep503-name>@<version>,
pkg:gem/<name>@<version>,
pkg:maven/<groupId>/<artifactId>@<version>); Maven
multi-module reactor support with POM inheritance + property
substitution. Per-ecosystem main-module matrix is now
COMPLETE: Go ✅, cargo ✅, npm ✅, pip ✅, gem ✅, maven ✅.
Empirical impact on this run:
polyglot-builder-image per-ecosystem coverage: mikebom
now reports full coverage on every ecosystem bucket
(cargo 11/11, gem 76/76, maven 108/108, pypi 2/2, golang 2/2,
rpm 529/529, binary 2/2). Pre-alpha.13 mikebom lacked main-
modules in cargo/gem/maven/pypi entirely. trivy on the same
fixture still shows gem 7/76 and maven 53/108.
PURL conformance: PURL_MISMATCH counts unchanged by this
bump (alpha.10–13 didn't touch deb/apk/rpm-side PURL behavior).
mikebom retains its 17-MISMATCH lead over trivy (1164),
cdxgen (684), bom (680), syft (280), scalibr (194).
mikebom resilience: 36/36 fixtures evaluated (1 tool
failure, 1 schema failure) — the most resilient tool in the
matrix. Compare syft 20 tool failures, trivy 17, cdxgen 10,
cyclonedx-py 15.
Total findings on mikebom rows: 12,165 across 36 fixtures
(9 finding kinds). 11,130 of those are
CROSS_FORMAT_INEQUIVALENCE — mikebom is the only tool
emitting all 3 formats (CDX 1.6 + SPDX 2.3 + SPDX 3), so it's
the only tool the framework can perform cross-format equivalence
on. By design, this is mikebom-only and reflects element-
fidelity gaps the multi-format work is incrementally closing.
Constitution Principle VI / polyglot-builder-image: still
silent-pass. mikebom emits [pre-build, post-build, operations]
on the image; strongest phase operations (rank 5) ≥
deployed (rank 4). The new alpha.13 main-module work doesn't
change the lifecycle envelope.
Open follow-ups carried over from alpha.9 (and where the
alpha.10–13 cycle landed them):
Bundled-jar role propagation in polyglot Maven (3 jars
previously FALSE_POSITIVE: maven-artifact, commons-cli,
slf4j-simple). Resolved by alpha.13 — every Maven jar
now carries the milestone-070 main-module identity, so the
bundled jars self-identify as their own packages with
pkg:maven/<g>/<a>@<v> PURLs and the
mikebom:component-role: main-module (C40) tag.
is_dev flag wiring in mikebom's Go resolve pipeline.
Subsumed by alpha.10/11's standards-native lifecycle-scope
tagging (CDX scope: "excluded" / SPDX *_DEPENDENCY_OF /
SPDX 3 lifecycleScope) — the legacy mikebom:dev-dependency
annotation no longer exists, and is_dev as a free-standing
field has been retired.
SPDX-side serialization parity for the new alpha.9
Go-annotation channels. Still partially open: cross-format
inequivalence count remains the single largest finding bucket
on mikebom rows (11,130). Each new main-module + lifecycle
channel adds new payloads that need symmetric emission across
all 3 formats.
alpha.8 → alpha.9 mikebom-side delta (2026-05-01): alpha.9
ships feat(049) Go transitive-closure resolution: mikebom now
reads go.sum for the full module closure rather than only
go.mod direct requires. ClearlyDefined Go-module license
enrichment also lands. Empirical impact:
Go fixtures gain transitive-closure components:
go-logrus-small mikebom emission goes from 3 components
(alpha.8 — go.mod direct only) to 7 components (alpha.9 —
go.sum transitive closure). On the apigatewayv2/config Go
service that motivated this work outside the fixture set:
6 (alpha.8) → 63 (alpha.9), finally aligning with trivy's
55 and syft's parity output. This is the first release where
mikebom's Go-side completeness meaningfully meets Constitution
Principle II for Go fixtures.
mikebom:source-files property now points at go.sum
rather than go.mod for Go fixtures (the canonical signal of
the transitive-closure shift).
mikebom total findings: 10,209 → 10,420 (+211): the
absolute increase is small because most of the new components
are well-formed (correct PURLs, no annotations missing). The
delta breakdown:
CROSS_FORMAT_INEQUIVALENCE9,580 → 9,752 (+172) —
new components surface SPDX-vs-CDX format-serialization
parity gaps that mikebom-side hasn't fully closed yet for
the new Go-side annotation channels.
FALSE_POSITIVE544 → 585 (+41) — alpha.9 finds the
test-only deps subgraph (testify-transitive: davecgh/go-spew,
pmezard/go-difflib) that ground-truth files don't yet list.
The feat(049) PR title mentions "test-vs-prod tagging via
is_dev" but the alpha.9 binary's resolve pipeline doesn't
yet populate is_dev: Some(true) — the field exists in the
Component model and emitters, but every emit-site constructs
is_dev: None. Filtering test-only deps from the default
emission (or surfacing via SBOM-side annotation) is a known
follow-up.
MISSING_COMPONENT16 → 14 (-2) — minor improvement
from the wider Go closure catching previously-missed deps.
Total all-tools findings: 28,598 → 28,809 (+211, all on
mikebom rows). Other tools' rows are byte-identical to the
alpha.8 baseline since none of the 9 non-mikebom tools changed
between runs.
Constitution Principle VI (feature 007) is the broader
shift this release operationalizes. The principle declares
that an SBOM MUST describe the software artifact, not merely
its manifest. alpha.9's Go transitive-closure work is the
first concrete tool-side step toward Principle VI compliance
for Go fixtures. The LIFECYCLE_SCOPE_MISMATCH finding kind
doesn't fire in this run (no fixture's expected_lifecycle
is yet set narrower than mikebom's emission); the canonical
polyglot-builder-image is annotated expected_lifecycle: deployed, mikebom emits [pre-build, post-build, operations],
and the strongest phase operations (rank 5) ≥ deployed
(rank 4) — silent pass.
Open follow-ups carried over from alpha.8:
Bundled-jar role propagation in polyglot Maven (3 jars still
FALSE_POSITIVE: maven-artifact, commons-cli,
slf4j-simple — the role lives on the parent RPM, the bundled
jars don't yet inherit).
is_dev flag wiring in mikebom's Go resolve pipeline (the
field exists; emitters always set None today; the deps-dev
match data is captured but not converted to is_dev=true).
SPDX-side serialization parity for the new alpha.9
Go-annotation channels (mikebom:cpe-candidates,
mikebom:deps-dev-match, mikebom:sbom-tier,
mikebom:source-files).
2026-05-01 (post-loader-fix correction): the alpha.7 →
alpha.8 review surfaced two bugs in the conformance suite's own
SPDX loaders that were inflating the CROSS_FORMAT_INEQUIVALENCE
count for years. Both fixed (commits 3a1dda6 + e9195be):
Bug 1: SPDX 2.3 loader expected mikebom:<key>=<value>
comment form; mikebom switched to a MikebomAnnotationCommentV1
JSON envelope at alpha.6. The loader was silently dropping
100% of mikebom's SPDX 2.3 annotations.
Bug 2: SPDX 3 loader looked for annotations at top-level
data["annotations"]; mikebom emits them as @graph[] entries
with type: "Annotation" (the canonical SPDX 3 JSON-LD shape).
Loader was dropping all 4,861 SPDX 3 annotations on polyglot.
Net impact on the published numbers:
CROSS_FORMAT_INEQUIVALENCE 11,692 → 9,580 (-18%, -2,112
findings, all on mikebom rows). Total findings 30,710 →
28,598. Other finding kinds unchanged. Clean_run pair count
unchanged (21/280) — none of the affected pairs were blocked
solely by cross-format. Mikebom's element-fidelity picture has
been quietly worse-than-real for the entire prior gist run; the
remaining 9,580 cross-format findings are real (genuinely
divergent values across formats, e.g., the new
mikebom:component-role is CDX-only — SPDX serializer in
alpha.8 doesn't yet emit it for ALL components, surfacing as a
finding per affected component).
alpha.7 → alpha.8 mikebom-side delta: alpha.8 ships
mikebom:component-role on RPM-level packages
(build-tool / language-runtime) on top of alpha.7's
baseline. Conformance gates unchanged (no FALSE_POSITIVE
reduction yet — see polyglot-Maven follow-up below).
Two follow-ups still open to close the polyglot Maven FP
loop that motivated alpha.8: (a) propagate component-role from
parent RPM (maven-lib) to bundled Maven jars (maven-artifact,
commons-cli, slf4j-simple) — currently the role lives at the
RPM layer only and the bundled jars still surface as
FALSE_POSITIVE; (b) (RESOLVED via the loader fix above —
alpha.7 already emitted component-role to all 3 formats; the
CDX/SPDX gap was on the conformance-suite side, not mikebom).
What changed in feature 006: 138 annotation
findings flipped from severity: blocking to severity: advisory
for mikebom:-namespaced vendor-extension keys (elf-build-id,
pe-machine, macho-uuid, etc.) plus the two extended-quality-
signal keys (evidence-kind, confidence). Constitution
Principle V says these "SHOULD be tested when feasible but are
not blocking criteria for initial conformance results"; the suite
previously enforced them as blocking, which systematically failed
non-mikebom tools for not emitting mikebom-specific extensions.
The signal is preserved (advisory findings still surface in the
report); the gate is constitutionally aligned. Pair-level
clean_run counts are mostly unchanged because the affected
pairs also have non-annotation blocking findings (PURL_MISMATCH,
MISSING_COMPONENT, etc.) — a separate concern. Section 6 (PURL
conformance) was last refreshed in feature 004; sections 1, 2,
3, 4, 5, 7, 8, 9, 10, 11, 12 are carried over from the 2026-04-20
baseline.
1. Completeness (No False Negatives)
Does the tool find everything that's actually present?
Directory Scans
Ecosystem
Fixture
Expected
syft
trivy
cdxgen
scalibr
sbom-tool
bom
mikebom
pypi
python-flask-small
7
7/7
7/7
7/7
7/7
7/7
0/7
7/7
npm
node-express-small
68
68/68
68/68
68/68
69/68*
68/68
0/68
68/68
golang
go-logrus-small
3
3/3
3/3
3/3
4/3*
3/3
3/3
7/3*
maven
java-maven-small
3
4/3*
12/3*
11/3*
3/3
27/3*
—
12/3*
cargo
rust-serde-small
3
12/3*
12/3*
12/3*
0/3
—
—
11/3*
gem
ruby-sinatra-small
7
7/7
7/7
7/7
0/7
—
—
7/7
deb
debian-bookworm
88
88/88†
0/88‡
88/88†
88/88
—
0/88
88/88
deb
ubuntu-24.04
92
—§
—§
—§
92/92
—
—
92/92
rpm
rocky-9
142
—§
—§
—§
0/142
—
—
142/142
apk
alpine-3.20
14
14/14
0/14‡
—§
14/14
—
—
14/14
* Over-reports: includes transitive deps or project root beyond ground-truth direct deps
† syft emits PURLs from dir scan when os-release is present
‡ trivy needs image or rootfs mode for OS package PURL emission from directories
§ Not tested in this configuration
Image Scans (Container Fixtures)
Fixture
Ecosystem
Expected
syft
trivy
cdxgen
mikebom
multi-ecosystem
deb
140
140
140
206
140
multi-ecosystem
pypi
12+
12/12
12/12
12/12
15/15
multi-ecosystem
npm
68+
272
272
258
70
multi-ecosystem
binary (jq)
1
1/1
0/1
0/1
1/1
go-binary-scratch
golang
3
3/3
3/3
0/3
3/3
base-image-layers
deb
88
88/88
88/88
88/88
88/88
polyglot-builder
golang
2
—¶¶
2/2
2/2
23/2*
polyglot-builder
maven
46
—¶¶
46/46
0/46**
7/46¶
polyglot-builder
cargo
11
—¶¶
11/11
11/11
11/11
polyglot-builder
gem
12
—¶¶
7/12
12/12
12/12
polyglot-builder
rpm
529
—¶¶
528/529
0/529¶¶¶
526/529
java-maven-image
maven
10
10/10
10/10
0/10**
10/10 (+2 FP)
* cdxgen over-reports on Fedora (373 pkg:generic/ fallback entries)
§ Not tested in this configuration
¶ mikebom finds 7 maven coords in packed JARs via JAR-embedded POM walk;
trivy additionally reads the .m2 cache and picks up all 46 build-time
deps. Scope decision: mikebom intentionally reports what's packed.
¶¶ syft wrapper currently fails on polyglot (exit 1, colima docker-socket
compatibility); tool is capable on this fixture when the environment
works — historical syft polyglot scans see 0 cargo, 12 gem, 2 golang,
46 maven, 528 rpm.
¶¶¶ cdxgen emits pkg:rpm/rpm/… instead of pkg:rpm/fedora/… on Fedora
— produces 621 rpm PURLs but 0 match GT's canonical pkg:rpm/fedora/…
form per the rpm type definition.
Note on npm counts: syft/trivy report 272-273 npm packages because they
include npm's own internal dependencies (/usr/local/lib/node_modules/npm/ node_modules/). mikebom reports 70 — the application's express dependency
tree only. Both are valid views; mikebom's is scoped to what the
application uses.
Key findings (updated with comprehensive ground truths):
polyglot-builder-image (600 components across 5 ecosystems): trivy
and mikebom detect all 5 ecosystems. syft misses Cargo. cdxgen fails.
mikebom exact match: 11/11 cargo, 12/12 gem, 2/2 golang, 7/46 maven
(37 maven coords pruned by shade plugin), 526/529 rpm.
java-maven-image (146 components: 136 deb + 10 maven): All tools
detect Maven deps in shaded JARs. mikebom uses deb/ubuntu/ namespace
and distro=ubuntu-24.04 matching the ground truth on Ubuntu images.
native-linkage / multi-stage-build (111-115 components): mikebom
achieves near-clean runs (2 FP each from binary residuals). All three
tools (mikebom, syft, trivy) now emit distro=debian-12
({ID}-{VERSION_ID} form from /etc/os-release). This follows
Philippe Ombredanne's explicit recommendation in purl-spec
#423, even
though the deb type definition examples and the KubeCon Europe 2026
slide's "Standard" reference both showed the codename form
(distro=bookworm). The KubeCon slide does not reflect the path
forward the spec is converging on — it shows the codename form,
while the maintainer recommendation is the debian-12-style version
number. The spec has not yet normatively landed this; it remains an
open question targeting PURL spec v1.1.
Binary analysis: mikebom detects the curl'd jq binary (only
syft + mikebom can). Binary scanner deduplicates against package-db
owned files — 2-3 pkg:generic/ residuals per Debian image.
go-binary-scratch: mikebom 3/3 Go modules in scratch (+ 1 generic FP).
2. Accuracy (No False Positives)
Does the tool report things that aren't really there?
Source of false positives
syft
trivy
cdxgen
scalibr
sbom-tool
mikebom
Project root as component (npm)
1
0
0
1
1
0
Project root as component (Go)
0
1
0
1
0
0
pkg:generic/ fallback (Fedora rpm)
0
0
373
0
0
0
Artefact entry (requirements.txt)
filtered
filtered
filtered
0
0
0
SWID self-reference
0
0
0
0
1
0
Version mismatches
0
0
0
0
0
1
Key finding: Near-zero version mismatches across all tools on all
fixtures. The single mikebom mismatch is slf4j-api@1.7.36 vs 1.7.32
on polyglot (Maven's transitive resolution picked a different version
than pom.xml declared — this is a GT/packed-JAR alignment issue, not a
mikebom bug). False positives otherwise come from structural
disagreements (project-root-as-component, generic fallbacks), not from
wrong data.
3. Dependency Relationships
Can you build a dependency graph from the SBOM?
Directory Scans
Source format
syft
trivy
cdxgen
scalibr
mikebom
package-lock.json (npm)
0 entries
70 entries, 29 with dependsOn
69 entries, 28 with dependsOn
0 entries
69 entries, 28 with dependsOn
go.mod (Go)
0 entries
6 entries, real tree
0 entries
0 entries
8 entries, 4 with dependsOn
requirements.txt (Python)
0 entries
9 entries, 2 with dependsOn (flat/fake)
1 entry, 0 dependsOn
0 entries
8 entries, 1 with dependsOn
pom.xml (Maven)
0 entries
14 entries, 5 with dependsOn
—
0 entries
13 entries, 4 with dependsOn
Cargo.lock (Rust)
0 entries
—
—
0 entries
12 entries, 8 with dependsOn
Gemfile.lock (Ruby)
0 entries
—
—
0 entries
8 entries, 4 with dependsOn
Image Scans (deb packages, base-image-layers 88-package deb subset)
Tool
Dependency entries
With dependsOn
trivy
88
67
mikebom
88
67
syft
—†
—†
scalibr
0
0
† syft currently fails on image fixtures in this run; historical syft
deb-image scans emit 88 entries each with dependsOn.
Dependency Tree Reality
Scenario
trivy
syft
mikebom
Others
npm from lockfile
Real tree (127 refs)
Absent
Real tree (28 nodes with edges)
cdxgen: partial
Go from go.mod
Real tree
Absent
Real tree (4 nodes with edges)
Absent
Maven from pom.xml
Real tree (5 nodes)
Absent
Real tree (4 nodes, full transitive)
Absent
Cargo from Cargo.lock
Absent
Absent
Real tree (8 nodes, always complete)
Absent
Ruby from Gemfile.lock
Absent
Absent
Real tree (4 nodes with edges)
Absent
Python from requirements.txt
Fake flat (all direct)
Absent
Fake flat (all direct)†
Absent
deb from image scan
Real (67 entries with dependsOn)
— (failed this run)
Real (67 entries with dependsOn)
Absent
† mikebom produces real Python dep trees from installed .dist-info/METADATARequires-Dist on image scans — the fake-flat output here is the dir-scan case where only requirements.txt is present (which by its nature lists only direct deps).
Key finding: mikebom now produces real dependency trees for 7
ecosystems from directory scans: npm (from package-lock.json), Go
(from module cache .mod files), Maven (from .m2 cache POM walk +
JAR-embedded POMs), Cargo (from Cargo.lock — always complete), Ruby
(from Gemfile.lock indent structure), and deb/rpm from package
databases. trivy produces dep trees for npm and Go from dir scans.
No other tool produces dep trees for Maven, Cargo, or Ruby from dir
scans. On image scans, syft, trivy, and mikebom all produce real
dependency trees from dpkg's Depends: fields. mikebom's Maven
image-scan dep tree is structured (real parent→child edges) vs
trivy's flat dump (all deps as children of the image root). No tool
except mikebom declares when its tree data is complete vs incomplete
(via compositions).
4. Metadata Quality
Beyond the package list: what else does the SBOM contain?
Image Scans (deb packages, base-image-layers — apples-to-apples, same 88-package deb subset)
Field
trivy
mikebom
syft†
scalibr‡
Licenses
85/88
88/88
—
—
Hashes
0/88
88/88
—
—
Supplier
88/88
88/88
—
—
CPE
0/88
88/88
—
—
VEX
0/88
0/88
—
—
Evidence + confidence
0/88
88/88
—
—
File occurrences
0/88
88/88
—
—
† syft currently fails on image fixtures in this run (wrapper exit 1,
colima docker-socket compatibility); historical syft image scans
produced 88/88 licenses + 88/88 CPEs + 0/88 hashes + 0/88 supplier.
‡ scalibr is directory-only by design.
Key finding: Directory scans produce zero metadata beyond the
package list (except syft's CPEs). Image scans produce rich metadata
from syft, trivy, and mikebom — licenses, hashes, supplier, dependency
trees — because the full dpkg database and copyright files are
available. mikebom is the only tool providing evidence with confidence
scores and per-file occurrence data, and is now 88/88 on CPE for the
deb subset where it previously was 0/88.
VEX is absent from every tool in every context.
5. SBOM Spec Compliance
Does the output validate against CycloneDX/SPDX schemas?
Tool
Format
Schema valid on all fixtures?
syft
CycloneDX 1.6
Yes
trivy 0.69
CycloneDX 1.6
Yes
trivy 0.55
CycloneDX 1.6
Yes
cdxgen
CycloneDX 1.6
Yes
sbom-tool
SPDX 2.2
Yes
bom
SPDX 2.3
Yes
cyclonedx-gomod
CycloneDX 1.6
Yes
cyclonedx-py
CycloneDX 1.6
Yes
scalibr
CycloneDX 1.6
Yes
mikebom
CycloneDX 1.6
Yes
Every tool produces structurally valid output. The schema is a
necessary but insufficient quality check — it validates structure,
not content correctness.
6. Sub-Spec Compliance (PURL and License Identifiers)
Are the PURLs inside the SBOM correct per the PURL spec?
PURL Conformance by Ecosystem
Ecosystem
Ground truth
syft
trivy
cdxgen
scalibr
bom
mikebom
pypi (7 pkgs)
7
7/7
7/7
7/7
7/7
—
7/7
npm (68 pkgs)
68
68/68
68/68
68/68
68/68
—
68/68
golang (3 pkgs)
3
3/3
3/3
3/3
0/3*
3/3
3/3
maven (3 pkgs)
3
3/3
3/3
0/3**
3/3
—
3/3
cargo (3 pkgs)
3
3/3
3/3
3/3
—
—
3/3
gem (7 pkgs)
7
7/7
7/7
7/7
—
—
7/7
apk (14 pkgs)
14
14/14
14/14
14/14
14/14
—
14/14
rpm-Fedora (polyglot, 529 pkgs)
529
—§
—§
—§
—
—
526/529
rpm-Rocky (142 pkgs)
142
0/142
142/142
0/142
0/142
—
141/142
deb-Debian (88)
88
77/88
88/88
0/88
77/88
0/88
88/88
deb-Ubuntu (92)
92
24/92
0/92
0/92
0/92
—
92/92
§ Not tested in this configuration — tools other than mikebom/trivy do
not emit rpm PURLs from dir scans of rpm-based fixtures (polyglot is
image-mode, so only tools with image support are measured there)
Polyglot-builder-image rpm-Fedora detail (529 Fedora rpms, against
fixtures/polyglot-builder-image/ground-truth.yaml, run
2026-04-21T15-38-39Z):
The ghost pkg:rpm/rpm/... entries (50 in prior runs) and the broad
version-string mismatches (93 in prior runs) have been resolved. Three
packages still emit in a form that doesn't exact-match GT; worth
isolating separately.
Note on deb-Ubuntu measurement: numbers reflect exact PURL string match
against the corrected ground truth (run 2026-04-20T22-16-29Z). The
Ubuntu GT was regenerated to percent-encode + as %2B per purl-spec
docs/standard/specification.md § Character encoding (+ is not in
the allowed set) and the deb-definition.md example
pkg:deb/debian/attr@1:2.4.47-2%2Bb1. syft's 24/92 reflects %3A epoch
encoding + upstream= qualifier on 68 packages; trivy/cdxgen/bom emit
no deb PURLs from dir scans; scalibr emits distro=noble instead of
the GT's distro=ubuntu-24.04.
* scalibr encodes / as %2F in Go module paths and drops v prefix
** cdxgen adds ?type=jar qualifier not in spec
Deb PURL Violations (88 Debian packages)
Deb PURL Conformance vs Reference Implementation
Measured by parsing each PURL with packageurl-python and checking
if to_string() round-trips identically. The reference implementation
percent-encodes + as %2B in both name and version.
Tool
Match reference impl
Non-conformant PURLs
Violations
trivy
88/88 (100%)
0
None — matches reference impl exactly
mikebom
88/88 (100%)
0
None — matches reference impl exactly
syft
77/88 (88%)
11
Epoch : encoded as %3A (ref impl keeps literal)
scalibr
77/88 (88%)
11
Same epoch %3A encoding issue as syft
Note: the 77/88 for syft/scalibr means the 11 packages with epochs
fail; all other PURLs match. The distro value and + encoding —
which we previously flagged as violations — actually match the
reference implementation's conventions (syft uses debian-12 which
packageurl-python accepts; %2B for + is what the reference
impl produces).
KubeCon Slide Validation
Tested with the exact package from the KubeCon Europe 2026 slide:
python3-magics++ version 2:1.5.8-1 — a package with ++ in its
name AND an epoch in its version (the hardest PURL test case).
mikebom matches the slide on %2B%2B name encoding and literal epoch
2: in the version string. The slide's distro=bookworm does not
match what any current tool emits, including mikebom — every tool
uses some form of the version number (debian-12 / debian-12.13)
rather than the codename.
⚠️The KubeCon slide's "Standard" used the wrong distro form. The
slide showed distro=bookworm (codename), but per purl-spec
#423 Philippe
Ombredanne (purl-spec maintainer) explicitly recommends
{ID}-{VERSION_ID} from /etc/os-release — i.e. distro=debian-12.
The codename-form example in the slide is not the path the spec
is converging on. The deb type definition page in the spec repo also
still shows codename examples (distro=jessie), but the open
recommendation in #423 is to move to version numbers. mikebom, syft,
and trivy all emit the version-number form today; none reproduce the
slide's codename literally. The spec is still silent normatively, but
bookworm should be treated as historical/non-recommended once #423
lands.
Key finding: trivy and mikebom both achieve 100% reference impl
conformance on the standard 88-package debian:bookworm set. On the
KubeCon slide's specific test case (epoch + ++), mikebom matches the
slide's name + epoch encoding (literal 2: in version) more closely
than the others — but on the distro= value, all three tools diverge
from the slide and instead follow the #423 maintainer recommendation
(version numbers). syft and scalibr fail on epoch encoding.
Application ecosystems (pypi, npm, Go, Cargo, Gem) are 100% conformant
across all tools that support them.
Cross-Tool PURL Agreement (qualitative)
Ecosystem
Do any two different tools produce identical PURLs?
pypi
Yes — all tools agree
npm
Yes — all tools agree
golang
Yes — 5/6 agree (cyclonedx-gomod adds qualifiers)
maven
No — cdxgen adds ?type=jar
cargo
Yes — all tools agree
gem
Yes — all tools agree
apk
No — distro value differs (syft: alpine-3.20.9, trivy: 3.20.9)
rpm
No — syft adds upstream qualifier
deb
No — every tool produces different PURLs
PURL_MISMATCH counts vs spec-correct ground truth (quantitative)
Framework feature (added 2026-04-29): When the matcher pairs a
ground-truth component with a tool-emitted component via the
(ecosystem, name, version) fallback path AND the two PURLs
normalize to different strings, the framework now emits a
PURL_MISMATCH finding with both raw and normalized PURLs in the
details payload. Before this addition, such divergences were silently
bridged by the matcher and only documented in
specs/003-binary-intro-fixtures/disagreements.md as a side-channel.
The numbers below are the first run that surfaces them as discrete
findings.
Each cell is the count of PURL_MISMATCH findings the tool emits
against ground truth in that ecosystem across all fixtures it ran. A
zero means the tool's emission matches the spec-cited form.
Ecosystem
bom
cdxgen
mikebom
scalibr
syft
trivy
apk
0
0
0
14
8
14
cargo
0
0
4 †
0
0
8
deb
680
680
0
180
130
1114
gem
0
0
0
0
0
4
golang
0
0
0
0
0
4
maven
0
4
0
0
0
16
npm
0
0
13 ‡
0
0
2
rpm
0
0
0
0
142
2
Total
680
684
17
194
280
1164
(cyclonedx-gomod and cyclonedx-py rows omitted: 0 PURL_MISMATCHes,
both tools failed on most fixtures because they require ecosystem-
specific manifests their target fixtures didn't have — see Tool
Failures in cross-tool-summary.md.)
† mikebom emits ?source=crates.io on registry-source cargo crates.
The PURL spec's cargo-definition.md lists crates.io as the default
repository and provides examples without the qualifier
(pkg:cargo/rand@0.7.2); the qualifier is non-canonical. This is
disagreement D1 in specs/003-binary-intro-fixtures/disagreements.md.
‡ mikebom's 13 npm PURL_MISMATCHes are all from base-image-layers —
duplicate package versions in nested node_modules/ (e.g. minipass
appearing as 3.3.6 AND 7.1.2). The matcher's fallback by
(ecosystem, name) only finds one; the chosen pairing then disagrees
on version. This is a matcher-side semantic, not a mikebom emission
flaw.
Patterns by tool
How each tool's deb PURL form diverges from the spec-correct GT form
(pkg:deb/<distro>/<name>@<version>?arch=<arch>&distro={ID}-{VERSION_ID},
e.g. ?distro=debian-12 per purl-spec#423):
For rpm (Rocky/Fedora), syft is the heaviest emitter of non-canonical
PURLs (142): every component carries an &upstream=<src.rpm> vendor
extension. Per the rpm-definition.md examples, rpm PURLs should be
pkg:rpm/<distro>/<name>@<version>?arch=<arch> — no upstream.
mikebom and trivy emit the spec-cited form.
7. Transparency
Does the tool honestly declare its own limitations?
Signal
syft
trivy
cdxgen
scalibr
sbom-tool
bom
cx-gomod
cx-py
mikebom
Uses compositions?
No
No
No
No
No
No
No
No
Yes (3–5 entries per SBOM)
Emits metadata.lifecycles[] (CDX 1.6)?
No
No
No
No
No
No
No
No
Yes (3 phases on polyglot: operations, post-build, pre-build)
Yes (mikebom:sbom-tier annotation on every component, all 3 formats)
Per-component role classification?
No
No
No
No
No
No
No
No
Yes (mikebom:component-role — build-tool / language-runtime / absent — alpha.8+; CDX-only today, SPDX parity is a follow-up)
Records detection method?
Yes†
Partial††
No
No
No
No
No
No
Yes†††
Evidence + confidence?
No
No
No
No
No
No
No
No
Yes (0.85/0.70)
Signals "deps unknown"?
No
No
No
No
No
No
No
No
Yes (via compositions)
Signals "ecosystem not scanned"?
No
No
No
No
No
No
No
No
Yes (incomplete_first_party_only)
Distinguishes real vs fake dep tree?
No
No
No
No
No
No
No
No
Yes (complete vs incomplete)
† syft records syft:package:foundBy per component (e.g., dpkgdb-cataloger, python-installed-package-cataloger, binary-cataloger)
†† trivy records aquasecurity:trivy:PkgType per component (e.g., os, library)
††† mikebom records technique (manifest-analysis, filename, instrumentation) + numerical confidence per component via evidence.identity
Key finding: mikebom emits three independent self-describing
channels for SBOM scope: CDX metadata.lifecycles[] (lifecycle
phases observed), CDX compositions[] (per-aggregate completeness
level), and a per-component mikebom:sbom-tier annotation visible
in all three output formats. As of alpha.7, SPDX 2.3 readers see
the same scope info CDX readers do via creationInfo.comment —
closing the cross-format parity gap that existed in alpha.6 (the
audit conversation 2026-04-30 surfaced this).
The remaining 9 tools provide no self-assessment of completeness,
confidence, or lifecycle scope. mikebom is also the only tool
providing per-component confidence scores via the evidence block.
8. Context Awareness
Does the tool handle real-world scenarios correctly?
Scenario
Fixture
syft
trivy
cdxgen
scalibr
mikebom
Dev deps excluded from prod scan
node-dev-vs-prod
Pass (68 prod)
Pass (67 prod)
Fail (309 all)
Fail (322 all)
Pass (68 prod)
Multi-stage: final image only
multi-stage-build
Fail (gcc leaked)
Fail (gcc leaked)
Fail (gcc leaked)
—
Fail (gcc leaked)
Native linkage (pip→libssl)
native-linkage
Fail (no link)
Fail (no link)
Fail (no link)
—
Fail (no link)
Monorepo: all ecosystems found
monorepo-mixed
Pass (3/3 eco)
Pass (3/3 eco)
Pass (3/3 eco)
Pass (3/3 eco)
Pass (3/3 eco)
npm workspaces: both members
node-workspace
Pass
Pass
Pass
Pass
Pass
Vendored Go: no double-count
go-vendored
Pass
Pass
Pass
Pass
Pass
Unpinned manifest: any output
node-unpinned
Fail (0 pkgs)
Fail (0 pkgs)
Pass (68 pkgs†)
Fail (1 pkg)
Pass (68 pkgs)
Base image layer attribution
base-image-layers
Pass (layer IDs)
Pass (layer IDs)
Fail (no metadata)
—
Pending
Go binary in scratch image
go-binary-scratch
Pass (3/3)
Pass (3/3)
Fail (0/3)
—
Pass (3/3)
† cdxgen achieves this by running npm install internally, which modifies the project directory
Tool
Scenarios passed
Scenarios applicable
syft
6/9
9
trivy
6/9
9
mikebom
7/9
9
cdxgen
4/9
9
scalibr
3/5
5
Universal failures (no tool passes):
Multi-stage Docker: all tools leak build-stage deps
Native library linkage: all tools see both sides, none connects them
9. Ecosystem Breadth
How many of the 10 tested ecosystems does each tool support?
Ecosystem
syft
trivy
cdxgen
scalibr
sbom-tool
bom
cx-gomod
cx-py
mikebom
pypi
Yes
Yes
Yes
Yes
Yes
No
No
Yes
Yes
npm
Yes
Yes
Yes
Yes
Yes
No
No
No
Yes
golang
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
maven
Yes
Yes
Yes
Yes
Yes
No
No
No
Yes
cargo
Yes
Yes
Yes
No
—
No
No
No
Yes
gem
Yes
Yes
Yes
No
—
No
No
No
Yes
deb
Yes
Yes
Yes
Yes
—
Yes
No
No
Yes
rpm
Yes
Yes
Yes
No
—
No
No
No
Yes
apk
Yes
Yes
Yes
Yes
—
No
No
No
Yes
binary
Yes
Yes
No
No
No
No
No
No
Yes
Total
10/10
10/10
9/10
6/10
4/10
2/10
1/10
1/10
10/10
10. External Quality Score (sbomqs)
How does each tool's output score against Interlynk's sbomqs?
sbomqs is an external SBOM
quality scorer that rates each SBOM on a 0–10 scale across 8 categories.
It measures presence of fields — not correctness — so it complements
(rather than replaces) the measured counts in sections 1–9. Included
here as a secondary data point for cross-reference.
Data: run 2026-05-05T18-49-30Z (alpha.14 canonical run via
scripts/run_with_sbomqs.py). sbomqs run against every successful
SBOM from the full matrix; pair counts below reflect each tool's
applicability + image-mode availability + tool_failure rate.
mikebom row reflects alpha.14 numbers; non-mikebom rows are
byte-identical to the alpha.13 baseline (runs/2026-05-04T02-50-56Z/).
Overall score (higher is better, max 10.0)
Tool
Mean
Min
Max
Pairs scored
mikebom
7.63
5.04
9.29
35
syft
5.22
4.69
5.62
16
cyclonedx-gomod
4.89
4.20
5.58
2
sbom-tool
4.77
4.77
4.77
14
cdxgen
4.67
2.22
6.39
27
bom
4.43
3.15
5.28
35
trivy-docker
4.11
3.58
4.52
20
trivy
4.00
3.58
4.52
35
cyclonedx-py
3.76
3.76
3.76
1
scalibr
3.04
1.82
3.47
15 †
† scalibr is directory-only by design.
Pair counts differ: tools that fail on image fixtures or are
ecosystem-specific score over a narrower subset. Compare means with
that caveat. mikebom and bom and trivy hit the full 35 schema-valid
pair count (mikebom is the only multi-format wrapper, so its 35 reflects
CDX-only scoring).
Category means (each 0–10)
Tool
Identification
Provenance
Integrity
Completeness
Licensing
Vulnerability
Structural
bom
7.58
5.50
4.22
3.21
1.61
1.55
10.00
cdxgen
7.53
8.50
3.18
2.75
0.17
4.26
9.93
cyclonedx-gomod
10.00
5.50
3.88
3.87
0.00
5.00
10.00
cyclonedx-py
10.00
5.50
0.00
1.00
0.00
5.00
10.00
mikebom
9.46
10.00
4.82
6.30
6.13
9.10
9.94
sbom-tool
10.00
7.50
0.00
4.67
1.00
5.00
10.00
scalibr
8.00
3.00
0.00
2.73
0.00
3.67
8.00
syft
9.30
5.50
4.66
3.00
0.26
7.95
10.00
trivy
9.11
5.50
0.00
3.54
0.69
4.29
9.43
trivy-docker
8.82
5.50
0.00
3.68
0.75
4.70
10.00
Component Quality scored 0 for every tool and is omitted — sbomqs
expects fields none of these tools emit.
NTIA Minimum Elements (2021) profile
Tool
Score
Pairs
sbom-tool
9.69
14
mikebom
9.13
35
cyclonedx-gomod
8.57
2
trivy-docker
8.37
20
trivy
8.36
35
cdxgen
7.45
27
cyclonedx-py
7.14
1
syft
6.57
16
bom
5.64
35
scalibr
5.29
15
NTIA Minimum Elements (2025) — RFC profile
Tool
Score
Pairs
mikebom
8.81
35
sbom-tool
7.53
14
cdxgen
6.93
27
trivy
6.49
35
trivy-docker
6.46
20
cyclonedx-gomod
6.44
2
bom
5.84
35
cyclonedx-py
5.38
1
syft
5.31
16
scalibr
4.77
15
13 required fields in the 2025 RFC profile. mikebom leads at 8.81
mean across 35 pairs (every (mikebom, fixture) pair that produced a
schema-valid SBOM). All other tools cluster in the 4.7–7.5 range.
sbom-tool's NTIA 2021 lead persists (9.69 vs mikebom 9.13) but only
across 14 directory-mode pairs; mikebom is the leader on the broader
35-pair set including image scans.
Caveats:
sbomqs measures field presence, not field correctness — a tool that
emits a wrong license still scores for having a license field.
The Vulnerability category rewards VEX/vulnerability metadata, which
section 4 shows no tool produces meaningfully.
Pair counts vary by tool; cyclonedx-gomod (2) and cyclonedx-py (1)
have too few samples to generalise from.
mikebom leads in 5 of 7 categories on the new run (Provenance 10.00,
Integrity 4.82, Completeness 6.30, Licensing 6.13, Vulnerability 9.10)
alongside near-perfect Structural (9.94) and a strong Identification
(9.46 vs 10.00 leaders).
mikebom's overall score moved 7.59 → 7.62 → 7.63 across the
alpha.9 → alpha.13 → alpha.14 bumps. The +0.01 alpha.13→alpha.14
delta is float-precision noise driven by the 2-finding
mikebom:sbom-tier drift on polyglot-builder-image; alpha.14
is otherwise emission-byte-identical to alpha.13. Other tools'
scores are byte-identical across both bumps since none of the
9 non-mikebom tools changed between runs.
11. Scope Ambiguities
A recurring pattern across these fixtures is that tools disagree not because
any one is wrong about what's on disk, but because there is no universal
answer to "what belongs in an SBOM". The SBOM's target dictates scope, and
the same image can yield multiple correct SBOMs with different scopes.
The NTIA/CISA "SBOM Types" framework recognises this — design, source,
build, analyzed, deployed, and runtime SBOMs describe the same
software at different lifecycle stages. A tool's "missing" finding against a
ground truth may be a tool-vs-GT scope mismatch, not a real omission. This
report treats scope as a first-class consideration rather than a quality
deduction; the bullets below document the recurring edge cases.
Vendoring (Go, Rust, JS, Python)
go-vendored fixture: dependencies are copied into vendor/. Source is
present; go.mod still names the modules; the compiler reads from vendor/
and never resolves remotely.
Is a vendored copy still a "component"? Arguments both ways:
Yes: the identity is known (go.sum hash, directory name maps to module
path), and supply-chain consumers care whether a known-vulnerable version
is present in the tree.
No: nothing is "installed" in the package-manager sense — the upstream
module isn't referenced at build time; it's static source included in the
local artifact.
In practice all tools here treat vendored copies as components (6/6 match on
this fixture). No SBOM spec normatively distinguishes "vendored copy" from
"resolved dependency", so the convention has emerged by tool behavior rather
than by specification.
Shaded / fat JARs (Java)
polyglot-builder-image ships a sbom-fixture-1.0.0.jar built by Maven's
shade-plugin — it merges the class files of runtime deps into a single JAR.
By default the plugin preserves only oneMETA-INF/maven/…/pom.properties
(for the project itself), not per-dep entries.
After shading, is the fat JAR one component or one + all merged deps?
File identity says one (one file, one SHA). Supply-chain identity says all
(the dep classes are still present as bytecode).
If the shade step stripped per-dep POM metadata, what's the tool's source of
truth for identifying merged deps? Bytecode fingerprinting? Re-reading the
build-time pom.xml? Querying a dep-graph service?
Tools diverge sharply here: mikebom reads the build-time pom.xml and
declared-not-cached transitive trees from deps.dev; trivy reports only
file-level JAR metadata; cdxgen parses POMs more aggressively. Counts on
the polyglot fat JAR: trivy 54, cdxgen 106, mikebom 576. None
are "wrong" — they answer different questions about what a fat JAR
"contains".
Statically-linked / embedded runtimes
Go binaries compile all deps (plus stdlib) into a single ELF. Trivy emits a
synthetic pkg:golang/stdlib@<version> entry to name the Go runtime; syft and
mikebom do not. Electron bundles Chromium+Node; JREs ship alongside Java apps;
PyInstaller embeds the Python interpreter.
Is a compiled-in runtime a component? It has a version and a vulnerability
surface, so arguably yes. But it's not separately installable, replaceable,
or upgradable — it's part of the same file as the application.
Build tooling in builder images
polyglot-builder-image has Maven installed (~54 JARs under
/usr/share/maven/lib/). Those JARs are physically present in the image.
If the SBOM target is "what's in the image", they're in scope. If the
target is "what the deployed application uses at runtime", they're build
tooling and out of scope — the app is a fat JAR that doesn't reference them.
Either interpretation is defensible. It's the NTIA SBOM-type distinction
made concrete: a "deployed-image SBOM" includes the Maven JARs; a
"runtime SBOM for the app" does not.
OS-package-owned files vs source-level identity
A JAR at /usr/share/maven/lib/maven-core-3.9.6.jar is owned by the Fedora
rpm maven-lib-3.9.6-6.fc40. Every JAR in that directory has both an rpm
identity (authoritative, package-manager-tracked) and a Maven identity
(from its META-INF/maven/.../pom.properties). Emitting both is arguably
double-counting the same artifact; emitting only the rpm loses the
upstream-ecosystem signal; emitting only the Maven coord loses the
distro-provenance signal.
No SBOM format has a normative "this component is identified two ways" idiom.
CycloneDX components[].evidence can list multiple identities, but few tools
use it.
Native library linkage (cross-ecosystem)
native-linkage fixture: a pip-installed Python package that dynamically
links libssl.so (a system library installed via apt). All tools see the
Python package; all tools see the deb package; no tool connects them —
no SBOM expresses "this pypi package's runtime requires this deb package".
The information is available (pip metadata + ldd output), but SBOM formats
describe static/declared dependencies, not runtime dynamic-linkage. Section 8
confirms this is a universal tool failure, not a gap in any one tool.
Multi-stage Docker builds
multi-stage-build fixture: FROM maven AS build ... FROM slim COPY --from=build ....
The deployed layer should contain only slim + the copied artifacts. In
practice no tool correctly scopes to the final stage — all leak build-stage
deps because image-layer traversal or docker save retains both. Scope
ambiguity: should the SBOM reflect the image's filesystem at docker run
time, or everything the image manifest references? The difference matters
for vulnerability assessment.
Direct-downloaded binaries
multi-ecosystem fixture includes jq downloaded via curl from GitHub
releases — no package manager. Only syft and mikebom identify it. The binary
has a known upstream and a hash, but no conventional package-manager identity.
A supply-chain consumer cares, a strict-PURL consumer has nothing to match on.
Why this matters
These ambiguities are the reason this report evaluates tools by multiple
measurements — exact PURL match (syntactic), completeness against GT
(semantic), and human-reviewed "did the tool answer the useful question"
(pragmatic) — rather than a single quality score. A tool that answers
question X well may answer question Y poorly depending on the scope it
chose, and reasonable tools can disagree on scope without either being
wrong.
12. Polyglot Convergence Case Study (mikebom × polyglot-builder-image)
The polyglot-builder-image fixture has been the most-evolved test bed
in this report and the place where the locked image-scope policy
(Section 11) meets a real Fedora-based builder image with multiple
language ecosystems, shade-plugin'd JARs, sidecar Maven POMs, and a
dual-identity OS-package overlay. This section documents the
convergence trajectory and the tool/GT changes that produced it.
Per-ecosystem final state on polyglot-builder-image (730 GT components):
Ecosystem
GT
mikebom matched
Extras
Missing
cargo
11
11/11 ✓
0
0
gem
76
76/76 ✓
0
0
golang
2
2/2 ✓
0
0
pypi
2
2/2 ✓
0
0
rpm
529
529/529 ✓
0
0
binary
2
2/2 ✓
0
0
maven
108
108/108 ✓
3
0
Mikebom matches 730/730 GT components on this fixture; the 3
remaining FPs are heuristic edge cases in the shaded-class-presence
matcher (artifact-name fragments that match unrelated class paths in
unrelated JARs). Trivy on the same fixture matches 601/730 with 154
findings — its 119 misses are the embedded-pom-properties Maven coords
shaded into other JARs and the Fedora-rebuilt JARs that ship metadata
in /usr/share/maven-poms/ rather than embedded.
Shade-relocation handling — what changed in tools and harness
The polyglot fixture exposed a real-world pattern that no other tool in
this matrix handles: maven-shade-plugin with package relocation.
Surefire (and several other Maven internals) ship JARs containing
relocated shaded copies of common deps — e.g.,
surefire-shared-utils-3.2.2.jar contains commons-compress 1.23.0's
bytecode at the path org/apache/maven/surefire/shared/compress/*,
with the original META-INF/maven/org.apache.commons/commons-compress/ pom.properties preserved as documentation.
Per the locked image-scope policy and the principle that vulnerabilities
in the original code apply to relocated copies (the algorithms are
unchanged, only class names rewritten), this is in-scope. Three changes
landed:
mikebom: mikebom_shade_relocation property tag on emissions
derived from embedded pom.properties whose classes are physically
present (verified by class-presence check); nested under the host
JAR's components[] per CycloneDX 1.4+ pedigree convention.
mikebom (filter): class-presence verification — pom.properties for
deps whose classes are absent from the JAR (excluded by shade
minimize/filter config) are NOT emitted. Cuts ~95% of would-be
phantom shaded entries.
harness loader (src/sbom_conformance/loaders/cyclonedx.py):
recursive walk of components[].components[] with PURL dedup, so
the matcher can see nested shaded-dep emissions. Without recursion
the matcher would never observe the shaded coords.
What remains a known capability gap
Trivy and syft do not read META-INF/maven/*/pom.properties from
inside JARs at all — they identify Maven coords by JAR filename or
package-manager metadata only. This is why their match rate on
polyglot is ~80% vs mikebom's 100%. The gap isn't a bug in those
tools so much as a deliberate-but-conservative scope choice: they
report what's directly identifiable as a shipped artifact, not what's
shaded inside one. Consumers running CVE scanners that need to flag
relocated shaded deps should be aware of this gap when choosing
between tools.
Summary
Per-Tool Profile (all fractions, no grades)
syft 1.27.0 (Anchore)
Completeness: broadest detection; finds curl'd binaries (mikebom now also does)
Accuracy: 1 false positive (project root on npm)
Dep trees: absent on dir scans; real on image scans (81 entries)
Metadata: 88/88 licenses + 88/88 CPEs on image scans; 0/7 on dir scans
PURL: 100% on pypi/npm/Go/Cargo/Gem/apk/rpm; 0/88 on deb
Transparency: records foundBy per component; no compositions
Accuracy: binary scanner deduplicates against package-database-owned files (dpkg .list, rpm header file lists) — only 5-8 pkg:generic/ residuals per Debian image (CPython runtime, soname evidence). polyglot-builder-image pending RPM version alignment (93 VERSION_MISMATCH)
PURL format: uses {ID}-{VERSION_ID} distro qualifier (distro=debian-12, distro=ubuntu-24.04, distro=alpine-3.20.9, distro=rocky-9.3, distro=fedora-40), following Philippe Ombredanne's explicit recommendation in purl-spec#423 — version numbers, not codenames. Notably this means mikebom does not reproduce the KubeCon Europe 2026 slide's distro=bookworm literally, even though that slide is referenced as the "Standard" — the slide's codename form predates and contradicts the maintainer recommendation in #423. Percent-encodes + as %2B per reference implementation
Dep trees: Real dependency trees for 7 ecosystems from dir scans — Go (module cache .mod), Maven (.m2 + JAR POMs, full transitive), Cargo (Cargo.lock, always complete), Ruby (Gemfile.lock), deb (dpkg Depends), rpm (rpmdb REQUIRES), apk (apk depends). No other tool produces dep trees for Maven, Cargo, or Ruby from dir scans. Maven image dep trees are structured (real parent→child) vs trivy's flat dump
Metadata (image): 88/88 deb licenses, 70/70 npm licenses, 14/15 pypi licenses; 88/88 deb hashes + suppliers; evidence on all components
Metadata (dir): 68/68 npm licenses (reads package.json license fields — no other tool does this from dir scans), 88/88 deb suppliers, 142/142 rpm (clean run); pypi licenses from .dist-info
Transparency: only tool using compositions — per-ecosystem declarations; only tool with evidence + confidence (0.85 manifest-analysis, 0.70 filename)
Ecosystems: 10/10
What No Tool Does (updated with mikebom)
0/10 correctly scope multi-stage Docker builds to the final image
1/10 produces real Python dependency trees from .dist-info/METADATARequires-Dist fields (Flask → Jinja2 → MarkupSafe — trivy produces fake flat trees from requirements.txt)
1/10 achieves 88/88 (100%) on every metadata field on debian:bookworm image
1/10 matches the KubeCon slide "Standard" on the hard parts of python3-magics++@2:1.5.8-1 — %2B%2B name encoding and literal 2: epoch in version (the slide's distro=bookworm codename is itself non-recommended per purl-spec#423, so no current tool reproduces it; mikebom emits distro=debian-12 like syft and trivy)
1/10 declares its generation context in SBOM metadata
