gistfile1.md

Notes from a JBrowse pangenome-visualization experiment

Measurements and friction points from ~Feb–May 2026, while trying to drive pangenome visualization in JBrowse from odgi. The prototype has since been removed from our tree; these notes are what's worth keeping.

Environment for all numbers: odgi v0.9.4-2-g405be8f6 built from source, vg v1.69.0, single 16-core box. Test data: HPRC chr20 (hprc-v1.1-mc-grch38.chr20, 919 paths, 1.13 GB .og) and a 51-path volvox toy pangenome.

---

What we were trying to do

JBrowse is built around a linear backbone — the whole UI assumes a reference coordinate system with tracks stacked against it. Making a pangenome visible in that model means one of two things, and we prototyped both:

1. Linearize the graph — project every haplotype path onto the reference and render one row per haplotype, so 90 haplotypes appear as 90 stacked stripes in the normal linear view. Preprocessed offline into tabix-indexed files that a static web server can range-query.
2. Show the graph itself, on demand — let the user drill into a locus and get a real node/edge or tube-map view of just that subgraph, rendered in the browser. This needs subgraph extraction at interactive latency, so it ran as a live server shelling out to odgi.

On the rendering side of (2), I've also been working on sequenceTubeMap (https://github.com/cmdcolin/sequenceTubeMap), and had a tube-map view running inside JBrowse that consumed GFA directly — so the browser-side story for "draw this subgraph" was largely solved. Given a subgraph, we can draw it. The bottleneck was never rendering; it was getting the subgraph back fast enough, which is why the extract numbers below are the part I'd most like to see move.

Path 1 is where most of the odgi usage lived; path 2 is where the performance problems were. Attending MemPanG and talking to Scott (PangyPlot) convinced me linearization is harder than I'd assumed and that BubbleGun-style bubble decomposition is probably the more promising direction — but that's future work.

---

The commands, and why

Building the .og

odgi build -g <source.gfa> -O -s -o <graph.og>

Why: every other odgi command in the pipeline consumes .og, so this was the one-time entry point from whatever the source was (usually vg convert -f out of an HPRC .gbz).

Why those flags specifically: -O compacts the node ID space and -s topologically sorts. Both are load-bearing rather than optional — without them odgi extract fails outright with "node IDs are not compacted", and sorting is what makes an extracted subgraph cover a contiguous ID range. This took a while to discover; it'd be worth making the docs or the extract error message say so.

Path enumeration

odgi paths -i <graph.og> -Ll     # name + start + end
odgi paths -i <graph.og> -L      # names only
odgi paths -i <graph.og> -f      # path sequences as FASTA

Why: the UI needs to know what it's about to draw before it draws anything — which haplotypes exist, and how long each one is — to build the row list and set up coordinate ranges. -Ll gave us that in one call.

-L ended up doing double duty as a health check (see the W-line bug below): if it came back empty, the .og was silently broken and we rebuilt.

-f was the intended sequence source for deriving per-base detail — you need actual haplotype sequence to compute alignments, and the graph is the only place it lives.

Subgraph extraction — the interactive path

odgi extract -i <graph.og> -r <path:start-end> -c <context> -t <threads> -o sub.og
odgi view -i sub.og -g

Why: this is the whole of use case 2. User clicks a locus, server extracts that region plus context, converts to GFA, browser lays it out and draws it as a tube map. Extract because we want a region not the whole graph; view -g because GFA is what both our tube-map view and sequenceTubeMap already parse.

-t defaulted to the core count. That mattered: chr20 extract went from 47 s single-threaded to 8 s at -t 16. Good scaling.

The problem — a fixed deserialize cost dominates:

Region	extract	view -g
1 bp	6.4 s	0.02 s
10 kb	8.5 s	0.02 s
100 kb	9.3 s	0.02 s
1 Mb	9.2 s	0.02 s

View is free; extract is the entire cost. About 6 s of it is fixed — pulling the 1.13 GB .og off disk into the in-memory graph — and only ~3 s scales with the region. A 1 bp query pays the full 6 s, because extract is a one-shot binary with no resident-process mode in v0.9.4.

For comparison, since it's the same ecosystem — vg find -x chr20.xg -p <region> -c 1 | vg view -g:

Region	odgi	vg
1 bp	6.4 s	0.7 s
10 kb	8.5 s	0.9 s
100 kb	9.3 s	2.9 s
1 Mb	9.2 s	25.8 s

xg wins ~9× on everything small and loses only past ~1 Mb, where the algorithmic cost of pulling many haplotype walks starts to dominate and odgi's in-memory structure pays off. That reads as a layout tradeoff rather than an algorithmic gap: odgi is optimized for whole-graph traversal and pays a constant to get there; xg's memory-mapped succinct format amortizes it.

Why this matters for visualization: the small-region case is the workload. A browser asks for a 10 kb window and needs it back in well under a second, over and over, as the user pans. Nothing above ~100 kb is on the interactive path at all — zoomed-out views get served from precomputed coarse tiers. A resident/daemon mode or an mmap-able on-disk layout would close the gap entirely and make odgi usable as a live backend. As it stood I added a backend: 'vg' option, which I'd rather not have had to do.

Linearization — untangle

odgi untangle -i graph.og -R <ref-paths> -Q <query-paths> -n 1 -p \
  | sort | bgzip > <ref>.synteny.paf.gz
tabix <ref>.synteny.paf.gz

Why: this is use case 1, and it's the part that worked well. Untangle projects haplotype paths against the reference into reference-relative PAF blocks — exactly the star topology (n alignments against one reference, not n²) that a haplotype-row display needs. The graph already encodes the alignment; untangle reads it out instead of re-deriving it. Tabix on top means the browser range-queries one static file at every zoom level, no server required.

We used -j 0.5 -m 1000 for the zoomed-out macro-block tier and -j 0 -m 0 for full detail on the toy dataset.

Why we switched to it: it replaced a home-grown Rust linearizer, and the comparison wasn't close (chr20, 919 paths):

	ours	odgi untangle
Wall time	~1 h	1 m 39 s
Peak RSS	7.9 GB	2.1 GB
Output	167–890 MB	11 MB (24k blocks at -j 0.5 -m 1000)
Crashes	—	0 / 4 runs

Two friction points:

• The strand column is graph-internal, not biological. ~51% of chr20 bp came out - strand even after odgi groom, because the threshold is "majority of node traversals are inverted in the graph representation" — largely a construction artifact. We ignored the column and projected unconditionally, which is a workaround, not a fix. If there's a principled way to recover biological orientation here, I'd be interested.
• No cigar/cs output (confirmed via --help), so untangle gives block structure only. Per-base detail — the SNP ticks and indel glyphs users actually want when zoomed in — had to come from elsewhere. We used vg deconstruct -a -u and projected its variants into cs:Z: tags offline. But untangle blocks and deconstruct snarls decompose the graph differently and don't line up at boundaries, so the projection was heuristic and lossy: on chr20 it dropped ~282k overlapping variants just to keep the cs string monotonic, and a snarl straddling two untangle blocks gets clipped. That's real graph-aware information thrown away. If untangle could emit per-base detail directly, that whole class of problem disappears.

Coarsening — bin (evaluated, not shipped)

odgi bin -w <N>

Why: zoomed-out views need fewer, bigger features — you can't draw 24k blocks per haplotype at whole-chromosome scale. We first tried linear-chain contraction as the coarsening primitive and it failed its reduction gate on HPRC graphs, so odgi bin was the recommended replacement: it bins nodes by pangenome position, output is already reference-anchored, and picking N gives a guaranteed N-fold compression. Lossy (topology within a bin is summarized), but the runtime coarsener we already had was effectively doing the same thing. We never got to measure it before the effort wound down.

groom

Why: an attempt to fix untangle's strand column, on the theory that grooming would normalize orientation before untangle read it. It didn't help — see above.

---

Bugs

Both in v0.9.4-2-g405be8f6, built locally from source.

• unchop crashed on every graph — std::length_error: cannot create std::vector larger than max_size(), including on a 278-node fixture. build and the stats commands were fine; only unchop and view crashed. Never rooted it out; possibly a libhandlegraph ABI mismatch in my build, so it may not reproduce upstream.
• odgi build silently dropped W-lines, producing an .og with zero paths and no error at all. Only caught downstream when odgi paths -L came back empty; we ended up converting W→P ourselves and rebuilding. Failing loudly here would have saved a day.

---

Summary of what I'd ask for

1. A resident mode, or an mmap-able .og. The 6 s fixed deserialize is the single thing standing between odgi and interactive visualization use. The rendering end is in decent shape (sequenceTubeMap, and the tube-map view I had in JBrowse); it's the extract latency that makes the loop unusable.
2. Per-base output from untangle. It would remove an entire lossy offline projection step and the ~282k dropped variants that come with it.
3. Loud failure on dropped W-lines, and ideally an extract error that names -O -s as the fix.
4. BubbleGun-style bubble decomposition, if it isn't there already — this is speculative on my part, but it's where I'd point next based on MemPanG and the PangyPlot work.