Skip to content

Instantly share code, notes, and snippets.

@sini
Last active August 27, 2026 20:43
Show Gist options
  • Select an option

  • Save sini/2f87b59e28ed2aec10d210e226ca9c46 to your computer and use it in GitHub Desktop.

Select an option

Save sini/2f87b59e28ed2aec10d210e226ca9c46 to your computer and use it in GitHub Desktop.
gen / den — Project Status (2026-08-27): delivered surface, design of record, remaining horizon, and the documentation/diagram plan

gen / den — Project Status

Date: 2026-08-27

Who this is for. You know Nix. You do not know this project. This report tells you what exists, what is decided, and what is left.

How to read the numbers. Every figure comes from a command run against the live repositories on the date above. No figure is copied from a document. Where a claim could not be checked, the report says so.

Terms come from the project glossary, gen/TERMINOLOGY.md.


1. What this project is

gen is a set of small Nix libraries. Each library does one job. Together they hold a graph. The graph has nodes and edges. Attributes are computed values on the nodes. Nix laziness computes an attribute only when something asks for it.

den is the product. It builds machine configurations. den version 1 works today. den version 2 will use gen.

The project has two phases:

  1. The toolkit phase. Build and consolidate the gen libraries. This phase runs now.
  2. The design phase. Write den version 2's architecture on top of gen.

One rule governs the order. gen must be consolidated before den is designed. The rule is ADR-0001. An earlier attempt built den first. That attempt reached 40,000 lines and is now frozen. The lesson: the toolkit was under-specified, so integration failed.


2. The delivered surface

2.1 The library ecosystem

The roster file gen/lib/mkGenLibs.nix lists every library gen ships. It holds 22 libraries. Each library is marked with its layer:

Layer Count Libraries
substrate 12 prelude, identity, algebra, scope, memo, graph, bind, schema, select, dispatch, product, view
module system 2 types, merge
aspect layer 3 aspects, link, class
framework 4 settings, assemble, program, delivery
retiring 1 resolve

All 22 have a public repository. None is archived. One, gen-resolve, is marked retiring: ADR-0008 rules that it retires, but it has not left the roster yet.

Five libraries retired and are archived. Their content moved to a library that stays. Nothing was dropped.

Retired Ruled by Content moved to
gen-edge ADR-0010 §3 gen-view
gen-pipe ADR-0010 §3 gen-view
gen-demand ADR-0008 §4 gen-scope
gen-rebuild ADR-0008 gen-memo
gen-flake ADR-0031 the hub, gen-memo, gen-delivery, an adapter set

Archive state was checked against GitHub directly, not against a document.

2.2 A pure module system

Nix's own module system needs all of nixpkgs to run. gen ships a second one that does not.

  • gen-types checks values against types. It has no nixpkgs dependency.
  • gen-merge merges definitions from many files. It reproduces nixpkgs' merge output byte for byte. A standing test checks this.
  • gen-schema and gen-aspects now sit on gen-merge and gen-types. Real nixpkgs enters at one named boundary only. There, resolved values cross into a consumer's build. gen types never cross.

This matters for one reason. A pure library can now offer module-style configuration without pulling in nixpkgs.

2.3 One minting authority for identity

Several libraries used to compute their own identity for a value. Some disagreed at the edges. Two of those disagreements were real defects, not style:

  • gen-schema's encoding let two different key sets hash to one identity.
  • gen-types compared four kinds of type declaration by name only. Two different declarations with one name read as equal, silently.

Both are fixed. gen-identity is a new library. It holds the one minting function. ADR-0034 rules the general law: identity for anything gen builds is structural, through that one mint. A name-only comparison survives only where content is sealed, and each such case must argue why structural identity is impossible.

2.4 Consolidations

  • Ordering. gen-graph owns all graph ordering. gen-prelude keeps plain sorting only. ADR-0009 rules this. A silent-drop defect in the ordering path is closed: an undeclared "run after X" name now refuses by name.
  • Content movement. ADR-0010 rules that moving data between graph positions is one shape — a query scoped at a root, with a dual for the reverse. Three libraries collapsed into that one shape.
  • Execution. ADR-0006 and ADR-0008 rule gen-scope the sole evaluator. One incremental plane, gen-memo, sits over it.

2.5 The deliveries

This is the item list behind the themes above. Each row landed. Each row names the library it changed and what it changed. Every row traces to a closed item in the project tracker, which holds the commit that landed it and the check that confirmed it.

New libraries published this cycle

Six libraries did not exist at the start of this work. All six are public. All six were created between 2026-08-18 and 2026-08-24 — first-commit dates read from the repositories themselves, against a control library dating to June.

Library What it gives you
gen-identity The one function that mints identity. Pulled out of gen-schema, and the old duplicate deleted, so two libraries can no longer disagree about what a value is.
gen-view The single query shape for moving data between graph positions. It absorbed two libraries.
gen-program Turns policy rules into a runnable program and solves it through the one evaluator, instead of each framework hand-rolling a rule engine.
gen-delivery Hands a declared target's collected content to a terminal the caller supplies.
gen-assemble The toolkit a configuration framework assembles with.
gen-differential Compares two runs, or two versions of a construction. It is now gen-merge's own correctness oracle, with 120 checked comparison cells.

Defects closed, by kind

What was wrong What it is now
gen-merge diverged from nixpkgs in four places: an unsound type merge, a wrong empty-value case, a silent check on deferred modules, and stub sub-option handling All four closed. A standing test holds the byte-for-byte result.
gen-schema's identity encoding let two different key sets produce one identity Closed by a self-delimiting encoding.
gen-types compared four kinds of type declaration by name only, so two different declarations with one name read as equal Closed. Identity now comes from the full declaration.
gen-graph dropped an undeclared "run after X" name in one path and threw an unrelated error in another Both closed. It now refuses by name before ordering runs.
gen-aspects accepted an unrecognised configuration key and treated it as a new nested aspect Closed. It refuses by name.
gen-settings resolved two batch members that shared one identity to the first member's value, silently Closed this week. The batch is now indexed once, by identity, and a collision refuses by name.
gen-product's node enumeration scaled far worse than the estimate, which was itself wrong Closed. Missing view operations were added, plus one grouping function in gen-prelude.
The project's own test harness erased a test's expected error before the check saw it, so 12 of 18 gate cases could never fire Closed. Found by deliberately re-breaking the source to confirm the check now catches it.

Retirements completed

Five libraries retired. Each one's content moved to a library that stays, and the move was tracked item by item. gen-flake alone dissolved into four named successors across seven planned units. Nothing was dropped to make a count smaller.

Boundary work

A foreign-protocol adapter lets one nixpkgs-shaped surface of 14 fields cross into gen-merge without gen-merge importing a single nixpkgs type. One named boundary replaced scattered special cases.

Rulings that turned into code

Five architecture records are not decisions on paper. They are carried out: gen-scope as the sole evaluator, ordering consolidation, content movement as one query shape, gen-flake's dissolution, and identity through one mint.

2.6 Measurement tools

The project builds tools that check its own work, because several process failures were found to be silent.

  • A CI gate used to report one true/false result covering 18 known-broken tests. A new regression could hide inside that lump. Each test now carries its own status.
  • STATUS/handoff-gate.sh blocks a session handover that would silently drop an open task, commit a stale tracker export, or leave a task marked in progress that nobody holds.
  • STATUS/c10-sweep.sh searches prior work before new work starts. It refuses to report "nothing found" when every search term returns zero. A clean search needs a term that hits, in the same run, as a control.

3. The design of record

34 Architecture Decision Records hold the ruled design. Each has two files:

  • A ledger. The full history. Appended to, never edited.
  • A law file. A short statement of the ruling as it stands today. Regenerated.

The counts match one to one: 34 law files, 34 ledgers. A reader who wants what is decided reads the law file. A reader who wants why reads the ledger.

How settled the design is, from each law file's own status line:

Status Count Meaning
RE-AFFIRMED 17 Confirmed again at the point where it was used
CONDITIONAL 16 Ruled, but not yet re-confirmed against a reviewed base
SUPERSEDED 1 Replaced by a later record

Half the design is re-confirmed. Half stands as made. That is by design: a decision is re-confirmed when a later piece of work depends on it, not on a schedule.


4. What remains

4.1 The gate

One bead, den-hoag-jnr6, is the entry gate to the den design phase. Its blockers are the remaining work.

Count
Total blockers 242
Closed 209
Still open 33

The tracker holds 737 closed, 181 open, and 83 deferred items in total. Deferred items mostly belong to the frozen first attempt. They are not queued work.

4.2 The 33 remaining blockers, grouped

Theme Count State
Premise document sign-off 1 Content settled. One owner reading is the only act left.
Engine consolidation 1 Nearly done. Routes through export rows and one pin bump.
Identity and equality rules 9 Core rule ruled 2026-08-19. Mostly wiring plus one record to write.
Retiring gen-resolve and gen-edge 3 Narrowed to four export rows this week.
Query library and toolkit wiring 5 Mechanism landed and gated 2026-08-19.
Silent-failure defects 6 Each understood. Each has a fix or a stated direction.
Tooling, process, other 8 Mixed.

Most remaining work is execution, not design. The rules are ruled. The code follows.

4.3 Open design questions

Four questions need the owner. Two are substantial.

  1. Merge semantics. Does provenance shadowing replace gen's priority-merge algebra, or work with it? This decides whether matching nixpkgs on merge priorities is the right target at all. A sitting is commissioned and not yet held.
  2. The engine's cyclic arm. Can the evaluator host lattice-shaped ascent without a second entry point? The cost of the answer cannot be priced before the sitting.
  3. A figure shown to the owner was later found to be 55 times too large. It needs re-presenting with the correction.
  4. One procedural fork: does an item need a measurement spike first, or can it be specified directly?

5. Documentation and diagrams

5.1 The ratified document list

On 2026-08-07 the owner ratified a 13-item list of design-phase documents. Ten are absent. Nine of those ten are correctly absent: they sit behind the gate, or behind the architecture description, by the project's own sequencing rule. A per-system specification may not precede the architecture description.

State Items
Present Ordered backlog (lives as the tracker graph), capability cheatsheets, the tracker itself
Partial Component-level entries; the harness shipped, the den-specific parts are deferred
Absent, blocked den PRD, NFR register, den architecture description, context diagram, container diagram, per-system specs, design records
Absent, not blocked The version 1 capability assessment

One ruled action never happened. The same ruling moved five items to a new directory. That directory does not exist. The ruling stands. No writer has acted on it.

5.2 Library documentation coverage

Surface Coverage
AGENTS.md cheatsheet, per roster library 22 of 22
Glossary entry in gen/TERMINOLOGY.md 16 of 22
Reference tree under gen-specs/ 20 of 22

Six libraries have no glossary entry: gen-identity, gen-memo, gen-view, gen-program, gen-delivery, gen-assemble. Two have no reference tree: gen-identity and gen-program. These two are the least documented libraries on the roster. Both are new.

5.3 Diagrams

No architecture diagram exists today. A search for mermaid diagrams across all 32 gen repositories returned zero. The same search against this repository returned 14 files, which proves the search works. Those 14 hold three unique diagrams. All three predate this effort and cover a different subject.

The governing plan sets three levels and excludes a fourth:

  • Context and container diagrams come from the architecture review.
  • Component diagrams come one per den system.
  • Code-level diagrams are excluded by rule. A design record names the binding in text.

Diagrams are written as mermaid, inside the document they explain. They are not separate files.

The plan, in order:

  1. Write the architecture description. This is the one true blocker. No diagram can start before this document names den's systems and its node, edge, and identity model.
  2. Draw the context diagram inside that document. It shows den as one box with its external actors.
  3. Draw the container diagram next, in the same document. It opens the box into its major parts.
  4. Draw one component diagram per den system, inside that system's design record. Order the systems so a foundational system comes before a system built on it.
  5. Draw no code-level diagram.

6. How far to the design phase

The honest answer has two parts.

The rules are nearly done. 209 of 242 gate blockers are closed. Of the 33 that remain, most are execution against rules that are already ruled. Two open design questions remain, and both have a commissioned sitting rather than open research.

The design-phase documents have not started. That is correct, not late — they sit behind the gate by the project's own rule. But it means the design phase starts with a writing effort, not with a running start. The first artefact is the architecture description. Every diagram and every per-system specification waits on it.

The main risk is not the count. It is that two of the remaining questions are genuine design forks. Their answers change how much work follows. The merge-semantics question is the larger one, and its scope is narrower than an early reading of it suggested.

The measured position: nixpkgs puts merge rules on its own types — 21 of them. gen splits the same job across two libraries. gen-types checks values and defines no merge rules, which is correct for a checker. gen-merge defines the structural merge rules, and its composite types already recurse into their element types the same way nixpkgs' do.

So the open question is not "gen must grow a whole set of merge rules." It is narrower: what happens at a leaf — a value whose type carries no merge rule of its own. gen refuses unless the definitions agree. nixpkgs combines them by shape. That difference, and where the rule that settles it should live, is what the sitting decides.


7. How this project measures itself

Three habits explain the figures above, and they are worth stating because they are unusual.

  1. A claim needs a control. Any statement that something is absent must show, in the same run, that the same check finds something that is present. A search that could not have matched is not evidence of absence. This report followed that rule, and one of its own checks failed it and had to be re-run.
  2. A report about work is not the work. Where an agent reported a figure, the figure was re-derived independently before it entered this report. One figure was wrong and was corrected.
  3. A decision is recorded where it is used. Rulings live in the ADR corpus. State lives in the tracker. A document that copies either goes stale, so documents point instead of repeating.

8. What this report does not claim

  • Performance figures are quoted from the closing record of the work that produced them. They were not re-run for this report.
  • The claim that the module system stack is fully free of nixpkgs rests on the project glossary and one corroborating defect report. It is not traced end to end through a single commit.
  • One tracker item records itself as complete while its status says open. The discrepancy is reported, not resolved.
  • The grouping of the 33 remaining blockers into seven themes is one defensible reading. It is not the only possible grouping.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment