Skip to content

Instantly share code, notes, and snippets.

@jalehman

jalehman/2026-05-25-agent-workflow-pagedrop.html

Last active May 25, 2026 13:46
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save jalehman/30b1f9fad7e35883951155cfb73c4bc9 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save jalehman/30b1f9fad7e35883951155cfb73c4bc9 to your computer and use it in GitHub Desktop.
Download ZIP
Agent workflow architecture - Lobster and multi-session pattern #pagedrop
Raw
2026-05-25-agent-workflow-pagedrop.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Agent Workflow Architecture</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Source+Serif+4:ital,wght@0,300;0,400;0,600;0,700;1,400&family=IBM+Plex+Mono:wght@400;500;600&display=swap" rel="stylesheet">
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
:root {
--bg: #0d0d0d;
--surface: #111;
--surface-2: #151515;
--border: #222;
--border-soft: #171717;
--text: #d4d0c8;
--text-strong: #e8e4dc;
--text-soft: #b0aca4;
--muted: #777;
--accent: #6b8aad;
--warn: #c4a035;
--good: #5c9a5c;
}
body {
background: var(--bg);
color: var(--text);
font-family: 'Source Serif 4', Georgia, serif;
font-weight: 300;
font-size: 17px;
line-height: 1.75;
-webkit-font-smoothing: antialiased;
text-rendering: optimizeLegibility;
}
.container {
max-width: 780px;
margin: 0 auto;
padding: 3rem 1.5rem 5rem;
}
.masthead {
display: flex;
justify-content: space-between;
align-items: baseline;
gap: 1rem;
flex-wrap: wrap;
font-family: 'IBM Plex Mono', monospace;
font-size: 11px;
font-weight: 500;
text-transform: uppercase;
letter-spacing: 0.12em;
color: var(--accent);
border-bottom: 1px solid var(--border);
padding-bottom: 1.25rem;
margin-bottom: 2.5rem;
}
.masthead .date { color: #555; }
h1 {
font-family: 'Source Serif 4', Georgia, serif;
font-weight: 700;
font-size: 2.35rem;
color: var(--text-strong);
line-height: 1.2;
margin-bottom: 1rem;
max-width: 15ch;
}
.lede {
font-size: 1.08rem;
color: var(--text-soft);
max-width: 64ch;
margin-bottom: 1.5rem;
}
.pullquote {
font-style: italic;
font-size: 1.16rem;
color: #978f82;
border-left: 2px solid var(--accent);
padding-left: 1.1rem;
margin: 2rem 0;
}
h2 {
font-family: 'IBM Plex Mono', monospace;
font-size: 12px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.1em;
color: var(--accent);
border-bottom: 1px solid var(--border);
padding-bottom: 0.5rem;
margin-top: 3rem;
margin-bottom: 1.5rem;
}
h3 {
font-family: 'Source Serif 4', Georgia, serif;
font-weight: 600;
font-size: 1.15rem;
color: #c8c2b6;
margin-top: 1.75rem;
margin-bottom: 0.75rem;
}
p { margin-bottom: 1.15rem; }
strong { font-weight: 600; color: var(--text-strong); }
ul, ol { margin: 0 0 1.15rem 1.35rem; }
li { margin-bottom: 0.45rem; }
li::marker { color: #555; }
code {
font-family: 'IBM Plex Mono', monospace;
font-size: 13px;
background: #1a1a1a;
color: #c8c2b6;
padding: 0.15em 0.4em;
border-radius: 3px;
}
.grid {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: 1rem;
margin: 1.25rem 0 2rem;
}
.card {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 8px;
padding: 1rem;
min-height: 160px;
}
.card .eyebrow,
.lane .eyebrow,
.step .num {
font-family: 'IBM Plex Mono', monospace;
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--accent);
margin-bottom: 0.55rem;
}
.card h3,
.step h3,
.lane h3 {
margin-top: 0;
margin-bottom: 0.5rem;
}
.card p,
.lane p {
color: var(--text-soft);
font-size: 15px;
margin-bottom: 0;
}
.diagram {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 8px;
padding: 1rem;
margin: 1.25rem 0 2rem;
overflow-x: auto;
}
.diagram-title {
font-family: 'IBM Plex Mono', monospace;
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--accent);
margin-bottom: 0.85rem;
}
.pipeline {
display: grid;
grid-template-columns: repeat(5, minmax(112px, 1fr));
gap: 0.65rem;
min-width: 620px;
align-items: stretch;
}
.pipe-node {
position: relative;
border: 1px solid var(--border-soft);
background: var(--surface-2);
border-radius: 6px;
padding: 0.85rem;
min-height: 118px;
}
.pipe-node:not(:last-child)::after {
content: "";
position: absolute;
right: -0.48rem;
top: 50%;
width: 0.3rem;
height: 0.3rem;
border-top: 1px solid var(--accent);
border-right: 1px solid var(--accent);
transform: translateY(-50%) rotate(45deg);
background: var(--surface);
}
.pipe-node .label,
.worker-node .label {
font-family: 'IBM Plex Mono', monospace;
font-size: 10px;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--accent);
margin-bottom: 0.45rem;
}
.pipe-node strong,
.worker-node strong {
display: block;
font-size: 15px;
line-height: 1.35;
margin-bottom: 0.35rem;
}
.pipe-node span,
.worker-node span {
display: block;
color: var(--text-soft);
font-size: 13px;
line-height: 1.45;
}
.fanout {
display: grid;
grid-template-columns: 1fr 1.4fr 1fr;
gap: 0.8rem;
min-width: 640px;
align-items: center;
}
.worker-stack {
display: grid;
gap: 0.55rem;
}
.worker-node {
border: 1px solid var(--border-soft);
border-radius: 6px;
background: var(--surface-2);
padding: 0.75rem 0.85rem;
}
.hub {
border: 1px solid var(--border);
border-radius: 8px;
background: #101010;
padding: 1rem;
text-align: center;
}
.hub .label {
font-family: 'IBM Plex Mono', monospace;
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--accent);
margin-bottom: 0.45rem;
}
.hub strong {
display: block;
color: var(--text-strong);
margin-bottom: 0.35rem;
}
.hub span {
display: block;
color: var(--text-soft);
font-size: 14px;
line-height: 1.45;
}
.comparison {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 0.8rem;
min-width: 620px;
}
.track {
border: 1px solid var(--border);
border-radius: 8px;
overflow: hidden;
background: var(--surface-2);
}
.track-header {
font-family: 'IBM Plex Mono', monospace;
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.08em;
padding: 0.75rem 0.9rem;
border-bottom: 1px solid var(--border);
color: var(--text-strong);
}
.track-header.peter { color: var(--good); }
.track-header.pepper { color: var(--warn); }
.track-step {
display: grid;
grid-template-columns: 1.4rem 1fr;
gap: 0.6rem;
padding: 0.75rem 0.9rem;
border-bottom: 1px solid var(--border-soft);
font-size: 14px;
color: var(--text-soft);
line-height: 1.45;
}
.track-step:last-child { border-bottom: none; }
.dot {
width: 1.1rem;
height: 1.1rem;
border: 1px solid var(--accent);
border-radius: 999px;
margin-top: 0.12rem;
font-family: 'IBM Plex Mono', monospace;
font-size: 9px;
line-height: 1rem;
text-align: center;
color: var(--accent);
}
.flow {
display: grid;
gap: 0.85rem;
margin: 1.25rem 0 2rem;
}
.flow-row {
display: grid;
grid-template-columns: 160px 1fr;
gap: 0.9rem;
align-items: stretch;
}
.flow-label {
font-family: 'IBM Plex Mono', monospace;
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--accent);
border: 1px solid var(--border);
border-radius: 6px;
padding: 0.8rem;
background: var(--surface);
}
.flow-body {
border: 1px solid var(--border-soft);
border-radius: 6px;
background: var(--surface);
padding: 0.8rem 1rem;
color: var(--text-soft);
font-size: 15px;
}
.step {
background: var(--surface);
border: 1px solid var(--border-soft);
border-radius: 8px;
padding: 1.05rem 1.15rem;
margin-bottom: 1rem;
}
.step p:last-child,
.step ul:last-child,
.step ol:last-child { margin-bottom: 0; }
.lane {
border: 1px solid var(--border);
border-radius: 8px;
background: var(--surface);
padding: 1rem;
margin-bottom: 1rem;
}
.lane.peter { border-left: 3px solid var(--good); }
.lane.pepper { border-left: 3px solid var(--warn); }
.matrix {
width: 100%;
border-collapse: collapse;
margin: 1rem 0 1.5rem;
}
.matrix th {
font-family: 'IBM Plex Mono', monospace;
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.05em;
color: var(--accent);
text-align: left;
padding: 0.55rem 0.75rem;
border-bottom: 2px solid var(--border);
vertical-align: top;
}
.matrix td {
padding: 0.75rem;
border-bottom: 1px solid var(--border-soft);
vertical-align: top;
font-size: 15px;
color: var(--text);
}
.note-list {
list-style: none;
padding-left: 0;
}
.note-list li {
padding: 0.65rem 0;
border-bottom: 1px solid var(--border-soft);
}
.note-list li:last-child { border-bottom: none; }
.note-list .name {
font-weight: 600;
color: var(--text-strong);
}
.note-list .detail {
color: var(--text-soft);
font-size: 15px;
}
.footer {
font-family: 'IBM Plex Mono', monospace;
font-size: 11px;
color: #555;
border-top: 1px solid #1a1a1a;
padding-top: 1.5rem;
margin-top: 3rem;
text-align: center;
}
@media (max-width: 720px) {
body { font-size: 16px; }
.container { padding: 2rem 1rem 3rem; }
h1 { font-size: 1.9rem; }
.grid,
.flow-row { grid-template-columns: 1fr; }
.masthead { flex-direction: column; gap: 0.25rem; }
.diagram { margin-left: -0.25rem; margin-right: -0.25rem; }
}
</style>
</head>
<body>
<div class="container">
<div class="masthead">
<div>Martian Engineering · OpenClaw</div>
<div class="date">May 25, 2026 · Redacted architecture note</div>
</div>
<h1>How We Structure Agent Workflows</h1>
<p class="lede">The shared pattern is not specific to any one agent. Lobster coordinates a workflow, bounded agent sessions perform narrow phases, each phase writes schema-backed artifacts, and final outputs are assembled from those artifacts with review gates before consequential actions.</p>
<div class="pullquote">Lobster is the spine. Agent sessions are the muscles. Durable packets are the connective tissue.</div>
<div class="grid">
<div class="card">
<div class="eyebrow">Control plane</div>
<h3>Lobster</h3>
<p>Defines stage order, arguments, retries, dependencies, and approval checkpoints.</p>
</div>
<div class="card">
<div class="eyebrow">Execution plane</div>
<h3>Agent sessions</h3>
<p>Independent workers run scoped tasks: extract, enrich, score, gather evidence, assemble, or verify.</p>
</div>
<div class="card">
<div class="eyebrow">Memory plane</div>
<h3>Artifacts</h3>
<p>JSON packets, citations, markdown drafts, gaps, and decisions are persisted between stages.</p>
</div>
</div>
<h2>System diagram</h2>
<div class="diagram" aria-label="Shared agent workflow architecture diagram">
<div class="diagram-title">Lobster-driven workflow loop</div>
<div class="pipeline">
<div class="pipe-node">
<div class="label">Context</div>
<strong>Resolve run</strong>
<span>Inputs, period, policy, route, redaction.</span>
</div>
<div class="pipe-node">
<div class="label">Orchestrate</div>
<strong>Lobster plan</strong>
<span>Stage order, args, retries, gates.</span>
</div>
<div class="pipe-node">
<div class="label">Execute</div>
<strong>Worker sessions</strong>
<span>Bounded tasks in parallel or sequence.</span>
</div>
<div class="pipe-node">
<div class="label">Persist</div>
<strong>Packets</strong>
<span>JSON, citations, gaps, drafts.</span>
</div>
<div class="pipe-node">
<div class="label">Control</div>
<strong>Review gate</strong>
<span>Approve, reject, retry, or publish.</span>
</div>
</div>
</div>
<h2>The shared architecture</h2>
<div class="flow">
<div class="flow-row">
<div class="flow-label">1. Runtime context</div>
<div class="flow-body">Resolve the run target once: source material, reporting period, user request, approval route, source policy, and redaction rules.</div>
</div>
<div class="flow-row">
<div class="flow-label">2. Fan-out</div>
<div class="flow-body">Launch parallel agent sessions for bounded work. Each worker owns one slice and writes a packet instead of trying to finish the entire job.</div>
</div>
<div class="flow-row">
<div class="flow-label">3. Normalize</div>
<div class="flow-body">Turn messy source material into typed, reviewable data with confidence, citations, gaps, and blocked claims.</div>
</div>
<div class="flow-row">
<div class="flow-label">4. Assemble</div>
<div class="flow-body">Downstream sessions consume packets only. They do not invent missing facts or silently re-query sources outside their phase contract.</div>
</div>
<div class="flow-row">
<div class="flow-label">5. Review gate</div>
<div class="flow-body">Human review happens before writes, publication, external communication, or source-of-record changes.</div>
</div>
</div>
<h2>Fan-out / fan-in</h2>
<div class="diagram" aria-label="Fan-out and fan-in worker diagram">
<div class="diagram-title">Multiple sessions, one durable output</div>
<div class="fanout">
<div class="hub">
<div class="label">Lobster</div>
<strong>Fan out</strong>
<span>Launch scoped workers from one run context.</span>
</div>
<div class="worker-stack">
<div class="worker-node">
<div class="label">Worker A</div>
<strong>Evidence</strong>
<span>Extract or gather one slice.</span>
</div>
<div class="worker-node">
<div class="label">Worker B</div>
<strong>Enrichment</strong>
<span>Resolve missing or ambiguous facts.</span>
</div>
<div class="worker-node">
<div class="label">Worker C</div>
<strong>Assembly</strong>
<span>Compile a section from packets.</span>
</div>
</div>
<div class="hub">
<div class="label">Lobster</div>
<strong>Fan in</strong>
<span>Validate packets, retry failures, synthesize only from artifacts.</span>
</div>
</div>
</div>
<h2>Two workflows, one pattern</h2>
<div class="lane peter">
<div class="eyebrow">Implemented pattern</div>
<h3>Founder intake</h3>
<p>Raw inbound material becomes extraction packets, enrichment packets, scoring packets, a review packet, and then an approval-gated write plan. The final action is not "the agent decided"; it is "the human approved a narrow write plan."</p>
</div>
<div class="lane pepper">
<div class="eyebrow">Aspirational pattern</div>
<h3>Quarterly letters</h3>
<p>Each portfolio company gets an evidence worker. Section workers assemble the letter from company and fund packets. The final writer drafts from packets only, with finance-sensitive claims left flagged until reviewed.</p>
</div>
<div class="diagram" aria-label="Peter and Pepper workflow comparison diagram">
<div class="diagram-title">Same skeleton, different domain</div>
<div class="comparison">
<div class="track">
<div class="track-header peter">Peter founder intake</div>
<div class="track-step"><div class="dot">1</div><div>Inbound material becomes an intake artifact.</div></div>
<div class="track-step"><div class="dot">2</div><div>Extraction, enrichment, tiering, and scoring run as bounded phases.</div></div>
<div class="track-step"><div class="dot">3</div><div>Review packet proposes a narrow create/update plan.</div></div>
<div class="track-step"><div class="dot">4</div><div>Human approval gates any source-of-record write.</div></div>
</div>
<div class="track">
<div class="track-header pepper">Pepper quarterly letter</div>
<div class="track-step"><div class="dot">1</div><div>Run context defines the quarter and source policy.</div></div>
<div class="track-step"><div class="dot">2</div><div>Company workers gather evidence packets in parallel.</div></div>
<div class="track-step"><div class="dot">3</div><div>Section workers assemble draft sections from packets.</div></div>
<div class="track-step"><div class="dot">4</div><div>Human review gates publication or LP-facing use.</div></div>
</div>
</div>
</div>
<h2>What unifies them</h2>
<table class="matrix">
<thead>
<tr>
<th>Element</th>
<th>Founder intake</th>
<th>Quarterly letter</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Trigger</strong></td>
<td>Inbound company material or operator request.</td>
<td>Quarterly drafting request or scheduled reporting window.</td>
</tr>
<tr>
<td><strong>Fan-out unit</strong></td>
<td>Extraction, enrichment, founder tiering, scoring, review.</td>
<td>One company per evidence worker, then one worker per letter section.</td>
</tr>
<tr>
<td><strong>Artifact contract</strong></td>
<td>Extraction packets, enrichment packets, scoring packets, write plan.</td>
<td>Company evidence packets, fund packets, section packets, final draft packet.</td>
</tr>
<tr>
<td><strong>Source discipline</strong></td>
<td>Separate provided material, public context, internal records, and judgment.</td>
<td>Separate system-of-record facts, meeting color, finance review gaps, and prose.</td>
</tr>
<tr>
<td><strong>Approval surface</strong></td>
<td>Review packet plus approve/reject controls before source-of-record writes.</td>
<td>Draft plus verification report before publication or LP-facing use.</td>
</tr>
</tbody>
</table>
<h2>Why multiple sessions matter</h2>
<div class="step">
<div class="num">Parallelism</div>
<h3>One worker per independent question</h3>
<p>If a workflow has twenty companies, one session should not gather all evidence serially. Twenty bounded sessions can gather company packets in parallel, each with its own citations and gaps.</p>
</div>
<div class="step">
<div class="num">Containment</div>
<h3>A bad worker cannot poison the whole run</h3>
<p>Failures become local: one company packet fails validation, one enrichment step is incomplete, one section has gaps. Lobster can retry or mark that stage without corrupting the rest of the workflow.</p>
</div>
<div class="step">
<div class="num">Reviewability</div>
<h3>Humans inspect packets, not hidden reasoning</h3>
<p>Every stage leaves a durable trail: inputs, supported claims, citations, unresolved questions, and blocked material. That is the difference between agent output and accountable workflow output.</p>
</div>
<h2>Packet anatomy</h2>
<table class="matrix">
<thead>
<tr>
<th>Field</th>
<th>Purpose</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>inputs</code></td>
<td>What the worker was allowed to read for this phase.</td>
</tr>
<tr>
<td><code>supportedClaims</code></td>
<td>Facts that can be used downstream because they have citations.</td>
</tr>
<tr>
<td><code>citations</code></td>
<td>Source references, excerpts, timestamps, and provenance.</td>
</tr>
<tr>
<td><code>needsVerification</code></td>
<td>Useful but incomplete items that require human or finance review.</td>
</tr>
<tr>
<td><code>blockedClaims</code></td>
<td>Claims that should not be published, written, or used as facts.</td>
</tr>
<tr>
<td><code>draftMarkdown</code></td>
<td>Optional prose generated from supported packet contents.</td>
</tr>
</tbody>
</table>
<h2>Safety model</h2>
<ul class="note-list">
<li><span class="name">Narrow phase contracts</span> <span class="detail">Each worker gets a specific job and a known output schema.</span></li>
<li><span class="name">No secret-bearing public output</span> <span class="detail">Internal paths, IDs, credentials, account names, private source names, and sensitive values are omitted or replaced with placeholders.</span></li>
<li><span class="name">No unsupported claims</span> <span class="detail">Claims without citations remain gaps, caveats, or blocked items.</span></li>
<li><span class="name">No automatic side effects</span> <span class="detail">Writes, publication, and external messaging happen after explicit approval.</span></li>
<li><span class="name">Source hierarchy is explicit</span> <span class="detail">Some sources can support qualitative context; others are required for financial or source-of-record facts.</span></li>
</ul>
<h2>Build order</h2>
<div class="step">
<div class="num">Phase 1</div>
<h3>Define the skeleton</h3>
<p>Write the Lobster workflow, runtime context, artifact layout, schemas, and dry-run mode. No external side effects.</p>
</div>
<div class="step">
<div class="num">Phase 2</div>
<h3>Implement the first worker</h3>
<p>Build one bottom-up worker skill and one packet schema. Prove a single phase works before adding fan-out.</p>
</div>
<div class="step">
<div class="num">Phase 3</div>
<h3>Add fan-out and retries</h3>
<p>Run multiple independent workers, validate packets, record failures, and support targeted retries.</p>
</div>
<div class="step">
<div class="num">Phase 4</div>
<h3>Assemble and review</h3>
<p>Build downstream assemblers, generate a review artifact, and keep approval as a separate stage from generation.</p>
</div>
<h2>What this deliberately does not show</h2>
<p>This is a high-level redacted view. It omits private source names, account identifiers, endpoint details, filesystem paths, credentials, company-specific evidence, customer data, financial values, and operational secrets. The goal is to communicate the workflow pattern without exposing the deployment.</p>
<div class="footer">High-level architecture view · confidential details redacted</div>
</div>
</body>
</html>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment