Skip to content

Instantly share code, notes, and snippets.

@jrecursive
Last active September 1, 2026 12:30
Show Gist options
  • Select an option

  • Save jrecursive/dd7934073c6ff081d7d867c252fa85ff to your computer and use it in GitHub Desktop.

Select an option

Save jrecursive/dd7934073c6ff081d7d867c252fa85ff to your computer and use it in GitHub Desktop.
ChatGPT + MCP Server Instruction for automated experimentation

Use the Culture MCP experiment tools to run a 10-case Luna circuit-synthesis study, wait for all ten experiments to finish, inspect every resulting circuit, and produce a comparative report.

This is an exploratory planner evaluation, not a transport acceptance test. The accepted-mode transport has already been validated.

Setup

First inspect Culture read-only state and resolve the exact persisted IDs for:

  • the current Luna model configuration used for logical planner experiments;
  • PLANNER_5, or the newest installed planner lineage if that exact lineage is unavailable;
  • the logical-2 schema.

Use exact persisted IDs. Do not guess.

Use the same:

  • model configuration;
  • planner lineage;
  • schema;
  • turn timeout;

for all ten experiments.

Use:

wait_mode = "accepted"
turn_timeout_seconds = 600

for every run.

Use a unique descriptive idempotency_key for each case.

Do not use arbitrary SQL, direct HTTP calls, database mutation, or other backdoors to launch or continue experiments. Launch only through the controlled experiment MCP tools.

Execution strategy

Phase 1 — submit all ten

Submit all ten experiments in accepted mode before beginning sustained polling.

The objective is to allow independent Luna syntheses to execute concurrently rather than serializing them behind observation.

For each successful admission, retain:

case_id
conversation_id
turn_id
initial_status
retry_after_ms

Surface a compact case-to-conversation-ID mapping to the human at the earliest host-supported opportunity so any individual experiment can be recovered manually if orchestration fails.

Do not launch duplicate experiments because of observation failures.

Phase 2 — observe the whole set

After all ten have been admitted, poll the nonterminal experiments using:

experiment_get(
    conversation_id=<conversation_id>,
    limit=1
)

Do not supply pagination cursors.

Use each experiment's advisory retry_after_ms approximately.

You may interleave polls across experiments intelligently rather than sleeping according to one experiment at a time.

Do not tight-loop.

Continue until all ten experiments reach canonical terminal states.

A polling failure does not imply experiment failure. Preserve the affected conversation_id and retry observation later.

Phase 3 — inspect every synthesis

For each completed experiment, inspect the complete planner result and its persisted evidence.

Do not judge a circuit only by whether logical-2 schema validation succeeded. Evaluate its actual semantics.

Phase 4 — comparative report

After all ten have been inspected, synthesize the findings across the set.

Do not produce the final report until every experiment is terminal or clearly irrecoverable.


The ten synthesis cases

Use the exact prompts below as the experiment prompts.

CASE 1 — Persistent event gate

Purpose: persistent state, PULSE control, data/control separation, gate semantics.

Prompt:

Build a persistent circuit that controls whether incoming Event messages may pass through.

It starts CLOSED.

Inputs:

  • Event input events
  • PULSE input open
  • PULSE input close
  • PULSE input toggle

Output:

  • Event output output

open forces the path open, close forces it closed, and toggle switches state.

While open, events pass through unchanged. While closed, they are suppressed.

Control pulses must never appear on the Event output.

State persists until another control input changes it.


CASE 2 — Correlate orders with delayed payments

Purpose: joins/correlation, out-of-order arrival, persistent unmatched state, deduplication.

Prompt:

Build a circuit that receives customer orders and payment events on separate inputs.

Orders and payments can arrive in either order and may be retried.

Correlate them by order_id.

Emit paid_orders only after both a valid order and a successful payment for that order have been seen.

Do not emit the same paid order twice if either input is retried.

Keep unmatched orders and unmatched payments persistently until their counterpart arrives.

Also expose an output for payment failures.

Do not require the two streams to arrive in lockstep.


CASE 3 — Debounced configuration watcher

Purpose: register/watch semantics, change detection, timers, suppression of redundant work.

Prompt:

Build a persistent circuit that watches a stream of configuration values.

Repeated identical values should do nothing.

When the value actually changes, wait until it has remained unchanged for 5 seconds before emitting the new stable configuration.

If another change arrives during those 5 seconds, restart the waiting period.

Expose:

  • the latest configuration immediately through persistent state;
  • a stable_changes output containing only debounced changes.

Do not emit duplicate stable-change events for byte-identical values.


CASE 4 — Sliding failure-rate alert

Purpose: stream/window aggregation, counters, timer/window semantics, threshold transitions.

Prompt:

Build a circuit that consumes request-result events containing:

  • service
  • timestamp
  • success

Maintain a rolling view of the last 60 seconds for each service.

If a service has at least 20 requests in the current window and its failure rate exceeds 10%, emit an alert.

Do not repeatedly emit the same alert on every subsequent event while the service remains in the alerting condition.

Emit a recovery event when the service returns below the threshold.

The circuit runs continuously.


CASE 5 — Priority work queue with pause and resume

Purpose: queueing, priority, control signals, persistent backlog, scheduling semantics.

Prompt:

Build a persistent work-dispatch circuit.

It receives tasks containing:

  • task_id
  • priority
  • payload

Higher-priority tasks should be dispatched before lower-priority tasks.

Inputs also include PULSE controls:

  • pause
  • resume

While paused, incoming work must continue accumulating but nothing may be dispatched.

When resumed, queued work should continue in priority order.

A retried task_id must not create duplicate work.

Expose:

  • dispatched work;
  • current backlog state.

CASE 6 — Bounded retry workflow

Purpose: feedback loop, iteration state, finite bounds, timers/backoff, terminal routing.

Prompt:

Build a circuit that receives jobs to be processed by an unreliable external worker.

For each job:

  • dispatch the job;
  • receive a later success or failure result correlated by job_id;
  • on failure, retry after a delay;
  • allow at most 3 total attempts;
  • stop immediately on success.

Emit successful jobs on completed.

Emit jobs that fail all allowed attempts on dead_letter.

Retries must preserve attempt number and job identity.

The feedback cycle must be explicitly bounded and must not create duplicate simultaneous attempts for one job.


CASE 7 — Fan-out semantic analysis and deterministic fusion

Purpose: parallel inference, broadcast/fan-out, independent semantic branches, join/fusion.

Prompt:

Build a circuit that receives a customer-support message.

Analyze each message along three independent semantic dimensions:

  1. topic/category;
  2. urgency;
  3. sentiment/tone.

These analyses may run independently and should be able to execute in parallel.

After all required analyses for the message are available, combine them into one structured routing decision.

Deterministic correlation and joining should be used where possible rather than asking another model merely to assemble fields.

Emit:

  • ordinary routed messages;
  • urgent escalations.

Preserve message identity through every branch.


CASE 8 — Human approval with latest-version semantics

Purpose: asynchronous approval, supersession, persistent state, stale-result rejection.

Prompt:

Build a circuit for publishing document drafts that require human approval.

Each document can have multiple versions.

A proposed version is sent for approval and may receive an approval or rejection later.

If version 3 has replaced version 2, a late approval for version 2 must not publish the stale version.

Only the latest currently pending version of a document may become published.

Preserve approval/rejection history.

Emit:

  • published versions;
  • rejected versions;
  • stale approval results that were ignored.

Inputs and approval results may be retried without producing duplicate publication.


CASE 9 — Business-hours router with urgent bypass

Purpose: time/state gating, branching, merge semantics, priority bypass.

Prompt:

Build a persistent circuit for incoming operational messages.

Each message contains:

  • message_id
  • urgency
  • payload

The circuit also receives the current business-hours state.

Ordinary messages arriving outside business hours should be held until business hours reopen.

Urgent messages must pass immediately even when the business is closed.

When business hours reopen, release the accumulated ordinary messages.

Do not duplicate a message if business-hours state changes repeatedly.

Expose:

  • normal work;
  • urgent work;
  • current held backlog.

CASE 10 — Stateful household inbox

Purpose: mixed deterministic/inference workflow, multiple outputs, state, human escalation, practical end-to-end synthesis.

Prompt:

Build a persistent circuit for a family's shared household inbox.

It receives messages such as:

  • bills;
  • school notices;
  • appointment reminders;
  • deliveries;
  • invitations;
  • ordinary personal messages;
  • spam;
  • messages that are unclear or important enough that a person should inspect them.

Classify incoming messages and route them appropriately.

Keep useful persistent state for things that remain actionable.

Repeated delivery of the same message should not create duplicate household tasks.

Use deterministic routing and state where possible, and inference only where understanding the message actually requires semantic judgment.

Expose useful outputs for:

  • bills/tasks;
  • calendar-like reminders;
  • deliveries;
  • ordinary informational messages;
  • spam;
  • human review.

Prefer a simple understandable circuit over unnecessary machinery.


Inspection rubric

For every case, record at least:

case_id
conversation_id
turn_id
final_status
elapsed_ms
schema_valid
provider invocation count
provider attempt count

Then inspect the synthesized circuit for the following.

A. Basic correctness

Does the circuit actually satisfy the natural-language task?

Identify:

  • correct behavior;
  • missing requirements;
  • incorrect semantics;
  • ambiguous semantics.

Do not count schema validity as semantic correctness.

B. Circuit structure

Record:

  • number of terminals;
  • number of devices/operators;
  • major device/operator kinds used;
  • persistent/stateful devices;
  • deterministic devices;
  • inference/model devices;
  • feedback loops;
  • explicit gates;
  • timers;
  • joins/barriers;
  • queues;
  • registers/watchers;
  • storage/state constructs;
  • forks/broadcasts/merges.

Use the actual planner output rather than assuming these constructs exist.

C. State ownership

Determine whether persistent state has a clear owner.

Look for:

  • duplicated state;
  • unclear state ownership;
  • state hidden unnecessarily inside model nodes;
  • correct persistence boundaries;
  • reasonable update semantics.

D. Data versus control semantics

Check whether the planner correctly separates:

  • data messages;
  • PULSE/control signals;
  • state changes;
  • timers;
  • completion behavior.

Flag control information incorrectly routed as ordinary data.

E. Deterministic versus semantic work

Evaluate whether Luna places inference only where semantic judgment is useful.

Identify:

  • inference that should have been deterministic;
  • deterministic logic that was incorrectly delegated to a model;
  • good inference boundaries.

F. Parallelism

Where the task contains independent work, determine whether Luna exposes useful concurrency.

In particular inspect Case 7 carefully.

Do not reward parallelism where ordering/state dependencies make it incorrect.

G. Correlation, replay, and idempotency

For cases involving retries or asynchronous streams, inspect:

  • stable IDs;
  • deduplication;
  • joins;
  • stale-result handling;
  • exactly-once-like release behavior;
  • whether retries could multiply outputs.

H. Feedback and termination

For Case 6 and any other generated loops:

  • identify the loop controller;
  • identify progress/attempt state;
  • confirm a finite bound or termination condition;
  • check that feedback cannot run indefinitely accidentally.

I. Continuous-circuit semantics

These circuits are generally persistent.

Check whether Luna mistakenly designs batch/DAG completion where continuous behavior was requested.

Conversely, note any unnecessary nontermination machinery.

J. Complexity

Judge whether the synthesis is:

  • underbuilt;
  • appropriately sized;
  • unnecessarily elaborate.

Record device count and explain any obvious overengineering.


Cross-case report

After inspecting all ten, produce a report with the following sections.

1. Executive summary

Give a concise assessment of how Luna performs as a logical circuit synthesizer across this set.

State the strongest and weakest aspects.

2. Results table

Produce one row per case with at least:

Case Primary feature Status Schema valid Devices Inference nodes Elapsed Semantic assessment

Use concise semantic ratings such as:

excellent
good
mixed
poor

but explain them later.

3. Case-by-case findings

For each of the ten cases:

  • summarize Luna's architecture;
  • identify what it got right;
  • identify concrete defects or questionable choices;
  • note any particularly interesting design decision.

Do not merely restate the generated plan.

4. Primitive coverage

Compile a matrix showing which cases exercised which important circuit concepts, based on what Luna actually synthesized.

Consider:

persistent state
gate
register
watch/change detection
queue
deduplication
join/correlation
fork/broadcast
merge
barrier
timer
window/aggregation
feedback loop
bounded retry
control/PULSE
inference
parallel inference
human review
stale-result suppression
continuous execution

Distinguish:

requested
actually synthesized
apparently synthesized correctly

where useful.

5. Failure-pattern analysis

Look across all cases for recurring planner weaknesses.

Examples might include:

  • excessive model use;
  • insufficient state ownership;
  • vague operators;
  • CUSTOM-like escape hatches;
  • missing deduplication;
  • weak completion semantics;
  • difficulty with feedback loops;
  • overengineering;
  • under-specification;
  • failure to exploit parallelism;
  • incorrect control/data separation.

Report only patterns actually supported by these ten outputs.

6. Strong-pattern analysis

Likewise identify architectural behaviors Luna performs reliably.

7. Complexity behavior

Compare requested task complexity against resulting circuit size.

Look for evidence that Luna's plans scale reasonably or explode unnecessarily.

Include device/operator counts.

8. Inference-node behavior

Across the ten syntheses, identify:

  • how often Luna uses inference;
  • what roles it assigns to inference;
  • whether those boundaries make sense;
  • whether inference nodes appear independently parallelizable;
  • repeated inference-node patterns that might justify first-class runtime primitives or reusable templates.

9. Runtime/planner observations

Report the distribution of synthesis runtimes and any obvious relationship between:

  • prompt complexity;
  • circuit size;
  • inference-node count;
  • synthesis latency.

Do not claim statistical significance from ten samples.

Include min, max, median, and mean elapsed synthesis time if all timing data is available.

10. Recommendations

Based on the observed results, recommend concrete next experiments or planner/runtime changes.

Separate recommendations into:

Planner prompt/schema improvements

Things that could plausibly improve synthesis by changing planner instructions or logical-2.

Runtime primitive opportunities

Repeated patterns that appear important enough to deserve better first-class primitives.

Evaluation gaps

Important circuit behaviors this ten-case suite did not exercise well enough.

Candidate regression suite

Identify which of these ten cases are especially useful as stable regression tests and why.

11. Overall conclusion

Answer:

Based on these ten syntheses, what does Luna appear to understand well about the circuit model, what does it still misunderstand, and what should we test next?

Be concrete and evidence-based.


Human recovery and orchestration safety

Throughout the study:

  • preserve all ten conversation IDs;
  • do not relaunch a case merely because polling fails;
  • if reliable observation of one case becomes impossible, report its conversation ID and continue observing the other cases;
  • use the original idempotency key only if the admission outcome itself is genuinely uncertain;
  • do not use experiment_continue during this study;
  • do not modify or repair Luna's outputs before evaluating them.

The report should evaluate exactly what each original Luna synthesis produced.

At the end, include a compact appendix mapping:

case_id → conversation_id → turn_id

so every experiment can be inspected again later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment