A cognitive science exploration for the kaizen project Round 1 — March 2026
The framing of "checklist vs. intuition" is a false dichotomy, and it points to the exact cognitive science literature we need. Gary Klein's Recognition-Primed Decision (RPD) model describes how experienced practitioners make decisions under time pressure and uncertainty: they don't evaluate options against criteria. They pattern-match a situation to a prototype, which immediately surfaces a candidate action, which they then mentally simulate. If the simulation runs without catastrophic failure, they act. They don't compare alternatives — they evaluate one option.
The "ultrathink" trigger is RPD pattern recognition firing. When Aviad looks at a proposed design and something feels wrong, he is not running through his five categories consciously. He is experiencing what Klein calls anomaly detection: the current situation fails to match the prototype for "this should work." The mismatch is felt before it is articulated. The five categories he named are not the cognitive process — they are a post-hoc rationalization of the pattern, a reverse-engineering of why the anomaly detector fired.
This is important: expert engineers don't recognize bad designs by checking conditions. They recognize good designs by pattern completion, and they notice when the pattern fails to complete. The "ultrathink" moment is not a positive trigger ("ah, this has property X that requires design thinking"). It is a negative signal ("something about this doesn't fit"). The absence of fit is the alarm.
Klein studied firefighters, nurses, and military commanders. The finding that transfers directly here: experts build a library of situation prototypes over years of experience. Each prototype is a bundle: a situation shape, an expectation about dynamics, a set of responses that worked, and — critically — a set of ways this class of situation went wrong. When a new situation arrives, the brain matches it to the closest prototype. When features of the current situation contradict the prototype's predictions, the anomaly detector fires. The expert pauses.
In Aviad's domain, the prototypes are things like: "distributed state ownership without clear contract = interface breaks under partial failure," or "multi-agent store pattern with late persistence = data loss on crash." He doesn't enumerate these at decision time. The prototype fires, the simulation runs, the anomaly is felt.
Using the vocabulary of cognitive science, Aviad's tacit knowledge appears to be a compound of at least four distinct types:
Episodic memory organized as a failure library. He has seen specific failures. Not "state resilience is important" in the abstract, but "that time we stored findings at the end and lost everything when the session died mid-review." These episodic memories are indexed by their failure signature, not their surface features. The "orchestrator-batch vs. agent-stores" case retrieved from episodic memory not because it looked the same but because its failure signature — late binding of critical state to session lifecycle — matched prior failures.
A causal model of system dynamics. Beyond episodic memory, Aviad has a working model of how this class of system fails. This is what Polanyi called "indwelling" — not explicit knowledge about the system, but tacit skill in inhabiting the system's perspective, seeing from within it. He can mentally simulate "what happens if this design runs and then the process dies at step N" because he has built a causal model through repeated exposure. This is different from recalling a specific incident; it is the ability to reason forward from a design to failure modes without having seen those specific failures before.
Option-space awareness. Senior engineers know there is almost always more than one design. They carry a rough map of the option space for common problem types. The agent in the concrete example never asked "what are the other ways to do this?" because it did not know there was an option space to explore. Aviad's anomaly detector partly fires because he sees that the agent has committed to one design without noticing the field. This is meta-cognitive: knowing that you need to check for alternatives before committing.
Irreversibility weighting. Perhaps the deepest piece: expertise includes a calibrated sense of which decisions are hard to undo. This is not just "can we refactor this?" It is a rich estimate of: how many other things does this decision constrain? If we discover this was wrong in three weeks, what breaks? Junior engineers treat most design decisions as reversible. Senior engineers have experienced enough "we built on the wrong foundation" events that irreversibility itself becomes a first-class property they track.
Polanyi's formulation — "we know more than we can tell" — is precisely apt here. Aviad's five categories are the parts he can tell. The part he can't tell is the shape of the prototypes themselves, the exact features that trigger pattern completion or anomaly detection. These are stored procedurally, not declaratively. You cannot list them; you can only exercise them.
The agent's failure in the concrete example was not a logic error or a missing step. It was an absence — it never entered the cognitive mode where design alternatives are considered. More precisely:
No anomaly detection trigger. The agent has no mechanism corresponding to "this doesn't fit the prototype." Every design looks equally normal to it, because it has no prototypes. Without a baseline of expected patterns, there is nothing to deviate from. The agent saw "store review findings" as a task to execute, not as a design decision with a shape. It found a valid solution and executed it. The normal-seeming surface of the orchestrator-batch pattern triggered no alarm, because the agent has no alarm to trigger.
No failure mode library. The agent has broad world knowledge about distributed systems failures, but it does not have indexed episodic memory of "what happened last time this class of design was used in this codebase." It knows that session crashes are theoretically possible. It does not have the visceral episodic weight of having watched data loss happen, which is what makes the failure mode salient rather than theoretical.
No option space initialization. The agent does not naturally ask "what are the other designs?" before committing. This is not because it cannot generate alternatives when asked — it clearly can. It is because the trigger for alternative generation is absent. The trigger for an expert is the felt sense that the first candidate solution is too easy, or that the problem has the shape of a class that typically has non-obvious alternatives. The agent has no such felt sense.
Commitment bias amplified. Once the agent begins implementing, it enters a regime where the current design is true and the task is to execute it correctly. This is normal for execution mode. The problem is the transition: expert engineers have a deliberate "design pause" phase before execution, triggered by anomaly detection. The agent lacks the trigger that would put it in design-pause mode before switching to execution mode.
No causal simulation. More precisely: the agent can simulate causally when prompted, but does not spontaneously run the simulation "what happens if this process dies at step N?" before choosing a design. The expert runs this simulation habitually, almost involuntarily, for certain classes of decision. It is a learned reflex.
The cognitive science literature on expert decision support suggests tools that externalize the cognitive operations experts perform internally.
Pre-mortem analysis (Klein). Klein developed this explicitly for teams whose members have insufficient experience to generate anomaly detection organically. The instruction: "Assume this design is implemented, deployed, and has failed catastrophically. It is six months later. What went wrong?" The pre-mortem forces backward simulation from failure, which is exactly the causal reasoning experts perform forward spontaneously. For an AI agent, this means: before committing to a design, generate two or three catastrophic failure narratives for it. If the failure narratives reveal state resilience issues, interface ambiguities, or irreversibility traps — pause and explore alternatives.
Inversion (Charlie Munger, via Hannibal Lecter's technique from military planning). "How would this design fail?" is structurally different from "will this design work?" The second question has a strong confirmation bias: the agent already found a path to "yes." The first question bypasses confirmation bias by asking for disconfirmatory evidence. Applied at design time: before implementing, generate the top three failure modes. If any of them are catastrophic and unrecoverable, that is the anomaly detector's result made explicit.
Option forcing. Before analyzing a design, require the agent to name at least two alternative designs. Not evaluate them — just name them. The act of generating the option space changes the epistemic stance from "execute this" to "choose among these." Research on naturalistic decision making shows that the most dangerous decision states are those where only one option is perceived. Option forcing breaks single-track commitment before it forms.
Causal chain tracing under adversarial conditions. Walk the happy path once, then walk the path assuming the most disruptive possible mid-execution interruption. In distributed/agent systems, the most natural adversarial condition is: the process dies after step N. This is not a generic fault injection; it is the specific failure mode that expertise in this domain has repeatedly surfaced. Formalizing it as a required step ("now trace the causal chain with a crash after each state transition") encodes the expert's learned reflex as a procedure.
Analogy to past failures (case-based reasoning). The agent lacks episodic memory, but the kaizen system accumulates documented incidents. A structured search for "which past kaizen issues have failure signatures similar to this design?" converts the incident archive into a functional analog of episodic memory. This is the cognitive science concept of case-based reasoning (Schank, Kolodner): retrieve the most similar past case, adapt the solution, verify the adaptation. The kaizen issues list is exactly this archive if the retrieval step is made explicit.
Cognitive science offers a sobering answer: partially, with irreducible residue.
Dreyfus and Dreyfus's skill acquisition model distinguishes five levels from novice to expert. The critical transition happens between competent and proficient: the competent practitioner applies rules analytically; the proficient practitioner sees the situation holistically and recognizes which rules apply. Expertise is the transition from rule-following to pattern recognition. Once expertise is achieved, the rules that were explicitly followed early in learning are replaced by recognition that cannot be fully re-translated back into rules without loss.
This is not mystical. It is the consequence of neural encoding. Procedural memory (how to ride a bike, how to read a face, how to recognize a bad design) uses different neural substrates than declarative memory (explicit propositions you can state). Procedural knowledge was built by repeated exposure, and its compression is one-way: the original rule set is discarded as the procedure is automatized. You cannot fully reverse this.
For Aviad's case: his five categories are the parts that could be articulated because they were, at some point, explicit hypotheses he tested. The residue that cannot be articulated is the specific feature-weight combination inside each prototype — exactly which design properties, in which combination, at what magnitude, trigger the "something is wrong" signal. This is stored as a multi-dimensional threshold function learned over hundreds of cases, and that function is not accessible to introspection.
The practical implication: checklists derived from expert articulation will always be incomplete. They capture the articulable structure but miss the continuous feature weighting that makes expert judgment sensitive and specific. A checklist trained on Aviad's five categories will catch the cases he can name and miss the cases he recognizes but cannot yet name. This is not a reason to abandon articulation — it is a reason to combine it with other mechanisms.
What cognitive science does suggest is tractable: critical incident debriefing (Klein's method) can surface tacit knowledge that experts cannot generate through direct introspection. By walking through specific past cases in detail — "what did you notice first? what didn't fit? what simulation ran through your head?" — the interviewer can extract feature details that the expert could not volunteer. Applied to kaizen: systematic debriefing of "Aviad said ultrathink" incidents, asking what specifically triggered it, could over time build a richer prototype library than direct articulation alone.
Here the cognitive science challenge becomes structural. Expertise in Klein's model requires thousands of hours of high-quality feedback loops: act, observe outcome, update prototype. The chess master's pattern recognition was built from tens of thousands of games. The firefighter's situational awareness was built from hundreds of fires. The senior engineer's design intuition was built from years of implementations that worked and failed.
AI agents, currently, have no persistent episodic memory across sessions. Each session begins at the same baseline. There is no mechanism by which seeing the orchestrator-batch failure this week makes next week's agent more sensitive to that failure mode. The "experience" that drives expertise cannot accumulate.
Cognitive science suggests several partial alternatives:
Compiled cases as substitute for episodic memory. Cognitive models of expertise distinguish between compilation (converting episodes into rules) and chunking (compressing patterns into retrievable units). Both happen inside individual human experts, but the compiled output can be externalized. Klein's approach to knowledge engineering is essentially this: extract compiled cases from experts through critical incident technique, encode them in retrievable form, and inject them at decision time. The kaizen incident archive, if structured and retrievable, is this mechanism.
Schema injection at task initialization. Developmental psychology (Piaget) and schema theory (Rumelhart) suggest that much expert performance can be scaffolded by activating the right schema at the right moment. The expert's anomaly detector fires because the current situation activates a schema that predicts a different configuration. If the schema is externalized and injected at design-time, the agent does not need to have learned it — it is given it. The question becomes: can we inject a "design review schema" that asks the right questions before implementation begins?
Outcome feedback through structured reflection. The kaizen reflection loop is the closest existing mechanism to experience accumulation. When a design choice is made and the outcome is documented in a kaizen issue, a feedback signal exists. The barrier is retrieval: the agent that made the choice is gone; the next agent does not automatically learn from the issue. Bridging this requires either retrieval augmentation (search the issue archive at design time) or schema update (generalize the incident into a design principle that is injected into future sessions). Both are tractable engineering problems.
Based on the foregoing, the most promising mechanism is not a checklist but a stance shift — a mandatory transition into a distinct cognitive mode before design commitment, triggered by recognizable surface features of the task.
The expert's anomaly detection is a sophisticated classifier that fires on subtle features. We cannot replicate that classifier directly. But we can replicate the consequence of that classifier firing: entering a phase of deliberate, adversarial evaluation before committing to a design. The expert does this automatically when the pattern doesn't fit; we need to trigger it by other means.
The proposal: a set of lightweight surface triggers that mandate design-pause mode, followed by a compact protocol that performs the cognitive operations experts perform internally.
Triggers (recognizable without expertise):
- The task involves storing or transmitting state that will be consumed by a different process or session boundary
- Multiple agents or sub-processes will read or write shared state
- The decision is described in terms of "how to implement X" rather than "whether and how to implement X" (option space is already collapsed)
- The problem has appeared in the kaizen issue archive before, regardless of how it was resolved
These triggers are deliberately conservative. They will fire on cases that do not need design exploration. That is acceptable — a false positive costs one design-pause conversation; a false negative costs a catastrophic failure.
The protocol (mimicking expert internal operations):
Step 1 — Option generation. Name at least two other ways to accomplish the same goal. Do not evaluate. Just name them. (Breaks single-track commitment before it forms.)
Step 2 — Failure-first simulation. For each named option, including the proposed one: what is the worst outcome if the process dies or is interrupted at its most vulnerable moment? (Encodes the expert's causal simulation under adversarial conditions.)
Step 3 — Irreversibility assessment. If this design is wrong, at what point does that become apparent? What would have to be undone? (Activates the irreversibility-weighting that experts carry implicitly.)
Step 4 — Archive retrieval. Does the failure signature of any option match a pattern in the kaizen issue archive? (Case-based reasoning as substitute for episodic memory.)
Step 5 — Design selection with explicit rationale. Choose a design with a one-sentence justification that names the failure modes of the alternatives it was chosen over. (Forces explicit comparison rather than first-candidate execution.)
This protocol does not replicate expert intuition. It replicates the output of expert intuition in cases where the intuition fired: a brief period of adversarial evaluation before commitment. The expert does this in seconds, almost unconsciously. The agent will do it in a few minutes, explicitly. The result — a considered design choice with documented failure-mode reasoning — should be equivalent.
The deeper point is epistemic: the agent does not need to have felt something was wrong in order to perform the cognitive work that follows from the expert feeling that something was wrong. The trigger can be exogenous. The operations can be explicit. The outcome — a more resilient design — is what matters.
The kaizen principle applies: instructions are necessary but never sufficient. A prompt that says "consider design alternatives" is L1. A trigger condition that mandates design-pause mode before state-ownership decisions is L2. An architecture that makes the design-pause protocol a required gate before implementation commits to a storage strategy is L3. The question is not which level is theoretically sufficient — it is which level the failure mode requires.
Given that the orchestrator-batch failure was not a logic error but an invisible option space, L2 is the minimum viable level: a hook or gate that detects the trigger conditions and requires the protocol before proceeding. L1 instructions will be forgotten. L3 architecture would catch it mechanistically regardless of instructions. L2 hooks are the achievable near-term target.
This document is a first-round cognitive ethnography. Its claims are hypotheses, not commandments. The kaizen principle applies: when practice contradicts a principle here, update this document.