Neon Pilot local extension system report - Thread Kanban

EXTENSION_SYSTEM_REPORT.md

Thread Kanban Extension System Report

Context

This was the first local user extension built in this Neon Pilot install. The extension lives at:

~/.local/state/neon-pilot/extensions/thread-kanban

It contributes a main page at /ext/thread-kanban and persists board data in extension storage.

Issues hit while creating the extension

1. Authoring docs point at APIs that are not consistently available

The CRUD template in packaged docs uses frontend calls like:

pa.actions.call('actionId', input)

But the runtime surface available to this extension did not include pa.actions, causing the page error:

Cannot read properties of undefined (reading 'call')

Existing packaged system extensions use:

pa.extension.invoke('actionId', input)

Fix applied: all frontend action calls were switched to pa.extension.invoke(...).

Impact: the documented/template API and the live runtime API are contradictory. A new extension author following the template can build an extension that passes packaged checks but fails immediately in the UI.

2. Build tooling is documented but not installed/discoverable in the packaged app

Docs and skill guidance say to run:

neon-pilot-extension build <extension-dir>
neon-pilot-extension doctor <extension-dir>

But neon-pilot-extension was not available on PATH and was not discoverable inside the packaged app resources. npx neon-pilot-extension also failed because it is not a public npm package.

Workaround used: the local source checkout at ~/personal/personal-agent provided:

pnpm --dir ~/personal/personal-agent run extension:build -- ~/.local/state/neon-pilot/extensions/thread-kanban

Impact: this is fine for Patrick with a repo checkout, but it is not a packaged-app local extension workflow.

3. The packaged neon-pilot CLI wrapper is broken

The installed CLI wrapper points to:

/Applications/Neon Pilot.app/Contents/Resources/server/dist/protocolCli.js

That file does not exist in the packaged app, so commands like validation/reload through the CLI failed with:

Cannot find module '/Applications/Neon Pilot.app/Contents/Resources/server/dist/protocolCli.js'

Impact: I could not use the documented CLI route to validate, reload, or administer the extension from the packaged runtime.

4. Static/package validation did not catch the runtime frontend API mismatch

The extension passed:

node ~/personal/personal-agent/scripts/check-packaged-extensions.mjs ~/.local/state/neon-pilot/extensions/thread-kanban

Result:

backend ok, frontend ok, manifest ok

But the UI still failed because validation did not execute the page against the real pa client shape.

Impact: current validation catches manifest/dist/export/import issues, but not smoke-level UI/runtime integration failures.

5. Worker action requirement was not surfaced by validation

After fixing the frontend action bridge, the runtime reported:

Extension "thread-kanban" action "threadKanbanLoad" must declare worker.enabled before it can run.

Fix applied: all backend actions now include:

"worker": { "enabled": true }

Impact: the packaged extension check still reported backend/frontend/manifest ok before this runtime failure. If user extensions now require worker-enabled actions, templates and validation should require or autofix this.

Follow-up: after adding worker.enabled and rebuilding, Extension Manager reload still surfaced the same error. The source manifest on disk was correct, so this points at stale manifest caching / reload not invalidating extension registry entries. In the source checkout, listExtensionEntries() has a process-level read cache; if reload does not call the registry cache invalidation path, actions can continue using the old manifest until full app restart.

6. Local extension discovery required a restart/manual registry edit

After writing the extension folder, it was not immediately discoverable. I had to add thread-kanban to:

~/.local/state/neon-pilot/extensions/registry.json

Then the app needed restart/reload to show the nav item.

Impact: local extension creation lacks a hot-discovery path. A user-created extension should appear in Extension Manager immediately, ideally with an explicit “install local folder” / “rescan local extensions” / “create extension” action that updates the registry and reloads contributions.

Missing information / unclear points

• The canonical frontend action API is unclear: docs/templates say pa.actions.call, packaged examples say pa.extension.invoke.
• The expected local-extension root and registry lifecycle are underdocumented for packaged-app users.
• The validation workflow assumes tooling exists (neon-pilot-extension, working neon-pilot CLI), but this install did not have it.
• It is unclear whether editing registry.json directly is acceptable, or whether Extension Manager should own all registration.
• There is no obvious command/tool from this agent environment to reload one extension after changing dist/.
• Validation docs say to visually inspect the route, but there is no reliable automated app-path smoke command exposed for a newly contributed route.

Why validation felt missing

There is a static/package validation step, but not an end-to-end extension smoke test. For this extension we need something like:

neon-pilot extensions create --template main-page --id thread-kanban
neon-pilot extensions build thread-kanban
neon-pilot extensions validate thread-kanban
neon-pilot extensions reload thread-kanban
neon-pilot extensions smoke thread-kanban --route /ext/thread-kanban

The smoke step should open the route in the running app or a headless Electron harness, mount the extension with the real pa client, call declared backend actions, and fail on visible runtime errors.

Why immediate discoverability failed

The app appears to read enabled extension IDs from registry.json and load contributions from that set. Creating a folder under ~/.local/state/neon-pilot/extensions did not automatically register/enable it in the running host. Without a working CLI reload/create action, the only route was direct registry editing plus restart/reload.

Expected behavior for first-class local extensions:

1. Extension Manager “Create” creates the folder, adds it to the registry, enables it, builds it, and reloads contributions.
2. Extension Manager watches the local extension directory or has a prominent rescan action.
3. Newly enabled main-page/nav contributions appear without full app restart.
4. Validation diagnostics and reload status are visible after every build.

Validation story needed without source-code access

Local extension authors and agents should not need the Neon Pilot source checkout to prove an extension works. The packaged app needs to ship validation utilities that operate against either a simulated extension host or the running app.

1. Packaged extension simulator

Ship a standalone utility, for example:

neon-pilot-extension simulate ./thread-kanban

The simulator should:

• read extension.json
• load dist/frontend.js in a browser-like React harness
• provide a mock but runtime-accurate pa client
• load dist/backend.mjs
• provide a mock ExtensionBackendContext
• support fixtures for conversations, storage, settings, workspace, selection, etc.
• fail on React render errors, missing frontend APIs, invalid backend calls, or runtime-policy violations

This would have caught the pa.actions.call(...) mismatch before the user opened the route manually.

Example fixture shape:

export default {
  conversations: [{ id: 'c1', title: 'Build kanban extension' }],
  storage: {},
  route: '/ext/thread-kanban',
  component: 'ThreadKanbanPage',
  smoke: async ({ screen, pa }) => {
    await screen.findByText('Thread Kanban');
    await pa.extension.invoke('threadKanbanLoad', {});
  },
};

2. Running-app smoke runner

Also ship a command that talks to the running packaged app, for example:

neon-pilot extensions smoke thread-kanban

The running-app smoke runner should:

• invalidate and reload the extension registry
• verify the runtime manifest hash matches the manifest on disk
• verify the extension is installed and enabled
• invoke backend actions through the real extension host
• open contributed routes in the real app
• collect route render errors, error-boundary output, and console errors
• print machine-readable diagnostics for agents

This would catch integration problems the simulator cannot, including stale manifest cache, missing permissions, worker requirements, nav/route contribution failures, and real pa client mismatches.

3. Minimum viable developer command

A first useful packaged command could be:

neon-pilot extensions dev ./thread-kanban --open --smoke

It should:

1. register the local folder
2. build if source changed
3. validate manifest/dist files
4. enable the extension
5. reload host contributions and invalidate manifest caches
6. open the contributed route
7. run a smoke action or self-test if declared
8. print diagnostics

4. Extension self-test contract

Extensions should be able to declare a self-test without the runner knowing domain details:

{
  "backend": {
    "actions": [
      {
        "id": "threadKanbanSelfTest",
        "handler": "selfTest",
        "worker": { "enabled": true }
      }
    ]
  },
  "contributes": {
    "dev": {
      "smokeAction": "threadKanbanSelfTest",
      "smokeRoute": "/ext/thread-kanban"
    }
  }
}

The self-test should use temporary or isolated storage and avoid destructive operations. The runner invokes it through the same backend action bridge used by the frontend.

5. Validation levels

The complete story should have three levels:

1. Static/package validation — manifest parses, dist files exist, component/action exports exist, forbidden imports are absent.
2. Simulator validation — frontend/backend run with a realistic mocked host and fixtures.
3. Running-app validation — the installed extension is hot-loaded, actions invoke through the real host, routes render in the real app, and diagnostics are clean.

Manual user testing should be optional confirmation, not the primary validation mechanism.

Concrete product/tooling recommendations

• Fix packaged neon-pilot CLI path so extension commands work from the installed app.
• Ship or expose neon-pilot-extension in the packaged app, or replace docs with neon-pilot extensions build/doctor commands that actually exist.
• Update templates to use pa.extension.invoke(...), or restore/support pa.actions.call(...); do not document both ambiguously.
• Add a runtime smoke validator for extension surfaces.
• Add a packaged extension simulator for local validation without a source checkout.
• Add a running-app smoke runner that can open routes and invoke backend actions through the real host.
• Add Extension Manager local create/import/rescan/reload actions that an agent can call without editing registry.json.
• Make new local extensions immediately discoverable and hot-load nav/view contributions after enablement.
• Have validation compare frontend bundle expectations against the actual runtime client API shape where possible.
• Ensure extension reload invalidates manifest/registry caches and verifies the runtime manifest hash changed.

Current fix applied

The UI error was fixed by replacing all pa.actions.call(...) calls with pa.extension.invoke(...) and rebuilding the extension.