Skip to content

Instantly share code, notes, and snippets.

@nerdalert

nerdalert/jul20-grid-summary.md

Last active July 21, 2026 15:11
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save nerdalert/811b237e6f08354742f400b2fa6519a7 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save nerdalert/811b237e6f08354742f400b2fa6519a7 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
jul20-grid-summary.md

AI Grid Progress Summary — July 20

Summary

AI Grid has moved from architecture sketch to repeatable Kind validation for the core routing story, including the llm-d-compatible provider-gateway path and native credential-injection path.

  • The routing foundation is now real enough to review: multi-cluster routing, API fallback, full-grid mixed backend routing, metrics-driven routing, GridSite discovery, SWIM encryption, lost-peer route-away, /v1/responses, llm-d-compatible routing, operator install/RBAC, and provider access-policy enforcement all have repeatable validation coverage.
  • The next work is not adding more demo surfaces. The highest-value work is hardening the implementation behind the demos: SWIM sender/origin binding, per-network SWIM isolation, explicit remote transport, mTLS readiness, metrics egress hardening, and recovery/GC semantics.
  • The main architecture split is now implemented: Praxis core stays generic, while AI/Grid route semantics live in the AI repo.
  • Temporary development images now make the current end-to-end stack reproducible by another developer while upstream Praxis AI PRs are still being merged.
  • The remaining design decisions are around production image ownership, production credential lifecycle, stale-state policy, GridSite lifecycle automation, metrics normalization, and deferred product surfaces like KV cache affinity, MCP federation, and budgets.

July 20 Snapshot

Current Grid baseline:

Grid repo: praxis-proxy/grid
Grid commit: 85f9ba9 feat(grid): enforce provider access policy

Temporary AI rollup branch for reproducible Grid E2E images:

The rollup branch contains the AI-side pieces needed by current Grid E2Es:

  • llm-d ext_proc moved into Praxis AI.
  • grid_route.
  • grid_credential_inject.
  • integrations/llmd/mock-epp, a deterministic mock EPP test server.
  • Containerfiles needed to build the temporary Grid E2E gateway and mock EPP images.

This branch is intentionally a temporary integration branch. It is useful for reproducible validation while the underlying upstream PRs are still being reviewed, but it is not the desired long-term dependency shape.

Fresh temporary GHCR images were built from the Grid commit and AI rollup branch and validated with zero local images:

Image Tag Digest prefix
ghcr.io/nerdalert/praxis-ai dev-20260720-a3e2a3b sha256:45a742db9fce
ghcr.io/nerdalert/praxis-ai-mock-epp dev-20260720-a3e2a3b sha256:2feec91585b6
ghcr.io/nerdalert/grid-mock-providers dev-20260720-a3e2a3b sha256:31221952aaf3
ghcr.io/nerdalert/grid-operator dev-20260720-a3e2a3b sha256:4ad8166b54a0

Temporary image override block:

export GRID_XTASK_GATEWAY_IMAGE=ghcr.io/nerdalert/praxis-ai:dev-20260720-a3e2a3b
export GRID_XTASK_MOCK_EPP_IMAGE=ghcr.io/nerdalert/praxis-ai-mock-epp:dev-20260720-a3e2a3b
export GRID_XTASK_MOCK_PROVIDER_IMAGE=ghcr.io/nerdalert/grid-mock-providers:dev-20260720-a3e2a3b
export GRID_XTASK_OPERATOR_IMAGE=ghcr.io/nerdalert/grid-operator:dev-20260720-a3e2a3b
export GRID_XTASK_IMAGE_PULL_POLICY=IfNotPresent

Validation result:

verify-llmd-compatible-routing:
  6/6 steps PASS
  24/24 assertions PASS
  0 pod restarts
  all provider/consumer pods used ghcr.io/nerdalert/* images

The prior llm-d failure was not a Grid/Praxis/mock-EPP bug. It was a host-level Kind issue: low fs.inotify.max_user_instances caused kube-proxy and CoreDNS failures, which broke cluster DNS and prevented the provider gateway from resolving mock-epp.

External replication note: these ghcr.io/nerdalert/* images are temporary development artifacts so reviewers can reproduce the current E2E behavior without rebuilding every local branch. They should move to project-owned image names after the Praxis AI PRs merge and publish.

Relevant upstream PRs:

Area Link Current role
llm-d ext_proc in AI praxis-proxy/ai#334 Canonical home for the AI-owned llm-d/ext_proc path
grid_route in AI praxis-proxy/ai#339 Canonical home for Grid overlay routing inside Praxis AI
grid_credential_inject in AI praxis-proxy/ai#386 Canonical home for request-time Secret-file credential injection
Peer identity trust in Praxis core praxis-proxy/praxis#803 Merged generic trust primitive used by Grid/Praxis mTLS hardening
Response-header lifecycle for ext_proc praxis-proxy/praxis#776 Merged generic ext_proc lifecycle support needed by the llm-d path

Architecture Shape

The system is settling into three layers:

  • Praxis core provides generic proxy infrastructure: filter runtime, routing hooks, TLS identity, config loading, and generic security primitives.
  • The AI repo owns AI-specific filters and protocol behavior, including Grid route inference/MCP routing.
  • The Grid operator owns control-plane state: provider inventory, scoring inputs, routing overlays, SWIM/CRDT membership, and Kubernetes-facing CRDs.
Layer Responsibility Current status
Praxis core Generic proxy runtime, filter registry, HTTP/TLS plumbing, peer identity, generic trust/security hooks Mostly established; should not grow Grid-specific semantics
AI repo AI filters, API adapters, llm-d/EPP integration, Grid route inference/MCP filters grid_route inference/MCP routing is now AI-owned and registered by praxis-ai-proxy
Grid operator CRDs, provider inventory, scoring, overlays, SWIM/CRDT state, Kind validation Core routing path is Kind-proven
Grid/xtask generated config Demo consumer/provider gateway config, route candidates, header injection Demo-proven, but some behavior still harness-owned
SWIM/CRDT control plane Membership, distributed provider propagation, stale detection Proven for core cases; recovery/TTL/seed management remain

Praxis Control-Plane Positioning

Praxis is the data plane. It can run from a static config file today, but that does not make static config the only control-plane story.

Mode Config source Current meaning
Static file Human, script, Ansible, GitOps, or test harness Local dev, VM/standalone deployments, and focused validation
Grid Grid Operator AI Grid-specific control plane for model/provider routing overlays
Istio / xDS-like Istio or compatible service-mesh control plane Intended general-purpose mesh/Gateway API integration path
Praxis operator praxis-proxy/operator Current Gateway API conformance/testing vehicle; not assumed as the long-term product control plane

Short flow:

Gateway API / Istio resources
  → Istio control plane
  → xDS-like config stream
  → Praxis gateway as an alternative data plane

GridNetwork / GridSite / InferenceProvider
  → Grid Operator
  → AI routing overlay / generated config
  → Praxis AI filters route model traffic

Grid should be described as a control plane for Praxis AI Grid deployments, not as the universal Praxis control plane. The broader service-mesh target is Istio/xDS or an Istio-compatible configuration source.

Claimable Demo Summary

These are the current engineering-review claims. The table intentionally separates “demo-proven” from production completeness.

Demo Current status What is proven Main caveat
Basic multi-cluster model routing Enabled / Kind-proven Consumer routes model-east and model-west through provider gateways, ext_proc, mock EPP, endpoint selector, and inference-sim Real production llm-d topology is not the claim
API provider fallback Enabled / Kind-proven, caveated Mock API fallback and gateway credential injection are proven; Secret-backed CRD-to-Kubernetes-Secret-to-Praxis config path is validated in xtask Production Secret delivery and gateway reload ownership still need integration
Demo 3: Full Grid — Clusters + Cloud + APIs Enabled / Kind-proven One consumer gateway routes across local, remote, cloud-managed mock, and API-provider mock backends Real cloud protocols/auth such as SigV4/OAuth2 are not exercised
Demo 4: Metrics-Driven Load Balancing Enabled / smaller version Kind-proven Prometheus-format queue-depth metrics change overlay order and request routing to the lower-queue backend API-provider spillover under saturation still needs policy semantics
Demo 7: Site Join and Discovery Enabled / discovery v1 + lifecycle v1 Kind-proven Operator creates GridSite from SWIM Alive membership; lifecycle preservation, routing metadata, and isolation are proven Cert exchange and automatic transition to Active remain
Demo 9: Failover Under Partition Advanced validation / route-away Kind-proven Lost peer becomes fresh=false; shared-model traffic routes through healthy east fallback Real network partition, recovery, and stale-record GC/TTL remain
/v1/responses routing Enabled / Kind-proven openai_responses_format parses Responses API requests and grid_route routes them through the same overlay model This is request-format support, not a separate Grid CRD feature
Operator install/RBAC Enabled / Kind-proven In-cluster operator Deployment, split RBAC, Secret/ConfigMap writes, and reconcile proof are validated Deployment package hardening and production image ownership remain
Provider access policy Implemented / unit-tested Local and remote CRDT providers are filtered before entering consumer overlays GridNetwork.spec.networkPolicy is still not implemented
Demo 5: KV Cache Affinity Deferred Not claimable yet Needs real prefix-cache signal source and provider-vs-endpoint routing design
Demo 8: MCP Tool Federation Deferred Not claimable yet Tool catalog gossip, policy filtering, session semantics, and failover are separate product work
Demo 6: Budget Enforcement Deferred Not claimable yet Needs budget/accounting/enforcement CRD/API design first

Praxis / AI Filter Boundary

The filter-location decision is no longer open: Grid route inference and MCP routing have moved to the AI repo.

  • Praxis should keep generic mechanics: filter registration, context propagation, mTLS identity, header/routing primitives, and generic trust enforcement.
  • AI should own model/provider/tool semantics: grid_route, inference candidates, freshness/locality policy, MCP tool candidates, and AI protocol behavior.
  • Closed Praxis PRs #802 and #804 were ported to AI rather than reopened in Praxis.
  • Praxis PR #803 is different: peer identity enforcement is a generic security primitive and can remain in Praxis if named/scoped generically.
Item Direction Reason
grid_route inference routing Moved to AI repo Model names, providers, locality, freshness, and overlays are AI/Grid semantics
grid_route MCP tool routing Moved to AI repo MCP tool routing is an AI/MCP product concern
mTLS peer identity enforcement Keep generic primitive in Praxis Allow/deny by peer identity is generally useful proxy security behavior
Temporary Grid proxy server Avoid unless blocked It creates a second data-plane path to explain, test, and later remove

Completed AI repo work in the temporary rollup:

  • Recovered the closed #802/#804 behavior and ported equivalent filters into praxis-ai-filters.
  • Registered grid_route through the AI registry so Praxis core does not need to know Grid semantics.
  • Updated AI’s pinned Praxis SHA to current upstream with the needed generic primitives.
  • Moved docs/examples into AI and verified Grid-generated config uses grid_route -> load_balancer without an intervening router.
  • Added a small mock EPP test-support image so the llm-d-compatible path can be validated without requiring a full production EPP service.

Grid Operator Capabilities

The operator now handles more than static overlay rendering. It has a working control-plane loop for local providers, remote CRDT providers, metrics, SWIM membership, stale state, and GridSite discovery.

  • Overlay generation is the central integration point. The operator ranks candidates and writes config that the data plane consumes.
  • SWIM and CRDT now provide real distributed state propagation for remote provider inventory.
  • Metrics are wired into scoring through configured endpoints, but the metric contract is still narrow.
  • GridSite discovery v1 is now operator-owned when explicitly enabled.
Capability Current implementation Remaining gap
Routing overlay generation Local and CRDT-remote providers become ordered route candidates Final data-plane ownership depends on AI-side grid_route migration
Candidate ordering Score ordering plus fresh=true before fresh=false tiebreaker Need recovery and explicit stale policy
Provider health/freshness HTTP/HTTPS probes and phase-to-fresh mapping mTLS health probing not wired
Metrics scoring metricsConfig scrapes Prometheus-format queue-depth during reconcile Raw vLLM/llm-d metric normalization undecided
SWIM membership foca-backed runtime forms membership and reports connected sites CRD-driven runtime seed management remains
CRDT propagation Provider snapshots gossip over SWIM and feed overlay input TTL/stale-age behavior remains
GridSite discovery Opt-in GridNetwork label creates GridSite for Alive SWIM peers Cert exchange and automatic phase transitions remain

Credential and API Provider Handling

The API-provider path has moved beyond the original harness-only shape, but production Secret delivery and rollout ownership still need operational integration.

  • InferenceProvider.spec.auth.secretRef is now controller-validated for the implemented bearer_token path.
  • The operator projects only credential references into the routing overlay. Token bytes are not written to Grid status, overlay ConfigMaps, routing candidates, or generated consumer Praxis ConfigMaps.
  • GatewayRef.consumerConfig can render a Praxis config that combines grid_route candidates with file-backed grid_credential_inject configuration.
  • The request-time injection filter lives in Praxis AI. Runtime deployments must use a Praxis AI image that includes grid_credential_inject.
  • Credential placement follows the final-hop rule: mount the Secret only where the final backend call is made. Direct API/cloud fallback usually injects at the ingress gateway because it is also the egress gateway. Remote Grid sites keep provider credentials in the provider site or provider-side component.
  • Kubernetes Secrets are the v1 backend, but the design still leaves room for Vault, External Secrets, OAuth2, SigV4, and workload identity.
Area Current state Next step
Credential source spec.auth.secretRef validated by the controller for bearer_token Keep resolver interface open for non-Kubernetes backends
Secret backend Kubernetes Secret v1 Add documented production delivery patterns such as External Secrets or Vault
Config delivery Native path writes only file: references into generated Praxis config Ensure deployed Praxis AI image includes grid_credential_inject
Injection scope Final-hop egress component injects; direct API fallback proven in Kind Add remote-provider credential-placement validation when provider-side injection is wired
Rotation Secret/config changes require gateway restart or reload by deployment owner Define rollout/reload automation

Site Join and Discovery

GridSite discovery has crossed the important threshold from harness-created to operator-created.

  • When the GridNetwork has the opt-in label, the operator observes remote Alive SWIM members and creates a GridSite.
  • The created GridSite includes network reference, egress address, and starts at phase=Discovered.
  • The lifecycle state model exists and phase preservation is proven, but automatic progression beyond Discovered is still future work.
  • The next major feature is trust bootstrap: certificate exchange, identity verification, and readiness-driven transition to Active.
Phase / component Current behavior Outstanding
Discovery trigger Remote Alive SWIM member Opt-in UX may become CRD field/default later
GridSite creation Operator applies {network}-{site_id} GridSite Need stale cleanup/TTL policy
Lifecycle Harness can patch phases; operator preserves them Operator does not drive transitions beyond Discovered
Trust bootstrap Not implemented Cert exchange and identity verification
Data-plane readiness Metadata exists Need readiness checks before Active

Failover and Partition Behavior

Failover is now more than an overlay-only signal, but it is still not the full partition story.

  • The operator observes lost SWIM peers and downgrades their CRDT providers to Degraded.
  • The overlay keeps stale candidates for observability but marks them fresh=false.
  • Shared-model route-away is proven: after west is killed, the healthy east candidate wins and the consumer request returns 200.
  • The production policy direction is now: deprioritize and retain stale candidates for observability, but add garbage collection for records that have been stale too long.
  • The fresh / stale naming is inherited from the current overlay shape. The production semantics are really liveness/eligibility signals: healthy candidates are preferred; degraded candidates remain visible but should not win while a healthy alternative exists.
Capability Current proof Remaining work
Lost peer detection Process kill drives SWIM Dead/Suspect behavior Add real network partition simulation
Stale signaling Remote candidates become fresh=false Add stale-age/TTL policy and GC
Route-away Shared model routes to healthy fallback Keep stale candidates deprioritized and retained, not hard-excluded
Recovery Not proven Prove rejoin restores fresh=true
Observability Stale candidate remains in overlay Add eventual cleanup for records that do not recover

Metrics and Scoring

Metrics-driven routing is proven for a bounded case, not yet a complete capacity policy.

  • The current validation uses a Prometheus-format metrics endpoint that exposes normalized queue depth.
  • The operator scrapes that endpoint during reconcile and changes route order based on queue depth.
  • This should not be described as “Prometheus server integration.” It is endpoint scraping of Prometheus-format metrics.
  • The preferred direction is destination-owned normalization for cluster-backed providers: each destination publishes normalized capability/metric signals that reflect its own hardware, scheduler, and backend behavior.
  • Cloud providers and third-party APIs are the exception: because they do not expose cluster-local metrics directly, the source side or an adapter will need to convert provider-specific signals into the same normalized contract.
Topic Current state Direction
Queue depth Prometheus-format endpoint scraped during reconcile Keep wording precise
Normalization Demo uses pre-normalized 0.0-1.0 values Prefer destination-owned normalization for clusters; use source/adapters for cloud/API providers
Invalid values Happy path works Handle NaN/out-of-range/stale values
Saturation spillover Not claimed Define when API provider beats self-hosted capacity
KV cache affinity Deferred Define signal ownership and routing layer split

Deferred Demo Surfaces

The deferred demos are valuable, but starting them now would pull effort away from hardening the validated routing/SWIM foundation.

  • KV cache affinity depends on signal ownership: Grid-level provider routing and llm-d/EPP endpoint routing must not fight each other.
  • MCP federation is a separate product surface. Moving grid_route MCP routing to AI is only the first step, not full federation.
  • Budget enforcement is too large to treat as a routing demo. It needs identity, accounting, policy, enforcement, and API design first.
Deferred area Why deferred Revisit trigger
KV Cache Affinity Needs prefix-cache signal source and provider-vs-endpoint routing design After metrics normalization is settled
MCP Tool Federation Needs distributed catalog, policy filtering, session semantics, failover After AI-side MCP routing and Grid capability records are stable
Budget Enforcement Needs budget/accounting/enforcement CRD/API design After credential and request-accounting ownership is clear

Validation and CI Posture

The project is using validation as the forcing function for architecture claims.

  • Static gates are consistently part of the workflow: fmt, check, test, lint, doc, and diff-check.
  • Production coverage is above the 80% gate, with xtask excluded because it is a Kind harness.
  • Kind validations are repeatable but should run sequentially, especially SWIM-heavy tests.
  • Generated artifacts such as coverage.json are ignored.
Area Current state Notes
Static gates fmt/check/test/lint/doc/diff-check used on implementation chunks Continue before commits
Coverage Production coverage gate passes above 80% xtask excluded from threshold
Kind validations Core demo commands are repeatable Run SWIM-heavy validations sequentially
CI hardening Coverage, MSRV, unicode safety workflows added Keep aligned with Praxis/AI practices

Test-Driven Development Strategy

The best pattern so far is: state the product claim, encode it as a repeatable validation command, then add unit tests around the logic seams that make the command reliable.

  • Start with a claim: what should the system prove after this change?
  • Add pure tests for policy logic before wiring Kubernetes or networking.
  • Add negative tests at production seams: missing Secret, invalid metric, stale peer, bad seed, empty candidates.
  • Keep xtask as the validation harness. If xtask owns product behavior, the claim is weaker.
  • Require a Kind proof for any externally stated demo claim.
  • Keep docs honest by saying what the validation does not prove.
Test layer Purpose Current examples
Pure unit tests Pin deterministic logic without Kubernetes/networking scoring, freshness ordering, CRDT merge, GridSite naming
Controller unit tests Validate controller helper behavior discovered-site extraction, grid ID resolution, staleness override
Schema/config tests Prove generated config matches runtime expectations Grid overlay-to-filter shape, AI/Praxis examples
Focused integration tests Validate one runtime seam AI registry tests, mock API credential proof
Kind E2E Prove user-visible product claims demo routing, metrics, discovery, failover commands
Regression suite Prevent prior demos from breaking sequential core Kind validations
Coverage gate Catch production logic with no tests production crates over 80%; xtask excluded

Current Decision List

These are the real remaining architecture decisions. Filter location is not listed because that decision is implemented: Grid route filters live in AI.

  • Temporary Grid proxy server: avoid a second proxy path by default. Temporary sites are valid, especially in forest-style architectures where providers regularly come and go and need a drain period before shutdown.
  • API credential projection: start with Kubernetes Secret, but keep internal mechanics generic so future backends can support Vault, External Secrets, OAuth2, SigV4, and workload identity.
  • fresh=false semantics: deprioritize and retain. Add garbage collection for records that stay degraded/stale beyond a defined window.
  • GridSite auto-discovery enablement: move from opt-in label to CRD-owned configuration for the long-term product shape.
  • Metrics normalization: prefer destination-owned normalized metrics for cluster-backed providers; use source-side adapters/conversion for cloud providers and third-party APIs.
  • Deployment packaging: start with raw YAML/Kustomize as the canonical installable base. Add Helm and OLM/OperatorHub packaging after image publishing, CRD packaging, RBAC shape, and examples stabilize.

Outstanding Questions

These are the main unresolved questions that affect product shape or production-readiness.

Question Why it matters
What is the canonical upstream shape for llm-d/EPP support in Praxis AI? The temporary rollup proves the path works, but the real work should land as maintainable AI PRs rather than one combined dev branch.
Should the mock EPP test server live in the llm-d PR or a separate AI test-support PR? Grid needs one reliable llm-d compatible smoke test, but the mock EPP is AI-owned test infrastructure, not Grid product code.
What is the production image namespace and release process? The ghcr.io/nerdalert/* images are temporary. Project-owned image names, tags, SBOM/signing, and publish permissions still need to be decided.
How should Grid prove a gateway loaded the generated config? Today Grid can render/apply a ConfigMap, but cannot prove Praxis loaded that version without a gateway status/reload handshake.
What is the exact split between Grid routing and llm-d endpoint selection? Grid should choose the eligible site/provider/service. llm-d should choose the local backend endpoint. The boundary needs to stay explicit to avoid competing schedulers.
Where should sticky/session affinity live? Likely as a later routing-policy layer using conversation/request identity, but it should not be mixed into the current correctness/security work.
What is the policy model for unknown health, unknown cost, and stale remote state? Current scoring can still be too optimistic. Production needs explicit policy for whether unknown or stale candidates are routable.
What is the production Secret delivery model? Kubernetes Secret works for v1, but Vault, External Secrets, cloud identity, OAuth2, and SigV4 need a clear integration story.
How should multi-network Grid tenancy be isolated at the SWIM layer? The current process-global SWIM runtime is acceptable for Kind validation but not a final multi-tenant transport model.

Open Praxis AI PR Blockers

These AI PRs are the current blockers for replacing the temporary ghcr.io/nerdalert/* validation images with project-owned Praxis AI images. The temporary rollup branch proves the pieces work together, but it should not be treated as the permanent integration branch.

  • praxis-proxy/ai#334 — move llm-d ext_proc compatibility into AI.

    • Status: Open.
    • Blocks: canonical AI-owned ext_proc path for llm-d/EPP-style routing.
    • Temporary workaround: included in nerdalert/ai:temp/grid-e2e-images-rollup and validated through verify-llmd-compatible-routing.

  • praxis-proxy/ai#339 — add grid_route.

    • Status: Open; depends on Praxis dependency/version alignment.
    • Blocks: canonical AI-owned Grid overlay routing filter.
    • Temporary workaround: included in the rollup image used by Grid routing, Responses API, and credential-injection E2Es.

  • praxis-proxy/ai#386 — add grid_credential_inject.

    • Status: Open; depends on #339.
    • Blocks: canonical request-time provider credential injection from mounted Secret files.
    • Temporary workaround: included in the rollup image used by the native credential-injection E2E.

  • AI mock EPP test-support image.

    • Status: Not yet a separate upstream PR.
    • Blocks: reproducible llm-d-compatible E2E without a real EPP deployment.
    • Temporary workaround: implemented in the rollup at integrations/llmd/mock-epp and published as ghcr.io/nerdalert/praxis-ai-mock-epp:dev-20260720-a3e2a3b.

Plain-language status:

  • Grid can validate the integrated behavior today using the temporary rollup images.
  • The long-term path is still to land the AI PRs separately and publish project-owned images.
  • The mock EPP is test scaffolding for the llm-d ext_proc path. It is not the real production EPP implementation.
  • grid-mock-providers is different: it is a Grid-owned fake backend provider image used so tests do not call real OpenAI, Anthropic, Bedrock, or Vertex services.

Next Steps

The next sequence should harden existing claims before opening more surfaces.

  1. Land or split the Praxis AI PRs so ext_proc, grid_route, grid_credential_inject, and mock EPP test support can be built from canonical project branches.
  2. Move temporary image references from ghcr.io/nerdalert/* to project-owned image names after upstream publishing is available.
  3. Add a deployable Grid package using raw YAML/Kustomize first: CRD packaging, install assets, examples, and OpenShift/RBAC hardening. Decide later whether Helm and OLM/OperatorHub packaging are needed once the base install shape is stable.
  4. Continue the Grid hardening track: SWIM sender/origin binding, per-network SWIM/key isolation, explicit remote transport, mTLS readiness conditions, and metrics/health egress hardening.
  5. Add stale-record garbage collection and recovery validation for lost-peer routing.
  6. Move GridSite auto-discovery enablement from label-only to CRD-backed configuration.
  7. Define the normalized metrics contract, with destination-owned normalization for cluster-backed providers and adapter conversion for cloud/API providers.
  8. Make Praxis AI grid_route overlay hot reload production-safe: reload grid-config.json in-process with last-known-good retention, no pod restart, no whole-pipeline reload, no request-path file I/O, and clear Kubernetes ConfigMap mount rules.
  9. Tighten the scoring contract between Grid and Praxis AI: document exactly when Grid-rendered order is authoritative, decide whether the overlay needs an explicit priority, rank, or score field, and add tests proving how freshness, locality, and Grid ordering interact.

How to Test This Yourself

This uses temporary development images under ghcr.io/nerdalert/* so reviewers can reproduce the current E2E behavior without rebuilding the pending Praxis AI branches locally.

Prerequisites:

  • Docker.
  • Kind.
  • kubectl.
  • Rust toolchain used by Grid.
  • Host file/inotify limits high enough for multiple Kind clusters. If CoreDNS or kube-proxy fail with "too many open files", raise fs.inotify.max_user_instances and recreate the Kind clusters.

Clone Grid:

git clone https://github.com/praxis-proxy/grid.git
cd grid
git checkout 85f9ba9

Use the temporary validated image set:

export GRID_XTASK_GATEWAY_IMAGE=ghcr.io/nerdalert/praxis-ai:dev-20260720-a3e2a3b
export GRID_XTASK_MOCK_EPP_IMAGE=ghcr.io/nerdalert/praxis-ai-mock-epp:dev-20260720-a3e2a3b
export GRID_XTASK_MOCK_PROVIDER_IMAGE=ghcr.io/nerdalert/grid-mock-providers:dev-20260720-a3e2a3b
export GRID_XTASK_OPERATOR_IMAGE=ghcr.io/nerdalert/grid-operator:dev-20260720-a3e2a3b
export GRID_XTASK_IMAGE_PULL_POLICY=IfNotPresent

Bring up the Kind environment:

cargo xtask env up -c tests/env/operator-routing-multisite.toml

Run the llm-d-compatible routing proof:

cargo xtask env verify-llmd-compatible-routing \
  -c tests/env/operator-routing-multisite.toml

Expected result:

6/6 steps PASS
24/24 assertions PASS
0 pod restarts

Optional additional checks:

cargo xtask env verify-responses-routing \
  -c tests/env/operator-routing-multisite.toml

cargo xtask env verify-operator-install-rbac \
  -c tests/env/operator-routing-multisite.toml

cargo xtask env verify-swim-encryption \
  -c tests/env/operator-routing-multisite.toml

Cleanup:

cargo xtask env down -c tests/env/operator-routing-multisite.toml

Notes:

  • These image references are temporary and should move to project-owned image names after the upstream Praxis AI PRs merge and publish.
  • GRID_XTASK_IMAGE_PULL_POLICY=IfNotPresent is required for these registry images. Without it, Grid xtask keeps the local-development default imagePullPolicy: Never.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment