The RHOAI 3.6 transition target is not for Praxis to call IPP. It is for Praxis
to replace IPP as Envoy's ExternalProcessor service while preserving the
MaaS body-based routing (BBR) behavior used by RHOAI 3.5. The gaps and
implementation phases in this document are the work required for RHOAI 3.6.
The planned release paths are:
A side-by-side analysis of the llm-d Inference Payload Processor (IPP) and Praxis, covering their architectures, design philosophies, integration models, and how they fit together in the broader llm-d and AI Grid ecosystems.
## Done vs Not Done
Done:
- Praxis has the core APIs needed for trusted pre-read routing mutations.
- The llm-d `ext_proc` compatibility filter has been moved into Praxis AI.
- Praxis AI can register the llm-d `ext_proc` filter behind a feature.
- The filter can open and manage EPP-style gRPC processor streams.
- Request-header and full-duplex request-body routing paths are implemented.
AI Grid has moved from architecture sketch to repeatable Kind validation for the core routing story, including the llm-d-compatible provider-gateway path and native credential-injection path.
/v1/responses, llm-d-compatible routing, operator install/RBAC, and provider access-policy enforcement all have repeatable validation coverage.AI Grid has moved from architecture sketch to repeatable Kind validation for the core routing story.
| Demo | Current status | Size to unblock | Deferred? | Notes |
|---|---|---|---|---|
| Demo 1: Basic Multi-Cluster Model Routing | Enabled / Kind-proven | Done | No | Exact two-provider llm-d-style path is proven. Consumer routes model-east and model-west through provider gateways, ext_proc, mock EPP, endpoint_selector, and inference-sim. |
| Demo 2: API Provider Fallback | Enabled / Kind-proven, with caveat | Done for demo; S for production hardening | No | Mock API fallback and gateway credential injection are proven. Packet 1 now validates Secret-backed CRD → Kubernetes Secret → Praxis config projection in xtask. Production controller-owned credential lifecycle remains future work. |
| Demo 3: Full Grid — Clusters + Cloud + APIs | Enabled / Kind-proven | Done | No | One consumer gateway routes across local, remote, cloud-managed mock, and API-provider mock. Selected-backend evidence is proven by matching response model fields. Real cloud/API protocols are not claimed. |
Praxis should keep three MCP operating modes available for now:
protocol_profile: current for the existing session-based broker behavior.protocol_profile: stateless for the MCP 2026-07-28 release-candidate broker behavior.
If the maas-api log shows a TLS handshake error like remote error: tls: bad certificate, this usually means the client side (Authorino) rejected the certificate rather than the other way around.
This looks like the Authorino-to-maas-api TLS path. maas-api serves HTTPS with an OpenShift service-serving cert, so Authorino needs the OpenShift service CA bundle when it calls the tier lookup endpoint. The Authorino CR does not expose generic env vars, so the current workaround is to mount the CA via the Authorino CR and set SSL_CERT_FILE / REQUESTS_CA_BUNDLE on the generated deployment.
There are two separate TLS connections in the MaaS auth flow:
Gateway -> Authorino (listener TLS) -- requires spec.listener.tls.enabled: true on the Authorino CR and the security.opendatahub.io/authorino-tls-bootstrap: "true" annotation on the gateway. On RHOAI/ODH this is usually already configured as a platform prerequisite.
Authorino -> maas-api (outbound TLS) -- Authorino makes HTTPS calls to maas-ap
| ```diff | |
| diff --git a/filter/src/builtins/http/ai/openai/responses/rehydrate/mod.rs b/filter/src/builtins/http/ai/openai/responses/rehydrate/mod.rs | |
| index 5e60a3f..8dc4ced 100644 | |
| --- a/filter/src/builtins/http/ai/openai/responses/rehydrate/mod.rs | |
| +++ b/filter/src/builtins/http/ai/openai/responses/rehydrate/mod.rs | |
| @@ -27,6 +27,7 @@ use crate::{ | |
| filter::{HttpFilter, HttpFilterContext}, | |
| }; | |
| +// ----------------------------------------------------------------------------- |
Praxis can sit between Codex and an upstream provider that supports the native OpenAI Responses API. Codex sends Responses requests to Praxis; Praxis rewrites the configured model name, adds effective-model routing metadata, and forwards the request to the selected backend.
The passthrough path preserves request fields such as input, instructions, tools, streaming flags, and function_call_output. Praxis does not run Codex tools or own the tool loop.