Created
June 3, 2026 21:19
-
-
Save bartoszmajsak/b949c31c7488f31b1013e9f8e8af94b4 to your computer and use it in GitHub Desktop.
LLMInferenceService: Pipeline-parallel well-known configs gap
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Pipeline-Parallel Well-Known Configs Gap | |
| ## Summary | |
| The LLMInferenceService API accepts pipeline-parallel configurations (`spec.parallelism.pipeline > 0`), | |
| the controller has code paths to handle them, but no well-known config YAML files are shipped. | |
| This means a user can create a valid-looking spec that passes webhook validation but fails | |
| at reconciliation time with `ConfigNotFound`. | |
| ## Evidence | |
| **API validation accepts it:** | |
| - `pkg/apis/serving/v1alpha2/llm_inference_service_types_func.go:57` - `IsPipelineParallel()` returns true when `spec.parallelism.pipeline > 0` | |
| - `pkg/apis/serving/v1alpha2/llm_inference_service_validation.go:262` - validation allows `worker` when either `IsDataParallel()` or `IsPipelineParallel()` is true | |
| **Controller has code paths:** | |
| - `pkg/controller/v1alpha2/llmisvc/config_merge.go:56-58` - config name constants defined for pipeline-parallel variants | |
| - `pkg/controller/v1alpha2/llmisvc/config_merge.go:85-87` - names registered in the Presets list | |
| - Decision tree in `combineBaseRefsConfig()` handles the `IsPipelineParallel()` branch | |
| **But no configs are shipped:** | |
| - `config/llmisvcconfig/` has no pipeline-parallel YAML files | |
| - The names are in the Presets list but NOT in `WellKnownDefaultConfigs` (lines 92-101) | |
| ## Impact | |
| A user who sets `spec.parallelism.pipeline: 2` with `spec.worker: {}` will: | |
| 1. Pass webhook validation | |
| 2. Hit `ConfigNotFound` during reconciliation | |
| 3. See `PresetsCombined: False` with no clear indication that pipeline-parallel configs don't exist | |
| ## Options | |
| 1. **Ship the configs** - create `config-llm-worker-pipeline-parallel.yaml` (and prefill/decode variants) | |
| 2. **Reject at validation** - add a webhook check that rejects `IsPipelineParallel()` until configs are shipped | |
| 3. **Document the limitation** - if this is intentionally deferred, document that users must provide their own config via `baseRefs` | |
| Option 2 is the safest short-term fix - fail early with a clear message instead of failing late with a generic `ConfigNotFound`. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment