Repository navigation
feat: conform /v1/decisions to the GA Decisions schema and align with vLLM's request fields - #17
Merged
Merged
Conversation
Re-pin the conformance tests to the general-availability revision of OpenAI's Decisions schema (openai-openapi 8243cc8f, 2026-10-24) and make the route accept what that revision allows and this adapter can serve: - up to 400 questions per request on both routes (was 200) - `input_json` parts in `input`, rendered as compact JSON the way an object `state` is on /v1/systemone - a question's `instructions` as a string or text and JSON parts Image parts, developer, system and assistant messages and tool-call items stay unsupported and answer 400 in OpenAI's error shape; the README lists them under "Deliberate exceptions on /v1/decisions", and the spec tests pin each one. Translation errors for these now return 400 instead of escaping the handler. scripts/probe_check.py now checks 400/401 questions, and e2e_check.py checks that `input_json` and instruction parts answer like their plain equivalents. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
…emone vLLM now ships its own /v1/systemone and /v1/decisions. Follow its request and response additions where they leave the prompt unchanged, so a client written for vLLM works here, and say plainly what is not supported. Accepted: - /v1/systemone takes `cache_salt`, `chat_template_kwargs`, `joint`, the media fields, `priority` and `request_id`, at their defaults. `model` stays required. `chat_template_kwargs` is validated and dropped, never forwarded: the 8b chat template renders some of its keys unescaped, and the adapter name is always the server's. Only `enable_thinking: false` is allowed. - `cache_salt` is validated with vLLM's rule and set on every engine read, so tenants can keep their prefix caches apart. /v1/decisions also takes it and `joint: false`, the two extensions vLLM adds on that route. - /v1/systemone responses carry `diagnostics` per question (`label_mass`, `argmax_is_label`), as vLLM's do. /v1/decisions does not, because OpenAI's response schema forbids extra fields. Refused, with a message that names the part and says what to do: - developer, system and assistant messages and tool calls and outputs on /v1/decisions, and request-level `instructions` on /v1/systemone. Each would add text the adapter was not trained to read, so the prompt stays exactly as it was. Put what the questions are about in a user message. - images, and two choices that would merge into one option. `_answer_question` now returns a typed `QuestionDiagnostics` as a fourth element. The README lists what is and is not supported, and tests/test_openai_spec.py pins each unsupported request as a named case. scripts/e2e_check.py gains a `vllm_fields` check, including that a `cache_salt` gets its own prefix cache. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
planetf1
force-pushed
the
feat/openai-ga-conformance-vllm-alignment
branch
from
October 26, 2026 19:55
ed61fca to
5196bcc
Compare
planetf1
marked this pull request as ready for review
October 26, 2026 20:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OpenAI's Decisions API changed last week, and vLLM now ships its own decision endpoints. This brings
/v1/decisionsup to date with OpenAI's schema for text, and lets/v1/systemoneaccept the extra fields vLLM's endpoint has, so clients written for either work here as they are.What you get
input_jsonparts for passing structured data without turning it into a string. A question'sinstructionscan also be text and JSON parts.cache_saltkeeps one tenant's prefix cache apart from another's, so cache timing andcached_tokenscan't reveal what someone else sent. It works on both routes./v1/systemoneresponses now includelabel_massfor each question, which drops when the marker or layout is wrong. Before, it was only logged./v1/systemoneacceptschat_template_kwargs,joint,images,audio,videos,priorityandrequest_idat their defaults, instead of answering 422.What it doesn't do
The adapter reads one block of text and a question, so the prompt it sees is unchanged and these get a clear error instead of an answer:
instructionson/v1/systemonechat_template_kwargsis checked and never passed on, because the 8b chat template renders some of its keys without escaping. Onlyenable_thinking: falseis accepted.The README lists all of this.
How it was checked
vllm_fieldscheck inscripts/e2e_check.pypasses with the existing wire and probe checks, and a freshcache_saltstarts with an empty cache.Not checked: the responses haven't been compared with OpenAI's live service, and
VLLM_BATCH_INVARIANT=1hasn't been tried with this plugin.The first commit stands alone (the schema update). The second adds the vLLM fields and
label_mass.