Repository navigation
Conversation
Introduce fit IR assign/append batches and lifecycle events with three inbound and four outbound SSE adapters. Preserve string boundaries and report final projections for a standalone IAT and per-protocol CBOR thin item implementation. Retain citation coordinate experiments and distinguish measured Responses code-point offsets from the provisional ChatCompletions interpretation. Cover incremental delivery, interleaving, Unicode ranges and lossless thin restoration in 34 tests.
Complete ordered image and text projection, finish IAT callbacks before terminal events, and preserve audio replay metadata and usage details. Keep selective thin references aligned with protocol item discriminators and retain unsafe JSON integers and original argument spelling. Add regression coverage for terminal consumption, pending images, JSON restoration, and invalid state propagation.
Centralize IR lifecycle traversal and defer audio classification until its streamed fields are known. Preserve generated image formats and out-of-range numeric lexemes, and keep empty text from claiming projection targets. Expose completed unnamed tools and invalid logprob ownership. Extend the regressions to 66 focused tests.
Preserve the full JSON wrapper as freeform tool input when its input field is absent or not a string, so clients can decide how to handle the call. Log the extraction failure without logging tool contents.
Replace nine pairwise return mappers with three native readers and four native writers inside translate. Keep request builders and public contracts unchanged, preserve source item identity and native error/refusal semantics, and keep IAT/thin utilities out of this change for a stacked draft. Move stream coverage to native async pipelines and cover the confirmed signature, refusal, custom input fallback, usage and terminal policies.
…pletion Keep structured ChatCompletions reasoning groups and source identities through late compatibility carriers. Preserve surrounding message order and add post-finish regression coverage. Remove test-only IR transport plumbing and redundant terminal checks after the private integration cleanup.
Use native response start and explicit choice completion in request-translation fixtures. Expect the upstream classifier explanation unchanged in translated Responses policy failures.
|
Codex: citation-coordinate investigation and probe evidence. Citation coordinate evidenceIn A via B, A is the client protocol and B is the upstream protocol; requests flow → and responses flow ←. Current interpretation
The IR coordinate unit is UTF-16 code units, preserving JavaScript string contents including lone surrogates. Converting an upstream code-point range into IR therefore requires calculating its corresponding UTF-16 offsets. The IR range type is ChatCompletions' code-point unit remains inferred. Responses' measured behavior is limited to the tested provider and URL-citation paths. Direct Copilot experimentsThe probes used The experiments ran on 2026-10-10 (Asia/Singapore). HTTP Retained Responses requestThe successful Unicode-prefix probe used this exact body at {
"model": "gpt-6-luna",
"input": [
{
"role": "user",
"content": [
{
"type": "input_text",
"text": "Use the built-in web search tool now to find the official Unicode character name for U+1F4D6. Begin the final answer exactly with the literal prefix A😀中B followed by one space. Then state the code point and official Unicode name in one short sentence, and include the literal symbol 📖 within that factual sentence. Do not type a URL, footnote, bracketed citation, or citation JSON; let the API provide server-generated source annotations."
}
]
}
],
"tools": [{ "type": "web_search", "search_context_size": "low" }],
"tool_choice": "required",
"include": ["web_search_call.action.sources"],
"reasoning": { "effort": "low" },
"text": { "verbosity": "low" },
"max_output_tokens": 1400
}The [
{
"role": "user",
"content": "Use the built-in web search tool to find the official Unicode source confirming the name for U+1F4D6. Begin the final answer exactly with the ASCII prefix ASCII followed by one space. Then state the code point and official Unicode name in one short sentence that contains the literal symbol 📖. Do not type a URL, footnote, bracketed citation, or citation JSON; allow the server to attach its source annotation."
}
]Responses: supplementary-character probeExact request model: {
"text": "A😀中B U+1F4D6 is officially named OPEN BOOK 📖. ([unicode.org](https://www.unicode.org/charts/PDF/U1F300.pdf?utm_source=openai))",
"annotation": {
"type": "url_citation",
"start_index": 46,
"end_index": 126,
"title": "The Unicode Standard, Version 17.0",
"url": "https://www.unicode.org/charts/PDF/U1F300.pdf?utm_source=openai"
}
}
The actual JavaScript code-unit slice at the reported coordinates was: Code-point slicing selected: Thus the reported offsets count code points, and the reported end equals the exclusive code-point text length. The supplementary characters precede the citation; no supplementary character inside the generated citation link was observed. Responses: independent checks
The ASCII-prefix control began Retained output and annotation for {
"text": "A😀中B U+1F4D6 is officially named OPEN BOOK 📖. ([unicode.org](https://www.unicode.org/Public/UCD/latest/charts/nameslist/1f300/?utm_source=openai))",
"annotation": {
"type": "url_citation",
"start_index": 46,
"end_index": 146,
"title": "Miscellaneous Symbols and Pictographs – Unicode 18.0.0",
"url": "https://www.unicode.org/Public/UCD/latest/charts/nameslist/1f300/?utm_source=openai"
}
}Retained output and annotation for the ASCII-prefix control: {
"text": "ASCII U+1F4D6 is officially named OPEN BOOK (📖). ([unicode.org](https://www.unicode.org/charts/PDF/U1F300.pdf?utm_source=openai))",
"annotation": {
"type": "url_citation",
"start_index": 49,
"end_index": 129,
"title": "The Unicode Standard, Version 17.0",
"url": "https://www.unicode.org/charts/PDF/U1F300.pdf?utm_source=openai"
}
}ChatCompletions: normal models
The successful normal The normal-generation prompt was The search attempts for {
"role": "user",
"content": "Use live web search now for an official Unicode source confirming the character name for U+1F4D6. Begin the response exactly with the literal prefix A😀中B followed by one space. State the code point and official Unicode name in one short factual sentence that includes the literal symbol 📖. Do not type a URL, footnote, Markdown citation, or citation JSON. If live search is unavailable, say so plainly."
}Their bodies set All three
These experiments establish ChatCompletions endpoint availability for the tested models. They do not measure citation offsets or prove absence of other citation mechanisms. Native ChatCompletions citation triggerThe inspected official web-search guide documents a dedicated search-model path: {
"model": "gpt-5-search-api",
"web_search_options": {},
"messages": [
{ "role": "user", "content": "What was a positive news story from today?" }
]
}Native returned citations are under The pinned official SDK associates message annotations with web search and declares the nested URL citation shape. Its request definition declares Public response capturesChatCompletionsActual SSE fixture, pinned revision: returned model After concatenating
The first two ends are followed by spaces; the final end equals the full character length. This establishes end-exclusive behavior in the capture. There are no supplementary characters in the body, so code point versus UTF-16 cannot be distinguished. A second ChatCompletions-shaped capture, with ResponsesOfficial Cookbook captured output, pinned revision: U+2019 appears at code-point position 531. Citation MessagesCaptured
These coordinates refer to the source document, not the generated assistant text. They cannot be copied into an IR output-text range. A Python consumer and another consumer slice source text using these character coordinates. Client and gateway interpretationsExplicit code-point interpretation: new-apiChatCompletions annotation conversion: runes := []rune(text)
return string(runes[start:end])The caller reads nested Conflicting direct client: PromptfooNative ChatCompletions annotation consumer: output.slice(citation.start_index, citation.end_index + 1).trim()The call site supplies Pass-through official clientOpenAI Agents JS ChatCompletions streaming adapter validates and retains Other consumers and limitations
Research included global web searches and global GitHub code searches across annotation field names, protocol names, slicing operations, coordinate encodings, and language-specific consumers. Schema declarations, pass-through adapters, URL-only displays, hand-authored annotations, and consumers of another protocol were not treated as proof of native ChatCompletions units. No inspected direct ChatCompletions client supplied a supplementary-character test establishing its coordinate interpretation. Remaining uncertaintyChatCompletions' unit remains inferred. Settling it requires an authentic native annotation response with a supplementary character before its range, or a primary upstream contract explicitly specifying the unit. The measured Responses behavior does not establish every provider or annotation type's contract. |
Move the detailed citation-coordinate investigation into the PR discussion and link the owning coordinate conversion comment directly to that evidence.
Keep core types and stream operations at the IR root and shared adapter support under ir/shared. Own native error conversion in each writer. Keep custom tool wrapping in the existing Responses translation helper outside IR.
Menci
changed the title
Stream scalar reasoning with native item interruptions, keep independent tools live, and retain text arrival order through buffered projections. Ignore incoming ChatCompletions reasoning_items and wait for stable tool identities. Preserve raw truncated arguments with native length/incomplete/max_tokens outcomes. GenerateContent keeps tool order, omits incomplete calls, and ends failures with compact Google JSON trailers. Recheck cancellation after writes and keep captured-stream copying consistent with the wire format. Validate with the complete repository verify pipeline (606 suites, 6761 tests), gated protocol-pair audits, actual SDK consumers, and converged comment/structure/defensive reviews.
# Conflicts: # apps/web/__tests__/components/requests/stream-render_test.ts # apps/web/src/components/requests/stream-render.ts
Expose known initial accounting before native start events and consume late cache breakdowns as whole snapshots. Preserve text arrival order across concurrent items and ordered tool buffering, rebase citations across interrupted source text, and keep logprobs with their original text owner. Normalize GenerateContent tier aliases at its output boundary and restore reasoning/output count validation. Add gated lifecycle, accounting, and ownership regressions; keep IR private to translate.
Emit the initial ChatCompletions role from the initialized start state and attach same-chunk citations and logprobs before tools close their text item. Verify both properties before pulling another upstream frame or relying on the terminal response snapshot.
|
Codex: accounting, timing and policy decisions with client evidence.
Downstream ChatCompletions supports continuous_usage_stats only together with include_usage. The mode follows vLLM's flag contract: latest reported cumulative usage is attached to actual choice chunks, including role, text, reasoning, tool, affinity carrier and finish chunks. It does not create usage progress frames. Unknown statistics stay null until upstream reports them. Responses frequently reports statistics only at completion, so this mode cannot invent intermediate counts. Per-choice counters remain distinct from the aggregate trailer. Default mode strips body statistics and emits only the latest empty-choices carrier immediately before DONE or normal EOF; failures do not flush pending statistics. Messages usage compatibility was reproduced with output 0 → 2 and progress/final snapshots of 2: Messages permits explanation:null; Responses requires error.message:string. Codex’s decoder supplies an OpenAI enrollment discriminator for absent/blank explanations; the truthful nonblank fallback prevents this provider-inappropriate UI. Initial non-null usage is accepted by the official SDK and appears in its own ResponseStream snapshots. The official SSE examples use null, but do not require discarding already-reported usage. Codex's catalog declares priority plus the additional fast spelling. Google has a native serviceTier field in its discovery schema. Nested cache replacement follows the nonstandard sender’s snapshot semantics. new-api constructs both TTL values, omits zero JSON members, and replaces the whole object including zero corrections, with a specific regression test. The Copilot gateway consumer also shallow-merges usage. Bifrost instead uses independent per-counter maxima; that can retain stale corrections and is a different accounting model. Native Messages delta usage does not declare the nested cache_creation field. TS/Python typed accumulators ignore this late extension; Go raw accumulation replaces each usage member rather than recursively merging it. The official TypeScript accumulator assigns cumulative output_tokens, so complete usage snapshots retain that count. Earlier envelopes are accepted, but do not inherently display earlier text. Real OpenAI 7.32.0 and Anthropic 0.133.0 gated SSE probes delivered one initial event callback and zero text callbacks; empty-body completion was accepted. Cline filters metadata/start parts, and its first-chunk timing measures normalized content. The start-before-block constraint has real client evidence in CLIProxyAPI #563. Copilot-api and CLIProxyAPI emit an immediate Messages start for response.created, while CLIProxyAPI's Chat translator waits for the first content delta. Both schedules are in use; this does not establish one required upstream-relative timing policy. Repeated cumulative ChatCompletions usage is unsafe as a universal extension. A mocked HTTP stream containing usage 10/0, 10/2, and final 10/4 produced correct final 10/4/14 in OpenAI Python 3.28.0 and Vercel OpenAI 4.0.91, but LangChain OpenAI 1.7.0 reported 30/6/36 both in its aggregate and on_llm_end callback. It converts each usage chunk and adds usage on aggregation. Official Chat stream usage is a final additional chunk. CLIProxyAPI emits repeated cumulative usage, but that example cannot override the reproduced client incompatibility. GenerateContent has a different consumer contract. Google JS 2.28.0 models and chats accepted usage-only frames, continued text AB, and retained chat history. LangChain Google 4.4.1 subtracts previous cumulative usage on content chunks; usage-only chunks instead produce empty-response warnings and are not exposed as application chunks. Vercel Google 4.0.93 records usage but exposes it only at final finish. This supports cumulative usage alongside actual content, but does not demonstrate a user-visible benefit from inventing pure usage frames. Known initial Responses usage is safe in the tested consumers: LangChain OpenAI 1.7.0 accepted created/in_progress 10/0/10 and final completed 10/4/14 without double counting. It only converts terminal usage into usage_metadata. This distinguishes initial response snapshots from repeated Chat usage chunks. The biology-prefix rule’s original rationale was Codex-specific UI. An injected prefix has a specific UX cost. Codex's special notice replaces the upstream explanation with fixed OpenAI Trusted Access enrollment links; its rendered snapshot confirms the displayed result. Anthropic has a separate Life Sciences Verification Program. An OpenAI enrollment cannot change the actual Anthropic classifier. Without the injected prefix, Codex still classifies bio_policy as non-retryable and retains the explanation. Copilot-api and Bifrost use content_filter rather than injecting the Codex prefix; no independently found Anthropic-to-Responses gateway used this prefix injection. Promptfoo and Inspect AI classify by code, without requiring it. The normative contract should not be confused with a universal first-party reporting cadence. Examined source recordings include genuine api.anthropic.com/v1/messages recordings in Braintrust's Claude Agent SDK cassettes, Anthropic SDK cassettes, and LangChain cassettes. All 25 examined streams carried one final usage delta after content blocks; the largest observed response was 475 output tokens. These third-party recordings support that common cadence, not a claim that long first-party generations can never report intermediate statistics. Streaming usage remains supported independent of that observation. |
Publish native initial envelopes immediately and capture downstream usage preferences before upstream accounting rewrites. Opted-in ChatCompletions choice chunks carry the latest reported cumulative counters, including affinity carriers and finish frames. Standard usage is coalesced into one terminal carrier. Validate both flag combinations through HTTP, per-trip preference ownership, unknown counters, gated initial envelopes, and per-choice affinity projections.
Preserve cumulative Messages usage progress. Supply nonblank policy error explanations when the upstream has none, and publish legacy reasoning carriers as each item completes before finish_reason. Reject replacements or parent-array deletions of text already emitted to scalar destinations, including closed blocks. Reconcile buffered text replacements and removals before delivery. Cover these policies with gated lifecycle and native-source snapshot regressions.
Introduce explicit initialize operations for unpublished skeletons and restrict append to strings. Array creation uses contiguous indexed initialize or assign operations. Track actual field publication independently from structural state so native writers never infer content from placeholders or frame grouping. Adapt all readers and writers, preserve native lifecycle envelopes, and cover skeleton suppression, actual empty values, strict array indexing, and regrouped frames.
|
Codex: final IR stream performance evidence. Workload: 4,000 ChatCompletions chunks with one streamed text token and one unique logprob token per chunk, plus 20 unique
The workload retains roughly 84,000 unique token/top-logprob objects. IR does not deep-copy those opaque leaf objects: the producer record and the two copy-on-write collectors share them. Most retained heap is therefore the source object graph whose lifetime is extended until terminal snapshots, plus array references and collector containers. Writer-specific retained state adds about 0.06–0.13 MiB. Separate 4k/16k measurements on the same architecture kept time per token broadly stable and retained heap proportional to token count, so the ordinary record path is linear. The final implementation also removes the semantically empty This evidence describes the final head |
Goal: replace untyped native events in the Responses writer. Result: events, output items, content parts, annotations and usage are protocol types; reasoning part events come from small builders instead of conditional key spreads, and logprob tokens are read from IR state. One cast remains per documented gap: IR logprobs may omit fields the wire type requires, and a cache write can be reported without cache reads.
…ders Goal: drop the untyped usage input of the two remaining usage readers. Result: ChatCompletions reads the protocol usage type and Responses reads a local view of the fields it consumes. The invalid-counter test calls each converter with a literal, since the converters no longer share one loosely typed parameter.
… kind Goal: replace the long if/else chains over unit kinds in the Responses writer. Result: message, image, reasoning and tool call units each have an opener returning the added item and its follow-up events, and a handler for operations on an open unit. openPendingUnits keeps the loop, the wait for a tool call ID and the dequeue; justOpened semantics are unchanged.
Goal: keep every statement in the IR, reader, writer and usage files readable. Result: long conditions, object literals and nested ternaries are split over several lines with named intermediates; behavior is unchanged.
Goal: replace the single-letter builder name in the three stream readers. Result: the builder is named ir in the ChatCompletions, Messages and Responses readers; no existing identifier used that name.
Goal: remove the near-identical IRJSONValue and IRValue definitions. Result: both derive from one generic value type; IRJSONValue has no extra leaf and IRValue adds the IRNone marker at any depth. Exported names are unchanged.
The custom tool input done event carries item_id, output_index and input only; the namespace restorer already strips a name when families change. Remove the name field the Responses writer added to it and the matching protocol type member.
A scratch script was staged by a bulk add and broke the lint job, which parses every TypeScript file against a project.
A response.failed event may carry a null error, and the previous translators answered it with a default message. Emit the same IR error instead of throwing.
The previous translators dropped citations that are not web or search results when writing Responses output. The conversion to file_citation was new behavior with an invented file index.
…served IR fields No writer reads the granularity of a quoted citation source, which left source_text a one-field wrapper; store the quoted text as a string. Note that choice-scoped logprobs and the audio MIME type are reserved for a GenerateContent upstream.
Declare the structure of the IR once, with the append, IRNone and lifecycle unit traits on its nodes, and check the value type inferred from it against the IR types at compile time. The path predicates, unit path builders, end event paths, record owners and the collector's validation now walk that schema instead of matching path shapes. Values written into the IR are checked against it, so a field the schema does not declare is rejected.
Goal: let upstream readers own writes through builder-managed unit handles instead of reconstructing lifecycle paths and state.\n\nResult: item, part, and choice handles build schema-derived paths, track open units, support idempotent closure and one-time initialization, and expose the two scoped accumulated-value reads needed by readers. Focused handle tests cover paths, lifecycle, and reads.
Goal: make the Messages upstream reader produce IR exclusively through builder-owned item, part, and choice handles.\n\nResult: block state now owns its write handles and citation count, all unit writes use relative handle paths, and handle lifecycle closure preserves the existing record order and errors.
Goal: remove accumulated IR reads and hand-built unit paths from the ChatCompletions upstream reader.\n\nResult: choice state now owns item, part, tool, annotation, and logprob bookkeeping through write handles; only the dedicated concatenated-text view is read for citation coordinates, while record order and lifecycle behavior remain unchanged.
Goal: make the Responses upstream reader own item, part, and reconciliation writes without inspecting accumulated IR structure.\n\nResult: reader state tracks content, reasoning, logprob, tool, image, and audio units through handles; snapshot comparisons use the scoped current-value API, full error paths resolve through the builder, and focused plus full translation tests preserve behavior.
Goal: stop rediscovering immutable IR unit kinds and per-record token values from accumulated state.\n\nDecision: retain only item, part, and logprob owner metadata at initialization; keep complete-state reads for envelopes, audio replay, terminal reasoning, citations, and late tool metadata.\n\nResult: per-record dispatch uses record values and registered discriminants without caching growing IR content.
Goal: avoid accumulated-state reads that only rediscover an initialized unit kind.\n\nDecision: register immutable content-block kinds from initialize records while keeping complete-state reads for ordered delayed opening and terminal reasoning or tool snapshots.\n\nResult: opened-unit append dispatch uses registry metadata without caching source content.
Goal: remove accumulated-state reads that only recover initialized unit kinds, grouping, or the current token.\n\nDecision: retain immutable part and logprob-owner metadata while preserving complete-state reads for delayed opening, coordinates, and terminal output snapshots.\n\nResult: registration and per-record emission use record values without mirroring growing IR content.
Goal: remove redundant accumulated-state dispatch and the writer-owned copy of pending IR text.\n\nDecision: retain immutable unit kinds and pending source offsets, then reconstruct delayed fragments from the collector when their target part can open.\n\nResult: original chunk boundaries and grounding mappings remain intact without mirroring growing source content.
Goal: share the target-independent rule that truncated tool completions do not require valid JSON.\n\nDecision: keep each protocol's waiting and streaming control flow local, and centralize only completion-time parsing and repair.\n\nResult: Messages, Responses, and GenerateContent preserve their distinct boundaries while using one tested completion helper.
Goal: prevent GenerateContent output from inventing inline-media MIME types absent from IR.\n\nDecision: remove unowned image and audio MIME fallbacks; retain Chat audio replay identity because it belongs to the newly emitted target stream.\n\nResult: reliable reader-provided MIME metadata is forwarded, and missing MIME continues to fail at the writer boundary.
Goal: keep the write-handle implementation compliant with repository lint.\n\nResult: equivalent optional-chain and nullish-assignment expressions remove the final lint violations without changing reader behavior.
Stream private IR records directly so grouping carries no false atomicity while preserving upstream backpressure. Resolve schema paths with one state cursor and replace repeated citation scans with lazy direction-specific coordinate indexes that follow later text appends.\n\nTrack native and emitted annotation progress separately, cover split surrogate boundaries and incremental citations, and retain the existing complete logprob state. Verified with translate typecheck, task-owned lint, and all 1,013 translate tests.
|
Codex: design archaeology for reasoning translation, Fit IR, and carried assistant-turn round trips. This comment records the design path that led to #594 and the dependent #599. It summarizes the decisions rather than preserving discussion transcripts. Earlier alternatives are included where they explain the final boundaries. The problem first appeared as ChatCompletions reasoning dialectsChatCompletions has no official reasoning-text or encrypted-reasoning contract. Real producers and clients independently use scalar text fields such as The first proposal was a configurable ChatCompletions codec: normalize an upstream-selected text/data dialect into one private message representation, then project it to the client's selected dialect. This was broader than pairwise translation because native ChatCompletions → ChatCompletions also needed normalization. The implementation used a private enumerable Symbol for normalized reasoning and exposed explicit source/target codec options. Client round-trip research showed why field renaming was not enough:
The resulting canonical split was conceptual rather than wire-specific:
For scalar text aliases, the first recognized field is selected per choice for the complete turn. Empty strings still establish the selected field. Simultaneous aliases use the stable preference The early codec PR was closed because the general problem was larger than ChatCompletions reasoning. Every protocol has native assistant-output structure another protocol cannot replay, and client compatibility can vary by API key rather than only by model/upstream. Its useful decisions survived in the later IR and sidecar designs; its protocol-specific carrier machinery did not. From a reasoning sidecar to a translation-wide sidecarThe original sidecar RFC was ChatCompletions-centric. It proposed extracting readable reasoning, retaining opaque fields, and encoding a compact encrypted object into That idea was generalized in two steps. First, the sidecar became a property of the complete translation system, not a reasoning codec. Any client protocol may carry one encrypted sidecar, and any upstream protocol may contribute native structure to it. The carrier is protocol-native:
Upstream opaque values are stored inside the authenticated sidecar when native replay needs them. The sidecar carrier is the only gateway-generated opaque value returned for one assistant turn; multiple sidecars make the turn unrecoverable. Second, "thin item" was generalized to a per-protocol thin assistant turn. A turn is the whole assistant result in that protocol: one assistant message for ChatCompletions/Messages, an interleaved output-item sequence for Responses, and potentially several model messages for GenerateContent. The protocol-specific code creates an intermediate thin turn containing references; generic code resolves it to a finalized thin turn. Fat IR and Fit IRTwo competing intermediate-representation directions were considered. Fat IR attempted to losslessly model the union of all protocol-native structures, possibly with unknown/native payloads embedded directly. In its strongest form, an A via B path would be Fit IR contains only the union of mutually translatable generated-output semantics. It is deliberately private to The initial Fit-IR discussion considered target-dependent gate-in: admit a semantic only if the selected target can express it. That was rejected. A reader should describe the source once; a writer describes what it actually emitted. The later IAT records which IR strings reached the client, so target capability does not change the source IR. The boundary is semantic equivalence or equivalent use, not shared spelling. Examples include encrypted reasoning versus thinking signatures. Source-only server tools, MCP/server-owned result objects, request-only configuration, and fields that no other output protocol can meaningfully consume stay outside IR and are retained only by thin capture where native replay needs them. Why the IR is an operation streamThe design started from the observation that most streamed output changes are either complete replacement or string accumulation. Numeric increments can always be represented as assigning the new cumulative value. Lifecycle cannot be represented as a value update, so records divide into operations and events. The current operation contract is: initialize(path, value) // create minimal structure at an absent path
assign(path, value) // publish a complete actual value
append(path, string) // publish an actual string fragment
Lifecycle records are An early Reader, accumulator, and writer ownershipReaders map native events directly into IR records. They keep only native facts IR does not hold or cannot yet place: source indexes, selected aliases, incomplete native wrapper text, and snapshot identity. Write handles remember the unit and initialized fields, so readers do not read IR back to discover what they wrote. Two reads remain intentional: ChatCompletions citation conversion reads accumulated generated text, and the Responses reader compares native done snapshots with accumulated IR values. Each producer and consumer owns its own copy-on-write accumulator. Record payloads are immutable; collectors borrow them until a write first touches the containing path. This preserves old record snapshots without repeated deep cloning. Writers react to the current record and retain only target-owned/out-of-sync facts: generated IDs, native ordinal indexes, pending-open ordering, emitted annotation progress, and target lifecycle. They read complete IR state only when the native protocol requires a complete opening/terminal snapshot or a coordinate calculation. Growing source strings, arguments, annotations, logprobs, and output items are not mirrored in writer state. Target ordering is explicit. A unit missing a target-required fact may delay later native starts. The queue stores units, not replayable IR records. When it opens, it reads the current accumulated value once. GenerateContent pending text stores source paths/ranges rather than copied strings. Protocol-specific lifecycle decisionsChatCompletions has no item lifecycle, so the reader infers it. Readable reasoning/text transitions close the previous scalar item. A function call closes when a later tool index starts or when the choice ends; syntactically complete JSON is not an end marker because real streams can append malformed/concatenated objects. A native custom call remains open until choice end. Updates for already closed calls are warned and ignored. Tool arguments remain strings in IR; targets requiring objects parse them at their own completion boundary, and truncation never forces successful JSON parsing. Messages blocks may be simultaneously open in IR. Client-specific reordering is an independent outer concern. Messages usage is cumulative; members update independently, while its nested cache snapshot follows the sender's snapshot semantics. Responses done events are snapshots, not unconditional replacements. Reconciliation is per field:
Done snapshots reconcile before closing their part/item. Terminal output may introduce an unannounced item, which is initialized and closed in output order. Type/ID mismatches and missing terminal items fail. Image generation allocates its output item independently and receives the image part when result data arrives. GenerateContent may require complete media/tool-call values and therefore holds later units behind an incomplete earlier unit. It preserves arrival order for delayed text. Its stream errors use a first-class raw frame so Google's compact bare-JSON error follows ordinary SSE chunks without pretending to be an SSE data field. Citations and coordinatesIR text ranges use UTF-16 code-unit offsets because IR stores JavaScript strings exactly, including isolated surrogates. Native protocols may use code points or UTF-8 bytes. The current implementation creates a direction-specific typed coordinate index lazily on the first citation and extends it on later text appends, including a surrogate pair split across deltas. ChatCompletions citation indices are treated as half-open code-point offsets based on the retained capture/community evidence. Responses probe evidence establishes code-point offsets. GenerateContent grounding uses UTF-8 byte offsets. Late ChatCompletions citations are intersected with the text that remains open after item interruption; a closed-only span is warned and dropped. The detailed probe evidence remains in the dedicated citation comment. Thin turns, IAT, and sidecar finalization (#599)#594 intentionally stops at returned-stream translation. #599 adds carried history restoration on top. While B SSE becomes Fit IR, protocol-B code also collects a native B assistant turn and replaces eligible source strings with intermediate references. Every IR-origin string is registered in an Intermediate Association Table (IAT). As A's writer emits content, it records how IR source ranges map onto A's client-visible strings. Concatenating or flattening several IR strings therefore produces ranges into the final A string rather than duplicating text in the sidecar. At completion, generic code resolves intermediate references through the IAT:
Final references use a predeclared table of binary hashes. A reference stores the table index and optional UTF-16 range, not a repeated hash string. Repeated strings share one hash entry. CBOR keeps binary hashes compact. Raw JSON numeric tokens use a dedicated tag, and isolated UTF-16 surrogates use WTF-8 in a dedicated tag; ordinary well-formed strings remain normal CBOR text. The sidecar contains the finalized thin assistant turn, referenced-content hashes, and a versioned replay check. It is signed/encrypted with deployment-owned keys. Versioning starts at 1 because the feature was not previously published; after publication, validators for historical promised versions must remain. On the next request, protocol-A code splits user and assistant turns according to A's native rules, strips/extracts the single sidecar, and constructs A's client-visible text/object view. The replay check covers the body the gateway actually returned and expects back: visible text/reasoning and tool-call ID/name/arguments in order, plus the item-kind sequence. Response-only metadata is excluded. User turns always take normal bare translation. The two assistant paths are mutually exclusive: Authentication failure or replay mismatch is treated as untrusted client history and falls back to bare translation. A sidecar that authenticates but cannot be decoded/hydrated indicates an implementation invariant failure and must surface an error rather than silently degrading. The early LiteLLM-style PR layering and final boundaries
The long-lived architecture is therefore: The IR remains deliberately fit: it describes the translatable generated-output semantics. Native residual structure belongs to the thin turn; client-visible integrity and reconstruction belong to IAT/replay; vendor/provider wire deviations are handled at their owning input boundary rather than becoming IR concepts. Detailed retained evidence:
|
Adopt direct IR record streams, schema-derived write handles, shared unit lifecycle, and the latest protocol adapter architecture from PR #594. # Conflicts: # packages/gateway/__tests__/data-plane/chat/openai-chat-completions/affinity/egress_test.ts # packages/translate/AGENTS.md # packages/translate/__tests__/ir/helpers.ts # packages/translate/__tests__/ir/initialize_test.ts # packages/translate/__tests__/ir/lifecycle_test.ts # packages/translate/__tests__/ir/streams_test.ts # packages/translate/__tests__/protocols/anthropic-messages/via-openai-chat-completions/request_test.ts # packages/translate/__tests__/protocols/anthropic-messages/via-openai-responses/request_test.ts # packages/translate/__tests__/protocols/gemini-generate-content/via-anthropic-messages/request_test.ts # packages/translate/__tests__/protocols/gemini-generate-content/via-openai-chat-completions/request_test.ts # packages/translate/__tests__/protocols/gemini-generate-content/via-openai-responses/request_test.ts # packages/translate/__tests__/protocols/openai-chat-completions/via-openai-responses/request_test.ts # packages/translate/__tests__/protocols/openai-responses/via-anthropic-messages/request_test.ts # packages/translate/__tests__/protocols/openai-responses/via-openai-chat-completions/request_test.ts # packages/translate/src/protocols/anthropic-messages/as-source/sse-to.ts # packages/translate/src/protocols/anthropic-messages/as-target/sse-from.ts # packages/translate/src/protocols/gemini-generate-content/as-source/sse-to.ts # packages/translate/src/protocols/gemini-generate-content/via-openai-chat-completions/request.ts # packages/translate/src/protocols/gemini-generate-content/via-openai-responses/request.ts # packages/translate/src/protocols/openai-chat-completions/as-source/sse-to.ts # packages/translate/src/protocols/openai-chat-completions/as-target/sse-from.ts # packages/translate/src/protocols/openai-responses/as-source/sse-to.ts # packages/translate/src/protocols/openai-responses/as-target/sse-from.ts # packages/translate/src/shared/json/object.ts
Emit each candidate termination at IR choice_end and send late usage at global finish without buffering terminal candidates. Keep protocol collection and gateway forwarding open after candidate completion so later candidates and usage survive. Preserve error, prompt blocking, and missing-completion behavior.
Append Responses delta logprobs at new IR array indexes while reconciling terminal snapshots. Compact nullable annotation indexes and validate required schema fields when units close. Track GenerateContent completion independently for every observed candidate in protocol collection and gateway streaming.
Model output text and refusal parts as distinct states so annotation, coordinate, and logprob bookkeeping cannot leak into refusal handling. Reject content-part type changes at their owning slot and remove the obsolete GenerateContent candidate-finished helper from public exports.
1 participant
Returned generation streams use a private Fit IR inside
packages/translate. Three upstream readers and four client writers compose the nine existing A via B routes. Native adapters and usage mappings remain under each protocol'sas-target/as-source; IR is not exported from the translate package.Readers map native events directly to self-describing IR records.
initializecreates the minimal terminal-valid structure and starts item/part lifecycle,assigncarries complete non-append values, andappendcarries string fragments only for statically declared fields.IRNonedistinguishes a determined absent call ID from a value that may still arrive. Response failure uses theerrorevent;finishonly represents completed or incomplete generation. Records stream directly without a batching/atomicity wrapper.The schema is the single source of truth for legal paths, optional/nullable value shape, append and
IRNonetraits, and choice/item/part lifecycle units. Its inferred value type is checked againstIRat compile time. The accumulator owns plain copy-on-write state, validates operation classes and closed lifecycles, and exposes narrow reads for native snapshots and pending opens.Writers react to the current record and retain state only for facts their target receives out of sync: ordered pending opens, generated identifiers, native ordinal mapping, emitted annotation progress and buffered complete tool calls. Native done and terminal snapshots are constructed from the accumulator only when emitted. There is no protocol namespace, publication index, filtered projection, rebuild-then-diff reader, writer-side scan to re-derive changes, or growing writer copy of IR content.
Responses done snapshots use per-field reconciliation: empty snapshots do not erase streamed content, equal values are ignored, append targets may add a missing suffix, non-prefix text throws, and assign fields may only be supplied once or match. Image generation items allocate independently at
output_item.addedand receive their image part at done. Late ChatCompletions citations are clamped to text that remains open after nonstandard item interruption; closed-only spans are warned and dropped. Lazy direction-specific coordinate indexes follow later text appends and preserve split surrogate boundaries. Responses delta logprobs append at consecutive IR array indexes before terminal reconciliation, nullable native annotation indexes compact into contiguous IR arrays, and closing a lifecycle unit validates every required schema field. GenerateContent completion tracks every observed candidate independently, so one finished candidate cannot terminate later candidates or usage.The complete design evolution—from ChatCompletions reasoning dialects and the discarded Fat IR direction through Fit IR and the stacked sidecar/IAT/thin-turn design—is recorded in the design archaeology. Detailed retained evidence covers accounting/policy, final performance, citation coordinates, ChatCompletions reasoning aliases and mixed deltas, and real ChatCompletions producer deviations.
Authenticated sidecars, replay checks and complete assistant-turn round trips remain scoped to dependent draft #599. Thin references remain scoped to its dependent draft #624.
Validation on current head
451205fb7: all five PR checks passed (lint, workspace/root typechecks, generated assets, web build, and 683 Vitest files / 7,976 passed tests with 3 expected skips). Focused local validation also passed the Gemini protocol/gateway suite, Responses snapshot-policy tests, all three owning TypeScript projects, and changed-file lint. The linked performance workload and results apply to the reviewed291a728fcbase.