Move the Flight rewriter into the Next.js module

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- **Breaking:** The React Server Components Flight rewriter is the middleware `framework.nextjs.flight` of the Next.js module, which runs on a `text/x-component` response where a `[[fetch]]` entry names it, where core rewrote every Flight response by media type. A deployment that fronts a Next.js site selects `[framework] module = "nextjs"` and adds the entry, as `trusted-server.example.toml` shows it. With the module selected and no entry, a Flight response passes through as the origin sent it, and an entry may now cover `text/x-component`. A middleware says with `Middleware::moves_origin_address` that it moves the origin's address itself, and core then leaves the addresses in a body of the media types it handles to it, whether or not an entry names it, because a plain replacement in a length prefixed row leaves the old length. **For crates outside core**, `trusted_server_core::rsc_flight` is removed.
 - **Breaking:** Where content comes from is a source module. `[source]` selects the publisher's, and each `[[proxy.asset_routes]]` entry selects its own in a `source` table of the same shape, with `module` naming the source and its settings in the table beneath. Core supplies `origin`, an HTTP origin, and a module a build links can offer another through `IntegrationRegistrationBuilder::with_source_module`. A source answers for each request where that request's content comes from, and core fetches, caches, rewrites the origin host and follows the redirect rule from that answer. `[source]` is required. The origin settings move into the `origin` table, and a document that still writes one where it was is refused at startup naming the new place. An environment variable that set the address is now `TRUSTED_SERVER__SOURCE__ORIGIN__URL`. The old name is refused where the build reads variables, and `ts config push` ignores it, as it ignores any variable for a key the document does not hold. A path in an asset route's `url` names a folder the target path is added under, where a path was refused, a target path with a `.` or `..` segment under a folder is answered `400`, and a route applies `directory_index` as the publisher does. Stored pages and the publisher's stored assets are fetched again after the upgrade, because their keys hold a code worked out from the settings, but an asset route's stored copies keep their keys. Where `host_header` is set, the page pipeline and every middleware are now told that `Host` as the page's origin host, where they were told the address's host, so the page's own addresses under that name are rewritten to the request's host. **For crates outside core**, `handle_asset_proxy_request` takes the route's source, which `IntegrationRegistry::route_source` gives, and `Publisher::origin_host`, `origin_host_header` and `origin_query_is_stripped` are replaced by `Settings::origin_source` and the `SourceLocation` a source answers.
 
   | Previous                                  | Now                                       |
@@ -105,6 +106,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `[auction].allowed_context_keys` now serializes in sorted, deduplicated order, so ESI template-cache fingerprints and `ts config diff`/`push` envelope hashes are stable across loads. Template fingerprints also sort object keys independently of `serde_json/preserve_order`. Existing envelopes may show a one-time allowlist reorder after upgrading; push once to settle it. The updated fingerprint format causes one template-cache miss per cached page after deployment.
 - TSJS-generated envelopes now send `trustedServer.params.storedRequest: false`, preventing accidental PBS stored lookups without suppressing eligible non-PBS demand. PBS filters unusable impressions after overrides; explicit `true` and omission in valid envelopes retain inline-first stored fallback. A malformed envelope disables stored fallback for the entire slot, including independent direct demand left unusable after overrides. Publisher intent survives repeated and refresh auctions. Deploy compatible server admission everywhere before serving the new JS, and retain it during rollback while cached clients remain. See the Prebid deployment guide.
 - Protocol-relative creative URLs now honor `rewrite.exclude_domains`, so excluded creative assets stay direct and excluded absolute or protocol-relative URLs submitted to `/first-party/sign` are rejected.
+- The origin's address in a React Server Components Flight response is now moved in one pass. A request host that begins with the origin host was matched again inside its own replacement, so `https://publisher.example/` served on `publisher.example.localhost` over `http` came out as `http://publisher.example.localhost.localhost.localhost/`, and on `publisher.example:8443` over `https` as `https://publisher.example:8443:8443/`.
 - Server-side ad template bids now always carry `hb_adid` in `window.tsjs.bids`. Bidders that return neither a Prebid Cache UUID nor an `adid` previously produced no `hb_adid` at all, so no `hb_adid` GPT targeting key was set and the Universal Creative render bridge had nothing to match — the winning creative never rendered. The OpenRTB bid `id`, which is mandatory per spec, is now the last-resort source; `cache_id` and `adid` still take priority where present. Blank `cacheId`/`adid` values no longer win that precedence and emit an unusable empty `hb_adid`, and `hb_cache_host`/`hb_cache_path` are now emitted only alongside a real Prebid Cache UUID — without one they pointed the Universal Creative at a guaranteed cache miss instead of letting it fall through to the inline creative.
 - The origin's address in a CSS, JavaScript, JSON or other text response that core rewrites without parsing it is now replaced in one pass. A request host that contains the origin host was matched again inside its own replacement, so `https://example.com/` served on `www.example.com` came out as `https://www.www.example.com/`, and `https://publisher.example/` served on `publisher.example.localhost` over `http` as `http://publisher.example.localhost.localhost.localhost.localhost/`.
 - A request that names its host only in its URI's authority, as an HTTP/2 request with no `Host` header may, is now processed. The request host was read from `Forwarded`, `X-Forwarded-Host` and `Host` alone, so such a request had an empty host and its page was returned unmodified, with no URL rewriting, middleware, injected script or auction. The authority is read last, so each of those headers still takes precedence ([#1118](https://github.com/IABTechLab/trusted-server/issues/1118)).
@@ -131,7 +133,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `PlatformValueCache`, the host's shared cache of small values with lifetimes, which `RuntimeServices` carries where the host supplies one. `lookup_or_reserve` obliges the caller that finds a key missing to fill it where the host can make other callers wait, and `remove` drops an entry. The Fastly adapter supplies one over Fastly's cache, answering the reservation with a transaction lookup and the removal with a purge of the entry's own surrogate key.
 - A file the origin does not have is made by the middleware a `[[fetch]]` entry with a `path`, for a media type other than HTML, names for it. When the origin answers `404 Not Found` for that path itself, which names a file and not a folder, the middleware runs on an empty document of the entry's media type and its output is the file, so `ads.txt` is made from the sellers lines alone where the publisher has none. With no such entry, or for a request that names no host, the origin's answer stands.
 - `[ads-txt] modules = ["sellers"]` adds the lines `[ads-txt.sellers] lines` gives to the publisher's `ads.txt`, from `crates/ads-txt/sellers`. The module is a middleware for `text/plain`, named in a `[[fetch]]` entry on the file's path, which passes the origin's file through and writes the lines after its last one, so the publisher's file stays authoritative, and a publisher whose origin has no file gets one made of the lines. Each line is checked as the settings load as a data record or a variable of the ads.txt grammar, and a selected module that no entry for `text/plain` on `/ads.txt` places is refused.
-- A `[[fetch]]` or `[[serve]]` entry may cover a media type other than `text/html`: another `text/` type except `text/x-component`, `application/javascript`, `application/json` or `image/svg+xml`, written as a bare `type/subtype` in lower case. A response that declares a charset other than UTF-8, or whose status is not `200`, gets no middleware. A middleware says which media types it works on with `handles_media_type`, HTML alone unless it says otherwise, and an entry naming one under a media type it does not handle is refused when the registry is built. On a media type other than HTML a middleware's stream processor alone runs on the body, after core has moved the origin's addresses, in the fetch phase and then, for a response that is not stored, the serve phase, and an action that decides about HTML there is refused by name. A response an entry covers loses the origin's `ETag` and `Last-Modified`, and one a `[[serve]]` entry covers is sent `private, no-store` when a module left state on the request.
+- A `[[fetch]]` or `[[serve]]` entry may cover a media type other than `text/html`: another `text/` type, `application/javascript`, `application/json` or `image/svg+xml`, written as a bare `type/subtype` in lower case. A response that declares a charset other than UTF-8, or whose status is not `200`, gets no middleware. A middleware says which media types it works on with `handles_media_type`, HTML alone unless it says otherwise, and an entry naming one under a media type it does not handle is refused when the registry is built. On a media type other than HTML a middleware's stream processor alone runs on the body, after core has moved the origin's addresses, in the fetch phase and then, for a response that is not stored, the serve phase, and an action that decides about HTML there is refused by name. A response an entry covers loses the origin's `ETag` and `Last-Modified`, and one a `[[serve]]` entry covers is sent `private, no-store` when a module left state on the request.
 - The InMobi Choice consent prompt as a module, `[cmp] module = "inmobi"` with `[cmp.inmobi] script_url`, from `crates/cmp/inmobi`. Where a `[[fetch]]` entry names `cmp.inmobi`, its middleware writes the IAB TCF stub, InMobi's GPP stub and the deferred Choice loader into the head, in that order, and the two stubs' behavior is tested with Node's test runner, which CI now runs for every module's `tests/*.test.mjs`. The script URL is refused unless it is `https` on an InMobi Choice host with no user name or password, `cmp_id` unless it is a whole number from 1 to 4095, and entries that leave any page without the prompt are refused. The new `Middleware::template_key` keys a stored page by what a middleware writes that no setting decides, so a change to a stub stores pages again. The Trusted Server maintainers hold the crate until InMobi adopt it.
 - The stock modules' fetch middleware give as their `template_key` the text compiled into their crates that they write into a page, so a change to it stores pages again. The Google ad tag gives its head scripts, the `tsjs.adInit` bootstrap among them, and the GPT script's first-party path, Prebid its configuration script and bundle tag, Didomi its settings script, and Sourcepoint its head scripts and its CDN's first-party path. Permutive, Lockr, DataDome and Google Tag Manager give the first-party path they point a vendor's address at.
 - `[robots-txt]` serves a publisher's `/robots.txt` from the settings. `modules` names what makes the file, in order. `refuse_all` and `allow_all` are Trusted Server's own, and any other name is a module that contributes rules through the new `RobotsTxtContributor` extension point, declared with `with_robots_txt_contributor`. Trusted Server writes the file, so a contributor cannot produce a malformed one, a refusal cannot be loosened, and a path in `always_allow` stays open to every crawler refused the whole site. With `refuse_all` selected every response an adapter finalizes carries `X-Robots-Tag: noindex, nofollow`. A module is asked for the host the request was addressed to, never one a forwarded header names, and is given nothing of the crawler's connection. Its answer is held for that host by the loading cache until its refresh is due. A module that cannot answer has its last held answer served, and with none held, or with no rule in the whole file, the answer is `503` and never an empty file. On Fastly the file is answered before the Edge Cookie steps and the request filters. Without the section `/robots.txt` reaches the publisher's origin as before.

crates/framework/nextjs/src/fixtures/page-change.settings.toml

@@ -7,3 +7,7 @@ rewrite_attributes = ["href", "link", "url"]
 [[fetch]]
 media_type = "text/html"
 middleware = ["framework.nextjs"]
+
+[[fetch]]
+media_type = "text/x-component"
+middleware = ["framework.nextjs.flight"]