Use the token overlay helper for the remaining structure-merging code

[LeaVerou]

"Merge one tokenized structure with another over the same text" is reimplemented or worked around in many places. The $inner selectors PR adds the primitives (splitTokenStream() and insertTokens() in src/util/token-stream.js) and uses them for meta-languages. These are the remaining instances found by scanning the repo, roughly in order of payoff:

DOM / string level today, should be token level

• keep-markup (src/plugins/keep-markup/keep-markup.js:62-168): records text offsets of existing elements, then re-wraps them after highlighting with Range.extractContents(), which cuts through token spans. → wrap ranges on after-tokenize with splitTokenStream. Its node-identity test (tests/plugins/keep-markup/test.js) must keep passing. normalize-whitespace's special branch for keep-markup (normalize-whitespace.js:213-223) goes away with it.
• command-line (src/plugins/command-line/command-line.js:60-131): stashes output lines and strips continuation prefixes on env.code, re-inserts on the highlighted HTML string split by \n (breaks any multi-line token). → the same shape as diff: select the command lines.
• markdown code blocks after autoload (src/languages/markdown.js:298-337): re-highlights via element.innerHTML = Prism.highlight(element.textContent).
• inline-color (src/plugins/inline-color/inline-color.js:92): recovers text by stripping tags from HTML.

Per-line / per-node grammar workarounds

• xml-doc /// lines (src/languages/xml-doc.js:11-26), javadoclike / jsdoc * prefixes (javadoclike.js:5-15, jsdoc.js, javascript.js:93-98): every pattern is /m and anchored per line because the prefixes are never removed, so nothing can match across lines. → select the prefix-stripped lines, tokenize as one document (exactly what diff does now).
• markdown blockquotes (markdown.js:49-52): > prefix per line, the remainder is not re-parsed as markdown.
• shell-session (src/languages/shell-session.js:58-71): each command highlighted as bash separately; output is one opaque token.

Per-string overlays that miss matches across token boundaries

• tokenizeStrings() (src/shared/tokenize-strings.js) and its users show-invisibles, autolinker, data-uri-highlight: a URL or data URI split by a token boundary is never matched. → :text-style selection of all strings at any depth, highlighted as one whole.

Same algorithm twice

• jsx and xquery walkTokens (src/languages/jsx.js:24-109, src/languages/xquery.js:8-105): concatenate selected nodes into plain-text tokens. → select + an empty inner language.

Ordering hazards that exist only because these overlays don't compose: command-line.js:82-84, line-highlight.js:348-370.

[DmitrySharabin]

One more for the list, found while reviewing #4110/#4112 — and this one crashes rather than just losing a token.

tokenizeStrings() at src/shared/tokenize-strings.js:39 does tokens.splice(i, 1, ...result). That passes every item as a call argument, so it throws RangeError: Maximum call stack size exceeded once result is longer than the engine's argument limit. #4116 fixes the same shape at the two sites inside embed/insertTokens and adds replaceRange(array, start, count, items) to src/util/token-stream.js for it; this site can use the same helper, whether or not it is migrated to the selectors first.

Worth checking the other splice(…, …, ...x) call sites at the same time:

grep -rn '\.splice([^)]*\.\.\.' src/