Skip to content

lmd: structural desugaring contract for AST built outside the reader (bd-7gqr148v, #846) - #850

Merged
cscheid merged 9 commits into
mainfrom
feature/bd-7gqr148v-lmd-ast-desugar-contract
Oct 16, 2026
Merged

cscheid merged 9 commits into
mainfrom
feature/bd-7gqr148v-lmd-ast-desugar-contract

Conversation

@cscheid

@cscheid cscheid commented Oct 16, 2026

Copy link
Copy Markdown
Member

Fixes #846. Braid strand bd-7gqr148v. Plan: claude-notes/plans/2026-10-15-lmd-ast-desugar-contract.md.

The problem

A .list-table or .definition-list div that an {lmd} cell built from nodes reached the writer as a div of nested bullet lists, because the reader's div desugar only ever ran over blocks the qmd reader produced. The fragment workaround failed from the other side: {=: ::: {.list-table} … :=} parses to a Table, and the lmd emitter had no Table arm.

The contract

The reader's postprocess mixes concrete-syntax clean-up with three structural rewrites that an author can also produce by constructing nodes: list-table div → Table, definition-list div → DefinitionList, single-image paragraph with alt text → Figure (pandoc's implicit_figures; applied at any depth, as pandoc 3.11 and the reader both do). This PR names that set as a contract: every block list that enters the pipeline other than through the reader is rewritten by it, exactly once. It is owed to lmd cell results, lmd fragments, pandoc.read(…, "json"), pampa's --from json, and document import. It is not owed to what user Lua filters return (pandoc and Quarto 1 parity). Written up in dev-docs/syntax-notes.md § Structural desugaring contract.

Changes, one commit per phase

  1. Fixture desugar.qmd pinning the gap, snapshotted through the native HTML pipeline (--to markdown runs Quarto 1's own list-table filter and hides the bug).

  2. pampa: new pandoc::desugar module (desugar_blocks / desugar_inlines / desugar_document) replacing transform_divs; the reader shares the per-node helpers so the two cannot drift; the Lua pandoc.read(…, "json") arm applies the set like the CLI and import paths.

  3. lmd cells: execute_lmd_cells runs the set per cell over what the cell returned (block and inline cells, so a Note's blocks are covered). A Q-2-35 from a malformed cell-built list-table is relocated onto the cell's code with an info line:

    Warning: [Q-2-35] Invalid List-Table Structure
        ╭─[ desugar.qmd:94:1 ]
        │
     94 │ return pandoc.Div({ pandoc.Para({ pandoc.Str("not a list") }) }, pandoc.Attr("", {"list-table"}))
        │ ────────────────────────────────────────────────┬────────────────────────────────────────────────
        │                                                 ╰────────────────────────────────────────────────── list-table div's last block must be a bullet list (the rows)
    ────╯
    ℹ The node was built by this lmd cell.
    ℹ Check the list-table documentation for the correct structure?
    
  4. Emitter: Block::Table is printed as pandoc.Table(caption, colspecs, head, bodies, foot, attr) in both modes, so pipe tables and list-tables work inside fragments. The tests and the errors.qmd fixture that used a pipe table as their unsupported example use a footnote definition instead.

  5. Docs: the contract in dev-docs/syntax-notes.md; a section in the Lua filters guide; a note on the Q-2-35 page.

Tests

Eight unit tests for pandoc::desugar, six for the execute stage, three for the emitter, JSON and pandoc.read round-trips, and the desugar fixture (13 sections: literal controls, node-built shapes, fragment tables, invalid list-table, pandoc.read markdown and json, a list-table inside an inline cell's Note, a block cell nested in a fragment). Full cargo xtask verify green locally.

🤖 Generated with Claude Code

cscheid and others added 8 commits October 15, 2026 13:55
…eader (bd-7gqr148v, #846)

Diagnosis and phased plan for GH #846: list-table and definition-list
divs built from nodes in an lmd cell are never desugared, and a
list-table inside a fragment hits the emitter's missing Table arm.
No code changes yet; the plan is up for review.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
D1: implicit figure is in the set, applied anywhere a Paragraph appears;
verified against pandoc 3.11 and q2's reader that neither restricts the
transform to the top level (table in the plan). D2: relocate Q-2-35 onto
the cell, with a rendered example due after Phase 3. D3: emit Table in
fragments. D4: user filters unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…bles (bd-7gqr148v, #846)

Phase 1 of the structural desugaring contract plan. A new desugar.qmd
fixture covers the issue's four sections plus the probes (fragment
list-table / pipe table / definition-list, a node-built single-image
paragraph, an invalid node-built list-table, pandoc.read of markdown
and json, a list-table inside an inline cell's Note, a block cell nested
in a fragment). It is snapshotted through the native HTML pipeline:
the markdown leg hands the AST to pandoc with Quarto 1's filters, whose
own list-table filter converts the divs and hides the bug.

The harness gains HTML_FIXTURES / render_html / render_to, and copies
non-qmd resources (pixel.png) into the scratch directory so the figure
probe renders without a missing-resource warning.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…c.read json (bd-7gqr148v, #846)

Phase 2 of the structural desugaring contract plan. A new
pandoc::desugar module owns the three shape rewrites the qmd reader
applies and that every non-reader AST entry point is promised:
list-table div -> Table, definition-list div -> DefinitionList, and
single-image paragraph with alt text -> Figure, at any depth.

- desugar_blocks / desugar_inlines / desugar_document replace
  transform_divs; the reader's postprocess shares the per-node
  rewrites (desugar_div, implicit_figure) so the two cannot drift.
- The Lua pandoc.read(..., "json") arm now applies the set, matching
  the qmd arm (through the reader), pampa's --from json and import.
- Tests: eight unit tests for the module, the JSON transform suite
  gains the figure and nested cases, readwrite gains a round-trip
  test; fixture section 11 (pandoc.read of JSON) now renders a table.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…bd-7gqr148v, #846)

Phase 3 of the structural desugaring contract plan. A list-table or
definition-list div, or a single-image paragraph, that an lmd cell
builds from nodes now becomes the Table, DefinitionList or Figure the
reader would have made from the markdown. The pass runs per cell over
the cell's result (block cells and inline cells alike, so a Note's
blocks are covered), and a diagnostic it raises (Q-2-35 for a malformed
list-table) is relocated onto the cell's text with a line saying the
node was built there, since a constructed node's own location is a Lua
chunk position and a pandoc.read result's is an offset into a private
source context.

Six new unit tests; the desugar fixture's sections 2, 4, 8, 9, 12 and
13 now render as intended.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Phase 4 of the structural desugaring contract plan. A fragment whose
markdown holds a pipe table, or a list-table the reader already
desugared, used to fail the cell with "Table nodes are not supported
inside markdown fragments yet". The emitter now prints pandoc.Table
with its caption, colspecs, head, bodies and foot through the existing
constructors, in both pretty and line-preserving modes; the caption
printer is shared with Figure.

The tests and the errors fixture that used a pipe table as their
unsupported example use a footnote definition instead.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Phase 5 of the plan. dev-docs/syntax-notes.md gains the contract: the
two layers of the reader's postprocess, the three-member set, which
AST entry points are owed it and where each applies it, and the
filter-return boundary. The Lua filters guide explains what that means
for filter and lmd authors, the Q-2-35 page covers the cell-built case,
and the lmd exploration plan no longer lists Table as unsupported in
fragments. A doc comment on the diagnostic relocation says what it
keeps and drops.

Full cargo xtask verify green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@posit-snyk-bot

posit-snyk-bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@cscheid
cscheid merged commit 5dbf0bc into main Oct 16, 2026
11 checks passed
@cscheid
cscheid deleted the feature/bd-7gqr148v-lmd-ast-desugar-contract branch October 16, 2026 17:46
cscheid pushed a commit that referenced this pull request Oct 16, 2026
…hanges

Two lmd changes landed on main while PR #855 was open, and the page had
documented both as limitations:

- #850 (bd-7gqr148v): tables inside fragments are emitted as pandoc.Table,
  so Q-26-2 no longer lists them and the limitation bullet is gone.
- #852 (bd-iqd6niat): a top-level inline cell writes numbers and booleans
  as tostring would. The metadata examples drop tostring(); the page, Q-26-6
  and the catalog say where it is still needed: constructor arguments and
  inline cells spliced inside a fragment (bd-q4gnd5os tracks lifting that).

Example 02, the patterns fixture and its snapshot follow.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

lmd: list-table and definition-list divs built from AST nodes in a cell are not desugared

2 participants