Skip to content

fix(pampa): pandoc.read from Lua no longer leaks the reader's verbose trace to stderr (bd-okfxemnj) - #849

Merged
cscheid merged 8 commits into
mainfrom
feature/bd-okfxemnj-pandoc-read-delimiter-dash-noise
Oct 16, 2026
Merged

cscheid merged 8 commits into
mainfrom
feature/bd-okfxemnj-pandoc-read-delimiter-dash-noise

Conversation

@cscheid

@cscheid cscheid commented Oct 16, 2026

Copy link
Copy Markdown
Member

Fixes #847

What

Calling pandoc.read(text, "markdown") from an {lmd} cell or a Lua filter on text containing a pipe table printed one unformatted [TOP-LEVEL MISSING NODE] Warning: Unhandled node kind: - line per delimiter-row dash to stderr. Two independent causes, both fixed:

  1. The Lua binding pointed the reader's verbose trace at stderr. readers::qmd::read takes an output stream that is a pampa -v trace, not a diagnostic channel; every pipeline caller sinks it. pandoc_read in lua/readwrite.rs was the only production caller passing std::io::stderr(). It now passes VerboseOutput::Sink, like lua/config_value.rs. This alone closes the issue.
  2. Four marker tokens had no arm in the tree-sitter dispatch. The -, pipe_table_align_left and pipe_table_align_right children of pipe_table_delimiter_cell, and the ![ opener of pandoc_image, are consumed by their parent processors by node name but fell through to the catch-all, which wrote the trace line. They now get explicit arms in the existing marker-token pattern (atx markers, task-list markers, math delimiters). Behavior-preserving: the catch-all already returned the same IntermediateUnknown. After this pampa -v is quiet on well-formed input, so the catch-all means what it says.

Tests (written first, both red on the unfixed tree)

  • crates/pampa/tests/integration/test_trace_buffer_silence.rs: the reader's trace buffer stays empty for a pipe table (all four alignments, alignments asserted) and an inline image.
  • crates/quarto/tests/integration/pandoc_read_stderr_e2e.rs: q2 render of the issue's repro leaves stderr free of the trace lines and still emits both tables.

Also checked on the unfixed tree that hyphens in text, a - b, link destinations and titles, autolinks, code spans, math, list markers and thematic breaks never produce a bare - node, so the new - arm only ever sees delimiter-row dashes (probe and output under claude-notes/plans/pandoc-read-delimiter-dash-noise-investigation/).

Out of scope, filed as follow-ups

  • bd-fwswbxnl: pandoc.read still discards the reader's real diagnostics (the third tuple element).
  • bd-yi613i1r: make the dispatch catch-all visible to tests instead of only to pampa -v.

Plan: claude-notes/plans/2026-10-15-pandoc-read-delimiter-dash-noise.md. cargo xtask verify --skip-hub-build green on the branch.

🤖 Generated with Claude Code

cscheid and others added 7 commits October 15, 2026 13:59
…e for pipe-table delimiter dashes (GH #847)

Plan skeleton + repro/probe artifacts. Two confirmed causes: the Lua
pandoc.read binding is the only production caller that points the qmd
reader's verbose trace buffer at stderr, and native_visitor has no arms
for the '-', pipe_table_align_left/right and '![' marker tokens, so
they hit the catch-all before their parent consumes them by name.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Scope is Phase 1 (sink) + Phase 2 (explicit marker arms) + reader-level
and e2e tests. Forwarding pandoc.read's reader diagnostics is filed as
bd-fwswbxnl; making the dispatch catch-all test-visible as bd-yi613i1r.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…emnj, GH #847)

Phase 0 of the plan. A reader-level test asserts the qmd reader's
verbose trace stays empty for pipe tables (all alignments) and inline
images; an e2e q2 render test asserts the issue's repro leaves stderr
free of '[TOP-LEVEL MISSING NODE]' lines. Both fail on this tree.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…o stderr (bd-okfxemnj, GH #847)

The qmd reader's output stream is its verbose trace (what 'pampa -v'
shows); every pipeline caller sinks it, but the Lua pandoc.read binding
handed it std::io::stderr(). Any node kind reaching the tree-sitter
dispatch catch-all therefore printed an unformatted
'[TOP-LEVEL MISSING NODE] Warning: Unhandled node kind: ...' line, one
per pipe-table delimiter dash, on every render that parsed a table
through pandoc.read. Use VerboseOutput::Sink like lua/config_value.rs.

The reader's real diagnostics are still discarded here; that is
bd-fwswbxnl.

Fixes #847

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…licit visitor arms (bd-okfxemnj)

The '-', pipe_table_align_left and pipe_table_align_right children of
pipe_table_delimiter_cell, and the '![' opener of pandoc_image, are
consumed by their parent processors by node name, but had no arm in
native_visitor, so each one fell through to the catch-all and wrote an
'Unhandled node kind' line to the reader's verbose trace. Add arms in
the existing marker-token pattern (atx markers, task-list markers, math
delimiters). Behavior-preserving: the catch-all already returned the
same IntermediateUnknown; only the trace line goes away, so 'pampa -v'
is quiet on well-formed input and the catch-all means what it says.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@posit-snyk-bot

posit-snyk-bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

✅ Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
✅ Open Source Security 0 0 0 0 0 issues
✅ Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@cscheid
cscheid merged commit 54f766d into main Oct 16, 2026
11 checks passed
@cscheid
cscheid deleted the feature/bd-okfxemnj-pandoc-read-delimiter-dash-noise branch October 16, 2026 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pandoc.read from Lua leaks "[TOP-LEVEL MISSING NODE] Unhandled node kind: -" to stderr for every pipe-table delimiter dash

2 participants