[JasonBuildAI]

What this fixes

Four core_text operators build their prompt batch by skipping rows whose
input_key cell is falsy, and then write the model's responses back as a
whole column:

dataframe[output_key] = generated_outputs

Because the skipped rows never made it into the batch, that list is shorter
than the frame, so any empty cell takes the run down with:

ValueError: Length of values does not match length of index

Affected: PromptedGenerator, PromptedEvaluator, PromptedRefiner and
RetrievalGenerator.

This is easy to hit in practice — it's normal for a frame to arrive with a few
blank cells after a source read or an upstream filter — and the worse variant
is silent misalignment rather than a crash: if the lengths happen to work out,
row N ends up with row N+1's answer. Either way the if raw_content guard
already anticipates empty rows, so I don't think the crash on the write side
was the intent.

How it's fixed

Each operator now remembers which row positions actually produced a prompt and
writes responses back at those positions, so the frame keeps exactly one value
per row. For the default value of a skipped row I kept each operator's existing
semantics rather than inventing a new rule:

• PromptedGenerator / RetrievalGenerator — empty string in the output column.
• PromptedRefiner — keeps its original text, since it overwrites input_key in
  place and dropping the value would be data loss.
• PromptedEvaluator — reuses the 0 that _parse_scores already returns for
  an unparseable response, so "no score produced" and "score parsed as
  unparseable" stay the same thing.

PromptedFilter calls PromptedEvaluator.eval() on a pre-filtered frame, so it
is unaffected; there is a test for it to keep that honest.

A batch that comes back with a response count that doesn't match the number of
prompts now raises instead of shifting rows. I followed #525 here, which does
the same thing for Text2QAGenerator — mismatched counts are a serving problem,
and silently reallocating them isn't a useful answer.

I also corrected three log lines that said Running PromptGenerator... inside
the evaluator and the refiner, and updated PromptedRefiner.get_desc, which
used to document the length-mismatch error as expected behaviour.

Testing

test/cpu_only/test_core_text_empty_row_alignment.py covers all four operators
against a stub serving and an in-memory storage, parametrised over which row is
empty (first / middle / last / none), and asserts the untouched rows keep their
identity — the fixture deliberately uses non-contiguous index labels [3, 7, 11]
so a positional/label mix-up can't pass. pytestmark = pytest.mark.cpu, no API
key or GPU needed.

$ pytest test/cpu_only -m "cpu or api"

Running that on upstream main and on this branch gives the same set of
failures (word2number and datasketch are missing from my local venv); the
only delta is the 14 new passes. Ruff over the four changed files reports the
same 10 pre-existing F401/F841 findings before and after.

One caveat on the new tests: RetrievalGenerator imports dataflow.serving at
module level, which needs the Google Cloud extras, so those four cases are behind
pytest.importorskip. They run in the CI matrix. I verified them locally by
stubbing that import, and the alignment holds for all four empty-row positions.
Making that import lazy seemed like a separate change, so I left it alone here.

Notes

No existing issue for this, so no Fixes #. Happy to open one first if you'd
prefer it tracked. The change is confined to core_text and doesn't touch any
operator signature, so test_register.py's conventions are unchanged.

[SunnyHaze]

LGTM, sincerely thanks!

[yinyueyuan]

能别给我发邮件了吗？烦不烦

…

On Mon, Oct 12, 2026 at 04:08 Ma, Xiaochen ***@***.***> wrote: Merged #536 <#536> into main. — Reply to this email directly, view it on GitHub <#536?email_source=notifications&email_token=BXVY4YG2TQS3SXPLZO4FBOL5TSGORA5CNFSNUABQM5UWIORPF5TWS5BNNB2WEL2JONZXKZKFOZSW45CON52GSZTJMNQXI2LPNYXTGMZQGI4TGMRTGA3TRJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#event-33029323078>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BXVY4YCJKGGKLVZEW7R4PU35TSGORAVCNFSNUABFKJSXA33TNF2G64TZHM4DOMRQGA2TSOBVHNEXG43VMU5TKNZVHA4TANJSHAYKC5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/BXVY4YF3OTMNPF6BBH6PXB35TSGORA5CNFSNUABQM5UWIORPF5TWS5BNNB2WEL2JONZXKZKFOZSW45CON52GSZTJMNQXI2LPNYXTGMZQGI4TGMRTGA3TRJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2KUZTPN52GK4S7NFXXG> and Android <https://github.com/notifications/mobile/android/BXVY4YBNQK6D6U5SS7SZEHT5TSGORA5CNFSNUABQM5UWIORPF5TWS5BNNB2WEL2JONZXKZKFOZSW45CON52GSZTJMNQXI2LPNYXTGMZQGI4TGMRTGA3TRJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2K4ZTPN52GK4S7MFXGI4TPNFSA>. Download it today! You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>