Skip to content

Three ways a prepared query gives a wrong answer without raising #324

Open

Description

@alxmrs

🌀 Claude: Three ways a prepared query gives a wrong answer and doesn't raise. CodeRabbit found these while reviewing #323. I reproduced each one on main at 447791a (0.7.1), with DataFusion 55.0.0, using these helpers:

import numpy as np, xarray as xr
from xarray_sql import Context

def ds(v):
    return xr.Dataset({"v": ("i", v)}, coords={"i": np.arange(len(v))})

def multi(x):  # two groups of dims, so from_dataset makes a table for each
    return xr.Dataset(
        {"s": ("i", x), "t": (("i", "j"), np.ones((3, 2)))},
        coords={"i": np.arange(3), "j": np.arange(2)},
    )

1. A rejected registration still replaces the Dataset that writes go to. from_dataset records the Dataset before DataFusion registers its table. So when DataFusion refuses a name that's taken, the record points at a Dataset no table reads, and later writes don't reach the query:

ctx = Context(); ctx.from_dataset("a", ds(np.arange(3.0)))
ctx.from_dataset("a", ds(np.zeros(3)))  # raises: the name is taken
q = ctx.prepare("SELECT v FROM a ORDER BY i")
q(a=ds(np.array([7.0, 8, 9])))["v"]     # [0.0, 1.0, 2.0]

2. Registering a split Dataset again isn't counted. register_schema replaces the old schema silently, so a prepared query of one of its tables keeps reading the old one:

ctx = Context(); ctx.from_dataset("g", multi(np.arange(3.0)), table_names={("i",): "s1"})
q = ctx.prepare("SELECT s FROM g.s1 ORDER BY i")
ctx.from_dataset("g", multi(np.array([7.0, 8, 9])), table_names={("i",): "s1"})
ctx.sql("SELECT s FROM g.s1 ORDER BY i")  # [7.0, 8.0, 9.0]
q()["s"]                                  # [0.0, 1.0, 2.0], with no error

3. A write of the wrong size writes some of the tables. The checks compare dims by name, not by size. So a dim of another size, with no coordinate on it, fails only during the copy, after earlier tables are written:

ctx = Context()
ctx.from_dataset("a", xr.Dataset({"v": ("i", np.arange(3))}))
ctx.from_dataset("b", xr.Dataset({"v": ("k", np.arange(3))}))
q = ctx.prepare("SELECT a.v + b.v AS s FROM a JOIN b ON a.i = b.k")
q(b=xr.Dataset({"v": ("k", np.arange(10, 13))}), a=xr.Dataset({"v": ("i", np.arange(5))}))
# ValueError: could not broadcast input array from shape (5,) into shape (3,)
ctx.sql("SELECT v FROM b ORDER BY k")  # [10, 11, 12]: b was written

#323 fixes all three:

  • from_dataset records a Dataset only after its table is registered.
  • Registering a split Dataset again counts each table its old schema had.
  • write checks each dim's size before it writes anything.

Activity

  1. added a commit that references this issue on Nov 17, 2026
    9c0b995
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions