Skip to content

feat: ai charts no sql - #1334

Draft
nicolaskempf57 wants to merge 51 commits into
mainfrom
feat/ai-charts-no-sql
Draft

nicolaskempf57 wants to merge 51 commits into
mainfrom
feat/ai-charts-no-sql

Conversation

@nicolaskempf57

@nicolaskempf57 nicolaskempf57 commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Front of the AI chart assistant.

What's included

  • Chart configurator in two tabs.
    • "Assistant": chat built on useChat, with reasoning and tool traces. The charts it creates or updates are applied to the form.
    • "Édition manuelle": the DSFR form. Saving happens only there.
  • Preview (ECharts):
    • a veil and a warning when a series has more than the 50 values a chart shows;
    • readable api-tabular errors, with a retry button only when retrying can help.
  • Saving:
    • the chart image is captured off screen at 448×320, once the preview shows the latest change;
    • a second click while saving is ignored;
    • a chart whose series differ in type or filter, which only the API can create, cannot be saved from the form, since the form would overwrite them.
  • Errors: a failed answer shows the server's reason, which says whether waiting helps.
  • CSP: the chat panel and the AI SDK are loaded on demand, and zod is set to jitless, so that no eval probe runs under the site's CSP.
  • Search encoding: TabularExplorer encodes searches like api-tabular main expects. This also affects the public explorer, the CADA page and the filtered download.
  • @datagouv/components-next:
    • new: chartContentToForm, hasMixedSeries, CHART_PAGE_SIZE, chart image size;
    • XAxis.limit added to the chart types.

To fix :

  • put DSFR input back
  • saved chart card images size
  • dont show charts with > 50 rows : now shown with a huge overlay
  • remove "schéma de la ressource"

Changes from the prototype :

  • input DSFR
  • sound effects
  • Game of life loading
  • color pickers
  • "interprétation des résultats"
  • tools ask for precision
  • 2 chart types instead of 5: no area, donut or scatter (no udata field)
  • no palettes, "show values", "show legend" or X-axis label (no udata field)
  • no "Créer manuellement" button: the chart is drawn as soon as a resource is picked
  • no "Données" tab in the preview
  • no model picker or token counter in the composer
  • no "En savoir plus" link under "L'assistant peut faire des erreurs."
  • assistant welcome aligned to the top instead of the bottom
  • "Contexte utilisé par l'assistant" block is not collapsible
  • step-by-step source band (producer, then dataset, then resource), with comboboxes and different labels ("Avec qui souhaitez-vous publier ?")
  • header: breadcrumb plus "Sauvegarder le graphique" at the bottom of the panel, instead of "Enregistrer le brouillon / Publier" at the top
  • X axis: "Continu (valeurs)" instead of "Temporel (date)", and a richer "Trier par" (column + direction)
  • "Nombre de valeurs" empty by default (the prototype uses 12), capped at 50
  • no "Spécification du graphique" block (JSON + Appliquer) and no SQL
  • truncation warning as a veil over the chart: not in the prototype

Next:

  • line for avg and median
  • api-tabular: merge the 4 PRs (fix/pagination-params, then feat/exact-next-link, which builds on it, plus fix/cors-on-errors and fix/quoted-column-operators), then drop the probe on the udata and cdata side
  • udata: add fields for colors, values, legend and X-axis label, if we want to catch up with the prototype
  • how to handle langfuse opt-in ?

Later:

  • SQL branches (udata#4066, cdata#1310): rewire them to the future /sql route without PostgREST
  • ASGI port of the chat endpoint: a worker stays blocked for up to 30 s if the client leaves during an api-tabular call

The fake services now always start with the e2e run and answer
/api/1/assistant/chat/ with a Vercel AI UI message stream: a
create_chart call for the request's resource on the first turn, an
update_chart call setting the Y unit on the next ones.
NUXT_PUBLIC_ASSISTANT_API_BASE (runtime config assistantApiBase) points
the chat at it; its port follows FAKE_SERVICES_URL.
Source band (producer, dataset, resource, column count) above the
preview card and a right panel switching between the form and the
assistant, whose conversation stays mounted across switches. The form
keeps main's controls in sections: Contenu with the collapsed resource
schema, a chart type button group, Axe X with « Nombre de valeurs »
(1 to 50, empty for no limit), Séries, Filtres and Axe Y without the
Styles accordion. The chart sent to the assistant is the current form.

A resource whose columns load after another one was selected no longer
resets the form, and the no-organization notice is hidden here.
Each create_chart / update_chart result is applied once per tool call
id, so manual edits made afterwards survive re-renders and tab
switches. A create_chart for another resource than the selected one
(the resource changed while the assistant answered) is ignored.
A count series points at the X column by convention, which hid the Y
sort options and made the sort watcher reset an assistant's top-N by
count. The Y sorts are now hidden only for a non-count series on the X
column, and a count is labelled « Compter » in the sort select. Column
counts use singular forms and the blank state a typographic apostrophe.
- key the fetched rows by series index so several series on the same
  resource each plot their own values (columns stay keyed by resource)
- watch the resource ids as a string: presentation changes no longer
  refetch the profile nor show the loader
- request the X values in descending order for a descending X sort,
  ascending otherwise; a Y-sort change still does not refetch
- show the chart from page 1, then run the 51st-group probe
- show a dash in the tooltip for a missing value
A loaded chart can have several count series: changing the X column
updated only the first one.
- the visualization sent to the assistant keeps only the chart fields
  (title, description, axes, series, filter, chart type) plus whether the
  chart exists; never ownership, visibility or extras
- the tool trace button is named by its label and summary
- the message list is a polite live log
- update the stale double-fetch comment of the assistant e2e
The admin-only entries were filtered on a value computed once at setup,
so they stayed hidden after a client-side login until a reload. The
filter now evaluates show() lazily inside the computed.
The chat transport does its own fetch: send X-API-KEY when devApiKey is set (instead of the session cookie) and the lang parameter, as plugins/api.ts does.
useChat keeps the user message of a failed request, and a partial reply when the stream breaks. Restoring the text in the composer and resending it showed and sent the message twice.
The chat endpoint now rejects oversized requests with a 413. The
composer already caps a message at the server limit of 4,000
characters, so in the UI a 413 means the conversation history itself
grew too large: say so instead of the generic error.
The wrapper kept the shadow-md and teal focus ring of the Headless UI example, so a Listbox stood out from the DSFR fields next to it. The button keeps its own input styles.
Filters come first, then content, chart type, X axis, Y axis (formerly series) and styles (formerly Y axis). Section titles become 12px h3 headings, like the prototype and the source band.
The Y axis asks for the calculation first (count rows, sum, average, minimum, maximum or raw values), then the column, hidden when counting. The payload sent for each calculation is unchanged. Styles becomes a collapsible section with the prototype labels; its content gets box-border because details passes content-box down through the inherited box-sizing.
…ar errors

Game-of-life loader on the first load and a veiled chart on slow reloads, French Y ticks and unseparated years on the X axis, readable series labels, a styled truncation note, bounds from the shown points, one shared page size, and api-tabular failures shown instead of an empty chart. ChartViewer now keys series data by series index: breaking for external consumers of the package.
…hter streaming

Animated welcome mark (from the datagouv-agents prototype), chart-oriented suggestions built from the schema, context block, disclaimer and thinking indicator, a new conversation button (offered after a 413), and memoized message rendering.
…red-field validation

Accessible Assistant / Édition manuelle tabs (manual editing first for an existing chart), bounded height with scrolling panels, mobile layout, the form filter as the single source of truth, one guard applying tool results once, and useForm validation that warns next to the save button and leads to the missing field.
Routes matched the exact URL while the transport adds ?lang=: a local run reached the real assistant. Every chat test now routes the scripted stream.
Quote or()/and() values and double-encode what api-tabular parses, plus % and #, identically to udata; double-encode top-level filter values like udata's quote(quote()); give the explorer search one layer, ufo adding the second, with _ standing in for the , and ) PostgREST cannot read.
Raw series read meta.total, unfiltered aggregates the profile's distinct count (now exposed as ColumnDefinition.distinctValues); only filtered aggregates still probe the 51st group, a 416 meaning no further group.
…ues, exact truncation detection

Column names and values go through one encoding for api-tabular's two decodings (columns unquoted inside or()/and()), in charts, the explorer filters and its global search. Truncation comes from the first page: meta.total for raw series, links.next otherwise, with a page-51 probe only when ambiguous; the profile's distinct count is gone.
…nd announcements

DSFR series colors at 3:1 or more with per-series markers, dashes and bar patterns; a role=img label summing up the chart; loader, errors and notes announced through status and alert regions; years unseparated in tooltips with integer ticks; X bounds from the shown points under a filter; long labels truncated.
…ignature played once, refocus, safe links

Fieldset legends, RequiredExplanation, an announced source status, readable placeholders and labelled remove buttons; the save warning clears once nothing is missing. The welcome signature plays once under 5 s, focus returns to the composer after a new conversation, assistant links open aside with a safe rel, and a 429 no longer mentions a quota.
…n the chat path

The CADA fixtures are keyed by the exact request URL, which the new encoding changes: re-recorded against the public api-tabular, orphans removed. The scripted chat route now matches the path whatever the origin.
…he chart

The muted note became a DSFR warning banner starting with "Graphique tronqué", still in the status region, so a chart cut at 50 values is not mistaken for the whole data.
… show readable tabular-api errors

A truncated chart gets a white veil over its whole area with the warning on top, so it is not read as the whole data.

fetchTabularData rejects with a TabularApiError decoded from tabular-api's error body (status, reason, retryable): unknown column, wrong-type filter value, impossible aggregate, forbidden aggregation and non-indexed column, parser refusal, data not available. The internal table name is masked like in udata's assistant client. The chart alert shows that message, or the generic one for a network/CORS failure, with a retry button except for refused queries.
…a block

The scoped prototype styles forced white bordered 11px fields, a custom
select arrow and a red-border error over every fr-input, fr-select and
input of the form. Drop them so the fields render like every other DSFR
form.

Also remove the collapsible resource schema from the content section:
the columns stay available in the pickers.
The saved image followed the preview size, wide on desktop and tall on
mobile, and ChartCard cropped it with object-cover into its 250x190 box.
Render it off screen at 500x380 and show stored images with
object-contain.
The title and description are filled in the manual tab, so the save
button now lives there only.

The footer warning of a refused save also vanished as soon as the
missing field lost focus: touch() empties a field's errors while it
revalidates, and the warning was cleared on an empty error list. Clear
it once the values pass validation instead.
…elds

The "Ajouter une règle" and "Ajouter un groupe" buttons keep their
tertiary look with a blue text. A new rule's empty column listbox shrank
to 32px: the Listbox value keeps at least one line of height, and the
value input matches the listboxes at 40px.
# Conflicts:
#	datagouv-components/src/components/ChartCard.vue
#	pnpm-lock.yaml
The harmonized cards show chart images in a 224x160 box: capture them at
448x320 instead of 500x380, sized for the former ChartCard.
chartContentToForm turns a chart's content into form fields for both a
chart loaded from the API and one created by the assistant. It applies
the count rule everywhere (a count series counts the x column) and only
copies known series fields.
- limit the chart form to the manual tab and make the chat composer a form;
- share the collapsible trace of reasoning and tool calls;
- skip malformed assistant chart outputs, share isRecord;
- join lists with the translated "et";
- keep the Styles details content border-box, use the blue-france-contrast
  token, tidy the configurator markup;
- drop dead branches, duplicated tests and unused fixture parameters.
The AI SDK brings zod, which probes eval with `new Function('')` when it
builds an object schema. Under the site's CSP (no 'unsafe-eval'), Firefox
reports that probe as a violation, failing every e2e test on a console
error, and the SDK sat in the entry chunk, so every page paid for it.

Load ChatPanel on demand so the SDK only comes with the chart
configurator, and set zod to jitless before the SDK builds its schemas.
The form edits one chart type and one filter, applied to every series on
save. A chart created through the API may give its series different ones:
saving it from the form would overwrite them. The form series keep their
own type and filter to tell such a chart, and the save button is disabled
with a banner explaining why. Charts from the assistant share one type and
one filter, and are saved as before.
api-tabular links an aggregated page to the next one only when that page
has rows (datagouv/api-tabular#129): the truncation note comes with the
data, without probing the 51st group. It is also reset when no series is
left to load, where it stayed on an empty chart.
A second click before the first answer created a second chart. The save
button shows its loading state and saveChart returns while a save runs.
A save right after an edit captured the previous chart, still shown while
the debounced reload ran. capture() waits for the chart of the latest
change, and gives no image when it does not load in time.
The profile of a resource left for another could answer last and replace
the columns of the current one, like the data request already guarded.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants