Skip to content

FIX: key model-less objective targets by content hash in analytics v1 - #3171

Merged
Roman Lutz (romanlutz) merged 5 commits into
microsoft:mainfrom
u7k4rs6:fix/target-identity-no-model
Oct 16, 2026
Merged

Roman Lutz (romanlutz) merged 5 commits into
microsoft:mainfrom
u7k4rs6:fix/target-identity-no-model

Conversation

@u7k4rs6

@u7k4rs6 Utkarsh Bahuguna (u7k4rs6) commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Analytics keyed objective targets by model name, temperature and top_p, so every target without a model name (HTTP, Gandalf and so on) got lumped into one unlabeled group per class. As Roman suggested, this corrects v1 instead of adding a v2. Targets with a model name keep their key, and the rest fall back to their content hash with a readable label like HTTPTarget instead of a blank one.

A new migration reruns the v1 backfill, so databases that already ran the old one get the fixed keys too. The tests cover different HTTP endpoints and Gandalf levels, blank model names, wrappers and an already migrated database.

Fixes #3087

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: db31a497-470d-44b7-ac37-934ab6362724
Match Python str.strip when choosing model-less target labels on SQLite and SQL Server. Cover normalized and embedded identifiers through SDK reports, facets, matrices, and drill-downs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: db31a497-470d-44b7-ac37-934ab6362724
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: db31a497-470d-44b7-ac37-934ab6362724
Move the label regression into the analytics suite and reuse its finite correctness budgets. Keep real SQLite, all identity assertions, and production timeout defaults unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: db31a497-470d-44b7-ac37-934ab6362724
@romanlutz
Roman Lutz (romanlutz) added this pull request to the merge queue Oct 16, 2026
Merged via the queue into microsoft:main with commit 18f5548 Oct 16, 2026
64 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: attack analytics merges non-LLM targets into one unlabeled objective_target group

2 participants