Repository navigation
TEST: freeze query clocks in analytics identity checks - #3221
Merged
Richard Lundeen (richlundeen) merged 2 commits intoOct 17, 2026
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 18f4b401-48f9-4728-995f-db1431157452
Richard Lundeen (richlundeen)
marked this pull request as ready for review
October 15, 2026 16:00
Roman Lutz (romanlutz)
approved these changes
Oct 16, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Oct 16, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Oct 16, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: edeb4a16-e95f-4da7-8893-9032522d7ada
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Oct 17, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Oct 17, 2026
Richard Lundeen (richlundeen)
deleted the
merge-queue-fix/identity-query-clock-20261015
branch
October 17, 2026 02:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Freeze the query-control clock in the analytics identity unit tests. Keep the event-loop clock, database operations, 20-second query deadline values, and identity assertions unchanged.
The tests check grouping, filters, facets, pagination, and identifier replacement. They do not check query latency. Their
_control()helper uses a real monotonic deadline. CI can consume that budget during correct SQLite work, soQueryControl.check()raisesAnalyticsTimeoutExceptionbefore the identity assertions run.The fixture replaces only the
timereference inpyrit.memory.query_control. It restores the reference after each test. It does not replace the process-wide clock or theremainingandcheckimplementations. A regression test advances the runner clock past the deadline and checks that the identity-test budget stays unchanged.Category:
test-defect. Confidence:high, from the CI traceback, code, and a deterministic local clock-jump reproduction. The defect is present onorigin/main.Five queue entries for five PRs were affected during the window starting at 2026-10-14T00:13:34Z. Example: https://github.com/microsoft/PyRIT/actions/runs/38622454726/attempts/1, attempt 1,
build_and_test,main-job (macos-latest, 3.11, pyrit, dev_all), stepRun unit tests.Fingerprints:
tests/unit/memory/test_attack_analytics_eval_identity.py::test_evaluation_groups_filters_facets_and_pages_share_result_identitytests/unit/memory/test_attack_analytics_eval_identity.py::test_replacing_atomic_identifier_updates_evaluation_groups_and_graphError:
AnalyticsTimeoutException: Status Code: 504, Message: Analytics query timed out. Narrow the filters or retry.Duplicate search: searched both test functions, the test filename, both full fingerprints, and the timeout error; checked open PR file lists, merged PRs from the past 14 days, open issues, and all-state PRs with the automation fingerprint line. No existing PR freezes this test clock. PR #3171 changes target identity rules and adds tests, but leaves the real-time control unchanged. PR #3220 only changes a message-send test. No related open issue was found.
Tests and Documentation
uv run --no-sync pytest -n 4 --dist=loadfile tests\unit\backend\test_message_send_service.py tests\unit\memory\test_attack_analytics_eval_identity.py -q: one run, 177 passed, zero failed runs. The CI load-dependent failure did not reproduce in this run.uv run --no-sync pytest -n 4 --dist=loadfile tests\unit\memory\test_attack_analytics_eval_identity.py::test_identity_query_deadline_does_not_follow_runner_clock -q: one run, one expected failure with the sameAnalyticsTimeoutException.uv run --no-sync pytest -n 4 --dist=loadfile tests\unit\memory\test_attack_analytics_eval_identity.py -q: 20 runs of the final file, six passed per run, zero failed runs. An earlier repeat series had seven complete passing runs, zero observed test failures, and one interrupted run. It was stopped to correct a type-only import; the full 20-run series was then started again.uv run --no-sync ruff check tests\unit\memory\test_attack_analytics_eval_identity.pyanduv run --no-sync ruff format --check tests\unit\memory\test_attack_analytics_eval_identity.py: passed after moving the type-onlyIteratorimport into the existing type-checking block.uv run --no-sync pre-commit run --all-files: could not complete. Hook environment installation fails when Windows creates a long Jedi stub path for nbQA. Standard, short-drive, and extended-path attempts during this run did not remove the hook setup blocker. No hook was disabled.This PR is a draft because the full pre-commit run is blocked by the local Windows hook environment.
merge-queue-fix fingerprint: tests/unit/memory/test_attack_analytics_eval_identity.py::test_evaluation_groups_filters_facets_and_pages_share_result_identity