Skip to content

[Tinker API] Cache validated sampling targets - #2058

Closed
yapdianang wants to merge 3 commits into
NovaSky-AI:mainfrom
yapdianang:dydx/cache-sampling-session-route-independent
Closed

yapdianang wants to merge 3 commits into
NovaSky-AI:mainfrom
yapdianang:dydx/cache-sampling-session-route-independent

Conversation

@yapdianang

Copy link
Copy Markdown
Contributor

Summary

  • Cache the validated external sampling target for each sampling session.
  • Serialize the first lookup so a concurrent request burst performs one validation.
  • Keep internal sampling and requests without a sampling session unchanged.

Depends on #2043.

Before / After

Case Database validation reads
Before: 512 concurrent requests for one sampling session 512
After: 512 concurrent requests for one sampling session 1

The cache stores only a target that passed sampling-session, model, and checkpoint validation. A failed validation is not cached.

Testing

uv run --extra dev --extra jax --extra tinker pytest \
  tests/tinker/test_sampling_target_cache.py \
  tests/tinker/test_api.py
uv run --extra dev pre-commit run --all-files

The API tests pass: 20 passed. All pre-commit hooks pass.

@yapdianang

Copy link
Copy Markdown
Contributor Author

Closing because #2061 includes this sampling-target validation change together with the training-request storage fix.

@yapdianang yapdianang closed this Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant