Skip to content

[Async AE 1] Add async key cache - #5077

Open
cheenamalhotra wants to merge 3 commits into
mainfrom
dev/cheena/async-ae-key-cache
Open

cheenamalhotra wants to merge 3 commits into
mainfrom
dev/cheena/async-ae-key-cache

Conversation

@cheenamalhotra

@cheenamalhotra cheenamalhotra commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Description

Layer 1/5 of the split of #4585: async symmetric-key cache foundation only. Adds per-key load gates, cancellable waits, cache-disabled concurrency, and shared sync/async publication. Provider invocation stays serialized without holding the gate across outstanding async I/O.

Extracted from source head 5f1b860410426f8c1af3eb55bfac0e7354bd2ac7 onto main 2bda3c9edb53ce59589cdd401f10ae4415f331a5, preserving newer main changes. Execution-path integration follows in later layers; no public API or dependency changes.

Changes added while addressing review feedback:

  • Comments explaining why _publicationLock and _pendingKeyLoads are needed.
  • Race fix: LoadKeyAsync rechecks the cache after acquiring the provider gate, so a key published by a sync caller during the wait is reused instead of decrypted again (which could fail).

Issues

Relates to #3672. Split from #4585; neither is closed by this layer.

Testing

All source GetKeyAsync_* coverage moved to SymmetricKeyCacheTests (sync/async identity, cancellation, provider serialization, failed-owner retry, zero-TTL concurrency), plus a regression for sync publication while async waits. Provider fixtures live under UnitTests/Utilities.

Local (macOS, head 16919af):

dotnet test src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj --framework net10.0 --configuration Debug --no-restore --filter 'FullyQualifiedName~SymmetricKeyCacheTests' --blame-hang-timeout 2m --verbosity quiet
# Passed: 17, failed: 0, skipped: 0.

dotnet test src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj --framework net10.0 --configuration Release --no-restore --filter 'FullyQualifiedName~SymmetricKeyCacheTests' --blame-hang-timeout 2m --verbosity quiet
# Passed: 17, failed: 0, skipped: 0.

dotnet build src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj --framework net462 --configuration Debug|Release --no-restore --verbosity quiet
# Passed (both): 0 warnings, 0 errors.

dotnet build build.proj -t:BuildSqlClient -p:Configuration=Debug|Release -verbosity:quiet
# Passed (both): implementation, reference, unsupported-platform; 0 warnings/errors.

The race regression (GetKeyAsync_SynchronousOwnerPublishesWhileWaiting_ReusesPublishedKey) failed before the fix and passes after.

CI:

  • This PR: sqlclient-pr build 192526 on 16919af; all 39 checks pass.

  • Full stack: [Async AE 5] Load enclave keys asynchronously and route package retries #5085 build 192590 on aa9fadbf (includes this layer). UnitTests: Linux net10 2668 passed / 0 failed; Windows net10 2692 / 0; net462 2588 / 0; simulated-server tests ran and no stack tests were skipped. AE manual tests: 246 passed / 0 failed; only enclave tests skipped (no enclave server in CI).

  • Tests added or updated

  • Public API changes documented (none)

  • Verified against customer repro (not applicable to this foundation layer)

  • Ensure no breaking changes introduced

Guidelines

Please review the contribution guidelines before submitting a pull request:

Extract the cache foundation from #4585 with per-key async loading, cancellation, and shared sync/async publication. Relocate cache regressions into focused cache tests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 18:59
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Nov 16, 2026
@cheenamalhotra cheenamalhotra added this to the 8.1.0-preview1 milestone Nov 16, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human verification is needed because the current PR identity, feedback and CI evidence, and a trusted revision of supplemental review policies were not established.

No open issues 🎉

What changed in this PR

Adds the asynchronous Always Encrypted symmetric-key cache foundation for #3672, as the first of five layers split from #4585. Command execution integration is deferred; this layer changes no public APIs or dependencies.

Changes:

  • Adds per-key load gates, cancellable waits, and concurrent uncached loads when the cache lifetime is zero.
  • Shares sync/async cache publication while serializing provider invocation without holding the gate during outstanding async I/O.
  • Adds 16 unit tests covering cache reuse, cancellation, retries, provider serialization, and publication races.
File Description
src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​Utilities/​SymmetricKeyCacheTestProviders.cs Adds observable synchronous and asynchronous test providers.
src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​Utilities/​PendingDecryptionKeyStoreProvider.cs Adds a controllable provider for overlapping decryption tests.
src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​TestIsolationCollections.cs Isolates tests that change the process-wide cache lifetime.
src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​Microsoft/​Data/​SqlClient/​AlwaysEncrypted/​SymmetricKeyCacheTests.cs Adds async cache behavior and concurrency regression coverage.
src/​Microsoft.Data.SqlClient/​src/​Microsoft/​Data/​SqlClient/​AlwaysEncrypted/​SymmetricKeyCache.cs Implements async loading, per-key coordination, and shared cache publication.

Review effort: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 19:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Required policy checks could not be completed because trusted supplemental review policies or references were unavailable, and head CI is still pending.

No open issues 🎉

Review effort: Balanced

Avoid repeating decryption when a synchronous caller publishes the key while an async caller waits for the provider gate. Cover the race with a deterministic regression test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 19:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Required policy checks could not be completed without trusted review references, while head CI remains pending and Windows/.NET Framework runtime coverage is unconfirmed.

No open issues 🎉

Review effort: Balanced

@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.46018% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.07%. Comparing base (17bef7b) to head (16919af).
⚠️ Report is 11 commits behind head on main.

Files with missing lines Patch % Lines
...ata/SqlClient/AlwaysEncrypted/SymmetricKeyCache.cs 96.46% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5077      +/-   ##
==========================================
+ Coverage   68.99%   70.07%   +1.08%     
==========================================
  Files         314      314              
  Lines       45857    69893   +24036     
==========================================
+ Hits        31638    48981   +17343     
- Misses      14219    20912    +6693     
Flag Coverage Δ
CI-SqlClient ?
sqlclient-pr 70.07% <96.46%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cheenamalhotra
cheenamalhotra marked this pull request as ready for review November 17, 2026 06:28
@cheenamalhotra
cheenamalhotra requested a review from a team as a code owner November 17, 2026 06:28
@cheenamalhotra cheenamalhotra changed the title Add async Always Encrypted key cache (1/5) [Async AE 1] Add async key cache Nov 17, 2026
@cheenamalhotra cheenamalhotra added the Hotfix 8.0.0 PRs that must be backported to release/8.0 label Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Hotfix 8.0.0 PRs that must be backported to release/8.0

Projects

Status: To triage

Development

Successfully merging this pull request may close these issues.

4 participants