Repository navigation
[Async AE 5] Load enclave keys asynchronously and route package retries - #5085
cheenamalhotra wants to merge 4 commits into
Conversation
Generate enclave packages with the async key store APIs after the describe round trip, revalidate the connection after each await, and route retryable package faults into the bounded enclave retry without transferring an already released reservation. Final layer of the #4585 split. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
A critical concurrency gap during the key-store await allows connection reassignment or overlapping execution to redirect pending work or overwrite encryption state.
1 open issue
What changed in this PR
Completes the fifth layer of async Always Encrypted integration in SqlClient, building on #5081’s metadata-cache work to load enclave keys asynchronously and route package failures through the existing retry mechanism.
Changes:
- Adds sequential async enclave-key loading while sharing validation and package construction with synchronous execution.
- Awaits package generation before command execution, revalidates the connection, and handles retryable package faults without transferring an already-released reservation.
- Adds provider, retry, and connection-revalidation tests; re-enables debug stale-enclave retry coverage.
| File | Description |
|---|---|
src/Microsoft.Data.SqlClient/tests/UnitTests/SimulatedServerTests/SqlCommandEnclaveAsyncTests.cs |
Tests connection revalidation during key loading and retry reservation handling. |
src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/SqlCommandTests.cs |
Covers retry success, failure, cancellation, and non-retryable write faults. |
src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft/Data/SqlClient/EnclaveDelegateTests.cs |
Tests provider dispatch, sequential key loading, cancellation, and errors. |
src/Microsoft.Data.SqlClient/tests/ManualTests/AlwaysEncrypted/ApiShould.cs |
Re-enables debug package-failure retry scenarios. |
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.Reader.cs |
Adds async package generation and an await-based post-describe execution path. |
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlCommand.cs |
Routes retryable package faults through bounded retry with adjusted reservation ownership. |
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveDelegate.cs |
Adds async key loading and shared key-info validation. |
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/EnclaveDelegate.Crypto.cs |
Adds async package generation and shared package assembly; clarifies session-lock behavior. |
Review effort: Balanced
Merge dev/cheena/ae-async-metadata-cache (9fee7e7) and reuse its in-progress guard on the async enclave path. Successful async preparation now hands the flag to the reader continuation, which holds it through package key loading and the RunExecuteReaderTds reservation, revalidates the connection lifetime after each await, and clears it on success, failure or cancellation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pass the captured database to ValidateConnectionLifetime after describe and after the package key await. Share UseDatabaseQueryEngine and add a ChangeDatabase test while the package key is pending. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Async enclave retry changes affect connection-reservation ownership and command reuse, while unexecuted simulated-server and real enclave stale-session retry tests leave key behavior needing CI/runtime confirmation.
1 open issue
Review effort: Balanced
BeginExecuteXmlReader now attaches its callback to and returns the global completion task, like Reader and NonQuery, so callers observe the bounded enclave retry instead of the faulted first attempt. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Async enclave changes span command guards, reservation ownership, and XML retry completion, while simulated-server and live SQL Server/enclave validation remain unverified and require human confirmation.
No open issues 🎉
1 resolved since last review
Review effort: Balanced
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev/cheena/ae-async-metadata-cache #5085 +/- ##
======================================================================
- Coverage 72.81% 70.68% -2.13%
======================================================================
Files 314 314
Lines 71546 70348 -1198
======================================================================
- Hits 52093 49724 -2369
- Misses 19453 20624 +1171
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|

Description
Layer 5 of 5 split from #4585. Stacked on #5081.
EnclaveDelegate: addsGenerateEnclavePackageAsyncandGetDecryptedKeysToBeSentToEnclaveAsync. Keys load one at a time with async key store calls. Sync and async paths shareBuildEnclavePackageandCreateColumnEncryptionKeyInfo.SqlCommand.Reader: after describe,ContinueRunExecuteReaderTdsAsync(on aTask.Runworker) generates the enclave package asynchronously, then runs the TDS execution. This replaces the syncGenerateEnclavePackagecall. The connection is revalidated after each await. Cancellation propagates unwrapped.SqlCommand.CreateLocalCompletionTask: a retryable enclave package fault before the read stage now goes through the same bounded retry as read-stage failures. This is the first production reader of_asyncExecutionCancellationToken.Fixes from review feedback:
_enclaveAttestationInProgress) is held from the post-describe continuation through package key loading, package generation and theRunExecuteReaderTdshandoff. It is cleared on failure or cancellation. ChangingConnectionor starting another execution while package keys load now throws.ValidateConnectionLifetimegets the database captured before dispatch, after describe and after the package-key await. A database change during key loading throws.BeginExecuteXmlReadernow attaches its callback to and returns the global completion task, like Reader and NonQuery. Callers see the retry's outcome, not the faulted first attempt.Deviations from #4585:
EnclaveDelegate.Crypto.cscomments mention the session-lock wait on a cache miss.Issues
Part of #4585.
Testing
EnclaveDelegateTests(6 tests): async vs sync provider API, all keys resolved, sequential loading, cancellation, provider failure.SqlCommandTests: newCreateLocalCompletionTask_*tests (retryable write fault retries, retry fails, retry canceled, non-retryable fault, canceled write).SimulatedServerTests/SqlCommandEnclaveAsyncTests:Execute_PendingPackageKey_*(revalidation, connection setter, second execution, database change),Prepare_Completed_HandsOffInProgressFlag,Retry_PackageFault_DoesNotTransferReservation,BeginExecuteXmlReader_RetryablePackageFault_CompletesWithRetry.ManualTests/AlwaysEncrypted/ApiShould.cs: re-enables debug stale-enclave retry coverage.Local results at aa9fadb:
dotnet test src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj -f net10.0 -c Debug|Release --filter "FullyQualifiedName~SqlCommandEnclaveAsyncTests|FullyQualifiedName~SqlCommandQueryMetadataCacheAsyncTests|FullyQualifiedName~EnclaveDelegateTests|FullyQualifiedName~SqlCommandTests|FullyQualifiedName~SqlQueryMetadataCacheTests|FullyQualifiedName~SqlSecurityUtilityTests|FullyQualifiedName~SymmetricKeyCacheTests|FullyQualifiedName~AsyncKeyStoreProviderTests": passed, 178/178 in each configuration. Simulated-server tests were run with a local-only macOS listener workaround (not committed).dotnet buildof UnitTests and ManualTests with-f net462 -c Debug|Release: 0 warnings, 0 errors.dotnet build build.proj -t:BuildSqlClient -p:Configuration=Debug|Release: 0 warnings, 0 errors.CI (build 192590, aa9fadb):
Pending: enclave end-to-end testing against a real enclave server.
Guidelines
Please review the contribution guidelines before submitting a pull request: