Skip to content

[Async AE 3] Load keys asynchronously during parameter discovery - #5080

Open
cheenamalhotra wants to merge 4 commits into
dev/cheena/async-ae-crypto-utilitiesfrom
dev/cheena/ae-async-parameter-discovery
Open

cheenamalhotra wants to merge 4 commits into
dev/cheena/async-ae-crypto-utilitiesfrom
dev/cheena/ae-async-parameter-discovery

Conversation

@cheenamalhotra

@cheenamalhotra cheenamalhotra commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Description

Layer 3/5 split from #4585. Depends on #5079 (base branch).

  • The describe-parameter-encryption parser now records CMK signature checks and CEK decryptions, then runs them after parsing completes. The sync path calls the sync utilities; the async path awaits the async utilities with the caller's token. Metadata is cached only after the key operations succeed (CacheQueryMetadataIfNeeded). The parser documents its exception ordering and why keys load sequentially.
  • ExecuteReaderAsync, ExecuteNonQueryAsync and ExecuteXmlReaderAsync save the caller's token, then clear it on cleanup and on failed setup. On failed setup, the async call context or the token registration is disposed.
  • No public API changes.

Review feedback fix: added gated-provider manual tests for key-store I/O that is still pending (cdb22c8).

Integration is partial until later layers land:

  • Layer 4 makes the query metadata cache path async. The existing sync cache lookup stays for now.
  • Layer 5 makes enclave package loading async and is the first production reader of the saved token. Sync package loading stays for now.

Issues

Relates to #3672.

Testing

  • Unit tests added in tests/UnitTests/Microsoft/Data/SqlClient/SqlCommandTests.cs, moved there from the source PR's SqlCommandAsyncShould:
    • End*Async_CanceledTask_ThrowsOperationCanceled (Reader, NonQuery, XML)
    • *Async_FailedSetup_ClearsCancellationToken (Reader, NonQuery, XML; with and without a connection)
  • Manual tests added in ManualTests/AlwaysEncrypted/AsyncKeyStoreProviderTests.cs:
    • Sync execution uses the sync provider API.
    • Reader/NonQuery/XML async execution uses the async API.
    • The caller's token reaches the provider when it cancels.
    • With a gated provider, the command stays pending until the provider is released, then returns the result.
    • Cancelling while the provider is pending throws, and the command stays reusable.

Local validation (macOS, head 86b2cdc):

  • dotnet test -f net10.0 -c Debug|Release --filter "FullyQualifiedName~Microsoft.Data.SqlClient.UnitTests.SqlCommandTests|FullyQualifiedName~SqlSecurityUtilityTests|FullyQualifiedName~SymmetricKeyCache": passed, 74/74, in both configurations.
  • dotnet build -f net10.0 and -f net462 for ManualTests: 0 errors.
  • The manual AE tests were not run locally because they need SQL Server.

CI from the top of the stack (#5085, build 192590, commit aa9fadb, which contains this head):

  • UnitTests: Linux net10 passed 2668 with 0 failures, Windows net10 passed 2692 with 0 failures, and net462 passed 2588 with 0 failures. Simulated-server tests ran, and no stack tests were skipped.
  • AE manual tests: 246 passed and 0 failed, including AsyncKeyStoreProviderTests (not skipped). Only the enclave tests were skipped, because CI has no enclave server.

Guidelines

Please review the contribution guidelines before submitting a pull request:

Layer 3/5 of #4585. Defer CMK signature verification and CEK decryption
from the describe-parameter-encryption parser so the async path awaits the
async key-store utilities with the caller's token. Capture and reset the
caller token across Reader/NonQuery/XML async setup and cleanup.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 19:14
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Nov 16, 2026
@cheenamalhotra cheenamalhotra added this to the 8.1.0-preview1 milestone Nov 16, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The tests use a provider that throws synchronously, leaving pending key-store I/O, successful metadata-cache publication, and cancellation followed by command reuse unverified.

1 open issue
What changed in this PR

This third layer of the Always Encrypted async integration connects parameter discovery to asynchronous key-store utilities without changing public APIs. Async query-metadata cache and enclave-package key loading remain deferred to later layers.

Changes:

  • Shares describe-result parsing, defers signature verification and key decryption, and caches metadata only after key operations succeed.
  • Captures caller cancellation tokens and improves cleanup for Reader, NonQuery, and XML async execution.
  • Adds tests for provider dispatch, cancellation propagation, command reuse, and failed-setup cleanup.
File Description
src/​Microsoft.Data.SqlClient/​tests/​UnitTests/​Microsoft/​Data/​SqlClient/​SqlCommandTests.cs Adds cancellation and failed-setup token-cleanup tests.
src/​Microsoft.Data.SqlClient/​tests/​ManualTests/​AlwaysEncrypted/​AsyncKeyStoreProviderTests.cs Adds provider-dispatch, cancellation, and command-reuse integration tests.
src/​Microsoft.Data.SqlClient/​src/​Microsoft/​Data/​SqlClient/​SqlCommand.Xml.cs Captures and clears XML execution tokens; handles failed-setup disposal.
src/​Microsoft.Data.SqlClient/​src/​Microsoft/​Data/​SqlClient/​SqlCommand.Reader.cs Captures and clears reader execution tokens; refines failed-setup disposal.
src/​Microsoft.Data.SqlClient/​src/​Microsoft/​Data/​SqlClient/​SqlCommand.NonQuery.cs Captures and clears non-query execution tokens; handles failed-setup disposal.
src/​Microsoft.Data.SqlClient/​src/​Microsoft/​Data/​SqlClient/​SqlCommand.Encryption.cs Defers key operations, selects sync or async utilities, and delays metadata caching until success.
src/​Microsoft.Data.SqlClient/​src/​Microsoft/​Data/​SqlClient/​SqlCommand.cs Adds and documents the saved async-execution cancellation token.

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

SQL Server/key-store integration remains untested and CI is pending, leaving deferred provider completion, cancellation, command reuse, and enclave execution in need of human validation.

1 open issue

Review effort: Balanced

Add a gated provider that holds async CEK decryption pending and then
returns the real key. Cover successful completion after release and
cancellation while the provider is pending, followed by command reuse.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 16, 2026 20:20
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Async Always Encrypted discovery changes parser/key-loading ordering and cancellation lifetimes, requiring human review and SQL Server/key-store validation because integration tests for success, cancellation, and command reuse were not run.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings November 16, 2026 20:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Async Always Encrypted discovery changes key-loading and cancellation lifetimes, requiring human validation because SQL Server/key-store integration tests for pending-provider completion, cancellation/reuse, and enclave signature verification have not run.

No open issues 🎉

Review effort: Balanced

@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 61.81818% with 42 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.25%. Comparing base (c75f65f) to head (86b2cdc).

Files with missing lines Patch % Lines
.../Microsoft/Data/SqlClient/SqlCommand.Encryption.cs 42.64% 39 Missing ⚠️
...ent/src/Microsoft/Data/SqlClient/SqlCommand.Xml.cs 82.35% 3 Missing ⚠️
Additional details and impacted files
@@                           Coverage Diff                            @@
##           dev/cheena/async-ae-crypto-utilities    #5080      +/-   ##
========================================================================
- Coverage                                 70.27%   70.25%   -0.02%     
========================================================================
  Files                                       314      314              
  Lines                                     70018    70098      +80     
========================================================================
+ Hits                                      49202    49250      +48     
- Misses                                    20816    20848      +32     
Flag Coverage Δ
sqlclient-pr 70.25% <61.81%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cheenamalhotra
cheenamalhotra marked this pull request as ready for review November 17, 2026 06:28
@cheenamalhotra
cheenamalhotra requested a review from a team as a code owner November 17, 2026 06:28
@cheenamalhotra cheenamalhotra changed the title Load Always Encrypted keys asynchronously during parameter discovery [Async AE 3] Load keys asynchronously during parameter discovery Nov 17, 2026
@cheenamalhotra cheenamalhotra added the Hotfix 8.0.0 PRs that must be backported to release/8.0 label Nov 17, 2026
await CreateEnclaveSessionAsync(attestationInfo, isRetry, cancellationToken).ConfigureAwait(false);
}

IReadOnlyList<ColumnMasterKeySignatureVerification> verifications = pending.SignatureVerifications;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question (security): Should signature verification run before the enclave session is created?

Before this change, VerifyColumnMasterKeySignature (trusted key path + CMK signature check) ran while reading the keys result set, i.e. before attestation. Now the foreach above calls CreateEnclaveSession(Async) first, so a key we'd reject still triggers attestation and gets an enclave session cached, and an attestation failure masks the signature error. The Core doc comment says failure state is unchanged. Was the reorder intended? If not, could the pending verifications run before the first enclave session is created (sync path at L809 too)? Either way, the Core remarks should describe the new order.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Hotfix 8.0.0 PRs that must be backported to release/8.0

Projects

Status: To triage

Development

Successfully merging this pull request may close these issues.

4 participants