Skip to content

feat: add general experimentation infrastructure and telemetry attribution - #1873

Open
Stella Huang (StellaHuang95) wants to merge 1 commit into
microsoft:mainfrom
StellaHuang95:experimentation-infrastructure
Open

Stella Huang (StellaHuang95) wants to merge 1 commit into
microsoft:mainfrom
StellaHuang95:experimentation-infrastructure

Conversation

@StellaHuang95

Copy link
Copy Markdown
Contributor

Summary

Prepare reusable experimentation infrastructure for the Python Environments extension without enabling an experiment or changing feature settings.

  • Add vscode-tas-client with one activation-owned service, validated publisher configuration, isolated globalState caching, defaulted treatment queries, bounded HTTPS requests, and consent-aware cancellation and disposal.
  • Reuse a lifecycle-owned telemetry reporter and attach abexp.assignmentcontext to ordinary events, explicitly reported errors, and automatically captured exceptions. Preserve consent, data cleaning, capture-time attribution, and existing event names without duplicate automatic-error reporting.
  • Add classified SDK diagnostics that distinguish cache initialization from successful assignment fetching, deterministic lifecycle and transport coverage, and a contract test using the real SDK with a fake transport.
  • Document the configuration contract, onboarding prerequisites, and existing measurements and gaps for a baseline experimentation scorecard.

Scope and rollout

The publisher-owned experimentation block is intentionally absent. This build makes no extension TAS requests; existing consented telemetry continues, with a notConfigured initialization diagnostic. No feature setting is exposed or enabled, and no A/B test is launched.

Update TypeScript to support the current TAS SDK declarations and @vscode/extension-telemetry to 1.5.2 for its public automatic-error logger options. Generated native tooling and local verification artifacts are excluded.

Validation

Re-run for this PR:

  • npm run lint
  • npm run compile-tests
  • Full unit suite: 2,505 passing, 6 pending existing platform/symlink cases
  • npm run compile
  • git diff --cached --check

The implementation session also exercised Windows smoke/integration/discovery workflows, an isolated environment/package lifecycle, installed release VSIX behavior, and real-SDK/telemetry paths with controlled responses. Those broader checks were not repeated during this PR handoff.

Follow-up and limits

Live rollout still requires an approved endpoint, randomization identity and targeting contract, integration review, metrics/scorecard setup, and A/A verification. MachineId is not assumed to be DevDeviceId.

Production TAS authentication/proxies, non-Windows and remote hosts, and long-duration polling remain unverified. The earlier session reproduced independent multi-root selection and shutdown issues on the unchanged base commit; unrelated production behavior is deliberately outside this change.

…ution

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog no-merge PR is currently blocked from merging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant