Skip to content

feat: add general experimentation infrastructure and telemetry attribution - #32

Closed
StellaHuang95 wants to merge 1 commit into
mainfrom
experimentation-infrastructure
Closed

StellaHuang95 wants to merge 1 commit into
mainfrom
experimentation-infrastructure

Conversation

@StellaHuang95

Copy link
Copy Markdown
Owner

Summary

Prepare reusable experimentation infrastructure for the Python Environments extension without enabling an experiment or changing feature settings.

  • Add vscode-tas-client with one activation-owned service, validated publisher configuration, isolated globalState caching, defaulted treatment queries, bounded HTTPS requests, and consent-aware cancellation and disposal.
  • Reuse a lifecycle-owned telemetry reporter and attach abexp.assignmentcontext to ordinary events, explicitly reported errors, and automatically captured exceptions. Preserve consent, data cleaning, capture-time attribution, and existing event names without duplicate automatic-error reporting.
  • Add classified SDK diagnostics that distinguish cache initialization from successful assignment fetching, deterministic lifecycle and transport coverage, and a contract test using the real SDK with a fake transport.
  • Document the configuration contract, onboarding prerequisites, and existing measurements and gaps for a baseline experimentation scorecard.

Scope and rollout

The publisher-owned experimentation block is intentionally absent. This build makes no extension TAS requests; existing consented telemetry continues, with a notConfigured initialization diagnostic. No feature setting is exposed or enabled, and no A/B test is launched.

Update TypeScript to support the current TAS SDK declarations and @vscode/extension-telemetry to 1.5.2 for its public automatic-error logger options. Generated native tooling and local verification artifacts are excluded.

Validation

Re-run for this PR:

  • npm run lint
  • npm run compile-tests
  • Full unit suite: 2,505 passing, 6 pending existing platform/symlink cases
  • npm run compile
  • git diff --cached --check

The implementation session also exercised Windows smoke/integration/discovery workflows, an isolated environment/package lifecycle, installed release VSIX behavior, and real-SDK/telemetry paths with controlled responses. Those broader checks were not repeated during this PR handoff.

Follow-up and limits

Live rollout still requires an approved endpoint, randomization identity and targeting contract, integration review, metrics/scorecard setup, and A/A verification. MachineId is not assumed to be DevDeviceId.

Production TAS authentication/proxies, non-Windows and remote hosts, and long-duration polling remain unverified. The earlier session reproduced independent multi-root selection and shutdown issues on the unchanged base commit; unrelated production behavior is deliberately outside this change.

…ution

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@StellaHuang95

Copy link
Copy Markdown
Owner Author

Superseded by microsoft#1873, targeting the upstream repository. The same branch and changes are preserved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant