[xiangfu0]

PR flow

GET returns versioned envelope with redacted config; PUT requires envelope with expected version for atomic update.

flowchart TD
  N0["Client GET request for table config #40;F1#41;"]:::stModified
  N1["Controller returns envelope with responseType#44; tableName#44; baseVersions#44; and rede #40;F1#44; F27#41;"]:::stModified
  N2["Client PUT request with complete envelope #40;F1#41;"]:::stModified
  N3["Controller validates envelope#44; checks versions#44; and updates table config atom #40;F8#41;"]:::stModified
  N0 --> N1
  N1 --> N2
  N2 --> N3
  classDef stAdded fill:#dafbe1,stroke:#1a7f37,color:#1f2328,stroke-width:2px
  classDef stModified fill:#fff8c5,stroke:#9a6700,color:#1f2328,stroke-width:2px
  classDef stRemoved fill:#ffebe9,stroke:#cf222e,color:#1f2328,stroke-width:2px
  classDef stUnchanged fill:#f6f8fa,stroke:#656d76,color:#1f2328,stroke-width:1px

Loading

AI-generated · Green: added · Yellow: modified · Red: removed · Gray: existing

Partial evidence: 31 file patches omitted; 3 truncated.

Diff evidence

• F1: pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/admin/TableAdminClient.java — before · after
• F8: pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java — before · after
• F27: pinot-clients/pinot-java-client/src/test/java/org/apache/pinot/client/admin/PinotAdminClientTest.java — before · after

• Regenerate PR flow

Summary

• Return redacted, unresolved table-configuration representations from the table GET APIs by default.
• Apply the same representation policy to SHOW CREATE TABLE, SHOW CREATE MATERIALIZED VIEW, and materialized-view details.
• Cover credential-bearing ingestion, filesystem, decoder/provider/custom properties, structured login strings, and URI components while preserving ordinary values and ${...} placeholders.
• Require an explicit get_znode permission for raw ZooKeeper reads.
• Remove configuration bodies and SQL text from related failure diagnostics.

Update semantics

The GET APIs return a versioned envelope containing the redacted configs and the per-znode ZooKeeper data versions used to produce them. Clients editing a masked response must retain and submit the complete envelope.

• An unchanged ***** value restores the stored value only when it remains unambiguously bound to the same property and consumer identity.
• Literal replacements are applied as intentional updates.
• New, moved, version-mismatched, ambiguous, or reassociated markers are rejected instead of being persisted.
• Schema and existing OFFLINE/REALTIME configs are version-checked and committed in one ZooKeeper multi-operation.
• Hybrid responses can be edited and submitted together.
• A versioned edit cannot add a missing table type; use the normal table-creation flow and fetch a fresh envelope before editing.
• Legacy literal PUT bodies remain accepted when they do not contain a redaction marker.

Connection settings remain part of consumer identity when retaining masked values; stream-array reordering preserves the binding to the matched consumer. Version checks cover existing znode data versions; table-incarnation fencing across delete/recreate and transactional absence checks remain outside their scope.

Compatibility

This changes the default response shape for the two table-configuration GET APIs. The response now has an object-valued table identity, format version, base versions, and nested configs. The Java admin client, controller UI, segment-generation reader, copy flow, and affected integration-test consumers are updated accordingly.

The envelope is intentionally not deserializable as the previous top-level TableConfig or TableConfigs body, so replaying a complete masked response through an older controller fails before mutation. Mixed-version clients must preserve the envelope and send it to a controller that supports the new format. Copy requires the matching versioned response media type.

How to reproduce

1. Create a test table with representative credential properties, a credential-bearing URI, and an unresolved ${...} placeholder.
2. Fetch it through /tables/{name} and /tableConfigs/{name} as a table READ principal, and inspect both SHOW CREATE forms.
3. Verify literal credentials are shown as *****, the placeholder remains unchanged, and ordinary settings are preserved.
4. Edit an ordinary setting in the complete response envelope and submit it with PUT.
5. Fetch the stored config through an explicitly permitted administrative path and verify the original credential remains unchanged.
6. Repeat with a stale envelope and verify the update returns a conflict without a partial schema or hybrid-config write.
7. Attempt the raw ZooKeeper read as the READ-only principal and verify it is denied.

Validation

Validated source: 65626210bf, rebased onto Apache master 083f293738. All six original commits are retained. The controller conflict resolution preserves the upstream EnumSet validation API and instance-assignment skip gate alongside the versioned update paths.

• 192 targeted tests passed with zero failures, errors or skips across seven existing SPI/controller classes: redaction (28), stateless resource manager (31), redaction REST (21), read-only REST (2), table-config REST (15), task-config validation (4), and table-config utilities (91).
• Regression probes exercise retained masked values after connection changes, unchanged round trips, and valid stream-array reordering with distinct connection identities. The final source passes all three behaviors.
• Scoped Spotless, Checkstyle and license checks passed across all ten affected modules. Warning-enabled compilation covered the affected modules and their 64-module dependency reactor; the SPI/controller scope was rebuilt after the private restoration follow-up. Unchanged caller-compilation evidence was retained because public APIs and build configuration did not change. No compiler warnings matched added Java lines.
• Independent review of the full cumulative diff and the CAS/validation-API domain found no remaining actionable issues. Formal maintainer approval remains separate.

Hosted checks for current head 65626210bf completed successfully: both unit shards, both integration shards, linter, quickstart, Java 11 client and all five compatibility jobs. The current Unit workflow logs independently confirm the same 192 targeted invocations, including the credential-destination and stream-array-reordering regressions. Formal maintainer approval remains required.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 71.94526% with 574 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.45%. Comparing base (083f293) to head (dd1e179).
⚠️ Report is 87 commits behind head on master.

Files with missing lines	Patch %	Lines
...ot/spi/config/table/TableConfigRedactionUtils.java	73.05%	127 Missing and 174 partials ⚠️
...esources/PinotMaterializedViewRestletResource.java	53.50%	35 Missing and 18 partials ⚠️
...oller/api/resources/PinotTableRestletResource.java	72.39%	38 Missing and 15 partials ⚠️
...ntroller/helix/core/PinotHelixResourceManager.java	66.36%	23 Missing and 14 partials ⚠️
...ew/executor/GrpcMaterializedViewQueryExecutor.java	6.25%	30 Missing ⚠️
...ler/api/resources/TableConfigsRestletResource.java	89.56%	13 Missing and 6 partials ⚠️
...troller/api/resources/PinotDdlRestletResource.java	70.00%	11 Missing and 4 partials ⚠️
...rg/apache/pinot/client/admin/TableAdminClient.java	53.33%	11 Missing and 3 partials ⚠️
...ler/api/resources/RedactedTableConfigResponse.java	77.58%	2 Missing and 11 partials ⚠️
.../pinot/minion/taskfactory/TaskFactoryRegistry.java	20.00%	8 Missing ⚠️
... and 10 more

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #19236      +/-   ##
============================================
+ Coverage     69.82%   70.45%   +0.62%     
- Complexity     1519     1522       +3     
============================================
  Files          3558     3575      +17     
  Lines        235843   240648    +4805     
  Branches      37848    38993    +1145     
============================================
+ Hits         164678   169546    +4868     
+ Misses        58464    58061     -403     
- Partials      12701    13041     +340     

Flag	Coverage Δ
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-25	70.45% <71.94%> (+0.62%)	⬆️
lane-a	100.00% <ø> (ø)
lane-b	0.00% <ø> (ø)
temurin	70.45% <71.94%> (+0.62%)	⬆️
unittests	70.45% <71.94%> (+0.62%)	⬆️
unittests1	59.59% <71.32%> (+0.52%)	⬆️
unittests2	42.02% <52.59%> (+0.46%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

Introduces secure, versioned table-configuration representations while preserving safe edit workflows and atomic ZooKeeper updates.

Changes:

• Redacts credentials across APIs, DDL, materialized views, clients, UI, and diagnostics.
• Adds generation-checked envelopes and atomic schema/config updates.
• Adds explicit raw-ZooKeeper authorization and focused compatibility/security tests.

Reviewed changes

Copilot reviewed 64 out of 64 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
pinot-clients/.../TableAdminClient.java	Parses redacted envelopes.
pinot-clients/.../PinotAdminClientTest.java	Tests client parsing.
pinot-common/.../TableConfigVersionMismatchException.java	Defines version-conflict failure.
pinot-common/.../ZKMetadataProvider.java	Adds versioned schema reads.
pinot-common/.../SegmentGenerationUtils.java	Supports envelope responses.
pinot-common/.../AccessControlUserConfigUtils.java	Parses fine-grained permissions.
pinot-common/.../SegmentGenerationUtilsTest.java	Tests envelope compatibility.
pinot-common/.../AccessControlUserConfigUtilsTest.java	Tests permission parsing.
pinot-controller/.../AuthenticationFilter.java	Processes authorization-only endpoints.
pinot-controller/.../BasicAuthAccessControlFactory.java	Enforces fine-grained actions.
pinot-controller/.../ZkBasicAuthAccessControlFactory.java	Enforces ZooKeeper-backed actions.
pinot-controller/.../CopyTablePayload.java	Adds credential overrides.
pinot-controller/.../PinotDdlRestletResource.java	Redacts SHOW CREATE output.
pinot-controller/.../PinotMaterializedViewRestletResource.java	Sanitizes MV details.
pinot-controller/.../PinotTableRestletResource.java	Implements envelope GET/PUT and copy handling.
pinot-controller/.../RedactedTableConfigResponse.java	Defines single-config envelope.
pinot-controller/.../RedactedTableConfigsResponse.java	Defines combined envelope.
pinot-controller/.../TableConfigValidationUtils.java	Sanitizes validation diagnostics.
pinot-controller/.../TableConfigsRestletResource.java	Adds atomic combined updates.
pinot-controller/.../ZookeeperResource.java	Requires raw-read permission.
pinot-controller/.../PinotHelixResourceManager.java	Implements versioned atomic persistence.
pinot-controller/.../PinotTaskManager.java	Sanitizes task failures.
pinot-controller/.../TableConfigTunerUtils.java	Avoids configuration leakage.
pinot-controller/.../Services.ts	Handles versioned API responses.
pinot-controller/.../TaskQueueTable.tsx	Removes sensitive task details.
pinot-controller/.../TenantDetails.tsx	Handles envelope-shaped configs.
pinot-controller/.../api/PinotTableRestletResourceTest.java	Tests copy payload behavior.
pinot-controller/.../AuthenticationFilterTest.java	Tests authorization filtering.
pinot-controller/.../PinotDdlRestletResourceMaterializedViewUnitTest.java	Tests redacted MV DDL.
pinot-controller/.../PinotDdlRestletResourceUnitTest.java	Tests redacted table DDL.
pinot-controller/.../PinotMaterializedViewRestletResourceTest.java	Tests safe MV details.
pinot-controller/.../PinotReadOnlyTableConfigResourceTest.java	Tests read-only security.
pinot-controller/.../resources/PinotTableRestletResourceTest.java	Tests table resource behavior.
pinot-controller/.../TableConfigRedactionResourceTest.java	Tests envelope edit semantics.
pinot-controller/.../PinotHelixResourceManagerMaterializedViewBackfillTest.java	Updates MV backfill coverage.
pinot-controller/.../PinotHelixResourceManagerMaterializedViewListingTest.java	Updates MV listing coverage.
pinot-controller/.../PinotHelixResourceManagerStatelessTest.java	Tests atomic version checks.
pinot-controller/.../PinotTaskManagerStatelessTest.java	Tests sanitized task errors.
pinot-core/.../Actions.java	Defines raw ZooKeeper action.
pinot-core/.../BasicAuthPrincipal.java	Separates explicit permissions.
pinot-core/.../BasicAuthPrincipalUtils.java	Builds fine-grained principals.
pinot-core/.../FineGrainedAccessControl.java	Defaults unknown actions to denial.
pinot-core/.../ZkBasicAuthPrincipal.java	Supports ZooKeeper-defined actions.
pinot-core/.../BasicAuthTest.java	Tests explicit raw-read grants.
pinot-core/.../FineGrainedAuthUtilsTest.java	Tests fine-grained authorization.
pinot-core/.../ZkBasicAuthTest.java	Tests ZooKeeper-backed grants.
pinot-integration-tests/.../MultiStageEngineIntegrationTest.java	Adapts integration consumers.
pinot-materialized-view/.../MaterializedViewConsistencyManager.java	Removes sensitive diagnostics.
pinot-materialized-view/.../GrpcMaterializedViewQueryExecutor.java	Sanitizes query failures.
pinot-materialized-view/.../MaterializedViewMetadataCache.java	Uses safe config representations.
pinot-materialized-view/.../MaterializedViewTaskScheduler.java	Removes SQL/config logging.
pinot-materialized-view/.../GrpcMaterializedViewQueryExecutorTest.java	Tests safe failures.
pinot-materialized-view/.../MaterializedViewTaskSchedulerTest.java	Tests sanitized scheduling.
pinot-minion/.../TaskFactoryRegistry.java	Removes task-config diagnostics.
pinot-minion/.../TaskFactoryRegistryTest.java	Tests safe task diagnostics.
pinot-plugins/.../MaterializedViewTaskExecutor.java	Stops logging SQL/configs.
pinot-spi/.../ConfigUtils.java	Removes resolved config bodies.
pinot-spi/.../TableConfigRedactionUtils.java	Implements redaction and marker restoration.
pinot-spi/.../UserConfig.java	Adds fine-grained permissions.
pinot-spi/.../Obfuscator.java	Applies structured credential policy.
pinot-spi/.../UserConfigBuilder.java	Builds fine-grained permissions.
pinot-spi/.../ConfigUtilsTest.java	Tests diagnostic sanitization.
pinot-spi/.../TableConfigRedactionUtilsTest.java	Tests redaction and restoration.
pinot-spi/.../UserConfigTest.java	Tests permission serialization.
pinot-spi/.../ObfuscatorTest.java	Tests structured obfuscation.
pinot-sql-ddl/.../PropertyExtractor.java	Removes values from errors.
pinot-sql-ddl/.../CanonicalDdlEmitterTest.java	Tests safe DDL errors.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.