Inner reads return at most 50 documents #2599

Current behavior

• Each leg runs a regular find internally ([IntermediateCollectionReadTask.java#L75](

  data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/reranking/IntermediateCollectionReadTask.java

  Line 75 in 02d0423

  findCommandResolver.resolveCommand(commandContext, findCommand);

  )). An unsorted, vector, or BM25 inner find only reads the first page ([CollectionReadOperation.java#L102](

  data-api/src/main/java/io/stargate/sgv2/jsonapi/service/operation/collections/CollectionReadOperation.java

  Line 102 in 02d0423

  Iterator<Row> rowIterator = rSet.currentPage().iterator();

  )).
• For vector and BM25 reads, the page size gets there through a hack ([stargate/data-api#1961](findAndRerank inner reads do not select full hybridLimits number of records: #1961)):
  • the builder writes hybridLimits into the request-scoped CommandContext ([FindAndRerankOperationBuilder.java#L304-L311](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/service/resolver/FindAndRerankOperationBuilder.java

    Lines 304 to 311 in 02d0423

    	// Hack: See https://github.com/stargate/data-api/issues/1961
    	// copying the hybrid limits on the command context so the find command resolver can pick it up
    	// when the command runs later, so we can set the page size to be the same as the limit
    	commandContext.setHybridLimits(
    	getOrDefault(
    	command.options(),
    	FindAndRerankCommand.Options::hybridLimits,
    	FindAndRerankCommand.HybridLimits.DEFAULT));

    ));
  • FindCommandResolver reads it back ([FindCommandResolver.java#L131-L135](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/service/resolver/FindCommandResolver.java

    Lines 131 to 135 in 02d0423

    	// Hack: See https://github.com/stargate/data-api/issues/1961
    	int pageSize =
    	commandContext.getHybridLimits() == null
    	? operationsConfig.defaultPageSize()
    	: commandContext.getHybridLimits().vectorLimit();

    ), [#L152-L156](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/service/resolver/FindCommandResolver.java

    Lines 152 to 156 in 02d0423

    	// Hack: See https://github.com/stargate/data-api/issues/1961
    	int pageSize =
    	commandContext.getHybridLimits() == null
    	? operationsConfig.defaultPageSize()
    	: commandContext.getHybridLimits().lexicalLimit();

    )).
• Plain reads ignore the hack, and the filter leg will use plain reads:
  • unsorted reads use defaultPageSize, which is 50 ([FindCommandResolver.java#L191-L199](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/service/resolver/FindCommandResolver.java

    Lines 191 to 199 in 02d0423

    	// Hack: See https://github.com/stargate/data-api/issues/1961
    	// not commandContext.getHybridLimits() becuase there is no limit for a non ANN or BM25 query
    	return FindCollectionOperation.unsorted(
    	commandContext,
    	resolvedDbLogicalExpression,
    	command.buildProjector(),
    	pageState,
    	limit,
    	operationsConfig.defaultPageSize(),

    ), [OperationsConfig.java#L43](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/config/OperationsConfig.java

    Line 43 in 02d0423

    int DEFAULT_PAGE_SIZE = 50;

    ));
  • sorted reads cap the limit at defaultPageSize ([FindCommandResolver.java#L172-L181](

    data-api/src/main/java/io/stargate/sgv2/jsonapi/service/resolver/FindCommandResolver.java

    Lines 172 to 181 in 02d0423

    	// Hack: See https://github.com/stargate/data-api/issues/1961
    	// not commandContext.getHybridLimits() becuase there is no limit for a non ANN or BM25 query
    	return FindCollectionOperation.sorted(
    	commandContext,
    	resolvedDbLogicalExpression,
    	command.buildProjector(),
    	pageState,
    	// For in-memory sorting if no limit provided in the request will use
    	// documentConfig.defaultPageSize() as limit
    	Math.min(limit, operationsConfig.defaultPageSize()),

    )).

Why fix it

• A filter leg with rerankCandidateLimit: 100 silently reads only 50 documents. There is no error and no warning.
• Single vector and lexical legs would also have to push rerankCandidateLimit through the hack.
• Capping rerankCandidateLimit at 50 doesn't avoid the problem. defaultPageSize is configurable, so the real cap can change per environment.
• Hidden shared state makes it hard to see who writes the value and who reads it. Every new leg type adds one more place to remember.

Implementation split and acceptance criteria

This foundation keeps the current public request syntax and is split across two dependent PRs:

1. Resolve internal candidate reads through resolveCollectionCandidateCommand, requiring a positive explicit inner options.limit. Derive vector/BM25 page sizes from that command and remove CommandContext.hybridLimits coupling. Preserve asymmetric leg budgets.
2. Add an internal candidate read mode that follows continuation pages until its budget is filled or results are exhausted. Use fresh state per subscription and sequential query fanout within each leg. Sorted reads already scan continuation pages, so retain that full scan and maxDocumentSortCount safeguard while removing the ordinary result-page cap for candidate reads.

Acceptance coverage includes underfilled and empty pages with continuation, early exhaustion, stopping without an extra fetch, later-page failures, shared budgets across generated queries, repeated subscriptions, and sorted/unsorted candidate counts above 50. Ordinary find pagination and sorting remain unchanged. Candidate limits apply before passage filtering, with no replacement reads for discarded passages. Successful HTTP reranking coverage is tracked in #2600.

Public single-leg syntax and candidate-limit configuration remain in #2601/#2602, and public filter retrieval/scoring remains in #2603.

Draft PRs

PR #2606 makes candidate sizing explicit. PR #2607 completes bounded collection and depends on #2606, which depends on #2605.