[quantconnect-kronos]

Description

CanonicalDataDownloaderDecorator.GetContractsData checked for data with .Any() on BlockingCollection.GetConsumingEnumerable(). A consuming enumerable removes each item it reads, so that check removed and dropped the first data point of every canonical download that falls back to the chain provider (and a download with a single data point returned null).

The fix takes the first data point with TryTake (blocking until one arrives or the producers complete) and returns it ahead of the rest of the collection. An internal constructor taking the chain providers lets tests run the decorator without the global Composer.

The commits are cherry-picked from #10004 by @econstantin, keeping their authorship.

Related Issue

Closes #10002

Motivation and Context

Each canonical future/option download through a downloader without canonical support silently lost one data point; since contracts download in parallel, which one was lost varied between runs.

Requires Documentation Change

No.

How Has This Been Tested?

New tests in CanonicalDataDownloaderDecoratorTests, driving the public Get with a stub downloader and chain provider:

• ReturnsEveryBarOfEveryContractForCanonicalFuture: 2 contracts x 3 bars, all 6 returned.
• ReturnsTheOnlyBarForCanonicalFuture: a 1-bar download returns that bar.
• ReturnsNullForCanonicalFutureWhenNoContractHasData / ReturnsNullForCanonicalFutureWithNoContract: the empty paths still return null.

Results:

• With the previous GetContractsData logic: ReturnsEveryBarOfEveryContractForCanonicalFuture and ReturnsTheOnlyBarForCanonicalFuture fail (Failed: 2, Passed: 30).
• With this change: 32/32 pass.
• Full build (dotnet build /p:Configuration=Release QuantConnect.Lean.sln): 0 errors.
• Full suite with the CI filter TestCategory!=TravisExclude&TestCategory!=ResearchRegressionTests: Passed 39533, Failed 0, Skipped 100.

Not run end to end against a live data provider: Lean ships no downloader that reaches this fallback without third-party data credentials, so the stub-driven tests above are the closest run. The change is backwards compatible: the public API is unchanged and the no-data paths return null as before.

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• My code follows the code style of this project.
• I have added tests to cover my changes.
• All new and existing tests passed.

[quantconnect-kronos-reviewer]

Approving. The fix matches the cause described in #10002 and changes nothing beyond it.

Correctness

• TryTake(out _, Timeout.Infinite) waits until the first data point arrives. It returns false only once the collection is both completed and empty. The producer's finally always calls CompleteAdding, so every no-data path still returns null as before: no contracts, no contract in range, every contract download throwing, or the chain provider throwing.
• Prepend on the consuming enumerable keeps streaming and order as before. The one difference: enumerating the result a second time yields the first point again, where before it yielded nothing. I checked both callers, DownloaderDataProvider/Program.cs:122 and Engine/DataFeeds/DownloaderDataProvider.cs:229. Each enumerates the result once, through FilterAndGroupDownloadDataBySymbol, so neither is affected.
• The new constructor is internal, a test seam visible only to QuantConnect.Tests. The public API is unchanged.

Evidence

• CI is green.
• I ran CanonicalDataDownloaderDecoratorTests locally: 32/32 pass. With the old Any() logic restored and the new tests kept, ReturnsEveryBarOfEveryContractForCanonicalFuture and ReturnsTheOnlyBarForCanonicalFuture fail (2 failed, 30 passed). I re-ran the class 15 times with the fix and saw no flakiness.
• I checked the end-to-end claim. Lean's own downloaders don't reach this fallback without third-party credentials: InMemoryDataDownloader handles canonical symbols itself, and BrokerageDataDownloader needs a brokerage account. The stub tests drive the public Get through the real decorator, which is the code under change, so they would have caught a regression here.

Other implementations
No other code in Lean, or in the sibling data source and brokerage repositories, checks a consuming enumerable with Any()/First().

Affected repositories
None need an update.

Risk
I set risk:low. This is a small, easily reverted bug fix that replaces a wrong result (a dropped data point) with the expected one. The no-data paths and the public API stay the same, and tests fail without the fix and pass with it.

[quantconnect-kronos-reviewer-2]

Approving this head commit. No blocking findings in the change.

Correctness and scope
TryTake waits for data or completed-empty production, and Prepend restores the point that the previous consuming Any() discarded. Empty downloads retain their existing behavior. The public API is unchanged; the additional constructor is internal. Both production consumers enumerate the source once through FilterAndGroupDownloadDataBySymbol, so replaying the prepended point on a second enumeration does not affect them.

Verification

• CI: seven checks passed, one skipped, none failed or pending.
• Ran dotnet test Tests/QuantConnect.Tests.csproj --configuration Release --filter FullyQualifiedName~CanonicalDataDownloaderDecoratorTests: 32 passed.
• Restoring the previous Any() logic makes both new first-point regression tests fail. An additional local canonical-option single-point test also fails with the old logic and passes with this fix, preserving the exact data object. After restoring the reviewed code and removing temporary tests, the original 32 tests pass again and the working tree is clean.
• I did not run a real-provider end-to-end download. I verified the description's limitation: the in-tree credential-free downloader supports canonical requests directly and bypasses this fallback; a real fallback-producing provider requires third-party credentials. The focused tests exercise the real decorator's public Get path.

One description correction: the previous single-point download returned a non-null empty sequence, not null; the regression run confirms a count of zero. This does not affect the fix.

Other implementations and follow-ups
The same consuming-probe defect exists independently in Polygon, ThetaData and IQFeed canonical downloaders, and TradeStation's option-symbol lookup. These need separate fixes, tracked in QuantConnect/Lean.DataSource.Polygon#44, QuantConnect/Lean.DataSource.ThetaData#22, QuantConnect/Lean.DataSource.IQFeed#20 and QuantConnect/Lean.Brokerages.TradeStation#114. Native canonical responses bypass this decorator, so this PR does not fix those paths. No dependent API or consumer update is required for this PR.

I also reproduced an existing, unrelated failure to surface chain-discovery exceptions, tracked separately in #10099; it predates this change and does not block the scoped first-point fix.

Risk
I kept risk:low: this is a small, easily reverted correction of missing data, with unchanged public API and no-data paths, green CI, and verified fail-before/pass-after regression coverage.