From 915fa00485754b1c71720d4f7d7842382f48ca70 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sat, 23 May 2026 18:42:40 +0000 Subject: [PATCH 01/82] docs(oes): add Egil.Orleans.Messaging API design document MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Comprehensive API design document covering all 8 design sections: - §0: Scope, packaging (one NuGet, split-ready), name settled as Egil.Orleans.Messaging - §1: IStateManager atomic writes with in-flight recovery - §2: Outbox storage placement (co-located on grain state) - §3: Outbox sealed class shape, epoch semantics, fingerprint equality, configurable max depth - §3a: VersionedState for ImmutableArray equality trap - §4: MessageTracker sealed class, dual-dictionary dedup - §5: StreamManager fluent builder facade - §6: Opinionated functional-grain pattern (guidance, not enforced) - §7: OutboxProcessor grain-scoped timer+reminder dispatch with callback-based postmen, per-attempt error tracking - §8: Naming (flat namespace), serialization ([Alias] with fully qualified prefix, sequential [Id]), telemetry conventions All decisions settled via structured grilling sessions. Directory still named Egil.Orleans.CQRS — rename deferred to project scaffold. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.CQRS/api-design.md | 1230 +++++++++++++++++++++++++++++++ 1 file changed, 1230 insertions(+) create mode 100644 Egil.Orleans.CQRS/api-design.md diff --git a/Egil.Orleans.CQRS/api-design.md b/Egil.Orleans.CQRS/api-design.md new file mode 100644 index 00000000..77818374 --- /dev/null +++ b/Egil.Orleans.CQRS/api-design.md @@ -0,0 +1,1230 @@ +# Egil.Orleans.Messaging — API Design + +Working design doc for the abstractions in the `Egil.Orleans.Messaging` +library. + +Each section captures the decided shape, the rationale, and any explicit +non-goals. Decisions are settled top-down via grilling sessions; revisit +only when a new constraint surfaces. + +Status legend: **Settled** = won't change without a new force; **Open** = +still under design; **Deferred** = explicitly postponed past the spike. + +--- + +## 0. Scope & packaging + +**Status:** Settled. + +### What this library is + +A set of composable building blocks for Orleans grains that need: + +1. **Atomic, recoverable state writes** — grain's observable `State` is + never out of sync with what is durably persisted, even on ambiguous + write failures. +2. **Outbox pattern** — durable, co-located message buffer that commits + atomically with business-state changes. +3. **Outbox processing (postman)** — timer + reminder driven dispatch + with retry, telemetry, and failure callbacks. +4. **Receiver-side dedup** — `MessageTracker` that tracks high-water + positions from both outbox senders and Orleans streams. +5. **Stream subscription management** — fluent subscribe/resume/error + facade over Orleans implicit subscriptions. + +### What this library is NOT + +- Not CQRS in the read/write-model-separation sense. No read-model + projections, no query stores, no event sourcing. +- Not a replacement for `IGrainStorage`. It wraps `IPersistentState`, + not the provider layer. + +### Packaging + +One NuGet package for now. Internal boundaries (state management vs +messaging vs streaming) are kept clean so a future split is mechanical. +Split only when dependency weight becomes a concrete problem. + +### Name + +**`Egil.Orleans.Messaging`** — the outbox, postman, dedup, and stream +manager are all messaging infrastructure. `IStateManager` exists to make +the messaging atomic. Messaging is the reason the library exists; safe +state is the enabler. + +--- + +## 1. `IStateManager` — atomic state writes with in-flight recovery + +**Status:** Settled. + +### Goal + +Replace direct grain use of `IPersistentState` with a thin wrapper +that guarantees the grain's observable `State` is never out of sync with +what is durably persisted, even when `WriteStateAsync` fails ambiguously +(timeout, network drop, server 5xx, ETag conflict). + +Grain code injects `IStateManager`; the raw +`IPersistentState` stays internal to the wrapper. This is +non-negotiable — exposing both is the loophole that lets grain authors +read stale `storage.State` after a failed write. + +### Interface + +```csharp +public enum WritePolicy { Concurrent, Force } + +public interface IStateManager where T : class, IEquatable +{ + T State { get; } + Task ReadAsync(); + Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concurrent); + Task ClearAsync(); +} +``` + +`WritePolicy.Force` is a rare last-resort escape hatch — nulls +`storage.Etag` before writing so the underlying provider skips its +optimistic-concurrency check. Version stamping and the read-back-on- +failure recovery path still run identically. See §3a for why this lives +at the `IStateManager` layer rather than as a settable `Version` +property on `VersionedState`. + +Constraints on `T`: + +- `class` — atomic reference swap of `State` from `[AlwaysInterleave]` + handlers is a single pointer load, no torn reads. +- `IEquatable` — recovery path compares server-side state to + attempted write to decide swallow-vs-rethrow. Records implement this + for free. + +Users pick one of two paths for `T`: + +- **Path A — plain record + structural equality** + (`MyState : IEquatable`). User owns `Equals`. Works trivially + for record-of-primitives shapes. Breaks silently if state holds + `ImmutableArray<>` (reference-equality trap) — user must override + `Equals` themselves in that case. +- **Path B — inherit `VersionedState`** + (`MyState : VersionedState`). Framework owns equality via a + per-write `Guid Version`. Immune to any collection-equality issues in + the state graph. Recommended default for any non-trivial state. + +See §3a for the `VersionedState` base class. + +### Activation: state is auto-populated + +`IStateManager` behaves like `IPersistentState` from the grain's +point of view: by the time `OnActivateAsync` runs, `State` already +reflects what is durably stored. Grain code does **not** call `ReadAsync` +during activation. + +This is achieved by composition rather than reimplementation: the +default `StateManager` wraps an `IPersistentState` facet that +Orleans hydrates during the `SetupState` grain-lifecycle stage. The +manager exposes `State` as a proxy onto that underlying storage, so it +sees the hydrated value the moment Orleans's lifecycle hook fires — +before `OnActivateAsync` is invoked. + +`ReadAsync` is retained for the rare case where the grain wants to +force a re-read of authoritative state mid-activation (e.g. after a +known external mutation). It is not required by the activation flow. + +### `WriteAsync` semantics + +A single default `StateManager` handles both shapes. It branches on +the non-generic `VersionedState` marker (see §3a) at runtime: if `T` +derives from it, the manager stamps a fresh `Guid Version` before every +write and uses that version for the recovery-path equality check; +otherwise it falls back to `T.Equals(...)`. + +```csharp +internal sealed class StateManager(IPersistentState storage) : IStateManager + where T : class, IEquatable +{ + public T State => storage.State; + + public Task ReadAsync() => storage.ReadStateAsync(); + + public async Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concurrent) + { + var previous = storage.State; + + if (newState is VersionedState versioned) + { + versioned.Version = Guid.CreateVersion7(); + } + + if (policy == WritePolicy.Force) + { + storage.Etag = null; + } + + storage.State = newState; + + try + { + await storage.WriteStateAsync(); + } + catch (Exception ex) + { + try { await storage.ReadStateAsync(); } + catch + { + storage.State = previous; + throw; + } + + if (ex is not InconsistentStateException && IsEquivalent(storage.State, newState)) + { + return; // write actually persisted; lost-response, swallow + } + + throw; + } + } + + private static bool IsEquivalent(T persisted, T attempted) => + persisted is VersionedState pv && attempted is VersionedState av + ? pv.Version.Equals(av.Version) + : persisted.Equals(attempted); +} +``` + +Behaviour matrix: + +| Failure | After `WriteAsync` returns/throws | +| ---------------------------------- | ------------------------------------------------ | +| Success | `State == newState`, returns | +| Timeout, write actually persisted | `State == newState`, returns (silent recovery) | +| Timeout, write did not persist | `State == server's value`, throws original ex | +| 5xx / transient | Same as timeout — re-read decides | +| `InconsistentStateException` | `State == server's value`, **always rethrows** | +| Re-read also fails (double failure)| `storage.State` reverted, throws original ex | + +### Double failure behaviour + +When both `WriteStateAsync` and the recovery `ReadStateAsync` fail, the +manager reverts `storage.State` to `previous` and rethrows. After this +the grain holds correct data but a **stale ETag**. The next write +attempt may hit `InconsistentStateException` if the first write actually +persisted. + +**Library does not auto-recover from double failure.** The grain must +call `ReadAsync()` before its next write to refresh the ETag if it +suspects this state. This is a documented contract — the library +surfaces the failure, the grain decides the policy (retry, deactivate, +alert). + +### Notes + +- **No internal `Deactivate` call.** Grain code decides deactivation + policy. +- **Always rethrow on `InconsistentStateException`**, even if equality + matches. A coincidental match would silently swallow a real concurrent + write; the contract "if we threw conflict, your command read stale + data" must hold. +- **`newState`'s reference is mutated for versioned state.** `Version` + is `set` (not `init`), so `versioned.Version = ...` updates the + caller's reference. Documented contract. +- **No `ReferenceEquals` pre-write short-circuit.** Functional + `with { ... }` produces a fresh reference even when no fields changed. + +### Provider-specific implementations + +`IStateManager` wraps an existing `IPersistentState`, not a +replacement for grain storage providers. No new `IGrainStorage` +implementation is introduced. + +Each storage provider we care to optimise for may ship its own +`IStateManager` that: + +- Inspects provider-specific exception types to classify failures as + *DefinitelyDidNotPersist* (skip re-read, revert + rethrow) vs + *UnknownOutcome* (re-read). +- Optionally exploits provider features (conditional writes, blob + versions) to avoid the re-read. + +### Why a wrapper, not extension methods + +The wrapper was debated — recovery logic is stateless, could be an +extension method on `IPersistentState`. But the wrapper does a fourth +thing extensions cannot: **it hides `IPersistentState.State`**. + +`IStateManager.State` exposes only the **committed** snapshot — the +value after the last successful `WriteAsync`. During an in-flight write, +`IPersistentState.State` already holds the uncommitted value. Read +methods marked `[AlwaysInterleave]` that access `storage.State` directly +could observe uncommitted state — and if the write fails, they returned +data that never persisted. + +The wrapper is a **concurrency safety boundary**, not just convenience. +Extension methods can't provide this fence because the grain still holds +`IPersistentState` and any method can access `.State` directly. + +### Grain wiring + +```csharp +// User injects IPersistentState as normal, wraps in OnActivateAsync. +[PersistentState("state")] IPersistentState storage; +IStateManager stateManager; + +public override Task OnActivateAsync(CancellationToken ct) +{ + stateManager = storage.AsStateManager(); + // ... +} +``` + +`AsStateManager()` is an extension method on `IPersistentState`. +Internally: + +```csharp +public static IStateManager AsStateManager(this IPersistentState storage) + where T : class, IEquatable +{ + // Future: check DI for IStateManagerFactory to resolve + // provider-specific implementations. + return new StateManager(storage); +} +``` + +No DI registration needed for v1. When provider-specific overrides ship +later, the extension checks DI for a registered `IStateManagerFactory`, +falls back to default `StateManager`. No breaking change. + +--- + +## 2. `Outbox` storage placement + +**Status:** Settled. + +The `Outbox` collection lives **inside the grain's main state +record**. Atomicity is the whole point — a state change and the messages +announcing it commit in one ETag-protected write or neither commits. +Splitting into a separate `IPersistentState` blob would re-introduce +the "told one party, not the other" failure mode. + +### Write amplification + +Co-locating the outbox means every command's `WriteStateAsync` +re-serialises both halves. Accepted. + +Provider-level mitigation is allowed: a storage provider may detect +unchanged sub-graphs and skip writing unchanged bytes. That stays a +provider concern. + +### Non-goal: outbox-only writes + +The grain does not expose "enqueue without committing other state." +Every outbox change rides the same `WriteAsync(newState)` as the +business-state change that produced it. + +--- + +## 3. `Outbox` — message collection inside grain state + +**Status:** Settled. + +### Goal + +`Outbox` is the per-grain durable buffer of messages that have been +*announced* (committed alongside a state change) but not yet *delivered* +(handed off to a postman successfully). It lives as a property on the +grain's state record for atomic writes. + +The collection behaves like `ImmutableArray>` +— read-only iteration, indexer, value semantics, mutators return new +instances — with three additions: + +- A baked-in `Sender` (grain id). +- A monotonic `LatestSequenceNumber` that persists independently of the + message array contents. +- An `Add(T payload)` mutator that owns sequence assignment. + +### Shape + +```csharp +[GenerateSerializer] +public sealed record OutboxSequenceToken( + [property: Id(0)] long SequenceNumber, + [property: Id(1)] GrainId Sender, + [property: Id(2)] DateTimeOffset Timestamp, + [property: Id(3)] DateTimeOffset Epoch); + +[GenerateSerializer] +public sealed record OutboxMessageEnvelope( + [property: Id(0)] OutboxSequenceToken Token, + [property: Id(1)] T Message); + +[GenerateSerializer] +public sealed class Outbox : IReadOnlyList>, IEquatable> +{ + [Id(0)] private readonly GrainId _sender; + [Id(1)] private readonly long _latestSequenceNumber; + [Id(2)] private readonly ImmutableArray> _items; + [Id(3)] private readonly DateTimeOffset? _epoch; + + // Non-persisted; no [Id]. Mutable, registered post-construction. + [NonSerialized] + [JsonIgnore] + private TimeProvider _time = TimeProvider.System; + + internal Outbox( + GrainId sender, + long latestSequenceNumber, + ImmutableArray> items, + DateTimeOffset? epoch) + { + _sender = sender; + _latestSequenceNumber = latestSequenceNumber; + _items = items; + _epoch = epoch; + } + + public static Outbox Empty(GrainId sender) => + new(sender, latestSequenceNumber: 0, items: [], epoch: null); + + public void RegisterTimeProvider(TimeProvider time) => _time = time; + + public GrainId Sender => _sender; + public long LatestSequenceNumber => _latestSequenceNumber; + public DateTimeOffset? Epoch => _epoch; + public int Count => _items.Length; + public bool IsEmpty => _items.IsDefaultOrEmpty; + public OutboxMessageEnvelope this[int index] => _items[index]; + + public IEnumerator> GetEnumerator() + => ((IEnumerable>)_items).GetEnumerator(); + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + + public Outbox Add(T message) { /* increments seq, stamps epoch on first call */ } + public Outbox Remove(OutboxSequenceToken token) { /* removes by seq number */ } + public Outbox RemoveRange(IEnumerable tokens) { /* batch remove */ } + public Outbox Clear() { /* removes all items, preserves LatestSequenceNumber + Epoch */ } + + // O(1) fingerprint equality — see below. + public bool Equals(Outbox? other) { /* fingerprint check */ } + public override bool Equals(object? obj) => obj is Outbox o && Equals(o); + public override int GetHashCode() + => HashCode.Combine(_sender, _latestSequenceNumber, _items.Length, _epoch); +} +``` + +### Epoch semantics + +- `Empty(sender)` → `epoch = null`, `LatestSequenceNumber = 0`. +- First `Add()` → stamps `epoch = now`. Persisted with state. +- Subsequent `Add()` → same epoch, incrementing sequence number. +- `Clear()` → removes items, **preserves** `LatestSequenceNumber` and + `Epoch`. This is the normal "postman drained successfully" path. +- `Empty(sender)` again → **resets both** epoch (to null) and sequence + number (to 0). This is the nuclear option — the next `Add` starts a + fresh epoch. Receivers see `token.Epoch > stored.Epoch` and accept. + +**`Clear()` is the normal path.** Grains should almost never call +`Empty(sender)` on an active outbox. `Empty` is for construction-time +initialisation and deliberate ops-level sequence-space resets. +Document and warn. + +### Outbox depth telemetry and configurable maximum + +The outbox can grow unbounded if postman targets are down. Mitigation: + +- **Telemetry:** gauge for outbox depth per grain type, emitted on every + write. Operators see growth before it becomes a crisis. +- **Configurable maximum:** users can set a max outbox size when + configuring the outbox processor. When exceeded, oldest messages are + automatically dropped (FIFO). This is opt-in; default is no cap. +- **Documentation:** storage providers have entity size limits (e.g. + Azure Table = 1MB). Document the risk of unbounded growth. + +### O(1) fingerprint equality + +Two `Outbox` values are equal when +`(Sender, LatestSequenceNumber, Epoch, Count, _items[0].Seq, _items[^1].Seq)` +matches. Constant-time regardless of `_items.Length`. + +Invariant holds by construction: `Outbox` is a sealed class with +private readonly backing fields. The only mutation paths are the four +public methods (`Add` / `Remove` / `RemoveRange` / `Clear`), each of +which changes at least one fingerprint field. + +### Why these choices + +- **Sealed class, not record.** No `with`, no synthesized copy + constructor. Mutation through four methods only — fingerprint + invariant holds by construction. Reference type because + `RegisterTimeProvider` is a void mutator. +- **`Add(T payload)` not `Add(envelope)`.** Outbox owns sequence + assignment. Callers cannot fabricate sequence numbers. +- **`RegisterTimeProvider` not per-call argument.** Matches + `MessageTracker`. Successor instances carry the provider forward. +- **Lazy `Epoch`.** A grain that never sends doesn't burn a fresh epoch + on storage. +- **`Remove(token)` not `Remove(envelope)`.** Token is the identity. +- **`LatestSequenceNumber` is a separate field.** After flush, items are + empty; high-water mark persists independently. + +### Retry diagnostics + +Envelope stays immutable. `SendAttempts` and `LastException` do NOT +appear on `OutboxMessageEnvelope`. The postman tracks attempts +in-memory keyed on `OutboxSequenceToken`. On re-activation, counts +restart from zero. Acceptable for burst retry policies. + +--- + +## 3a. `VersionedState` — structural equality across `ImmutableArray` + +**Status:** Settled. + +### Problem + +`ImmutableArray.Equals` compares the underlying `T[]` reference, not +contents. Record-auto-generated `Equals` on state records holding +`ImmutableArray<>` returns false for identical-content different-backing +arrays. This breaks `IStateManager.WriteAsync`'s recovery path. + +### Resolution + +```csharp +[GenerateSerializer] +public abstract record VersionedState +{ + [Id(0)] public Guid Version { get; internal set; } = Guid.CreateVersion7(); +} + +[GenerateSerializer] +public abstract record VersionedState : VersionedState + where TSelf : VersionedState +{ + public virtual bool Equals(VersionedState? other) => + other is not null && Version == other.Version; + + public override int GetHashCode() => Version.GetHashCode(); +} +``` + +User state: + +```csharp +[GenerateSerializer] +public sealed record MyState : VersionedState +{ + [Id(1)] public Outbox Outbox { get; init; } = Outbox.Empty(sender); + [Id(2)] public ImmutableArray Items { get; init; } = []; +} +``` + +### Why this shape + +- **`Guid Version` with `internal set`.** Library code can write + `Version`; user code cannot. Hard compile-time fence. +- **`set`, not `init`.** Library mutates caller's reference during + `WriteAsync` to stamp new version. +- **`Guid.CreateVersion7()`.** Sortable UUID v7 (.NET 9+). Cheap, + collision-free at this rate. +- **Initialiser on property.** Fresh records get non-empty version + before any write. +- **Two-layer base.** Non-generic `VersionedState` gives `StateManager` + a single type to test against. Generic layer carries typed equality. + +### Why `Version` is internal-set, not user-settable + +`IPersistentState.Etag` is storage concurrency. `Version` is +library-internal recovery decoration. Conflating them erases a layer. +Force-write escape hatch lives on `IStateManager` via +`WritePolicy.Force`. + +--- + +## 4. `MessageTracker` + +**Status:** Settled. + +Receiver-side, persisted dedup state. Tracks high-water position from +each upstream source. Two source kinds: + +- **Orleans streams** — keyed by `StreamId`; position is a + `StreamCursor` wrapping `(StreamId, StreamSequenceToken)`. +- **Outbox messages** — keyed by sender `GrainId`; position is an + `OutboxSequenceToken`. + +### Shape + +**Sealed class** (not record — consistent with `Outbox`, avoids +`_time` field participating in record-synthesized equality). + +```csharp +[GenerateSerializer] +public sealed class MessageTracker +{ + [Id(0)] private ImmutableDictionary _stream; + [Id(1)] private ImmutableDictionary _outbox; + + // Non-persisted; no [Id]. Mutable. + [NonSerialized] + [JsonIgnore] + private TimeProvider _time = TimeProvider.System; + + public void RegisterTimeProvider(TimeProvider time) => _time = time; + + public bool ProcessMessage(StreamCursor cursor, out MessageTracker next); + public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next); + + public StreamCursor? LatestStream(StreamId stream); + public OutboxSequenceToken? LatestOutbox(GrainId sender); + + public MessageTracker Evict(DateTimeOffset olderThan); + public MessageTracker EvictStreams(DateTimeOffset olderThan); + public MessageTracker EvictOutboxes(DateTimeOffset olderThan); + public MessageTracker Evict(StreamId stream, DateTimeOffset olderThan); + public MessageTracker Evict(GrainId sender, DateTimeOffset olderThan); + + [GenerateSerializer] + private readonly record struct StreamEntry( + [property: Id(0)] StreamCursor LastPosition, + [property: Id(1)] DateTimeOffset Received); + + [GenerateSerializer] + private readonly record struct OutboxEntry( + [property: Id(0)] DateTimeOffset Epoch, + [property: Id(1)] long LastSequenceNumber, + [property: Id(2)] DateTimeOffset Received); +} +``` + +### Identity model (no OriginId) + +- Outbox identity: `OutboxSequenceToken.Sender` in the envelope. + Payloads stay clean. +- Stream identity: `StreamId` from runtime. Sufficient under the + one-provider-per-namespace convention (code-review enforced, not + runtime enforced). This is Orleans's own constraint — the library + doesn't make it worse. + +### `ProcessMessage(StreamCursor)` semantics + +| Prior entry | Decision | Effect | +| -------------------------------- | -------- | ---------------------------------------------- | +| None | Accept | Insert `(LastPosition = cursor, Received = now)` | +| `cursor > stored.LastPosition` | Accept | Update position + received | +| `cursor <= stored.LastPosition` | Reject | No change | + +### `ProcessMessage(OutboxSequenceToken)` semantics + +| Prior entry | Comparison | Decision | Effect | +| ----------- | -------------------------------------------- | -------- | ------------------------ | +| None | — | Accept | Insert | +| Exists | `token.Epoch > stored.Epoch` | Accept | Replace (sender reset) | +| Exists | Same epoch, `token.Seq > stored.LastSeq` | Accept | Update seq + received | +| Exists | Same epoch, `token.Seq <= stored.LastSeq` | Reject | Duplicate | +| Exists | `token.Epoch < stored.Epoch` | Reject | Stale epoch | + +### `Evict` — uniform cleanup + +Five overloads, one rule: remove entries where `entry.Received < olderThan`. +No separate `Forget` API — `Evict(id, DateTimeOffset.MaxValue)` is the +documented idiom for unconditional clear. + +### `RegisterTimeProvider` — void by design + +Mutable field, non-persisted, excluded from equality. After +deserialization, grain must re-register. Falls back to +`TimeProvider.System` if skipped — correct for production, breaks +fake-clock tests. + +--- + +## 5. `StreamManager` — fluent subscribe/resume/error facade + +**Status:** Settled. + +Grain-level facade around Orleans implicit subscriptions. Four +responsibilities: + +1. **Subscribe** to stream namespaces during `OnActivateAsync`. +2. **Resume** from last accepted `StreamCursor` per namespace. +3. **Dispatch** with projected `StreamCursor`. +4. **Per-subscription error handling** via `.OnError(...)`. + +### Why a facade, not a base class + +Extension/composition over inheritance. Grain inherits from `Grain`, +uses `StreamManager` as a field. + +### Shape + +```csharp +public sealed class StreamManager +{ + public static StreamManagerBuilder ForGrain( + Grain owner, + MessageTracker trackerSnapshot, + IServiceProvider services); +} + +public sealed class StreamManagerBuilder +{ + public StreamSubscriptionBuilder Subscribe( + string streamNamespace, + bool resumeUsingLatestSequenceToken = true); + + public StreamManager Build(); +} + +public sealed class StreamSubscriptionBuilder +{ + public StreamSubscriptionBuilder OnNext( + Func handler); + + public StreamSubscriptionBuilder OnError( + Func handler); + + public StreamManagerBuilder Done(); +} +``` + +### Typical wiring + +```csharp +public override async Task OnActivateAsync(CancellationToken ct) +{ + var state = await stateManager.ReadAsync(); + state.Tracker.RegisterTimeProvider(timeProvider); + + streamManager = StreamManager.ForGrain(this, state.Tracker, services) + .Subscribe("electricity-prices") + .OnNext(HandlePriceTickAsync) + .OnError(LogStreamErrorAsync) + .Done() + .Subscribe("tariff-events") + .OnNext(HandleTariffChangedAsync) + .Done() + .Build(); +} +``` + +### Resume semantics + +- `Build()` reads `trackerSnapshot.LatestStream(streamId)` once per + subscription. +- If cursor exists → hydrate `StreamSequenceToken`, pass to + `SubscribeAsync`. +- If null → subscribe without token (provider default, typically: + start from current). +- Tracker read once at `Build()` time, never again. Handler is the + only path that mutates the tracker. + +### Per-subscription `OnError` + +Signature: `Func`. Default when +omitted: log + emit counter, do NOT rethrow. + +### One-provider-per-namespace + +Accepted as a code-review convention. Orleans's own constraint — the +library doesn't make it worse. + +### `StreamCursor` projection constraint + +`StreamCursor` carries an opaque `StreamSequenceToken`. Library ships an +STJ converter for a closed set of known subtypes discriminated on +`$kind`: + +| Subtype | Source | +| ------------------------------- | ------------------------------- | +| `EventSequenceToken` | Orleans SimpleMessageStream | +| `EventHubSequenceToken` | Orleans EH provider v1 | +| `EventHubSequenceTokenV2` | Orleans EH provider v2 | +| `EnrichedEventHubSequenceToken` | Library-shipped v2 subclass | + +Unknown subtype throws at serialization time — silently dropping the +cursor would corrupt dedup. + +### Custom EH adapter for enriched tokens + +Optional pattern. To opt into `EnrichedEventHubSequenceToken` (carries +`EnqueuedTime`, see §5 projection constraint table), the silo wires a +custom `IEventHubDataAdapter` via `UseDataAdapter`: + +```csharp +internal sealed class EnrichedEventHubAdapter(Serializer serializer) + : EventHubDataAdapter(serializer) +{ + public override StreamSequenceToken GetSequenceToken( + EventHubMessage eventHubMessage, int eventIndex) + { + var seqNum = eventHubMessage.SequenceNumber; + return new EnrichedEventHubSequenceToken( + eventHubMessage.Offset, + seqNum, + eventIndex, + eventHubMessage.EnqueuedTimeUtc); + } +} +``` + +`StreamManager` is unaware of the adapter — the enrichment surfaces +through `StreamCursor.TryGetEnqueuedTime(...)` in the user's `OnNext`. + +### OpenTelemetry trace correlation + +Orleans streams lose `Activity.Current` across the queue boundary. To +correlate consumer-side spans with producer-side spans without creating +multi-hour distributed traces, `StreamManager` should use +`ActivityLink`s, not parent chaining: + +- Producer side (in a custom `ToQueueMessage` override on the adapter): + stash `Activity.Current?.Id` into `EventData.Properties["traceparent"]` + before the event hits EH. +- Consumer side (in `StreamManager`'s OnNext wrapper): read the + property, parse into `ActivityContext`, start the OnNext span with + `ActivityKind.Consumer` and `links: [new ActivityLink(parsedContext)]`. + +This produces separate traces per delivery, each with a link back to +the producer span. OTel backends render the cross-trace arrow without +collapsing weeks of traffic into one trace. + +The adapter override is the user's responsibility (it lives in their +custom `IEventHubDataAdapter`); the consumer-side parse+link is a +library concern owned by `StreamManager`. + +### Telemetry + +Track at minimum (see `OutboxProcessor` for the meter pattern): + +- Counter: messages delivered per `(streamNamespace, accepted|rejected)`. +- Counter: subscriptions established / torn down / errored. +- Histogram: handler latency per `streamNamespace`. +- Histogram (when `EnrichedEventHubSequenceToken` is available): + end-to-end lag = `now - cursor.TryGetEnqueuedTime()`. Surfaces + consumer lag against the broker, parallel to outbox sender-to-receiver + timing. + +--- + +## 6. Opinionated grain pattern — functional commands, interleaved reads + +**Status:** Settled (guidance, not enforced by library API). + +### The pattern + +Grains using this library should follow a functional-command model: + +1. **State is immutable.** Grain state is a `record` (or sealed record) + composed of immutable types (`ImmutableArray`, `Outbox`, + `MessageTracker`, value objects). No mutable collections, no mutable + fields. + +2. **Commands run sequentially.** Methods that mutate state ("commands") + produce a new state value from `(currentState, commandPayload)` and + call `IStateManager.WriteAsync(newState)`. They run under + Orleans' default non-reentrant turn-based concurrency — one at a + time, no interleaving. + +3. **Reads interleave freely.** Methods that only read committed state + can be marked `[AlwaysInterleave]`. They see the last + `WriteAsync`-committed snapshot via `IStateManager.State` — the + committed-state fence (§1) ensures they never observe in-flight + uncommitted values. Multiple reads execute in parallel. + +4. **No external I/O in command handlers.** Commands should not call + HTTP, query databases, or invoke other grains. All input needed for + the decision must arrive in the command payload. External data is + fetched by the caller *before* invoking the grain. If a command needs + to trigger downstream work, enqueue it via `Outbox.Add(...)` and + let the `OutboxProcessor` dispatch it after the write. + +### Why this works + +- **Deterministic commands.** Same state + same payload → same result. + Easy to test, easy to reason about, no ambient dependencies. +- **Safe concurrency.** Reads never block writes. Writes are serialised + by the runtime. No custom locks. +- **Recovery-friendly.** `IStateManager.WriteAsync` recovery path + relies on `VersionedState.Equals` — works because state is a + value type (record with `Version` equality). +- **Outbox replaces side effects.** Instead of "write state + call + service" (two failure points), it's "write state with outbox item" + (one atomic write) + "processor retries delivery" (idempotent). + +### What the library does NOT enforce + +- No compile-time prevention of injecting `HttpClient` or calling + external services in command methods. This is a documentation and + code-review concern. +- No base class. The pattern emerges from the types: `VersionedState` + is a record (immutable), `IStateManager.WriteAsync` takes a new + value (not mutation), `Outbox.Add` returns a new instance. +- Future: Roslyn analyzers could warn on external I/O inside methods + that call `WriteAsync`. Not in scope for v1. + +--- + +## 7. `OutboxProcessor` — timer + reminder driven dispatch + +**Status:** Settled. + +Grain-scoped component that owns the timer, reminder, and postman +dispatch lifecycle for draining `Outbox`. Modelled after the +[spike](https://gist.github.com/egil/2f3318d1bd22045268e11a5d988ba938) +in the `Clever.PricingEngine` codebase. + +### Architecture + +- **Grain-scoped, not silo-scoped.** Each grain with an outbox gets its + own `OutboxProcessor`. No external scan, no registry, no second store. +- **GrainTimer** for in-process fast retry while activated. +- **Durable Reminder** for cross-activation recovery. Reactivates the + grain if it deactivates with pending items. Timer arms on activation. +- **Non-reentrant assumption.** Orleans default turn-based concurrency + serialises calls; the processor adds no internal locks. Not safe on + `[Reentrant]` grains — document and enforce. + +### Postman dispatch + +**Callback-based.** The grain registers one or more postmen via +`AddPostman(...)`, each handling a subtype of `TOutbox`. Matching +is first-registered-wins against the item's runtime type — order from +most specific to least specific (like a `switch`). + +- Per-item exceptions are caught and surfaced through `OnPostErrorAsync` + with attempt count (in-memory, resets on reactivation) — the grain + decides: leave item in state to retry, or remove to dead-letter after + N attempts. +- Each item dispatches to exactly **one** postman (first-registered-wins). + Items whose runtime type matches no postman → reported as failed with + `NoPostmanRegisteredException`. +- `PostAsync` only throws `TimeoutException` (per-run timeout), + `OperationCanceledException` (caller token), or callback exceptions. + +### Grain integration pattern + +```csharp +// 1. Marker interface — DIM handles ReceiveReminder. +public interface IOutboxGrain : IRemindable +{ + Task IRemindable.ReceiveReminder(string reminderName, TickStatus status) + { + var grainBase = (IGrainBase)this; + var component = grainBase.GrainContext.GetComponent(); + return component is null + ? Task.CompletedTask + : component.ReceiveReminderAsync(reminderName, status).AsTask(); + } +} + +// 2. C# 14 extension for InitializeOutboxProcessor. +extension(TGrain grain) where TGrain : IOutboxGrain, IGrainBase +{ + public OutboxProcessor InitializeOutboxProcessor( + OutboxProcessorOptions options) where TOutbox : notnull + { + var services = grain.GrainContext.ActivationServices; + var processor = new OutboxProcessor( + grain, options, + services.GetRequiredService() + .CreateLogger($"OutboxProcessor<{typeof(TOutbox).Name}>"), + services.GetService() ?? TimeProvider.System, + services.GetRequiredService()); + processor.AttachToGrain(); + return processor; + } +} +``` + +### `OutboxProcessorOptions` + +```csharp +public sealed class OutboxProcessorOptions where TOutbox : notnull +{ + /// Snapshot of pending items. Called once per post run. + public required Func> GetPending { get; init; } + + /// Items successfully posted. Grain must remove from backing collection. + public required Func, CancellationToken, ValueTask> + OnPostCompletedAsync { get; init; } + + /// Failed items with exception and attempt count (in-memory, resets on + /// reactivation). Grain decides: leave to retry, or remove to + /// dead-letter after N attempts. If null, failed items retry silently. + public Func, + CancellationToken, ValueTask>? OnPostErrorAsync { get; init; } + + /// Max time per post run. Set below grain's response timeout. + public TimeSpan ProcessingTimeout { get; init; } = TimeSpan.FromSeconds(20); + + /// Timer + reminder period. Orleans reminders fire at most once/minute. + public TimeSpan RetryDelay { get; init; } = TimeSpan.FromMinutes(2); +} +``` + +### `OutboxProcessor` + +```csharp +public sealed partial class OutboxProcessor : IOutboxComponent + where TOutbox : notnull +{ + public OutboxProcessor AddPostman( + Func postman) where TSub : TOutbox; + public OutboxProcessor AddPostman( + Func postman) where TSub : TOutbox; + public OutboxProcessor AddPostman( + Func postman) where TSub : TOutbox; + + /// Posts pending items. Safe to call from grain's task scheduler. + /// Arms timer/reminder if items remain, unregisters if empty. + public ValueTask PostAsync(CancellationToken cancellationToken = default); + + /// Called by IOutboxGrain DIM. No-ops for unknown reminder names. + public ValueTask ReceiveReminderAsync(string reminderName, TickStatus status); +} + +internal interface IOutboxComponent +{ + ValueTask ReceiveReminderAsync(string reminderName, TickStatus status); +} +``` + +### Grain author experience + +Two obligations (both compiler-enforced): + +1. Implement `IOutboxGrain`. +2. Call `InitializeOutboxProcessor(...)` in `OnActivateAsync`. + +No `ReceiveReminder` override needed (DIM handles it). No manual +timer/reminder lifecycle. No telemetry wiring. + +Escape hatch for grains with their own reminders: + +```csharp +public async Task ReceiveReminder(string name, TickStatus status) +{ + if (name == MyOwnReminder) { await DoMyReminderWork(); return; } + await outbox.ReceiveReminderAsync(name, status); +} +``` + +### Why these choices + +- **Grain-scoped, not silo-scoped.** Grain already knows its own outbox + state. No external scan needed. Reminder ensures cross-activation + recovery. Timer handles fast retry. Pattern Orleans itself uses. +- **Callback-based, not DI service.** Grain controls dispatch logic, + can pass its own state to the postman. DI service adds indirection + without clear benefit. +- **First-registered-wins postman matching.** Simple dispatch model. + Order most-specific first. Unmatched items → `NoPostmanRegisteredException` + via `OnPostErrorAsync`. +- **`PostAsync` swallows per-item errors.** Grain observes failures via + `OnPostErrorAsync` with attempt count. Processor never drops items + silently unless the grain explicitly removes them. +- **One postman per item.** First-registered-wins, not broadcast. + Simpler error semantics, no partial-success ambiguity. +- **DIM on `IOutboxGrain`.** Zero ceremony. Grain author never writes + `ReceiveReminder` unless they have their own reminders. +- **C# 14 extension members.** Generic constraint on `TGrain : + IOutboxGrain, IGrainBase` means the extension won't compile on types + that don't implement the marker. Type-safe opt-in. + +--- + +## 8. Naming, serialization & telemetry conventions + +**Status:** Settled. + +### Namespace + +Flat: all public types in `Egil.Orleans.Messaging`. One `using` +statement. ~15-20 public types — not crowded enough to split. + +### Orleans `[Alias]` on serializable types + +All public serializable types get `[Alias]` for version-tolerant +serialization. Aliases are **globally scoped** — must be unique across +the entire application. + +**Pattern:** `egil.orleans.messaging.TypeName`. Generic types include +backtick + arity. + +Examples: + +``` +[Alias("egil.orleans.messaging.Outbox`1")] +[Alias("egil.orleans.messaging.OutboxMessageEnvelope`1")] +[Alias("egil.orleans.messaging.MessageTracker")] +[Alias("egil.orleans.messaging.OutboxSequenceToken")] +[Alias("egil.orleans.messaging.StreamCursor")] +[Alias("egil.orleans.messaging.VersionedState")] +``` + +### `[Id]` numbering + +Sequential per type, scoped per inheritance level (matches Orleans +convention). New fields get the next number. Never reuse removed IDs. + +```csharp +[GenerateSerializer] +public abstract record VersionedState +{ + [Id(0)] public Guid Version { get; internal set; } +} + +[GenerateSerializer] +public abstract record VersionedState : VersionedState +{ + // Inherits [Id(0)] from parent. Child IDs start fresh at [Id(0)]. +} +``` + +### System.Text.Json serialization + +All serializable types carry `[JsonConverter]` attributes referencing +library-shipped converters. STJ discovers them automatically — users +need no registration, no `JsonSerializerOptions` configuration. + +This ensures correct round-tripping through storage providers that use +STJ (e.g., Orleans's Cosmos, blob, or custom providers configured with +`System.Text.Json`). + +**Newtonsoft.Json is not supported.** Users whose storage providers use +Newtonsoft must either configure their provider for STJ or write their +own converters. Documented as a known limitation. + +| Type | Converter approach | +|------|-------------------| +| `Outbox` | `[JsonConverter(typeof(OutboxJsonConverterFactory))]` — factory creates closed `JsonConverter>` | +| `OutboxMessageEnvelope` | `[JsonConverter(typeof(OutboxMessageEnvelopeJsonConverterFactory))]` | +| `MessageTracker` | `[JsonConverter(typeof(MessageTrackerJsonConverter))]` | +| `OutboxSequenceToken` | `[JsonConverter(typeof(OutboxSequenceTokenJsonConverter))]` | +| `StreamCursor` | `[JsonConverter(typeof(StreamCursorJsonConverter))]` | +| `VersionedState` | No custom converter — `[JsonInclude]` on `Version` property makes `internal set` visible to STJ | + +**Why custom converters (not `[JsonInclude]` on private fields):** + +`Outbox` and `MessageTracker` are sealed classes with private +backing fields. Exposing them via `[JsonInclude]` would leak internals +and weaken the fingerprint invariant. Custom converters keep +encapsulation intact and control the exact wire format. + +**`VersionedState` exception:** `Version` is a single `Guid` property +with `internal set`. `[JsonInclude]` is sufficient — no encapsulation +risk, and a full custom converter for an abstract base class is +unnecessary complexity. + +**Generic converters:** STJ requires `JsonConverterFactory` for open +generic types. The factory's `CreateConverter` method creates the closed +`JsonConverter>` for the specific `T`. + +### Non-serialized fields + +Service-reference fields like `TimeProvider _time` get both +`[NonSerialized]` and `[JsonIgnore]` — belt-and-suspenders: + +- **No `[Id]`** → Orleans `[GenerateSerializer]` skips them. +- **`[NonSerialized]`** → .NET runtime serializers skip them. +- **`[JsonIgnore]`** → STJ skips them even if a storage provider + bypasses our custom converter and falls back to reflection. +- **Custom converters** also skip them explicitly. + +All four layers prevent accidental serialization of non-restorable +service references. `RegisterTimeProvider()` re-injects after +deserialization. + +### Telemetry + +**Meter name:** `egil.orleans.messaging` (matches package name). + +**Outbox-specific metrics only** — do not duplicate Orleans-provided +metrics for state read/write, activation lifecycle, messaging layer. + +| Instrument | Type | Description | +|---------------------------|-----------|--------------------------------------| +| `outbox.post.duration` | Histogram | Post run duration (ms) | +| `outbox.post.item.duration` | Histogram | Per-item postman dispatch duration | +| `outbox.post.items` | Counter | Items successfully dispatched | +| `outbox.post.errors` | Counter | Items that failed dispatch | +| `outbox.depth` | Gauge | Pending items per grain type | + +**Tags** (matching spike pattern): +- `grain.type` — owning grain type name +- `event.type` — outbox item type name +- `success` — `true`/`false` on per-item histograms + +**ActivitySource:** `egil.orleans.messaging` for distributed traces. + +### Public interface surface + +- `IOutboxGrain` — marker + DIM for `ReceiveReminder`. Kept: DIM saves + real boilerplate for the 80% case (grains with no other reminders). + Generic constraint on `InitializeOutboxProcessor` ensures type-safe + opt-in. +- `IOutboxComponent` — internal. Not part of public API. + +--- + +## 9. Test strategy + +**Status:** Settled. + +### Test runner + +All tests run through `Egil.Orleans.Testing`'s `InProcessTestCluster` — +real Orleans runtime, fast boot, no mocking of Orleans internals. Even +pure-logic types (`Outbox`, `MessageTracker`) are tested through +grain interactions to validate real serialization, persistence, and +concurrency behavior. + +### Coverage targets + +Matching `Egil.Orleans.Testing` convention: + +- **100% branch coverage** on core types: `Outbox`, + `MessageTracker`, `StateManager`, `VersionedState`, + `OutboxProcessor`. +- **95% branch coverage** on supporting types: `OutboxSequenceToken`, + `StreamCursor`, `StreamManager`, `OutboxMessageEnvelope`, + `WritePolicy`. + +### Test grain shape + +Purpose-built test grains in the test assembly, each targeting a +specific behavior: + +- **Write recovery grain** — exercises `StateManager.WriteAsync` + failure + recovery path, double failure, `WritePolicy.Force`. +- **Outbox drain grain** — exercises `Outbox` Add/Remove/Clear, + epoch reset, `OutboxProcessor` timer/reminder lifecycle, postman + dispatch + error callback with attempt count. +- **Dedup grain** — exercises `MessageTracker.ProcessMessage` for both + stream cursors and outbox tokens, epoch-aware acceptance, eviction. +- **Interleaved-read grain** — exercises `[AlwaysInterleave]` reads + seeing only committed state while a write is in-flight. +- **Stuck postman grain** — exercises `ProcessingTimeout` behavior, + `OnPostErrorAsync` with timeout exception. +- **Multi-reminder grain** — exercises `IOutboxGrain` DIM with grain + that also has its own reminders (DIM shadowing). + +### Serialization round-trip tests + +Dedicated tests for every type with `[GenerateSerializer]`: +- **Orleans serialization:** Serialize → bytes → deserialize → assert equal. +- **System.Text.Json:** Serialize → JSON string → deserialize → assert + equal. Validates `[JsonConverter]` attributes and converter correctness. +- Catches: missing `[Id]`, wrong `[Alias]`, `ImmutableArray` edge + cases, version-tolerance regressions, STJ converter bugs. + +Types covered: `Outbox`, `OutboxMessageEnvelope`, +`MessageTracker`, `OutboxSequenceToken`, `StreamCursor`, +`VersionedState` subtypes. + +### No mocks + +No mocking `IPersistentState` or Orleans internals. The cluster +provides real storage (in-memory), real timers, real reminders. Test +grains exercise the library through the same code path production grains +use. From beb6ac35db0352be4ce0f33738677e19a5e6beb2 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sat, 23 May 2026 19:15:59 +0000 Subject: [PATCH 02/82] build(oes): scaffold Egil.Orleans.Messaging project structure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rename directory from Egil.Orleans.CQRS to Egil.Orleans.Messaging. Add solution file, Directory.Packages.props, version.json, global.json, src csproj (library), and test csproj. Package description: toolbox of composable building blocks — not a framework. Pick what you need, leave the rest. [skip notes] Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Directory.Packages.props | 30 ++++++++++ .../Egil.Orleans.Messaging.slnx | 8 +++ .../api-design.md | 0 Egil.Orleans.Messaging/global.json | 9 +++ .../Egil.Orleans.Messaging.csproj | 55 +++++++++++++++++++ .../Egil.Orleans.Messaging.Tests.csproj | 34 ++++++++++++ Egil.Orleans.Messaging/version.json | 30 ++++++++++ 7 files changed, 166 insertions(+) create mode 100644 Egil.Orleans.Messaging/Directory.Packages.props create mode 100644 Egil.Orleans.Messaging/Egil.Orleans.Messaging.slnx rename {Egil.Orleans.CQRS => Egil.Orleans.Messaging}/api-design.md (100%) create mode 100644 Egil.Orleans.Messaging/global.json create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj create mode 100644 Egil.Orleans.Messaging/version.json diff --git a/Egil.Orleans.Messaging/Directory.Packages.props b/Egil.Orleans.Messaging/Directory.Packages.props new file mode 100644 index 00000000..ce414d35 --- /dev/null +++ b/Egil.Orleans.Messaging/Directory.Packages.props @@ -0,0 +1,30 @@ + + + true + false + $(NoWarn);NU1507 + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Egil.Orleans.Messaging/Egil.Orleans.Messaging.slnx b/Egil.Orleans.Messaging/Egil.Orleans.Messaging.slnx new file mode 100644 index 00000000..779ae7ba --- /dev/null +++ b/Egil.Orleans.Messaging/Egil.Orleans.Messaging.slnx @@ -0,0 +1,8 @@ + + + + + + + + diff --git a/Egil.Orleans.CQRS/api-design.md b/Egil.Orleans.Messaging/api-design.md similarity index 100% rename from Egil.Orleans.CQRS/api-design.md rename to Egil.Orleans.Messaging/api-design.md diff --git a/Egil.Orleans.Messaging/global.json b/Egil.Orleans.Messaging/global.json new file mode 100644 index 00000000..60fdc5d9 --- /dev/null +++ b/Egil.Orleans.Messaging/global.json @@ -0,0 +1,9 @@ +{ + "sdk": { + "rollForward": "latestMajor", + "allowPrerelease": false + }, + "test": { + "runner": "Microsoft.Testing.Platform" + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj new file mode 100644 index 00000000..21a83ea3 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj @@ -0,0 +1,55 @@ + + + + net10.0 + enable + enable + true + embedded + + + + Egil.Orleans.Messaging + Orleans Messaging Toolbox + Egil Hansen + Egil Hansen + + A toolbox of composable building blocks for Orleans grains that need + atomic state writes, transactional outbox, receiver-side dedup, and + stream subscription management. Pick what you need, leave the rest. + + README.md + orleans,messaging,outbox,dedup,cqrs,grain,egil + Egil Hansen + https://github.com/egil/framework + https://github.com/egil/framework + git + LICENSE + true + true + true + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + <_Parameter1>Egil.Orleans.Messaging.Tests + + + + + + True + \ + + + + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj new file mode 100644 index 00000000..a26f041e --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj @@ -0,0 +1,34 @@ + + + + enable + enable + true + Exe + Egil.Orleans.Messaging.Tests + net10.0 + true + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/Egil.Orleans.Messaging/version.json b/Egil.Orleans.Messaging/version.json new file mode 100644 index 00000000..5dc43310 --- /dev/null +++ b/Egil.Orleans.Messaging/version.json @@ -0,0 +1,30 @@ +{ + "$schema": "https://raw.githubusercontent.com/dotnet/Nerdbank.GitVersioning/main/src/NerdBank.GitVersioning/version.schema.json", + "version": "0.1-alpha", + "publicReleaseRefSpec": [ + "^refs/heads/main$", + "^refs/heads/release/egil-orleans-messaging/v\\d+(?:\\.\\d+)?$" + ], + "cloudBuild": { + "setAllVariables": true, + "buildNumber": { + "enabled": true + } + }, + "release": { + "tagName": "egil-orleans-messaging/v{version}", + "branchName": "release/egil-orleans-messaging/v{version}" + }, + "pathFilters": [ + "./", + "./Directory.Packages.props", + "../.github/workflows/egil-orleans-messaging-ci.yml", + "../.editorconfig", + "../.gitattributes", + "../.gitignore", + "../Directory.Build.props", + "../LICENSE", + "../version.json", + "../xunit.runner.json" + ] +} From bb85bd66f16fa54bdd2f725800f90b19e7a0b3fa Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sat, 23 May 2026 19:35:21 +0000 Subject: [PATCH 03/82] feat(om): stub types with XML docs --- .../Directory.Packages.props | 3 + Egil.Orleans.Messaging/api-design.md | 302 ++++++++-------- .../Egil.Orleans.Messaging.csproj | 2 + .../EnrichedEventHubAdapter.cs | 213 +++++++++++ .../EnrichedEventHubAdapterExtensions.cs | 63 ++++ .../EnrichedEventHubSequenceToken.cs | 127 +++++++ .../Egil.Orleans.Messaging/IOutboxGrain.cs | 51 +++ .../Egil.Orleans.Messaging/IStateManager.cs | 138 +++++++ .../Egil.Orleans.Messaging/MessageTracker.cs | 230 ++++++++++++ .../MessageTrackerJsonConverter.cs | 38 ++ .../NoPostmanRegisteredException.cs | 45 +++ .../src/Egil.Orleans.Messaging/Outbox.cs | 339 ++++++++++++++++++ .../OutboxJsonConverterFactory.cs | 35 ++ .../OutboxMessageEnvelope.cs | 45 +++ ...tboxMessageEnvelopeJsonConverterFactory.cs | 30 ++ .../Egil.Orleans.Messaging/OutboxProcessor.cs | 170 +++++++++ .../OutboxProcessorExtensions.cs | 66 ++++ .../OutboxProcessorOptions.cs | 98 +++++ .../OutboxSequenceToken.cs | 59 +++ .../OutboxSequenceTokenJsonConverter.cs | 32 ++ .../Egil.Orleans.Messaging/StateManager.cs | 124 +++++++ .../StateManagerExtensions.cs | 58 +++ .../Egil.Orleans.Messaging/StreamCursor.cs | 125 +++++++ .../StreamCursorJsonConverter.cs | 40 +++ .../Egil.Orleans.Messaging/StreamManager.cs | 161 +++++++++ .../StreamManagerExtensions.cs | 49 +++ .../Egil.Orleans.Messaging/VersionedState.cs | 64 ++++ .../src/Egil.Orleans.Messaging/WritePolicy.cs | 24 ++ .../Egil.Orleans.Messaging.Tests.csproj | 1 + .../OutboxTests.cs | 210 +++++++++++ .../xunit.runner.json | 5 + 31 files changed, 2801 insertions(+), 146 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/VersionedState.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/WritePolicy.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/xunit.runner.json diff --git a/Egil.Orleans.Messaging/Directory.Packages.props b/Egil.Orleans.Messaging/Directory.Packages.props index ce414d35..ea7f22ba 100644 --- a/Egil.Orleans.Messaging/Directory.Packages.props +++ b/Egil.Orleans.Messaging/Directory.Packages.props @@ -16,15 +16,18 @@ + + + diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 77818374..d380c42d 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -106,12 +106,13 @@ Users pick one of two paths for `T`: for record-of-primitives shapes. Breaks silently if state holds `ImmutableArray<>` (reference-equality trap) — user must override `Equals` themselves in that case. -- **Path B — inherit `VersionedState`** - (`MyState : VersionedState`). Framework owns equality via a - per-write `Guid Version`. Immune to any collection-equality issues in - the state graph. Recommended default for any non-trivial state. +- **Path B — inherit `VersionedState`** + (`MyState : VersionedState`). Library stamps a per-write `Guid Version`. + Recovery path pattern-matches on `VersionedState` and compares `Version` + directly — immune to collection-equality issues in the state graph. + Recommended default for any non-trivial state. -See §3a for the `VersionedState` base class. +See §3a for `VersionedState` and why the generic layer was removed. ### Activation: state is auto-populated @@ -361,15 +362,15 @@ public sealed record OutboxMessageEnvelope( [GenerateSerializer] public sealed class Outbox : IReadOnlyList>, IEquatable> { - [Id(0)] private readonly GrainId _sender; - [Id(1)] private readonly long _latestSequenceNumber; - [Id(2)] private readonly ImmutableArray> _items; - [Id(3)] private readonly DateTimeOffset? _epoch; + [Id(0)] private readonly GrainId sender; + [Id(1)] private readonly long latestSequenceNumber; + [Id(2)] private readonly ImmutableArray> items; + [Id(3)] private readonly DateTimeOffset? epoch; // Non-persisted; no [Id]. Mutable, registered post-construction. [NonSerialized] [JsonIgnore] - private TimeProvider _time = TimeProvider.System; + private TimeProvider time = TimeProvider.System; internal Outbox( GrainId sender, @@ -377,54 +378,53 @@ public sealed class Outbox : IReadOnlyList>, IEquata ImmutableArray> items, DateTimeOffset? epoch) { - _sender = sender; - _latestSequenceNumber = latestSequenceNumber; - _items = items; - _epoch = epoch; + this.sender = sender; + this.latestSequenceNumber = latestSequenceNumber; + this.items = items; + this.epoch = epoch; } - public static Outbox Empty(GrainId sender) => + public static Outbox Create(GrainId sender) => new(sender, latestSequenceNumber: 0, items: [], epoch: null); - public void RegisterTimeProvider(TimeProvider time) => _time = time; + public void RegisterTimeProvider(TimeProvider time) => this.time = time; - public GrainId Sender => _sender; - public long LatestSequenceNumber => _latestSequenceNumber; - public DateTimeOffset? Epoch => _epoch; - public int Count => _items.Length; - public bool IsEmpty => _items.IsDefaultOrEmpty; - public OutboxMessageEnvelope this[int index] => _items[index]; + public GrainId Sender => sender; + public long LatestSequenceNumber => latestSequenceNumber; + public DateTimeOffset? Epoch => epoch; + public int Count => items.Length; + public bool IsEmpty => items.IsDefaultOrEmpty; + public OutboxMessageEnvelope this[int index] => items[index]; public IEnumerator> GetEnumerator() - => ((IEnumerable>)_items).GetEnumerator(); + => ((IEnumerable>)items).GetEnumerator(); IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); public Outbox Add(T message) { /* increments seq, stamps epoch on first call */ } - public Outbox Remove(OutboxSequenceToken token) { /* removes by seq number */ } - public Outbox RemoveRange(IEnumerable tokens) { /* batch remove */ } + public Outbox Remove(OutboxSequenceToken token) { /* removes FIFO head if token matches */ } + public Outbox RemoveRange(IEnumerable tokens) { /* removes matching FIFO prefix */ } public Outbox Clear() { /* removes all items, preserves LatestSequenceNumber + Epoch */ } - // O(1) fingerprint equality — see below. - public bool Equals(Outbox? other) { /* fingerprint check */ } + // O(1) sequence equality — see below. + public bool Equals(Outbox? other) { /* metadata + first/last sequence check */ } public override bool Equals(object? obj) => obj is Outbox o && Equals(o); - public override int GetHashCode() - => HashCode.Combine(_sender, _latestSequenceNumber, _items.Length, _epoch); + public override int GetHashCode() { /* metadata + first/last sequence hash */ } } ``` ### Epoch semantics -- `Empty(sender)` → `epoch = null`, `LatestSequenceNumber = 0`. +- `Create(sender)` → `epoch = null`, `LatestSequenceNumber = 0`. - First `Add()` → stamps `epoch = now`. Persisted with state. - Subsequent `Add()` → same epoch, incrementing sequence number. - `Clear()` → removes items, **preserves** `LatestSequenceNumber` and `Epoch`. This is the normal "postman drained successfully" path. -- `Empty(sender)` again → **resets both** epoch (to null) and sequence +- `Create(sender)` again → **resets both** epoch (to null) and sequence number (to 0). This is the nuclear option — the next `Add` starts a fresh epoch. Receivers see `token.Epoch > stored.Epoch` and accept. **`Clear()` is the normal path.** Grains should almost never call -`Empty(sender)` on an active outbox. `Empty` is for construction-time +`Create(sender)` on an active outbox. `Create` is for construction-time initialisation and deliberate ops-level sequence-space resets. Document and warn. @@ -440,22 +440,28 @@ The outbox can grow unbounded if postman targets are down. Mitigation: - **Documentation:** storage providers have entity size limits (e.g. Azure Table = 1MB). Document the risk of unbounded growth. -### O(1) fingerprint equality +### O(1) sequence equality Two `Outbox` values are equal when -`(Sender, LatestSequenceNumber, Epoch, Count, _items[0].Seq, _items[^1].Seq)` -matches. Constant-time regardless of `_items.Length`. +`(Sender, LatestSequenceNumber, Epoch, Count, first sequence number, last sequence number)` +matches. Constant-time regardless of `items.Length`. -Invariant holds by construction: `Outbox` is a sealed class with -private readonly backing fields. The only mutation paths are the four -public methods (`Add` / `Remove` / `RemoveRange` / `Clear`), each of -which changes at least one fingerprint field. +This relies on the outbox invariant that sequence numbers are assigned +only by `Add` and pending items are removed only in FIFO order. Under +that invariant, matching first and last sequence numbers with matching +count identifies the same contiguous pending sequence window. Equality +therefore stays independent of outbox depth and does not need a separate +persisted fingerprint field. + +If two outboxes have the same sender, epoch, high-water mark, count, and +sequence-window endpoints but different payloads, the outbox was used in +a way that broke encapsulation/invariants. Equality does not attempt to +detect that invalid state. ### Why these choices - **Sealed class, not record.** No `with`, no synthesized copy - constructor. Mutation through four methods only — fingerprint - invariant holds by construction. Reference type because + constructor. Mutation through four methods only. Reference type because `RegisterTimeProvider` is a void mutator. - **`Add(T payload)` not `Add(envelope)`.** Outbox owns sequence assignment. Callers cannot fabricate sequence numbers. @@ -463,7 +469,9 @@ which changes at least one fingerprint field. `MessageTracker`. Successor instances carry the provider forward. - **Lazy `Epoch`.** A grain that never sends doesn't burn a fresh epoch on storage. -- **`Remove(token)` not `Remove(envelope)`.** Token is the identity. +- **`Remove(token)` not `Remove(envelope)`.** Token is the identity, but + removal is constrained to the FIFO head to preserve the contiguous + pending-sequence invariant. - **`LatestSequenceNumber` is a separate field.** After flush, items are empty; high-water mark persists independently. @@ -476,9 +484,9 @@ restart from zero. Acceptable for burst retry policies. --- -## 3a. `VersionedState` — structural equality across `ImmutableArray` +## 3a. `VersionedState` — version-based equality for recovery -**Status:** Settled. +**Status:** Settled. Generic `VersionedState` removed — see below. ### Problem @@ -495,41 +503,41 @@ public abstract record VersionedState { [Id(0)] public Guid Version { get; internal set; } = Guid.CreateVersion7(); } - -[GenerateSerializer] -public abstract record VersionedState : VersionedState - where TSelf : VersionedState -{ - public virtual bool Equals(VersionedState? other) => - other is not null && Version == other.Version; - - public override int GetHashCode() => Version.GetHashCode(); -} ``` User state: ```csharp [GenerateSerializer] -public sealed record MyState : VersionedState +public sealed record MyState : VersionedState { - [Id(1)] public Outbox Outbox { get; init; } = Outbox.Empty(sender); + [Id(1)] public Outbox Outbox { get; init; } = Outbox.Create(sender); [Id(2)] public ImmutableArray Items { get; init; } = []; } ``` -### Why this shape - -- **`Guid Version` with `internal set`.** Library code can write - `Version`; user code cannot. Hard compile-time fence. -- **`set`, not `init`.** Library mutates caller's reference during - `WriteAsync` to stamp new version. -- **`Guid.CreateVersion7()`.** Sortable UUID v7 (.NET 9+). Cheap, - collision-free at this rate. -- **Initialiser on property.** Fresh records get non-empty version - before any write. -- **Two-layer base.** Non-generic `VersionedState` gives `StateManager` - a single type to test against. Generic layer carries typed equality. +### Why the generic `VersionedState` was removed + +The original design had a two-layer hierarchy where the generic layer +overrode `Equals` to compare only `Version`, bypassing `ImmutableArray` +reference-equality. However: + +1. **The Equals override was broken.** When the user writes + `sealed record MyState : VersionedState`, the compiler + generates `MyState.Equals(MyState?)` that calls `base.Equals(other)` + (the version-only check) **AND** adds property-level checks for all + of `MyState`'s declared properties. So `ImmutableArray` comparisons + still happen in the generated code. +2. **The recovery path doesn't need it.** `IStateManager.WriteAsync` + does `if (newState is VersionedState v)` and compares `v.Version` + directly via pattern matching — it never relies on `T.Equals()` for + VersionedState-derived types. +3. **`IEquatable` on `IStateManager` is satisfied automatically** + by the record-generated equality for non-VersionedState types. + +The non-generic `VersionedState` provides everything the library needs: +the `Version` property, a single type for pattern matching at runtime, +and the `[JsonInclude]` + `internal set` fence. ### Why `Version` is internal-set, not user-settable @@ -555,21 +563,21 @@ each upstream source. Two source kinds: ### Shape **Sealed class** (not record — consistent with `Outbox`, avoids -`_time` field participating in record-synthesized equality). +`time` field participating in record-synthesized equality). ```csharp [GenerateSerializer] public sealed class MessageTracker { - [Id(0)] private ImmutableDictionary _stream; - [Id(1)] private ImmutableDictionary _outbox; + [Id(0)] private ImmutableDictionary streams; + [Id(1)] private ImmutableDictionary outbox; // Non-persisted; no [Id]. Mutable. [NonSerialized] [JsonIgnore] - private TimeProvider _time = TimeProvider.System; + private TimeProvider time = TimeProvider.System; - public void RegisterTimeProvider(TimeProvider time) => _time = time; + public void RegisterTimeProvider(TimeProvider time) => this.time = time; public bool ProcessMessage(StreamCursor cursor, out MessageTracker next); public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next); @@ -648,7 +656,7 @@ responsibilities: 1. **Subscribe** to stream namespaces during `OnActivateAsync`. 2. **Resume** from last accepted `StreamCursor` per namespace. 3. **Dispatch** with projected `StreamCursor`. -4. **Per-subscription error handling** via `.OnError(...)`. +4. **Per-subscription error handling** via the optional `onError` callback. ### Why a facade, not a base class @@ -660,31 +668,27 @@ uses `StreamManager` as a field. ```csharp public sealed class StreamManager { - public static StreamManagerBuilder ForGrain( - Grain owner, - MessageTracker trackerSnapshot, - IServiceProvider services); -} - -public sealed class StreamManagerBuilder -{ - public StreamSubscriptionBuilder Subscribe( + public StreamManager Subscribe( string streamNamespace, - bool resumeUsingLatestSequenceToken = true); + Func onNextAsync, + Action? onError = default, + bool passLatestSequenceTokenOnResume = true); - public StreamManager Build(); + public StreamManager Subscribe( + string streamNamespace, + Func onNextAsync, + Action? onError = default, + bool passLatestSequenceTokenOnResume = true); } -public sealed class StreamSubscriptionBuilder +public static class StreamManagerExtensions { - public StreamSubscriptionBuilder OnNext( - Func handler); - - public StreamSubscriptionBuilder OnError( - Func handler); - - public StreamManagerBuilder Done(); + public static StreamManager InitializeStreamManager( + this TGrain grain, + MessageTracker trackerSnapshot) + where TGrain : IGrainBase; } + ``` ### Typical wiring @@ -695,33 +699,30 @@ public override async Task OnActivateAsync(CancellationToken ct) var state = await stateManager.ReadAsync(); state.Tracker.RegisterTimeProvider(timeProvider); - streamManager = StreamManager.ForGrain(this, state.Tracker, services) - .Subscribe("electricity-prices") - .OnNext(HandlePriceTickAsync) - .OnError(LogStreamErrorAsync) - .Done() - .Subscribe("tariff-events") - .OnNext(HandleTariffChangedAsync) - .Done() - .Build(); + streamManager = this.InitializeStreamManager(state.Tracker) + .Subscribe("electricity-prices", HandlePriceTickAsync, LogStreamError) + .Subscribe("tariff-events", HandleTariffChangedAsync); } ``` +The `TEvent` generic argument is usually inferred from the handler method +group. Users only specify it for inline lambdas or ambiguous method groups. + ### Resume semantics -- `Build()` reads `trackerSnapshot.LatestStream(streamId)` once per - subscription. +- Each subscription reads `trackerSnapshot.LatestStream(streamId)` once + when `Subscribe(...)` activates it. - If cursor exists → hydrate `StreamSequenceToken`, pass to `SubscribeAsync`. - If null → subscribe without token (provider default, typically: start from current). -- Tracker read once at `Build()` time, never again. Handler is the +- Tracker read once at subscription time, never again. Handler is the only path that mutates the tracker. ### Per-subscription `OnError` -Signature: `Func`. Default when -omitted: log + emit counter, do NOT rethrow. +Signature: `Action`, where the string is the stream +namespace. Default when omitted: log + emit counter, do NOT rethrow. ### One-provider-per-namespace @@ -744,31 +745,39 @@ STJ converter for a closed set of known subtypes discriminated on Unknown subtype throws at serialization time — silently dropping the cursor would corrupt dedup. -### Custom EH adapter for enriched tokens +### Event Hub adapter for enriched tokens + +The library ships `EnrichedEventHubAdapter`, a public unsealed +`EventHubDataAdapter` subclass. It opts Event Hub streams into +`EnrichedEventHubSequenceToken`, which carries: + +- `EnqueuedTime` — broker-side enqueue time for lag measurement. +- `StreamProviderName` — provider identity for dedup and multi-provider + edge cases. +- `TraceParent` — W3C traceparent captured from the producer-side + `Activity.Current?.Id`. -Optional pattern. To opt into `EnrichedEventHubSequenceToken` (carries -`EnqueuedTime`, see §5 projection constraint table), the silo wires a -custom `IEventHubDataAdapter` via `UseDataAdapter`: +Users who want the built-in behavior register it with +`UseEnrichedDataAdapter()` on `IEventHubStreamConfigurator`: ```csharp -internal sealed class EnrichedEventHubAdapter(Serializer serializer) - : EventHubDataAdapter(serializer) +siloBuilder.AddEventHubStreams("orders", b => { - public override StreamSequenceToken GetSequenceToken( - EventHubMessage eventHubMessage, int eventIndex) - { - var seqNum = eventHubMessage.SequenceNumber; - return new EnrichedEventHubSequenceToken( - eventHubMessage.Offset, - seqNum, - eventIndex, - eventHubMessage.EnqueuedTimeUtc); - } -} + b.UseEnrichedDataAdapter(); + // ... other Event Hub config +}); ``` -`StreamManager` is unaware of the adapter — the enrichment surfaces -through `StreamCursor.TryGetEnqueuedTime(...)` in the user's `OnNext`. +Users who need custom adapter behavior can subclass +`EnrichedEventHubAdapter` and register their subclass via Orleans' +`UseDataAdapter` directly. The library intentionally provides no +generic registration helper for custom subclasses because those adapters +usually need extra services/options. + +`StreamManager` is unaware of Event Hubs specifically — enrichment +surfaces through `StreamCursor.TryGetEnqueuedTime(...)`, +`StreamCursor.TryGetStreamProviderName(...)`, and +`StreamCursor.TryGetTraceParent(...)`. ### OpenTelemetry trace correlation @@ -777,20 +786,24 @@ correlate consumer-side spans with producer-side spans without creating multi-hour distributed traces, `StreamManager` should use `ActivityLink`s, not parent chaining: -- Producer side (in a custom `ToQueueMessage` override on the adapter): - stash `Activity.Current?.Id` into `EventData.Properties["traceparent"]` +- Producer side (`EnrichedEventHubAdapter.ToQueueMessage`): stash + `Activity.Current?.Id` into `EventData.Properties["traceparent"]` before the event hits EH. +- Adapter ingest side (`EnrichedEventHubAdapter.GetStreamPosition`): + extract `EventData.Properties["traceparent"]` into + `EnrichedEventHubSequenceToken.TraceParent`. - Consumer side (in `StreamManager`'s OnNext wrapper): read the - property, parse into `ActivityContext`, start the OnNext span with + token's traceparent, parse into `ActivityContext`, start the OnNext span with `ActivityKind.Consumer` and `links: [new ActivityLink(parsedContext)]`. This produces separate traces per delivery, each with a link back to the producer span. OTel backends render the cross-trace arrow without collapsing weeks of traffic into one trace. -The adapter override is the user's responsibility (it lives in their -custom `IEventHubDataAdapter`); the consumer-side parse+link is a -library concern owned by `StreamManager`. +The built-in adapter owns producer-side propagation for users who call +`UseEnrichedDataAdapter()`. Custom adapters should preserve the same +`traceparent` property behavior if they want `StreamManager` to create +links. ### Telemetry @@ -845,8 +858,8 @@ Grains using this library should follow a functional-command model: - **Safe concurrency.** Reads never block writes. Writes are serialised by the runtime. No custom locks. - **Recovery-friendly.** `IStateManager.WriteAsync` recovery path - relies on `VersionedState.Equals` — works because state is a - value type (record with `Version` equality). + pattern-matches `VersionedState` and compares `Version` directly — + works because the library stamps a v7 UUID on every write. - **Outbox replaces side effects.** Instead of "write state + call service" (two failure points), it's "write state with outbox item" (one atomic write) + "processor retries delivery" (idempotent). @@ -856,7 +869,7 @@ Grains using this library should follow a functional-command model: - No compile-time prevention of injecting `HttpClient` or calling external services in command methods. This is a documentation and code-review concern. -- No base class. The pattern emerges from the types: `VersionedState` +- No base class. The pattern emerges from the types: `VersionedState` is a record (immutable), `IStateManager.WriteAsync` takes a new value (not mutation), `Outbox.Add` returns a new instance. - Future: Roslyn analyzers could warn on external I/O inside methods @@ -1073,14 +1086,11 @@ public abstract record VersionedState { [Id(0)] public Guid Version { get; internal set; } } - -[GenerateSerializer] -public abstract record VersionedState : VersionedState -{ - // Inherits [Id(0)] from parent. Child IDs start fresh at [Id(0)]. -} ``` +> **Note:** The generic `VersionedState` layer was removed. +> See §3a for rationale. Child record IDs start fresh at `[Id(0)]`. + ### System.Text.Json serialization All serializable types carry `[JsonConverter]` attributes referencing @@ -1091,9 +1101,9 @@ This ensures correct round-tripping through storage providers that use STJ (e.g., Orleans's Cosmos, blob, or custom providers configured with `System.Text.Json`). -**Newtonsoft.Json is not supported.** Users whose storage providers use -Newtonsoft must either configure their provider for STJ or write their -own converters. Documented as a known limitation. +**Newtonsoft.Json is not supported out of the box.** Users whose storage +providers use Newtonsoft can write and register their own converters. +Documented as a known limitation. | Type | Converter approach | |------|-------------------| @@ -1122,7 +1132,7 @@ generic types. The factory's `CreateConverter` method creates the closed ### Non-serialized fields -Service-reference fields like `TimeProvider _time` get both +Service-reference fields like `TimeProvider time` get both `[NonSerialized]` and `[JsonIgnore]` — belt-and-suspenders: - **No `[Id]`** → Orleans `[GenerateSerializer]` skips them. @@ -1184,7 +1194,7 @@ concurrency behavior. Matching `Egil.Orleans.Testing` convention: - **100% branch coverage** on core types: `Outbox`, - `MessageTracker`, `StateManager`, `VersionedState`, + `MessageTracker`, `StateManager`, `VersionedState`, `OutboxProcessor`. - **95% branch coverage** on supporting types: `OutboxSequenceToken`, `StreamCursor`, `StreamManager`, `OutboxMessageEnvelope`, diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj index 21a83ea3..6cffc4ca 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj @@ -35,8 +35,10 @@ all runtime; build; native; contentfiles; analyzers; buildtransitive + + diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs new file mode 100644 index 00000000..584fc178 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs @@ -0,0 +1,213 @@ +using System.Diagnostics; +using Orleans.Providers.Streams.Common; +using Orleans.Serialization; +using Orleans.Streaming.EventHubs; +using Orleans.Streams; + +namespace Egil.Orleans.Messaging; + +/// +/// An subclass that produces +/// instances carrying the +/// broker-side enqueue time, stream provider name, and W3C traceparent. +/// +/// +/// +/// Registration: Do not instantiate directly. Use +/// +/// on the IEventHubStreamConfigurator during silo setup: +/// +/// siloBuilder.AddEventHubStreams("my-provider", b => +/// { +/// b.UseEnrichedDataAdapter(); +/// // ... other config +/// }); +/// +/// +/// +/// Overrides: This adapter overrides three methods: +/// +/// and +/// +/// — to return instead of +/// . +/// — to stamp +/// Activity.Current?.Id into the outgoing +/// EventData.Properties["traceparent"] before the event hits +/// Event Hub, enabling cross-queue trace correlation via +/// s on the consumer side. +/// +/// All other adapter behavior (batch container, partition key) is +/// inherited unchanged from . +/// +/// +/// OTel trace correlation: Orleans streams lose +/// across the queue boundary. This adapter +/// uses the W3C traceparent propagation pattern: +/// +/// Producer side (): +/// stashes Activity.Current?.Id into +/// EventData.Properties["traceparent"]. +/// Consumer side (): +/// extracts the traceparent property and stores it in +/// . +/// StreamManager: reads +/// and creates an +/// — correlating consumer spans to producer +/// spans without creating multi-hour parent-child traces. +/// +/// +/// +/// Subclassing: This class is not sealed — users who need additional +/// adapter customization (custom partition keys, custom batch containers, +/// etc.) can inherit from instead of +/// from directly. This preserves the +/// enriched token behavior while allowing further overrides: +/// +/// public class MyAdapter : EnrichedEventHubAdapter +/// { +/// public MyAdapter(string streamProviderName, Serializer serializer) +/// : base(streamProviderName, serializer) { } +/// +/// public override string GetPartitionKey(StreamId streamId) +/// => streamId.GetNamespace(); +/// } +/// +/// When subclassing, register via UseDataAdapter directly on the +/// IEventHubStreamConfigurator with your custom adapter type. +/// +/// +public class EnrichedEventHubAdapter : EventHubDataAdapter +{ + /// + /// The Event Hub application property key used to propagate the W3C + /// traceparent header across the queue boundary. + /// + protected const string TraceParentPropertyKey = "traceparent"; + + /// + /// The name of the Orleans stream provider. Accessible to subclasses for + /// custom token construction or diagnostics. + /// + protected string StreamProviderName { get; } + + /// + /// Creates a new adapter that enriches tokens with the given + /// . + /// + /// + /// The name of the Orleans stream provider, passed through from the + /// UseDataAdapter factory delegate. Stored in every + /// produced by this adapter. + /// + /// + /// The Orleans used by the base + /// for batch container + /// serialization/deserialization. + /// + public EnrichedEventHubAdapter(string streamProviderName, Serializer serializer) + : base(serializer) + { + StreamProviderName = streamProviderName; + } + + /// + /// Overrides the base to stamp Activity.Current?.Id into the + /// outgoing EventData.Properties["traceparent"] before the event + /// is published to Event Hub. + /// + /// + /// + /// This is the producer side of the OTel trace correlation pattern. + /// The current (W3C format) is captured at + /// publish time and stored as an Event Hub application property. On the + /// consumer side, extracts it into the + /// property. + /// + /// + /// If no is active at publish time, no property + /// is set and the consumer-side token will have a null + /// . + /// + /// + /// The type of stream events. + /// The target stream identity. + /// The events to publish. + /// The sequence token (must be null for Event Hubs). + /// The Orleans request context dictionary. + /// + /// An with the + /// traceparent property stamped if an + /// is active. + /// + public override Azure.Messaging.EventHubs.EventData ToQueueMessage( + StreamId streamId, + IEnumerable events, + StreamSequenceToken token, + Dictionary requestContext) + { + throw new NotImplementedException(); + } + + /// + /// Overrides the base to produce an + /// carrying the cached message's enqueue time and the stream provider name. + /// + /// + /// + /// Called by the Event Hub cache when delivering messages from the in-memory + /// cache. The is available on every + /// cached message — no additional broker round-trip is needed. + /// + /// + /// Note: The + /// is not available on tokens reconstructed from cache — the + /// struct does not carry Event Hub application + /// properties. Trace correlation uses the token produced by + /// at initial ingest. + /// + /// + /// The cached Event Hub message. + /// + /// An with the enqueue time and + /// provider name populated. + /// is null for cache-reconstructed tokens. + /// + public override StreamSequenceToken GetSequenceToken(ref CachedMessage cachedMessage) + { + throw new NotImplementedException(); + } + + /// + /// Overrides the base to produce an + /// when the adapter first sees a raw EventData from the Event Hub + /// partition receiver. + /// + /// + /// + /// Called once per incoming EventData before it enters the cache. + /// The EventData.EnqueuedTime property carries the broker-stamped + /// UTC time; the is captured at adapter + /// construction time. + /// + /// + /// Trace correlation: If the incoming EventData has a + /// "traceparent" application property (stamped by + /// on the producer side), it is extracted + /// and stored in . + /// + /// + /// The Event Hub partition identifier. + /// The raw Event Hub message. + /// + /// A whose SequenceToken is an + /// with enqueue time, + /// provider name, and traceparent (if present). + /// + public override StreamPosition GetStreamPosition( + string partition, + Azure.Messaging.EventHubs.EventData queueMessage) + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs new file mode 100644 index 00000000..767f9663 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs @@ -0,0 +1,63 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Extension methods for registering the -producing +/// data adapter on an Event Hub stream configurator. +/// +/// +/// +/// Usage: +/// +/// siloBuilder.AddEventHubStreams("my-provider", b => +/// { +/// b.ConfigureEventHub(ob => ob.Configure(options => +/// { +/// options.ConfigureTableStorageCheckpointing(...); +/// })); +/// b.UseEnrichedDataAdapter(); +/// }); +/// +/// +/// +/// What it does: Registers the library's internal +/// EnrichedEventHubAdapter via UseDataAdapter. The adapter +/// overrides GetStreamPosition and GetSequenceToken to return +/// instances carrying the +/// broker-side enqueue time and the stream provider name. No user-written +/// adapter code is needed. +/// +/// +/// Downstream access: Once registered, the enrichment is transparently +/// available on every stream token. Use +/// for lag measurement and +/// for provider-aware dedup and diagnostics. +/// +/// +public static class EnrichedEventHubAdapterExtensions +{ + /// + /// Registers the library's -producing + /// data adapter on the given Event Hub stream configurator. + /// + /// + /// + /// Replaces the default EventHubDataAdapter. Only one data adapter + /// can be active per stream provider — calling this after a prior + /// UseDataAdapter replaces the previous registration. + /// + /// + /// The adapter is resolved per-provider: the Serializer is obtained + /// from the silo's , and the stream provider + /// name is captured from the configurator's factory delegate. + /// + /// + /// + /// The Event Hub stream configurator, obtained from + /// ISiloBuilder.AddEventHubStreams. + /// + public static void UseEnrichedDataAdapter( + this IEventHubStreamConfigurator configurator) + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs new file mode 100644 index 00000000..aed2805b --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs @@ -0,0 +1,127 @@ +using Orleans.Streaming.EventHubs; + +namespace Egil.Orleans.Messaging; + +/// +/// An subclass that carries the +/// broker-side and the , +/// enabling end-to-end lag measurement and provider-aware dedup in +/// and . +/// +/// +/// +/// Registration: Call +/// +/// on the Event Hub stream configurator during silo setup. This registers +/// the library's internal EnrichedEventHubAdapter that overrides +/// GetStreamPosition and GetSequenceToken to produce +/// instances automatically. +/// +/// +/// Transparent to StreamManager: The is +/// unaware of the adapter. The enrichment surfaces through +/// and +/// in the user's +/// OnNext handler. +/// +/// +/// Dedup and edge cases: The embedded +/// in the token enables to distinguish messages +/// arriving from different stream providers on the same stream namespace, +/// supporting multi-provider topologies and provider-specific eviction. +/// +/// +/// OTel trace correlation: The property +/// carries the W3C traceparent header captured at publish time by +/// . On the consumer side, +/// reads it via +/// and creates an +/// — correlating consumer +/// spans to producer spans without multi-hour parent-child traces. +/// +/// +/// Serialization: Inherits Orleans serialization from +/// . The library's STJ converter for +/// recognizes this subtype via a $kind +/// discriminator and round-trips , +/// , and fields. +/// +/// +[GenerateSerializer] +[Alias("egil.orleans.messaging.EnrichedEventHubSequenceToken")] +public class EnrichedEventHubSequenceToken : EventHubSequenceTokenV2 +{ + /// + /// The wall-clock time the event was enqueued at the Event Hub broker. + /// Used by to compute + /// end-to-end lag: now - EnqueuedTime. + /// + [Id(0)] + public DateTimeOffset EnqueuedTime { get; } + + /// + /// The name of the Orleans stream provider that delivered this event. + /// Enables provider-aware dedup and diagnostics in . + /// + [Id(1)] + public string StreamProviderName { get; } + + /// + /// The W3C traceparent header value from the producer-side + /// at publish time, or null + /// if no activity was active when the event was published. + /// + /// + /// Stamped by in its + /// ToQueueMessage override (producer side) and extracted in + /// GetStreamPosition (consumer side). + /// uses this to create + /// s — correlating consumer + /// spans to producer spans without creating multi-hour parent-child traces. + /// + [Id(2)] + public string? TraceParent { get; } + + /// + /// Creates a new enriched token with the broker-side enqueue time and + /// stream provider name. + /// + /// The Event Hub partition offset (string). + /// The Event Hub sequence number. + /// + /// The index of the event within a batch at the same sequence number. + /// + /// + /// The (UTC) when the event was enqueued at the + /// Event Hub broker. + /// + /// + /// The name of the Orleans stream provider. Passed through from the + /// EnrichedEventHubAdapter at construction time. + /// + /// + /// The W3C traceparent value captured at publish time, or + /// null if no was active. + /// + public EnrichedEventHubSequenceToken( + string offset, + long sequenceNumber, + int eventIndex, + DateTime enqueuedTime, + string streamProviderName, + string? traceParent = null) + : base(offset, sequenceNumber, eventIndex) + { + EnqueuedTime = new DateTimeOffset(enqueuedTime, TimeSpan.Zero); + StreamProviderName = streamProviderName; + TraceParent = traceParent; + } + + /// + /// Parameterless constructor for Orleans serializer use only. + /// + public EnrichedEventHubSequenceToken() + { + StreamProviderName = string.Empty; + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs new file mode 100644 index 00000000..3842c79b --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs @@ -0,0 +1,51 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Marker interface for grains that use . +/// Provides a default interface method (DIM) for +/// that routes reminder callbacks to the processor automatically. +/// +/// +/// +/// Zero ceremony: Implementing this interface is one of two obligations +/// for grains using the outbox pattern (the other is calling +/// InitializeOutboxProcessor in OnActivateAsync). No manual +/// ReceiveReminder override is needed — the DIM discovers the +/// registered on the grain context and +/// forwards the callback. +/// +/// +/// Escape hatch: Grains with their own reminders can override +/// explicitly, handle their own +/// reminder names, and forward unknown names to the processor: +/// +/// public async Task ReceiveReminder(string name, TickStatus status) +/// { +/// if (name == MyOwnReminder) { await DoMyWork(); return; } +/// await outboxProcessor.ReceiveReminderAsync(name, status); +/// } +/// +/// +/// +/// Compiler enforcement: The InitializeOutboxProcessor +/// extension method constrains TGrain : IOutboxGrain, IGrainBase, +/// so grains that forget to implement this interface get a compile error. +/// +/// +public interface IOutboxGrain : IRemindable +{ + /// + /// Default implementation that discovers the + /// on the grain context and forwards the reminder callback. Returns + /// if no processor is attached (safe + /// no-op for reminders that fire before activation completes). + /// + Task IRemindable.ReceiveReminder(string reminderName, TickStatus status) + { + var grainBase = (IGrainBase)this; + var component = grainBase.GrainContext.GetComponent(); + return component is null + ? Task.CompletedTask + : component.ReceiveReminderAsync(reminderName, status).AsTask(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs new file mode 100644 index 00000000..0aa1d3ab --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs @@ -0,0 +1,138 @@ +namespace Egil.Orleans.Messaging; + +/// +/// A thin wrapper around that guarantees +/// the grain's observable is never out of sync with what is +/// durably persisted, even when fails ambiguously +/// (timeout, network drop, server 5xx, ETag conflict). +/// +/// +/// +/// Committed-state fence: exposes only the last +/// successfully written value. During an in-flight write, the underlying +/// .State already holds the uncommitted +/// value. Methods marked [AlwaysInterleave] that read +/// through this interface are guaranteed to never observe uncommitted state. +/// This is the primary reason exists as a wrapper +/// rather than extension methods on . +/// +/// +/// Usage: Inject as normal via +/// [PersistentState], then wrap it during OnActivateAsync: +/// +/// stateManager = storage.AsStateManager(); +/// +/// The raw should not be accessed +/// directly after wrapping — doing so bypasses the committed-state fence. +/// +/// +/// Recovery: On ambiguous write failure, the manager re-reads from +/// storage. If the write actually persisted (detected via version or equality +/// check), it swallows the exception. If the write did not persist, it rethrows. +/// If both write and re-read fail (double failure), the manager reverts to the +/// previous state and rethrows — the grain must call +/// before its next write to refresh the ETag. +/// +/// +/// +/// The grain state type. Must be a reference type (atomic pointer swap for +/// interleaved reads) and implement (recovery path +/// compares server-side state to attempted write). Records satisfy both for free. +/// For state containing +/// or other types with reference-based equality, inherit from +/// — the recovery path pattern-matches against it +/// and compares directly, bypassing +/// Equals entirely. +/// +/// Deep immutability: Every type referenced from the state record +/// should also be immutable. The strength of this requirement depends on +/// how the grain is used: +/// +/// +/// Required when the grain uses [AlwaysInterleave] on any +/// read method. Interleaved readers access concurrently +/// with an in-flight command building the next state value. If the state +/// graph contains mutable reference types, a reader could observe a +/// partially mutated object even though the root reference hasn't +/// been swapped yet (the old state's inner mutable object is being modified +/// in place by the command). +/// +/// +/// Strongly recommended even without interleaving. Orleans default +/// turn-based concurrency prevents concurrent access, so torn reads cannot +/// occur. However, immutable state still provides value: it makes the +/// functional-command pattern (with { ... }) predictable, prevents +/// accidental mutation of the "previous" snapshot held by the recovery +/// path, and avoids subtle bugs if [AlwaysInterleave] is added later. +/// +/// +/// Use ImmutableArray<T>, ImmutableDictionary<K,V>, +/// records, and value objects throughout. Mutable collections +/// (, +/// ) inside +/// the state graph undermine these guarantees. +/// +/// +public interface IStateManager + where T : class, IEquatable +{ + /// + /// Gets the last successfully committed state snapshot. + /// + /// + /// Safe to read from [AlwaysInterleave] methods — returns only + /// committed values, never in-flight uncommitted state. This is the + /// committed-state fence that justifies the wrapper over raw + /// . + /// + T State { get; } + + /// + /// Re-reads state from durable storage, replacing the current + /// snapshot. + /// + /// + /// Not required during activation — + /// auto-hydrates before OnActivateAsync. Use only when the grain + /// needs to force a re-read mid-activation (e.g., after a known external + /// mutation or to recover from a double-failure scenario). + /// + Task ReadAsync(); + + /// + /// Atomically writes to durable storage. + /// On success, reflects . + /// + /// + /// + /// If derives from , + /// a fresh version (v7) is stamped on it before writing. + /// The caller's reference is mutated — this is a documented contract. + /// + /// + /// Recovery on failure: Re-reads from storage. If the write actually + /// landed (version or equality match), swallows the exception and returns + /// normally. If it did not land, rethrows the original exception. + /// InconsistentStateException always rethrows + /// even if equality matches — a coincidental match must not hide a real + /// concurrent write. + /// + /// + /// Double failure: If both write and re-read fail, reverts + /// to its pre-write value and rethrows. The grain + /// holds correct data but a stale ETag — call + /// before the next write. + /// + /// + /// The new state value to persist. + /// + /// Concurrency policy. (default) uses + /// the provider's ETag check. bypasses it. + /// + Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concurrent); + + /// + /// Clears the persisted state, resetting it to default. + /// + Task ClearAsync(); +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs new file mode 100644 index 00000000..12b8a5ba --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs @@ -0,0 +1,230 @@ +using System.Collections.Immutable; +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging; + +/// +/// Receiver-side, persisted dedup state. Tracks the high-water position from +/// each upstream message source so the grain can detect and reject duplicates. +/// +/// +/// +/// Two source kinds: +/// +/// Orleans streams — keyed by ; position +/// is a wrapping (StreamId, StreamSequenceToken). +/// Outbox messages — keyed by sender ; +/// position is an . +/// +/// +/// +/// Sealed class, not record. Consistent with — +/// avoids time field participating in record-synthesized equality, and +/// prevents with { ... } expressions that could bypass invariants. +/// +/// +/// ProcessMessage semantics (streams): +/// +/// No prior entry → Accept, insert (LastPosition = cursor, Received = now). +/// cursor > stored.LastPosition → Accept, update position + received. +/// cursor <= stored.LastPosition → Reject (duplicate), no change. +/// +/// +/// +/// ProcessMessage semantics (outbox): +/// +/// No prior entry → Accept, insert. +/// token.Epoch > stored.Epoch → Accept (sender reset), replace entry. +/// Same epoch, token.Seq > stored.LastSeq → Accept, update. +/// Same epoch, token.Seq <= stored.LastSeq → Reject (duplicate). +/// token.Epoch < stored.Epoch → Reject (stale epoch). +/// +/// +/// +/// Eviction: Five overloads, one rule — remove entries where +/// entry.Received < olderThan. No separate Forget API. +/// Evict(id, DateTimeOffset.MaxValue) is the documented idiom for +/// unconditional removal of a single source entry. +/// +/// +/// TimeProvider: Non-persisted ([NonSerialized], +/// [JsonIgnore], no [Id]). After deserialization, the grain +/// must call to inject a test-friendly +/// clock. Falls back to if skipped. +/// +/// +/// Serialization: Decorated with [GenerateSerializer] for Orleans +/// and [JsonConverter] for STJ. The custom converter keeps private backing +/// fields encapsulated. +/// +/// +[GenerateSerializer] +[Alias("egil.orleans.messaging.MessageTracker")] +// [JsonConverter(typeof(MessageTrackerJsonConverter))] +public sealed class MessageTracker : IEquatable +{ + [Id(0)] private readonly ImmutableDictionary streams; + [Id(1)] private readonly ImmutableDictionary outbox; + + /// + /// Non-persisted service reference. No [Id], no serialization. + /// Falls back to when not explicitly set. + /// + [NonSerialized] + [JsonIgnore] + private TimeProvider time = TimeProvider.System; + + /// + /// Creates an empty with no tracked sources. + /// + public MessageTracker() + { + streams = ImmutableDictionary.Empty; + outbox = ImmutableDictionary.Empty; + } + + private MessageTracker( + ImmutableDictionary streams, + ImmutableDictionary outbox) + { + this.streams = streams; + this.outbox = outbox; + } + + /// + /// Registers a for Received timestamps. + /// Must be called after deserialization to inject a test-friendly clock. + /// + public void RegisterTimeProvider(TimeProvider time) => this.time = time; + + /// + /// Evaluates a stream message for acceptance. Returns true if the + /// advances past the stored high-water mark. + /// + /// The stream cursor to evaluate. + /// + /// When accepted, a new with the updated + /// position. When rejected, equals this. + /// + /// true if accepted (new message); false if duplicate. + public bool ProcessMessage(StreamCursor cursor, out MessageTracker next) + { + throw new NotImplementedException(); + } + + /// + /// Evaluates an outbox message for acceptance. Returns true if the + /// advances past the stored position or carries + /// a newer epoch. + /// + /// The outbox sequence token to evaluate. + /// + /// When accepted, a new with the updated + /// position. When rejected, equals this. + /// + /// true if accepted; false if duplicate or stale. + public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) + { + throw new NotImplementedException(); + } + + /// + /// Returns the last accepted for the given + /// , or null if no messages from that + /// stream have been tracked. + /// + public StreamCursor? LatestStream(StreamId stream) + { + throw new NotImplementedException(); + } + + /// + /// Returns the last accepted for the + /// given , or null if no outbox messages + /// from that sender have been tracked. + /// + public OutboxSequenceToken? LatestOutbox(GrainId sender) + { + throw new NotImplementedException(); + } + + /// + /// Removes all entries (both stream and outbox) where + /// entry.Received < . + /// + public MessageTracker Evict(DateTimeOffset olderThan) + { + throw new NotImplementedException(); + } + + /// + /// Removes stream entries where + /// entry.Received < . + /// Outbox entries are unaffected. + /// + public MessageTracker EvictStreams(DateTimeOffset olderThan) + { + throw new NotImplementedException(); + } + + /// + /// Removes outbox entries where + /// entry.Received < . + /// Stream entries are unaffected. + /// + public MessageTracker EvictOutboxes(DateTimeOffset olderThan) + { + throw new NotImplementedException(); + } + + /// + /// Removes the entry for the given if + /// entry.Received < . + /// Use DateTimeOffset.MaxValue to unconditionally remove. + /// + public MessageTracker Evict(StreamId stream, DateTimeOffset olderThan) + { + throw new NotImplementedException(); + } + + /// + /// Removes the entry for the given outbox if + /// entry.Received < . + /// Use DateTimeOffset.MaxValue to unconditionally remove. + /// + public MessageTracker Evict(GrainId sender, DateTimeOffset olderThan) + { + throw new NotImplementedException(); + } + + /// + public bool Equals(MessageTracker? other) + { + throw new NotImplementedException(); + } + + /// + public override bool Equals(object? obj) => obj is MessageTracker o && Equals(o); + + /// + public override int GetHashCode() => HashCode.Combine(streams.Count, outbox.Count); + + /// + /// Internal entry tracking a stream source's last known position and + /// the wall-clock time it was received. + /// + [GenerateSerializer] + internal readonly record struct StreamEntry( + [property: Id(0)] StreamCursor LastPosition, + [property: Id(1)] DateTimeOffset Received); + + /// + /// Internal entry tracking an outbox source's last known epoch, + /// sequence number, and the wall-clock time it was received. + /// + [GenerateSerializer] + internal readonly record struct OutboxEntry( + [property: Id(0)] DateTimeOffset Epoch, + [property: Id(1)] long LastSequenceNumber, + [property: Id(2)] DateTimeOffset Received); +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs new file mode 100644 index 00000000..f06ecdf1 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs @@ -0,0 +1,38 @@ +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging; + +/// +/// STJ converter for . Serializes and +/// deserializes the tracker's internal stream-position and outbox-position +/// dictionaries without exposing private fields. +/// +/// +/// +/// is a sealed class with private +/// +/// backing fields. Exposing them via [JsonInclude] would leak +/// internal structure and weaken encapsulation. This converter controls +/// the exact wire format. +/// +/// +/// Registered on via [JsonConverter]. +/// STJ discovers the attribute automatically — no user-side +/// configuration needed. +/// +/// +internal sealed class MessageTrackerJsonConverter : JsonConverter +{ + /// + public override MessageTracker? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + throw new NotImplementedException(); + } + + /// + public override void Write(Utf8JsonWriter writer, MessageTracker value, JsonSerializerOptions options) + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs new file mode 100644 index 00000000..3ff3ec6a --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs @@ -0,0 +1,45 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Thrown (internally, via ) +/// when an outbox item's runtime type does not match any registered postman. +/// +/// +/// +/// This exception is never thrown out of . +/// It is surfaced through the +/// callback as the Error member of the failure tuple, allowing the grain +/// to decide how to handle unmatched items (log, dead-letter, remove, etc.). +/// +/// +/// Common cause: A new subtype of the outbox base type was added but no +/// corresponding +/// call was registered. Fix by adding the missing postman registration in +/// OnActivateAsync. +/// +/// +/// Postman ordering: Postmen are matched first-registered-wins. If a +/// less-specific postman is registered before a more-specific one, the +/// more-specific subtype may be consumed by the less-specific postman instead +/// of reaching this exception. Register from most specific to least specific. +/// +/// +public sealed class NoPostmanRegisteredException : InvalidOperationException +{ + /// + /// Creates a new for the + /// given . + /// + /// The runtime type of the unmatched outbox item. + public NoPostmanRegisteredException(Type itemType) + : base($"No postman registered for outbox item type '{itemType.FullName}'.") + { + ItemType = itemType; + } + + /// + /// The runtime type of the outbox item that could not be matched to any + /// registered postman. + /// + public Type ItemType { get; } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs new file mode 100644 index 00000000..2eba2bd9 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs @@ -0,0 +1,339 @@ +using System.Collections; +using System.Collections.Immutable; +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging; + +/// +/// A per-grain durable buffer of messages that have been announced +/// (committed alongside a state change) but not yet delivered +/// (handed off to a postman successfully). Lives as a property on the grain's +/// state record so it participates in atomic WriteStateAsync calls. +/// +/// +/// +/// Immutable-collection semantics: Behaves like +/// — read-only iteration, indexer access, +/// mutators (, , , +/// ) return new instances. The original is never +/// modified. Assign the return value back to the state property and write. +/// +/// +/// Sequence ownership: Only assigns sequence numbers. +/// Callers supply the payload; the outbox stamps +/// with a monotonically increasing and the +/// current . This is a hard invariant — there is no public +/// constructor that accepts a pre-built sequence number. +/// +/// +/// Epoch semantics: +/// +/// → Epoch = null, +/// LatestSequenceNumber = 0. Use at construction time or for deliberate +/// ops-level sequence-space resets. +/// First → stamps Epoch = now. Persisted with state. +/// Subsequent → same epoch, incrementing sequence number. +/// → removes all items but preserves +/// and . This is the normal +/// "postman drained successfully" path. +/// +/// Grains should almost never call on an active outbox. +/// +/// +/// Equality: Two instances are equal when +/// sender, sequence metadata, epoch, count, first sequence number, and last +/// sequence number are equal. Equality is O(1). +/// +/// +/// Unbounded growth risk: If postman targets are down, the outbox grows +/// without limit. Mitigations: telemetry (gauge for depth per grain type), +/// configurable max via (oldest +/// messages auto-dropped FIFO), and documentation of storage-provider entity +/// size limits (e.g., Azure Table = 1 MB). +/// +/// +/// Serialization: Decorated with [GenerateSerializer] for Orleans. +/// A [JsonConverter] attribute (via JsonConverterFactory) ensures +/// System.Text.Json round-trips without exposing private backing fields. +/// Newtonsoft.Json is not supported out of the box; users can write and +/// register their own Newtonsoft converter if needed. +/// +/// +/// TimeProvider: The time field is non-persisted +/// ([NonSerialized], [JsonIgnore], no [Id]). After +/// deserialization the grain must call to +/// inject a test-friendly clock. Falls back to +/// if skipped — correct for production, breaks fake-clock tests. +/// +/// +/// +/// The user-defined message payload type. Must be serializable by Orleans +/// ([GenerateSerializer]) and by System.Text.Json if the storage provider +/// uses STJ. +/// +[GenerateSerializer] +[Alias("egil.orleans.messaging.Outbox`1")] +// [JsonConverter(typeof(OutboxJsonConverterFactory))] +public sealed class Outbox : IReadOnlyList>, IEquatable> +{ + [Id(0)] private readonly GrainId sender; + [Id(1)] private readonly long latestSequenceNumber; + [Id(2)] private readonly ImmutableArray> items; + [Id(3)] private readonly DateTimeOffset? epoch; + + /// + /// Non-persisted service reference. No [Id], no serialization. + /// Falls back to when not explicitly set. + /// + [NonSerialized] + [JsonIgnore] + private TimeProvider time = TimeProvider.System; + + /// + /// Internal constructor used by mutation methods to produce new instances. + /// Not user-callable — use to create the + /// initial outbox, then to append messages. + /// + internal Outbox( + GrainId sender, + long latestSequenceNumber, + ImmutableArray> items, + DateTimeOffset? epoch) + { + this.sender = sender; + this.latestSequenceNumber = latestSequenceNumber; + this.items = items; + this.epoch = epoch; + } + + /// + /// Creates a fresh, empty outbox for the given . + /// is null and + /// is 0. The next stamps a new epoch. + /// + /// + /// Use at grain-state construction time (default property initializer). + /// Calling on an active outbox is a nuclear reset — the next + /// starts a fresh epoch. Receivers see the epoch change + /// and accept unconditionally. Prefer for the normal + /// "postman drained" path. + /// + /// + /// The of the grain that owns this outbox. Baked + /// into every produced by . + /// + public static Outbox Create(GrainId sender) => + new(sender, latestSequenceNumber: 0, items: [], epoch: null); + + /// + /// Registers a for timestamp generation. + /// Must be called after deserialization to inject a test-friendly clock. + /// + /// + /// This is a void mutator on a non-persisted field — it does not produce + /// a new instance. The provider is carried forward + /// to successor instances created by , , + /// , and . + /// + public void RegisterTimeProvider(TimeProvider time) => this.time = time; + + /// + /// The of the grain that owns this outbox. Baked + /// into every . + /// + public GrainId Sender => sender; + + /// + /// The highest sequence number ever assigned in this outbox, including + /// items that have been removed. Persists independently of item contents — + /// does not reset it. + /// + public long LatestSequenceNumber => latestSequenceNumber; + + /// + /// The epoch marker stamped on the first call. null + /// only for a freshly constructed () outbox that + /// has never had an item added. + /// + public DateTimeOffset? Epoch => epoch; + + /// Gets the number of pending messages. + public int Count => items.Length; + + /// + /// true when the outbox contains no pending messages. Note that + /// may be non-zero even when empty + /// (items were drained by the postman). + /// + public bool IsEmpty => items.IsDefaultOrEmpty; + + /// Gets the envelope at the specified index. + public OutboxMessageEnvelope this[int index] => items[index]; + + /// + public IEnumerator> GetEnumerator() + => ((IEnumerable>)items).GetEnumerator(); + + /// + IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); + + /// + /// Appends a message to the outbox, assigning the next sequence number + /// and stamping the epoch on first call. + /// + /// + /// Returns a new instance — the original + /// is not modified. Assign the result back to the state property: + /// + /// state = state with { Outbox = state.Outbox.Add(myEvent) }; + /// await stateManager.WriteAsync(state); + /// + /// + /// The user-defined payload to enqueue. + /// A new outbox containing the appended message. + public Outbox Add(T message) + { + var now = time.GetUtcNow(); + var epoch = this.epoch ?? now; + var sequenceNumber = latestSequenceNumber + 1; + var token = new OutboxSequenceToken(sequenceNumber, sender, now, epoch); + var next = new Outbox( + sender, + sequenceNumber, + items.Add(new OutboxMessageEnvelope(token, message)), + epoch); + + next.RegisterTimeProvider(time); + return next; + } + + /// + /// Removes the message identified by from the outbox. + /// + /// + /// Matches the full identity against the + /// first pending item. If the token is not the FIFO head, returns the same + /// instance unchanged. Does not affect + /// or . + /// + /// The token of the message to remove. + /// A new outbox without the specified message. + public Outbox Remove(OutboxSequenceToken token) + { + if (items.IsDefaultOrEmpty || items[0].Token != token) + { + return this; + } + + var next = new Outbox( + sender, + latestSequenceNumber, + items.RemoveAt(0), + epoch); + + next.RegisterTimeProvider(time); + return next; + } + + /// + /// Batch-removes messages identified by . + /// + /// + /// Removes the longest FIFO prefix whose tokens appear in + /// . Tokens not found at the head of the outbox + /// are silently ignored. + /// + /// The tokens of the messages to remove. + /// A new outbox without the specified messages. + public Outbox RemoveRange(IEnumerable tokens) + { + var tokenSet = tokens.ToHashSet(); + if (tokenSet.Count == 0) + { + return this; + } + + var removeCount = 0; + while (removeCount < items.Length && tokenSet.Contains(items[removeCount].Token)) + { + removeCount++; + } + + var remaining = items.RemoveRange(0, removeCount); + if (remaining.Length == items.Length) + { + return this; + } + + var next = new Outbox( + sender, + latestSequenceNumber, + remaining, + epoch); + next.RegisterTimeProvider(time); + return next; + } + + /// + /// Removes all pending messages but preserves + /// and . + /// + /// + /// This is the normal path after the postman has successfully drained all + /// items. The high-water mark persists so subsequent calls + /// continue the sequence without gaps. Receivers see monotonically increasing + /// sequence numbers within the same epoch. + /// + /// A new empty outbox preserving sequence metadata. + public Outbox Clear() + { + if (items.IsDefaultOrEmpty) + { + return this; + } + + var next = new Outbox( + sender, + latestSequenceNumber, + [], + epoch); + + next.RegisterTimeProvider(time); + return next; + } + + /// + /// O(1) equality over sender, sequence metadata, epoch, count, first + /// sequence number, and last sequence number. + /// + public bool Equals(Outbox? other) + { + return ReferenceEquals(this, other) + || (other is not null + && sender.Equals(other.sender) + && latestSequenceNumber == other.latestSequenceNumber + && epoch == other.epoch + && items.Length == other.items.Length + && FirstSequenceNumber == other.FirstSequenceNumber + && LastSequenceNumber == other.LastSequenceNumber); + } + + /// + public override bool Equals(object? obj) => obj is Outbox o && Equals(o); + + /// + public override int GetHashCode() + => HashCode.Combine( + sender, + latestSequenceNumber, + epoch, + items.Length, + FirstSequenceNumber, + LastSequenceNumber); + + private long FirstSequenceNumber => + items.IsDefaultOrEmpty ? 0 : items[0].Token.SequenceNumber; + + private long LastSequenceNumber => + items.IsDefaultOrEmpty ? 0 : items[^1].Token.SequenceNumber; +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs new file mode 100644 index 00000000..5e6b2258 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs @@ -0,0 +1,35 @@ +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging; + +/// +/// STJ converter factory that creates closed +/// instances for . +/// +/// +/// +/// Registered on via [JsonConverter]. STJ +/// discovers the attribute automatically — no user-side +/// configuration needed. +/// +/// +/// The converter serializes only the structural data needed to reconstruct +/// the outbox (sender, epoch, sequence numbers, items). Internal +/// bookkeeping fields (fingerprint cache, time provider) are excluded. +/// +/// +internal sealed class OutboxJsonConverterFactory : JsonConverterFactory +{ + /// + public override bool CanConvert(Type typeToConvert) + { + throw new NotImplementedException(); + } + + /// + public override JsonConverter? CreateConverter(Type typeToConvert, JsonSerializerOptions options) + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs new file mode 100644 index 00000000..f52b6675 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs @@ -0,0 +1,45 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Wraps a user-defined message with its +/// , forming the unit of storage and +/// dispatch within an . +/// +/// +/// +/// Immutability: Envelopes are immutable records. Retry diagnostics +/// (attempt count, last exception) are not stored on the envelope — +/// the tracks attempts in-memory, +/// keyed by . On grain reactivation, attempt counts +/// restart from zero. +/// +/// +/// Serialization: Decorated with [GenerateSerializer] for +/// Orleans and [JsonConverter] (via JsonConverterFactory) for +/// System.Text.Json. The factory creates a closed +/// JsonConverter<OutboxMessageEnvelope<T>> for the specific +/// at runtime. +/// +/// +/// Storage co-location: Envelopes live inside , +/// which lives inside the grain's state record. They are serialized as part +/// of the grain's single WriteStateAsync call — there is no separate +/// "outbox store." +/// +/// +/// +/// The user-defined message payload type. Must be serializable by both Orleans +/// ([GenerateSerializer]) and System.Text.Json if the storage provider +/// uses STJ. +/// +/// +/// The sequence token identifying this message. Assigned by +/// — never user-constructed. +/// +/// The user-defined message payload. +[GenerateSerializer] +[Alias("egil.orleans.messaging.OutboxMessageEnvelope`1")] +// [JsonConverter(typeof(OutboxMessageEnvelopeJsonConverterFactory))] +public sealed record OutboxMessageEnvelope( + [property: Id(0)] OutboxSequenceToken Token, + [property: Id(1)] T Message); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs new file mode 100644 index 00000000..1f4399d4 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs @@ -0,0 +1,30 @@ +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging; + +/// +/// STJ converter factory that creates closed +/// instances for . +/// +/// +/// +/// Registered on via +/// [JsonConverter]. STJ discovers the attribute automatically — +/// no user-side configuration needed. +/// +/// +internal sealed class OutboxMessageEnvelopeJsonConverterFactory : JsonConverterFactory +{ + /// + public override bool CanConvert(Type typeToConvert) + { + throw new NotImplementedException(); + } + + /// + public override JsonConverter? CreateConverter(Type typeToConvert, JsonSerializerOptions options) + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs new file mode 100644 index 00000000..60f17571 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs @@ -0,0 +1,170 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Grain-scoped component that owns the timer, reminder, and postman dispatch +/// lifecycle for draining an . +/// +/// +/// +/// Architecture: Each grain with an outbox gets its own +/// — no external scan, no registry, +/// no second store. +/// +/// GrainTimer for in-process fast retry while activated. +/// Durable Reminder for cross-activation recovery. Reactivates +/// the grain if it deactivates with pending items. +/// +/// +/// +/// Non-reentrant assumption: Relies on Orleans default turn-based +/// concurrency — adds no internal locks. Not safe on +/// [Reentrant] grains. Document and enforce via code review. +/// +/// +/// Postman dispatch: The grain registers one or more postmen via +/// , each handling a +/// subtype of . Matching is +/// first-registered-wins against the item's runtime type — register +/// from most specific to least specific (like a switch). +/// +/// Each item dispatches to exactly one postman. +/// Items whose runtime type matches no postman → reported as failed +/// with via +/// . +/// Per-item exceptions are caught and surfaced through +/// with +/// an in-memory attempt count that resets on grain reactivation. +/// +/// +/// +/// error contract: Only throws +/// (per-run timeout), +/// (caller token), or callback +/// exceptions from +/// / . Per-item +/// postman failures are swallowed and routed to the error callback. +/// +/// +/// Grain integration: Two obligations (both compiler-enforced): +/// +/// Implement . +/// Call InitializeOutboxProcessor(...) in OnActivateAsync. +/// +/// No ReceiveReminder override needed — the +/// DIM handles it. No manual timer/reminder lifecycle. No telemetry wiring. +/// +/// +/// Escape hatch: Grains with their own reminders can forward unknown +/// reminder names to : +/// +/// public async Task ReceiveReminder(string name, TickStatus status) +/// { +/// if (name == MyOwnReminder) { await DoMyWork(); return; } +/// await outboxProcessor.ReceiveReminderAsync(name, status); +/// } +/// +/// +/// +/// Telemetry: Emits to meter egil.orleans.messaging: +/// outbox.post.duration (histogram), outbox.post.item.duration +/// (histogram), outbox.post.items (counter), outbox.post.errors +/// (counter), outbox.depth (gauge). Tags: grain.type, +/// event.type, success. +/// +/// +/// +/// The base type of items in the outbox. Postmen can handle subtypes via +/// . +/// +public sealed partial class OutboxProcessor : IOutboxComponent + where TOutbox : notnull +{ + internal OutboxProcessor() { } + + /// + /// Registers a postman that handles items of type . + /// + /// + /// Order matters. Postmen are matched first-registered-wins against + /// the item's runtime type. Register from most specific to least specific. + /// Returns this for fluent chaining during OnActivateAsync. + /// + /// The subtype this postman handles. + /// + /// Async callback that delivers the item. Per-item exceptions are caught + /// and surfaced through . + /// + /// This processor for chaining. + public OutboxProcessor AddPostman( + Func postman) where TSub : TOutbox + { + throw new NotImplementedException(); + } + + /// + public OutboxProcessor AddPostman( + Func postman) where TSub : TOutbox + { + throw new NotImplementedException(); + } + + /// + /// + /// Async callback with cancellation support. The token is the per-run + /// timeout from . + /// + public OutboxProcessor AddPostman( + Func postman) where TSub : TOutbox + { + throw new NotImplementedException(); + } + + /// + /// Posts all pending items by dispatching each to its matching postman. + /// Arms timer/reminder if items remain; unregisters if the outbox is empty. + /// + /// + /// Safe to call from the grain's task scheduler (turn-based). Does not + /// throw for per-item postman failures — those are routed to + /// . Only + /// throws , + /// , or callback exceptions. + /// + /// Optional cancellation token. + public ValueTask PostAsync(CancellationToken cancellationToken = default) + { + throw new NotImplementedException(); + } + + /// + /// Called by the DIM when a reminder fires. + /// No-ops for reminder names not owned by this processor. + /// + /// The reminder name from Orleans. + /// The tick status from Orleans. + public ValueTask ReceiveReminderAsync(string reminderName, TickStatus status) + { + throw new NotImplementedException(); + } + + /// + /// Attaches this processor to the grain's as + /// a component so the DIM can discover it. + /// Called internally by InitializeOutboxProcessor. + /// + internal void AttachToGrain() + { + throw new NotImplementedException(); + } +} + +/// +/// Internal interface registered as a grain context component so the +/// DIM can forward reminder callbacks without +/// knowing the outbox generic type. +/// +internal interface IOutboxComponent +{ + /// Forwards a reminder callback to the outbox processor. + ValueTask ReceiveReminderAsync(string reminderName, TickStatus status); +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs new file mode 100644 index 00000000..ca6d69c4 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs @@ -0,0 +1,66 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Extension methods for wiring into +/// a grain's activation lifecycle. +/// +public static class OutboxProcessorExtensions +{ + /// + /// Creates and attaches an to the + /// grain. Call in OnActivateAsync and chain + /// + /// calls on the result. + /// + /// + /// + /// Registers the processor as an on the + /// grain context so the DIM can discover it + /// for reminder forwarding. + /// + /// + /// Usage: + /// + /// public override async Task OnActivateAsync(CancellationToken ct) + /// { + /// outboxProcessor = this.InitializeOutboxProcessor(new OutboxProcessorOptions<IMyEvent> + /// { + /// GetPending = () => stateManager.State.Outbox + /// .Select(e => e.Message).ToImmutableArray(), + /// OnPostCompletedAsync = async (items, ct) => + /// { + /// // remove delivered items and persist + /// }, + /// OnPostErrorAsync = async (failures, ct) => + /// { + /// // log, dead-letter, or leave for retry + /// }, + /// }) + /// .AddPostman<PriceCalculated>(PublishPriceCalculatedAsync) + /// .AddPostman<InvoiceReady>(SendInvoiceReadyAsync); + /// } + /// + /// + /// + /// + /// The grain type. Must implement (for reminder + /// DIM) and (for grain context access). Both + /// constraints are compiler-enforced. + /// + /// The base type of outbox items. + /// The grain instance (this). + /// Processor configuration. + /// + /// The processor instance. Chain AddPostman calls on it, then store + /// in a grain field for later + /// calls. + /// + public static OutboxProcessor InitializeOutboxProcessor( + this TGrain grain, + OutboxProcessorOptions options) + where TGrain : IGrainBase, IOutboxGrain + where TOutbox : notnull + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs new file mode 100644 index 00000000..96951a19 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs @@ -0,0 +1,98 @@ +using System.Collections.Immutable; + +namespace Egil.Orleans.Messaging; + +/// +/// Configuration for . Defines how the +/// processor reads pending items, reports successes/failures, and controls +/// timing. +/// +/// +/// +/// Callback model: The processor uses callbacks rather than DI services +/// so the grain controls dispatch logic and can pass its own state to the +/// handlers. This keeps the processor grain-scoped with no ambient dependencies. +/// +/// +/// Required callbacks: +/// +/// — snapshot of pending items, called once +/// per post run. +/// — items successfully posted. +/// The grain must remove them from its backing collection and persist. +/// +/// +/// +/// Optional callback: +/// +/// — failed items with exception and +/// attempt count. If null, failed items retry silently on the next +/// run. The grain decides: leave the item in state to retry, or remove it +/// to dead-letter after N attempts. +/// +/// +/// +/// +/// The base type of outbox items. Must match the type parameter of the +/// this options instance configures. +/// +public sealed class OutboxProcessorOptions + where TOutbox : notnull +{ + /// + /// Snapshot of pending items. Called once per post run. The processor + /// iterates the returned array and dispatches each item to its matching + /// postman. + /// + /// + /// Typically implemented as a lambda reading the grain's current outbox: + /// + /// GetPending = () => stateManager.State.Outbox.Select(e => e.Message).ToImmutableArray() + /// + /// + public required Func> GetPending { get; init; } + + /// + /// Called with the items that were successfully dispatched by their postmen. + /// The grain must remove these items from its backing collection (e.g., + /// ) and persist the updated state. + /// + /// + /// Exceptions thrown from this callback propagate out of + /// — the processor does + /// not catch callback failures. + /// + public required Func, CancellationToken, ValueTask> + OnPostCompletedAsync { get; init; } + + /// + /// Called with items that failed dispatch, along with the exception and + /// the in-memory attempt count (resets on grain reactivation). The grain + /// decides: leave the item in state to retry on the next run, or remove + /// it to dead-letter after N attempts. + /// + /// + /// If null, failed items retry silently on the next post run. + /// Exceptions thrown from this callback propagate out of + /// . + /// + public Func, + CancellationToken, ValueTask>? OnPostErrorAsync { get; init; } + + /// + /// Maximum time per post run. Set below the grain's response timeout to + /// avoid grain-level timeouts during dispatch. Default: 20 seconds. + /// + public TimeSpan ProcessingTimeout { get; init; } = TimeSpan.FromSeconds(20); + + /// + /// Timer and reminder period. Controls how frequently the processor + /// retries pending items. Default: 2 minutes. + /// + /// + /// Orleans reminders fire at most once per minute, so values below 1 + /// minute are effectively timer-only (the reminder still fires at its + /// minimum interval for cross-activation recovery). + /// + public TimeSpan RetryDelay { get; init; } = TimeSpan.FromMinutes(2); +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs new file mode 100644 index 00000000..5bb74d64 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs @@ -0,0 +1,59 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Identifies a specific message within an . Combines a +/// monotonic sequence number with the sender's identity and timing information +/// to form a globally unique, totally ordered token per sender. +/// +/// +/// +/// Identity: The token is the identity of an outbox message. Receivers +/// use and to detect duplicates +/// via . +/// +/// +/// Epoch semantics: changes only when the sender +/// calls (nuclear reset). When a receiver sees +/// a token with Epoch > stored.Epoch, it accepts unconditionally, +/// knowing the sender intentionally reset its sequence space. Same-epoch +/// tokens are compared by — higher wins, equal +/// or lower is a duplicate. +/// +/// +/// Ordering: Within a single sender, tokens are totally ordered by +/// (Epoch, SequenceNumber). Across different senders, tokens are not +/// comparable — use to partition. +/// +/// +/// Serialization: Decorated with [GenerateSerializer] for +/// Orleans and [JsonConverter] for System.Text.Json. Round-trips +/// correctly through any Orleans storage provider using either serializer. +/// +/// +/// +/// Monotonically increasing sequence number within a single . +/// Assigned by — callers cannot fabricate or choose +/// sequence numbers. Starts at 1 for each new epoch. +/// +/// +/// The of the grain that owns the outbox. Receivers use +/// this to partition dedup tracking per sender. +/// +/// +/// Wall-clock time when the message was added to the outbox. Informational — +/// not used for ordering or dedup. Stamped by the sender's +/// . +/// +/// +/// Opaque marker that changes only on (full +/// sequence-space reset). Receivers compare epochs to detect resets. A token +/// with a newer epoch supersedes all prior sequence numbers from the same sender. +/// +[GenerateSerializer] +[Alias("egil.orleans.messaging.OutboxSequenceToken")] +// [JsonConverter(typeof(OutboxSequenceTokenJsonConverter))] +public sealed record OutboxSequenceToken( + [property: Id(0)] long SequenceNumber, + [property: Id(1)] GrainId Sender, + [property: Id(2)] DateTimeOffset Timestamp, + [property: Id(3)] DateTimeOffset Epoch); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs new file mode 100644 index 00000000..a15ac735 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs @@ -0,0 +1,32 @@ +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging; + +/// +/// STJ converter for . Serializes and +/// deserializes the token's , +/// , and +/// properties. +/// +/// +/// +/// Registered on via [JsonConverter]. +/// STJ discovers the attribute automatically — no user-side +/// configuration needed. +/// +/// +internal sealed class OutboxSequenceTokenJsonConverter : JsonConverter +{ + /// + public override OutboxSequenceToken? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + throw new NotImplementedException(); + } + + /// + public override void Write(Utf8JsonWriter writer, OutboxSequenceToken value, JsonSerializerOptions options) + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs new file mode 100644 index 00000000..efc4b5e4 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs @@ -0,0 +1,124 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Default implementation of . Wraps an +/// and adds committed-state fencing, +/// version stamping for -derived types, and +/// automatic write recovery. +/// +/// +/// +/// Committed-state fence: always returns the +/// last successfully written value. During , the +/// underlying .State is mutated, but the +/// caller's view is updated only after the write succeeds. On failure, the +/// recovery path re-reads from storage to determine whether the write +/// actually landed. +/// +/// +/// Version stamping: If derives from +/// , the manager stamps a fresh +/// on the +/// property before every write. The recovery path then compares versions +/// directly (pattern-matched via is VersionedState) instead of +/// relying on , which avoids the +/// +/// reference-equality trap. +/// +/// +/// Recovery matrix: +/// +/// +/// Failure +/// Outcome after WriteAsync +/// +/// +/// Success +/// State == newState, returns normally. +/// +/// +/// Write throws, re-read succeeds, state matches +/// Lost-response — write landed. Exception swallowed. +/// +/// +/// Write throws, re-read succeeds, state does not match +/// Write genuinely failed. Original exception rethrown. +/// +/// +/// Write throws, re-read also throws +/// Double failure. State reverts to pre-write snapshot. +/// Original exception rethrown. Caller must call +/// before the next write to re-sync. +/// +/// +/// +/// +/// Concurrency: (default) uses +/// the storage provider's ETag for optimistic concurrency. Writes against +/// stale ETags fail with InconsistentStateException. +/// nulls the ETag before writing, bypassing +/// the concurrency check — use only for admin repair operations. +/// +/// +/// Thread safety: Relies on Orleans turn-based concurrency. Not safe +/// for [Reentrant] grains unless is only read +/// (never written) from interleaved calls. +/// +/// +/// +/// +/// +internal sealed class StateManager : IStateManager + where T : class, IEquatable +{ + private readonly IPersistentState storage; + + /// + /// Creates a new wrapping the given + /// facet. + /// + /// + /// The Orleans persistent state facet to wrap. The manager takes full + /// ownership — callers must not read or write + /// directly after wrapping. + /// + internal StateManager(IPersistentState storage) + { + this.storage = storage; + } + + /// + public T State + { + get => throw new NotImplementedException(); + } + + /// + public Task ReadAsync() + { + throw new NotImplementedException(); + } + + /// + public Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concurrent) + { + throw new NotImplementedException(); + } + + /// + public Task ClearAsync() + { + throw new NotImplementedException(); + } + + /// + /// Determines whether the persisted state matches the attempted write. + /// For -derived types, compares + /// directly. For plain types, falls + /// back to . + /// + private static bool IsEquivalent(T persisted, T attempted) + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs new file mode 100644 index 00000000..4df55e39 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs @@ -0,0 +1,58 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Provides the extension method that wraps an +/// in an . +/// +public static class StateManagerExtensions +{ + /// + /// Wraps the given in an + /// that provides atomic write recovery and a committed-state fence. + /// See for full behavioral contract, recovery + /// semantics, and deep-immutability requirements. + /// + /// + /// + /// Call site: Typically called once in OnActivateAsync: + /// + /// [PersistentState("state")] IPersistentState<MyState> storage; + /// IStateManager<MyState> stateManager; + /// + /// public override Task OnActivateAsync(CancellationToken ct) + /// { + /// stateManager = storage.AsStateManager(); + /// // ... + /// } + /// + /// + /// + /// Important: After calling this method, the grain should not access + /// directly. Doing so bypasses the committed-state + /// fence. See remarks for details. + /// + /// + /// Provider-specific overrides (future): In v1, this always returns + /// the default StateManager<T>. In a future version, this method + /// will check ActivationServices for a registered + /// IStateManagerFactory to resolve provider-specific implementations + /// that can classify exceptions more precisely (e.g., "definitely did not + /// persist" vs. "unknown outcome") and skip the re-read when safe. + /// + /// + /// + /// + /// The Orleans-managed persistent state facet, typically injected via + /// [PersistentState("name")]. Must already be hydrated (Orleans + /// hydrates it during the SetupState lifecycle stage, before + /// OnActivateAsync). + /// + /// + /// An wrapping . + /// + public static IStateManager AsStateManager(this IPersistentState storage) + where T : class, IEquatable + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs new file mode 100644 index 00000000..d1967d0a --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs @@ -0,0 +1,125 @@ +using Orleans.Streams; + +namespace Egil.Orleans.Messaging; + +/// +/// Wraps a and its associated +/// into a single value that uses for dedup and +/// uses for resume. +/// +/// +/// +/// Projection constraint: is an abstract +/// type. The library ships an STJ converter that handles a closed set of known +/// subtypes via a $kind discriminator: +/// +/// EventSequenceToken — Orleans SimpleMessageStream +/// EventHubSequenceToken — Orleans Event Hub provider v1 +/// EventHubSequenceTokenV2 — Orleans Event Hub provider v2 +/// — library-shipped v2 +/// subclass carrying +/// +/// Unknown subtypes throw at serialization time — silently dropping the cursor +/// would corrupt dedup state. +/// +/// +/// Enriched time: Use to extract the +/// broker-side enqueue time when the token is an . +/// Returns false for all other token types. This enables end-to-end lag +/// histograms in the telemetry without coupling the +/// cursor type to a specific streaming provider. +/// +/// +/// Serialization: Decorated with [GenerateSerializer] for Orleans +/// and [JsonConverter] for System.Text.Json. The STJ converter handles +/// the polymorphic via the discriminator. +/// +/// +/// The Orleans stream identity. +/// +/// The opaque sequence token for resumption. May be null when no prior +/// position exists (subscribe from provider default). +/// +[GenerateSerializer] +[Alias("egil.orleans.messaging.StreamCursor")] +// [JsonConverter(typeof(StreamCursorJsonConverter))] +public sealed record StreamCursor( + [property: Id(0)] StreamId StreamId, + [property: Id(1)] StreamSequenceToken? Token) +{ + /// + /// Attempts to extract the broker-side enqueue time from the underlying + /// . + /// + /// + /// Returns true only when the token is an + /// . Callers can use this to + /// compute end-to-end lag (now - enqueuedTime) without coupling + /// to a specific streaming provider. + /// + /// + /// The time the event was enqueued at the broker, or default if + /// the token type does not carry this information. + /// + /// + /// true if is an + /// and the time was extracted; + /// false otherwise. + /// + public bool TryGetEnqueuedTime(out DateTimeOffset enqueuedTime) + { + throw new NotImplementedException(); + } + + /// + /// Attempts to extract the stream provider name from the underlying + /// . + /// + /// + /// Returns true only when the token is an + /// . Enables provider-aware + /// dedup and diagnostics without coupling to a specific streaming provider. + /// + /// + /// The name of the stream provider that delivered this event, or + /// null if the token type does not carry this information. + /// + /// + /// true if is an + /// and the name was extracted; + /// false otherwise. + /// + public bool TryGetStreamProviderName([NotNullWhen(true)] out string? streamProviderName) + { + throw new NotImplementedException(); + } + + /// + /// Attempts to extract the W3C traceparent value from the underlying + /// . + /// + /// + /// Returns true only when the token is an + /// whose + /// is not null. + /// uses this to create + /// s — correlating consumer + /// spans to producer spans without creating multi-hour parent-child traces. + /// + /// + /// The W3C traceparent value from the producer-side + /// , or null if the token + /// type does not carry this information or no activity was active at + /// publish time. + /// + /// + /// true if is an + /// with a non-null + /// ; false + /// otherwise. + /// + public bool TryGetTraceParent([NotNullWhen(true)] out string? traceParent) + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs new file mode 100644 index 00000000..3fcee0b9 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs @@ -0,0 +1,40 @@ +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging; + +/// +/// STJ converter for . Serializes and +/// deserializes the cursor's and +/// sequence token. +/// +/// +/// +/// wraps a StreamId and an optional +/// StreamSequenceToken. The token is polymorphic — it may be an +/// , +/// EventHubSequenceTokenV2, +/// or another provider-specific subclass. The converter round-trips the +/// concrete type via a discriminator so deserialization restores the +/// original token type. +/// +/// +/// Registered on via [JsonConverter]. +/// STJ discovers the attribute automatically — no user-side +/// configuration needed. +/// +/// +internal sealed class StreamCursorJsonConverter : JsonConverter +{ + /// + public override StreamCursor? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + throw new NotImplementedException(); + } + + /// + public override void Write(Utf8JsonWriter writer, StreamCursor value, JsonSerializerOptions options) + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs new file mode 100644 index 00000000..31bd7a41 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs @@ -0,0 +1,161 @@ +using Orleans.Streams; + +namespace Egil.Orleans.Messaging; + +/// +/// Grain-level facade around Orleans implicit subscriptions. Manages subscribe, +/// resume, dispatch, and per-subscription error handling. +/// +/// +/// +/// Composition over inheritance: The grain inherits from Grain +/// and uses as a field — no base-class coupling. +/// +/// +/// Typical wiring: +/// +/// public override async Task OnActivateAsync(CancellationToken ct) +/// { +/// streamManager = this.InitializeStreamManager(state.Tracker) +/// .Subscribe("electricity-prices", HandlePriceTickAsync, LogStreamError) +/// .Subscribe("tariff-events", HandleTariffChangedAsync); +/// } +/// +/// +/// +/// Resume semantics: each subscription reads +/// trackerSnapshot.LatestStream(streamId) once per subscription. +/// If a cursor exists, the is passed to +/// SubscribeAsync. If null, subscribes without a token +/// (provider default, typically: start from current). The tracker is read +/// once at subscription time, never again — the handler is the only path that +/// mutates the tracker. +/// +/// +/// OTel trace correlation: The OnNext wrapper reads +/// when the stream +/// token is enriched, parses it into an ActivityContext, and starts +/// the handler span with ActivityKind.Consumer and an +/// ActivityLink back to the producer. This produces separate traces +/// per delivery without collapsing weeks of traffic into one distributed trace. +/// +/// +/// Telemetry: Emits counters for messages delivered per +/// (streamNamespace, accepted|rejected), subscriptions +/// established/torn-down/errored, histograms for handler latency, and +/// (when is available) +/// end-to-end lag. +/// +/// +public sealed class StreamManager +{ + private StreamManager() { } + + /// + /// Creates a for the given grain. + /// + /// The grain that owns this stream manager. + /// + /// A snapshot of the grain's at activation + /// time. Used to look up resume tokens. Read once per subscription — not + /// held afterward. + /// + /// A for fluent subscription configuration. + internal static StreamManager Create( + IGrainBase owner, + MessageTracker trackerSnapshot) + { + throw new NotImplementedException(); + } + + /// + /// Declares a subscription to the given . + /// + /// + /// The expected event type on this stream namespace. Must match the type + /// published by the producer. Usually inferred from + /// ; specify explicitly only for inline + /// lambdas or ambiguous method groups. + /// + /// + /// The Orleans stream namespace to subscribe to. Must follow the + /// one-provider-per-namespace convention (code-review enforced, not + /// runtime enforced). + /// + /// + /// Receives the deserialized event and the + /// representing this event's position. The handler should update the grain's + /// and persist state before returning. + /// + /// + /// Optional synchronous error handler. Receives the stream namespace and the + /// exception. If omitted, the default behavior is to log and emit a counter + /// without rethrowing. + /// + /// + /// When true (default), the subscription looks up the last accepted + /// from the snapshot + /// and passes its token to SubscribeAsync for resumption. When + /// false, subscribes from the provider's default position (typically: + /// current/latest). + /// + /// + /// This instance for fluent subscription chaining. + /// + public StreamManager Subscribe( + string streamNamespace, + Func onNextAsync, + Action? onError = default, + bool passLatestSequenceTokenOnResume = true) + { + throw new NotImplementedException(); + } + + /// + /// Declares a subscription to the given + /// using a -returning event handler. + /// + /// + /// Convenience overload for handlers that already return . + /// Implementations should adapt this to the overload. + /// + /// + /// The expected event type on this stream namespace. Must match the type + /// published by the producer. Usually inferred from + /// ; specify explicitly only for inline + /// lambdas or ambiguous method groups. + /// + /// + /// The Orleans stream namespace to subscribe to. Must follow the + /// one-provider-per-namespace convention (code-review enforced, not + /// runtime enforced). + /// + /// + /// Receives the deserialized event and the + /// representing this event's position. The handler should update the grain's + /// and persist state before returning. + /// + /// + /// Optional synchronous error handler. Receives the stream namespace and the + /// exception. If omitted, the default behavior is to log and emit a counter + /// without rethrowing. + /// + /// + /// When true (default), the subscription looks up the last accepted + /// from the snapshot + /// and passes its token to SubscribeAsync for resumption. When + /// false, subscribes from the provider's default position (typically: + /// current/latest). + /// + /// + /// This instance for fluent subscription chaining. + /// + public StreamManager Subscribe( + string streamNamespace, + Func onNextAsync, + Action? onError = default, + bool passLatestSequenceTokenOnResume = true) + { + throw new NotImplementedException(); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs new file mode 100644 index 00000000..cece3fc3 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs @@ -0,0 +1,49 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Extension methods for wiring into a grain's +/// activation lifecycle. +/// +public static class StreamManagerExtensions +{ + /// + /// Creates a for the grain and returns it for + /// fluent stream subscription configuration. + /// + /// + /// + /// Call from OnActivateAsync, then chain + /// Subscribe calls on the returned manager. + /// + /// + /// Usage: + /// + /// public override async Task OnActivateAsync(CancellationToken ct) + /// { + /// streamManager = this.InitializeStreamManager(state.Tracker) + /// .Subscribe("electricity-prices", HandlePriceTickAsync); + /// } + /// + /// + /// + /// + /// The grain type. Must implement so stream providers + /// and telemetry infrastructure can be resolved through activation services. + /// + /// The grain instance (this). + /// + /// A snapshot of the grain's at activation + /// time. Used to look up resume tokens. + /// + /// + /// The initialized . Store it in a grain field so + /// the subscriptions remain rooted for the activation lifetime. + /// + public static StreamManager InitializeStreamManager( + this TGrain grain, + MessageTracker trackerSnapshot) + where TGrain : IGrainBase + { + return StreamManager.Create(grain, trackerSnapshot); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/VersionedState.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/VersionedState.cs new file mode 100644 index 00000000..2ad14842 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/VersionedState.cs @@ -0,0 +1,64 @@ +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging; + +/// +/// Abstract base for version-stamped grain state. Provides the +/// property that +/// uses for its write-recovery comparison. +/// +/// +/// +/// detects this base type at runtime via +/// pattern matching (if (newState is VersionedState v)). If the state +/// derives from , the manager stamps a fresh +/// on before every +/// write and compares directly on the recovery path. +/// This means the recovery comparison does not use +/// T.Equals() for -derived types, avoiding +/// problems with types like +/// whose Equals uses reference equality. For non- +/// types the manager falls back to T.Equals(). +/// +/// +/// has internal set — library code can stamp it; +/// user code cannot. This is a hard compile-time fence. The set (not +/// init) accessor allows mutation of the caller's reference during +/// . +/// +/// +/// Usage: +/// +/// [GenerateSerializer] +/// public sealed record MyState : VersionedState +/// { +/// [Id(0)] public Outbox<MyEvent> Outbox { get; init; } +/// [Id(1)] public ImmutableArray<Something> Items { get; init; } = []; +/// } +/// +/// +/// +[GenerateSerializer] +[Alias("egil.orleans.messaging.VersionedState")] +public abstract record VersionedState +{ + /// + /// Gets the version identifier stamped by + /// on every successful write. UUID v7, sortable, collision-free at grain + /// write rates. + /// + /// + /// + /// Fresh records get a non-empty version from the initializer. The manager + /// overwrites it with a new v7 UUID before each WriteAsync call. + /// + /// + /// For System.Text.Json serialization: the internal set accessor + /// is invisible to STJ by default. The + /// makes it visible without requiring a custom converter. + /// + /// + [Id(0)] + [JsonInclude] + public Guid Version { get; internal set; } = Guid.CreateVersion7(); +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/WritePolicy.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/WritePolicy.cs new file mode 100644 index 00000000..c64c1fd9 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/WritePolicy.cs @@ -0,0 +1,24 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Controls how handles optimistic concurrency. +/// +[GenerateSerializer] +[Alias("egil.orleans.messaging.WritePolicy")] +public enum WritePolicy +{ + /// + /// Default. Uses the storage provider's ETag-based optimistic concurrency check. + /// If another writer changed the state since the last read, the write fails with + /// InconsistentStateException. + /// + Concurrent = 0, + + /// + /// Nulls the ETag before writing, bypassing the provider's concurrency check. + /// Use as a last-resort escape hatch when a grain needs to force-overwrite + /// state regardless of concurrent modifications (e.g., admin repair operations). + /// The recovery path and version stamping still run identically. + /// + Force = 1, +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj index a26f041e..01209a6d 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj @@ -24,6 +24,7 @@ + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs new file mode 100644 index 00000000..34dfc056 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs @@ -0,0 +1,210 @@ +using TimeProviderExtensions; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class OutboxTests +{ + [Fact] + public void Add_appends_message_with_next_sequence_sender_and_time() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var outbox = Outbox.Create(sender); + outbox.RegisterTimeProvider(time); + + var next = outbox.Add("created"); + + Assert.Empty(outbox); + Assert.Single(next); + Assert.Equal(1, next.LatestSequenceNumber); + Assert.Equal(now, next.Epoch); + Assert.Equal("created", next[0].Message); + Assert.Equal(new OutboxSequenceToken(1, sender, now, now), next[0].Token); + } + + [Fact] + public void Add_preserves_epoch_and_increments_sequence_for_later_messages() + { + var sender = GrainId.Create("test/sender", "one"); + var epoch = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var later = epoch.AddMinutes(5); + var time = new ManualTimeProvider(epoch); + var outbox = Outbox.Create(sender); + outbox.RegisterTimeProvider(time); + + var next = outbox.Add("first"); + time.Advance(TimeSpan.FromMinutes(5)); + next = next.Add("second"); + + Assert.Equal(2, next.Count); + Assert.Equal(2, next.LatestSequenceNumber); + Assert.Equal(epoch, next.Epoch); + Assert.Equal(new OutboxSequenceToken(1, sender, epoch, epoch), next[0].Token); + Assert.Equal(new OutboxSequenceToken(2, sender, later, epoch), next[1].Token); + } + + [Fact] + public void Remove_deletes_matching_sequence_and_preserves_sequence_metadata() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var outbox = Outbox.Create(sender); + outbox.RegisterTimeProvider(time); + outbox = outbox.Add("first").Add("second"); + + var next = outbox.Remove(outbox[0].Token); + + Assert.Single(next); + Assert.Equal("second", next[0].Message); + Assert.Equal(2, next.LatestSequenceNumber); + Assert.Equal(now, next.Epoch); + } + + [Fact] + public void Remove_ignores_token_from_different_sender_with_same_sequence() + { + var sender = GrainId.Create("test/sender", "one"); + var otherSender = GrainId.Create("test/sender", "two"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var outbox = Outbox.Create(sender); + outbox.RegisterTimeProvider(time); + outbox = outbox.Add("first"); + var otherToken = new OutboxSequenceToken(1, otherSender, now, now); + + var next = outbox.Remove(otherToken); + + Assert.Same(outbox, next); + Assert.Single(next); + } + + [Fact] + public void Remove_ignores_matching_token_that_is_not_fifo_head() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var outbox = Outbox.Create(sender); + outbox.RegisterTimeProvider(time); + outbox = outbox.Add("first").Add("second"); + + var next = outbox.Remove(outbox[1].Token); + + Assert.Same(outbox, next); + Assert.Equal(2, next.Count); + } + + [Fact] + public void RemoveRange_deletes_matching_fifo_prefix_and_ignores_missing_tokens() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var outbox = Outbox.Create(sender); + outbox.RegisterTimeProvider(time); + outbox = outbox.Add("first").Add("second").Add("third"); + var missing = new OutboxSequenceToken(99, sender, now, now); + + var next = outbox.RemoveRange([outbox[0].Token, outbox[1].Token, missing]); + + Assert.Single(next); + Assert.Equal("third", next[0].Message); + Assert.Equal(3, next.LatestSequenceNumber); + Assert.Equal(now, next.Epoch); + } + + [Fact] + public void RemoveRange_stops_at_first_token_gap_to_preserve_fifo_order() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var outbox = Outbox.Create(sender); + outbox.RegisterTimeProvider(time); + outbox = outbox.Add("first").Add("second").Add("third"); + + var next = outbox.RemoveRange([outbox[0].Token, outbox[2].Token]); + + Assert.Equal(2, next.Count); + Assert.Equal("second", next[0].Message); + Assert.Equal("third", next[1].Message); + } + + [Fact] + public void Clear_removes_items_and_keeps_sequence_metadata_for_next_add() + { + var sender = GrainId.Create("test/sender", "one"); + var epoch = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var later = epoch.AddMinutes(5); + var time = new ManualTimeProvider(epoch); + var outbox = Outbox.Create(sender); + outbox.RegisterTimeProvider(time); + outbox = outbox.Add("first").Add("second"); + + var cleared = outbox.Clear(); + time.Advance(TimeSpan.FromMinutes(5)); + var next = cleared.Add("third"); + + Assert.Empty(cleared); + Assert.Equal(2, cleared.LatestSequenceNumber); + Assert.Equal(epoch, cleared.Epoch); + Assert.Equal(new OutboxSequenceToken(3, sender, later, epoch), next[0].Token); + } + + [Fact] + public void Equals_treats_matching_sequence_window_as_same_pending_items() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var leftTime = new ManualTimeProvider(now); + var rightTime = new ManualTimeProvider(now); + var left = Outbox.Create(sender); + var right = Outbox.Create(sender); + left.RegisterTimeProvider(leftTime); + right.RegisterTimeProvider(rightTime); + + left = left.Add("left").Add("middle-left").Add("last"); + right = right.Add("right").Add("middle-right").Add("last"); + + Assert.Equal(left, right); + } + + [Fact] + public void Equals_returns_true_when_sender_sequence_epoch_and_envelopes_match() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var leftTime = new ManualTimeProvider(now); + var rightTime = new ManualTimeProvider(now); + var left = Outbox.Create(sender); + var right = Outbox.Create(sender); + left.RegisterTimeProvider(leftTime); + right.RegisterTimeProvider(rightTime); + + left = left.Add("first").Add("second"); + right = right.Add("first").Add("second"); + + Assert.Equal(left, right); + Assert.Equal(left.GetHashCode(), right.GetHashCode()); + } + + [Fact] + public void Equals_returns_false_when_sequence_metadata_differs() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var leftTime = new ManualTimeProvider(now); + var rightTime = new ManualTimeProvider(now); + var left = Outbox.Create(sender); + var right = Outbox.Create(sender); + left.RegisterTimeProvider(leftTime); + right.RegisterTimeProvider(rightTime); + + left = left.Add("first").Add("second"); + right = right.Add("first").Add("second").Add("third"); + + Assert.NotEqual(left, right); + } +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/xunit.runner.json b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/xunit.runner.json new file mode 100644 index 00000000..da63501b --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/xunit.runner.json @@ -0,0 +1,5 @@ +{ + "$schema": "https://xunit.net/schema/current/xunit.runner.schema.json", + "methodDisplay": "method", + "methodDisplayOptions": "all" +} \ No newline at end of file From 30773950ba8254fd8cf25fd44934857f58ead003 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sat, 23 May 2026 22:13:11 +0000 Subject: [PATCH 04/82] feat(om): Outbox implemented --- .../src/Egil.Orleans.Messaging/Outbox.cs | 2 +- .../OutboxJsonConverterFactory.cs | 79 ++++++++++++++++++- .../OutboxJsonConverterFactoryTests.cs | 55 +++++++++++++ .../OutboxTests.cs | 1 + 4 files changed, 133 insertions(+), 4 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs index 2eba2bd9..fdeb1694 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs @@ -73,7 +73,7 @@ namespace Egil.Orleans.Messaging; /// [GenerateSerializer] [Alias("egil.orleans.messaging.Outbox`1")] -// [JsonConverter(typeof(OutboxJsonConverterFactory))] +[JsonConverter(typeof(OutboxJsonConverterFactory))] public sealed class Outbox : IReadOnlyList>, IEquatable> { [Id(0)] private readonly GrainId sender; diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs index 5e6b2258..b9dce726 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs @@ -1,3 +1,4 @@ +using System.Collections.Immutable; using System.Text.Json; using System.Text.Json.Serialization; @@ -16,7 +17,7 @@ namespace Egil.Orleans.Messaging; /// /// The converter serializes only the structural data needed to reconstruct /// the outbox (sender, epoch, sequence numbers, items). Internal -/// bookkeeping fields (fingerprint cache, time provider) are excluded. +/// non-restorable service references such as the time provider are excluded. /// /// internal sealed class OutboxJsonConverterFactory : JsonConverterFactory @@ -24,12 +25,84 @@ internal sealed class OutboxJsonConverterFactory : JsonConverterFactory /// public override bool CanConvert(Type typeToConvert) { - throw new NotImplementedException(); + return typeToConvert.IsGenericType + && typeToConvert.GetGenericTypeDefinition() == typeof(Outbox<>); } /// public override JsonConverter? CreateConverter(Type typeToConvert, JsonSerializerOptions options) { - throw new NotImplementedException(); + var messageType = typeToConvert.GetGenericArguments()[0]; + var converterType = typeof(OutboxJsonConverter<>).MakeGenericType(messageType); + + return (JsonConverter?)Activator.CreateInstance(converterType); + } + + private sealed class OutboxJsonConverter : JsonConverter> + { + public override Outbox? Read( + ref Utf8JsonReader reader, + Type typeToConvert, + JsonSerializerOptions options) + { + var model = JsonSerializer.Deserialize>(ref reader, options); + if (model is null) + { + return null; + } + + return new Outbox( + GrainId.Create(model.Sender.Type, model.Sender.Key), + model.LatestSequenceNumber, + [.. model.Items.Select(static item => new OutboxMessageEnvelope( + new OutboxSequenceToken( + item.Token.SequenceNumber, + GrainId.Create(item.Token.Sender.Type, item.Token.Sender.Key), + item.Token.Timestamp, + item.Token.Epoch), + item.Message))], + model.Epoch); + } + + public override void Write( + Utf8JsonWriter writer, + Outbox value, + JsonSerializerOptions options) + { + var model = new OutboxJsonModel( + ToJsonModel(value.Sender), + value.LatestSequenceNumber, + value.Epoch, + [.. value.Select(static item => new OutboxMessageEnvelopeJsonModel( + new OutboxSequenceTokenJsonModel( + item.Token.SequenceNumber, + ToJsonModel(item.Token.Sender), + item.Token.Timestamp, + item.Token.Epoch), + item.Message))]); + + JsonSerializer.Serialize(writer, model, options); + } } + + private sealed record OutboxJsonModel( + GrainIdJsonModel Sender, + long LatestSequenceNumber, + DateTimeOffset? Epoch, + ImmutableArray> Items); + + private sealed record OutboxMessageEnvelopeJsonModel( + OutboxSequenceTokenJsonModel Token, + T Message); + + private sealed record OutboxSequenceTokenJsonModel( + long SequenceNumber, + GrainIdJsonModel Sender, + DateTimeOffset Timestamp, + DateTimeOffset Epoch); + + private sealed record GrainIdJsonModel(string Type, string Key); + + private static GrainIdJsonModel ToJsonModel(GrainId grainId) => + new(grainId.Type.ToString()!, grainId.Key.ToString()!); } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs new file mode 100644 index 00000000..8e46862e --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs @@ -0,0 +1,55 @@ +using System.Text.Json; +using TimeProviderExtensions; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class OutboxJsonConverterFactoryTests +{ + [Theory] + [InlineData("first", "second")] + public void JsonSerializer_round_trips_outbox_with_string_messages(string first, string second) + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var outbox = Outbox.Create(sender); + outbox.RegisterTimeProvider(time); + outbox = outbox.Add(first).Add(second); + + var json = JsonSerializer.Serialize(outbox); + var roundTripped = JsonSerializer.Deserialize>(json); + + Assert.NotNull(roundTripped); + Assert.Equal(outbox, roundTripped); + Assert.Equal(first, roundTripped[0].Message); + Assert.Equal(second, roundTripped[1].Message); + } + + [Fact] + public void JsonSerializer_round_trips_outbox_with_complex_messages() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var outbox = Outbox.Create(sender); + outbox.RegisterTimeProvider(time); + outbox = outbox.Add(new ComplexMessage( + "order-17", + 42, + new NestedMessage("north", true))); + + var json = JsonSerializer.Serialize(outbox); + var roundTripped = JsonSerializer.Deserialize>(json); + + Assert.NotNull(roundTripped); + Assert.Equal(outbox, roundTripped); + Assert.Equal("order-17", roundTripped[0].Message.OrderId); + Assert.Equal(42, roundTripped[0].Message.Quantity); + Assert.Equal("north", roundTripped[0].Message.Route.Name); + Assert.True(roundTripped[0].Message.Route.IsExpress); + } + + private sealed record ComplexMessage(string OrderId, int Quantity, NestedMessage Route); + + private sealed record NestedMessage(string Name, bool IsExpress); +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs index 34dfc056..0584cf7c 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs @@ -207,4 +207,5 @@ public void Equals_returns_false_when_sequence_metadata_differs() Assert.NotEqual(left, right); } + } From fac3c9559a09f2741f05965d91071cceb8eb18e0 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 08:49:30 +0000 Subject: [PATCH 05/82] feat(om): MessageTracker --- Egil.Orleans.Messaging/api-design.md | 4 +- .../EnrichedEventHubSequenceToken.cs | 6 +- .../Egil.Orleans.Messaging/MessageTracker.cs | 191 +++++++++++++-- .../MessageTrackerJsonConverter.cs | 222 ++++++++++++++++- .../MessageTrackerJsonConverterTests.cs | 128 ++++++++++ .../MessageTrackerTests.cs | 231 ++++++++++++++++++ 6 files changed, 756 insertions(+), 26 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index d380c42d..8fbd9737 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -567,6 +567,8 @@ each upstream source. Two source kinds: ```csharp [GenerateSerializer] +[Alias("egil.orleans.messaging.MessageTracker")] +[JsonConverter(typeof(MessageTrackerJsonConverter))] public sealed class MessageTracker { [Id(0)] private ImmutableDictionary streams; @@ -633,7 +635,7 @@ public sealed class MessageTracker ### `Evict` — uniform cleanup -Five overloads, one rule: remove entries where `entry.Received < olderThan`. +Five overloads, one rule: remove entries where `entry.Received <= olderThan`. No separate `Forget` API — `Evict(id, DateTimeOffset.MaxValue)` is the documented idiom for unconditional clear. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs index aed2805b..a2d8dda0 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs @@ -92,7 +92,7 @@ public class EnrichedEventHubSequenceToken : EventHubSequenceTokenV2 /// The index of the event within a batch at the same sequence number. /// /// - /// The (UTC) when the event was enqueued at the + /// The when the event was enqueued at the /// Event Hub broker. /// /// @@ -107,12 +107,12 @@ public EnrichedEventHubSequenceToken( string offset, long sequenceNumber, int eventIndex, - DateTime enqueuedTime, + DateTimeOffset enqueuedTime, string streamProviderName, string? traceParent = null) : base(offset, sequenceNumber, eventIndex) { - EnqueuedTime = new DateTimeOffset(enqueuedTime, TimeSpan.Zero); + EnqueuedTime = enqueuedTime; StreamProviderName = streamProviderName; TraceParent = traceParent; } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs index 12b8a5ba..0411013e 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs @@ -1,5 +1,6 @@ using System.Collections.Immutable; using System.Text.Json.Serialization; +using Orleans.Streams; namespace Egil.Orleans.Messaging; @@ -42,7 +43,7 @@ namespace Egil.Orleans.Messaging; /// /// /// Eviction: Five overloads, one rule — remove entries where -/// entry.Received < olderThan. No separate Forget API. +/// entry.Received <= olderThan. No separate Forget API. /// Evict(id, DateTimeOffset.MaxValue) is the documented idiom for /// unconditional removal of a single source entry. /// @@ -60,7 +61,7 @@ namespace Egil.Orleans.Messaging; /// [GenerateSerializer] [Alias("egil.orleans.messaging.MessageTracker")] -// [JsonConverter(typeof(MessageTrackerJsonConverter))] +[JsonConverter(typeof(MessageTrackerJsonConverter))] public sealed class MessageTracker : IEquatable { [Id(0)] private readonly ImmutableDictionary streams; @@ -83,7 +84,7 @@ public MessageTracker() outbox = ImmutableDictionary.Empty; } - private MessageTracker( + internal MessageTracker( ImmutableDictionary streams, ImmutableDictionary outbox) { @@ -109,7 +110,22 @@ private MessageTracker( /// true if accepted (new message); false if duplicate. public bool ProcessMessage(StreamCursor cursor, out MessageTracker next) { - throw new NotImplementedException(); + var now = time.GetUtcNow(); + + if (!streams.TryGetValue(cursor.StreamId, out var entry)) + { + next = CreateTracker(streams.Add(cursor.StreamId, new StreamEntry(cursor, now)), outbox); + return true; + } + + if (!IsNewer(cursor.Token, entry.LastPosition.Token)) + { + next = this; + return false; + } + + next = CreateTracker(streams.SetItem(cursor.StreamId, new StreamEntry(cursor, now)), outbox); + return true; } /// @@ -125,7 +141,28 @@ public bool ProcessMessage(StreamCursor cursor, out MessageTracker next) /// true if accepted; false if duplicate or stale. public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) { - throw new NotImplementedException(); + var now = time.GetUtcNow(); + + if (!outbox.TryGetValue(token.Sender, out var entry)) + { + next = CreateTracker(streams, outbox.Add(token.Sender, new OutboxEntry(token.Epoch, token.SequenceNumber, now))); + return true; + } + + if (token.Epoch > entry.Epoch) + { + next = CreateTracker(streams, outbox.SetItem(token.Sender, new OutboxEntry(token.Epoch, token.SequenceNumber, now))); + return true; + } + + if (token.Epoch == entry.Epoch && token.SequenceNumber > entry.LastSequenceNumber) + { + next = CreateTracker(streams, outbox.SetItem(token.Sender, new OutboxEntry(entry.Epoch, token.SequenceNumber, now))); + return true; + } + + next = this; + return false; } /// @@ -135,7 +172,7 @@ public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) /// public StreamCursor? LatestStream(StreamId stream) { - throw new NotImplementedException(); + return streams.TryGetValue(stream, out var entry) ? entry.LastPosition : null; } /// @@ -145,69 +182,185 @@ public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) /// public OutboxSequenceToken? LatestOutbox(GrainId sender) { - throw new NotImplementedException(); + return outbox.TryGetValue(sender, out var entry) + ? new OutboxSequenceToken(entry.LastSequenceNumber, sender, entry.Received, entry.Epoch) + : null; } /// /// Removes all entries (both stream and outbox) where - /// entry.Received < . + /// entry.Received <= . /// public MessageTracker Evict(DateTimeOffset olderThan) { - throw new NotImplementedException(); + var newStreams = FilterByReceived(streams, olderThan, static entry => entry.Received, out var streamsChanged); + var newOutbox = FilterByReceived(outbox, olderThan, static entry => entry.Received, out var outboxChanged); + + return streamsChanged || outboxChanged + ? CreateTracker(newStreams, newOutbox) + : this; } /// /// Removes stream entries where - /// entry.Received < . + /// entry.Received <= . /// Outbox entries are unaffected. /// public MessageTracker EvictStreams(DateTimeOffset olderThan) { - throw new NotImplementedException(); + var newStreams = FilterByReceived(streams, olderThan, static entry => entry.Received, out var changed); + return changed ? CreateTracker(newStreams, outbox) : this; } /// /// Removes outbox entries where - /// entry.Received < . + /// entry.Received <= . /// Stream entries are unaffected. /// public MessageTracker EvictOutboxes(DateTimeOffset olderThan) { - throw new NotImplementedException(); + var newOutbox = FilterByReceived(outbox, olderThan, static entry => entry.Received, out var changed); + return changed ? CreateTracker(streams, newOutbox) : this; } /// /// Removes the entry for the given if - /// entry.Received < . + /// entry.Received <= . /// Use DateTimeOffset.MaxValue to unconditionally remove. /// public MessageTracker Evict(StreamId stream, DateTimeOffset olderThan) { - throw new NotImplementedException(); + if (!streams.TryGetValue(stream, out var entry) || entry.Received > olderThan) + { + return this; + } + + return CreateTracker(streams.Remove(stream), outbox); } /// /// Removes the entry for the given outbox if - /// entry.Received < . + /// entry.Received <= . /// Use DateTimeOffset.MaxValue to unconditionally remove. /// public MessageTracker Evict(GrainId sender, DateTimeOffset olderThan) { - throw new NotImplementedException(); + if (!outbox.TryGetValue(sender, out var entry) || entry.Received > olderThan) + { + return this; + } + + return CreateTracker(streams, outbox.Remove(sender)); } /// public bool Equals(MessageTracker? other) { - throw new NotImplementedException(); + if (ReferenceEquals(this, other)) + { + return true; + } + + if (other is null || streams.Count != other.streams.Count || outbox.Count != other.outbox.Count) + { + return false; + } + + foreach (var item in streams) + { + if (!other.streams.TryGetValue(item.Key, out var otherEntry) || !item.Value.Equals(otherEntry)) + { + return false; + } + } + + foreach (var item in outbox) + { + if (!other.outbox.TryGetValue(item.Key, out var otherEntry) || !item.Value.Equals(otherEntry)) + { + return false; + } + } + + return true; } /// public override bool Equals(object? obj) => obj is MessageTracker o && Equals(o); /// - public override int GetHashCode() => HashCode.Combine(streams.Count, outbox.Count); + public override int GetHashCode() + { + return HashCode.Combine( + streams.Count, + AggregateHash(streams), + outbox.Count, + AggregateHash(outbox)); + } + + private MessageTracker CreateTracker( + ImmutableDictionary streams, + ImmutableDictionary outbox) + { + var next = new MessageTracker(streams, outbox) + { + time = time + }; + + return next; + } + + private static bool IsNewer(StreamSequenceToken? candidate, StreamSequenceToken? stored) + { + if (stored is null) + { + return candidate is not null; + } + + if (candidate is null) + { + return false; + } + + return StreamSequenceTokenUtilities.Newer(candidate, stored); + } + + private static ImmutableDictionary FilterByReceived( + ImmutableDictionary source, + DateTimeOffset olderThan, + Func receivedSelector, + out bool changed) + where TKey : notnull + { + var builder = ImmutableDictionary.CreateBuilder(); + changed = false; + + foreach (var item in source) + { + if (receivedSelector(item.Value) <= olderThan) + { + changed = true; + continue; + } + + builder.Add(item); + } + + return changed ? builder.ToImmutable() : source; + } + + private static int AggregateHash(ImmutableDictionary dictionary) + where TKey : notnull + { + var hash = 0; + + foreach (var item in dictionary) + { + hash ^= HashCode.Combine(item.Key, item.Value); + } + + return hash; + } /// /// Internal entry tracking a stream source's last known position and diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs index f06ecdf1..b1c09a3f 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs @@ -1,5 +1,11 @@ +using System.Collections.Immutable; +using System.Reflection; +using System.Text; using System.Text.Json; using System.Text.Json.Serialization; +using Orleans.Providers.Streams.Common; +using Orleans.Streaming.EventHubs; +using Orleans.Streams; namespace Egil.Orleans.Messaging; @@ -18,21 +24,231 @@ namespace Egil.Orleans.Messaging; /// /// /// Registered on via [JsonConverter]. -/// STJ discovers the attribute automatically — no user-side +/// STJ discovers the attribute automatically - no user-side /// configuration needed. /// /// internal sealed class MessageTrackerJsonConverter : JsonConverter { + private static readonly FieldInfo StreamsField = typeof(MessageTracker).GetField( + "streams", + BindingFlags.Instance | BindingFlags.NonPublic)!; + + private static readonly FieldInfo OutboxField = typeof(MessageTracker).GetField( + "outbox", + BindingFlags.Instance | BindingFlags.NonPublic)!; + /// public override MessageTracker? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { - throw new NotImplementedException(); + var model = JsonSerializer.Deserialize(ref reader, options); + if (model is null) + { + return null; + } + + var streams = ImmutableDictionary.CreateBuilder(); + foreach (var item in model.Streams ?? Array.Empty()) + { + streams.Add( + ParseStreamId(item.StreamId), + new MessageTracker.StreamEntry( + FromJsonModel(item.LastPosition), + item.Received)); + } + + var outbox = ImmutableDictionary.CreateBuilder(); + foreach (var item in model.Outboxes ?? Array.Empty()) + { + outbox.Add( + ParseGrainId(item.Sender), + new MessageTracker.OutboxEntry( + item.Epoch, + item.LastSequenceNumber, + item.Received)); + } + + return new MessageTracker(streams.ToImmutable(), outbox.ToImmutable()); } /// public override void Write(Utf8JsonWriter writer, MessageTracker value, JsonSerializerOptions options) { - throw new NotImplementedException(); + var streams = GetStreams(value); + var outbox = GetOutbox(value); + + var streamModels = new StreamEntryJsonModel[streams.Count]; + var streamIndex = 0; + foreach (var item in streams) + { + streamModels[streamIndex++] = new StreamEntryJsonModel( + item.Key.ToString(), + ToJsonModel(item.Value.LastPosition), + item.Value.Received); + } + + var outboxModels = new OutboxEntryJsonModel[outbox.Count]; + var outboxIndex = 0; + foreach (var item in outbox) + { + outboxModels[outboxIndex++] = new OutboxEntryJsonModel( + ToJsonModel(item.Key), + item.Value.Epoch, + item.Value.LastSequenceNumber, + item.Value.Received); + } + + JsonSerializer.Serialize( + writer, + new MessageTrackerJsonModel(streamModels, outboxModels), + options); + } + + private static ImmutableDictionary GetStreams(MessageTracker value) => + (ImmutableDictionary)StreamsField.GetValue(value)!; + + private static ImmutableDictionary GetOutbox(MessageTracker value) => + (ImmutableDictionary)OutboxField.GetValue(value)!; + + private static StreamId ParseStreamId(string value) => + StreamId.Parse(Encoding.UTF8.GetBytes(value)); + + private static GrainId ParseGrainId(GrainIdJsonModel value) => + GrainId.Create(value.Type, value.Key); + + private static StreamCursor FromJsonModel(StreamCursorJsonModel model) => + new(ParseStreamId(model.StreamId), FromJsonModel(model.Token)); + + private static StreamSequenceToken? FromJsonModel(StreamSequenceTokenJsonModel? model) + { + if (model is null) + { + return null; + } + + return model.Kind switch + { + StreamSequenceTokenKinds.EventSequenceToken => new EventSequenceToken(model.SequenceNumber, model.EventIndex), + StreamSequenceTokenKinds.EventSequenceTokenV2 => new EventSequenceTokenV2(model.SequenceNumber, model.EventIndex), + StreamSequenceTokenKinds.EventHubSequenceToken => new EventHubSequenceToken( + model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), + model.SequenceNumber, + model.EventIndex), + StreamSequenceTokenKinds.EventHubSequenceTokenV2 => new EventHubSequenceTokenV2( + model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), + model.SequenceNumber, + model.EventIndex), + StreamSequenceTokenKinds.EnrichedEventHubSequenceToken => new EnrichedEventHubSequenceToken( + model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), + model.SequenceNumber, + model.EventIndex, + model.EnqueuedTime ?? throw new JsonException("Missing EnqueuedTime."), + model.StreamProviderName ?? throw new JsonException("Missing StreamProviderName."), + model.TraceParent), + _ => throw new JsonException($"Unsupported stream sequence token kind '{model.Kind}'.") + }; + } + + private static StreamCursorJsonModel ToJsonModel(StreamCursor value) => + new( + value.StreamId.ToString(), + ToJsonModel(value.Token)); + + private static StreamSequenceTokenJsonModel? ToJsonModel(StreamSequenceToken? value) + { + if (value is null) + { + return null; + } + + return value switch + { + EnrichedEventHubSequenceToken token => new StreamSequenceTokenJsonModel( + StreamSequenceTokenKinds.EnrichedEventHubSequenceToken, + token.SequenceNumber, + token.EventIndex, + token.EventHubOffset, + token.EnqueuedTime, + token.StreamProviderName, + token.TraceParent), + EventHubSequenceTokenV2 token => new StreamSequenceTokenJsonModel( + StreamSequenceTokenKinds.EventHubSequenceTokenV2, + token.SequenceNumber, + token.EventIndex, + token.EventHubOffset, + null, + null, + null), + EventHubSequenceToken token => new StreamSequenceTokenJsonModel( + StreamSequenceTokenKinds.EventHubSequenceToken, + token.SequenceNumber, + token.EventIndex, + token.EventHubOffset, + null, + null, + null), + EventSequenceTokenV2 token => new StreamSequenceTokenJsonModel( + StreamSequenceTokenKinds.EventSequenceTokenV2, + token.SequenceNumber, + token.EventIndex, + null, + null, + null, + null), + EventSequenceToken token => new StreamSequenceTokenJsonModel( + StreamSequenceTokenKinds.EventSequenceToken, + token.SequenceNumber, + token.EventIndex, + null, + null, + null, + null), + _ => throw new NotSupportedException( + $"Unsupported stream sequence token type '{value.GetType().FullName}'.") + }; + } + + private static GrainIdJsonModel ToJsonModel(GrainId grainId) => + new(grainId.Type.ToString()!, grainId.Key.ToString()!); + + private sealed record MessageTrackerJsonModel( + StreamEntryJsonModel[]? Streams, + OutboxEntryJsonModel[]? Outboxes); + + private sealed record StreamEntryJsonModel( + string StreamId, + StreamCursorJsonModel LastPosition, + DateTimeOffset Received); + + private sealed record OutboxEntryJsonModel( + GrainIdJsonModel Sender, + DateTimeOffset Epoch, + long LastSequenceNumber, + DateTimeOffset Received); + + private sealed record StreamCursorJsonModel( + string StreamId, + StreamSequenceTokenJsonModel? Token); + + private sealed record StreamSequenceTokenJsonModel( + string Kind, + long SequenceNumber, + int EventIndex, + string? EventHubOffset, + DateTimeOffset? EnqueuedTime, + string? StreamProviderName, + string? TraceParent); + + private sealed record GrainIdJsonModel( + string Type, + string Key); + + private static class StreamSequenceTokenKinds + { + public const string EventSequenceToken = "event-sequence"; + public const string EventSequenceTokenV2 = "event-sequence-v2"; + public const string EventHubSequenceToken = "event-hub"; + public const string EventHubSequenceTokenV2 = "event-hub-v2"; + public const string EnrichedEventHubSequenceToken = "enriched-event-hub"; } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs new file mode 100644 index 00000000..eb9f6a3e --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs @@ -0,0 +1,128 @@ +using System.Text.Json; +using Orleans.Providers.Streams.Common; +using Orleans.Streaming.EventHubs; +using Orleans.Streams; +using TimeProviderExtensions; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class MessageTrackerJsonConverterTests +{ + public static TheoryData SupportedStreamTokens => new() + { + new EventSequenceToken(7, 1), + new EventSequenceTokenV2(8, 2), + new EventHubSequenceToken("42", 10, 3), + new EventHubSequenceTokenV2("43", 11, 4), + new EnrichedEventHubSequenceToken( + offset: "44", + sequenceNumber: 12, + eventIndex: 5, + enqueuedTime: new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero), + streamProviderName: "provider-a", + traceParent: "00-abc-xyz-01") + }; + + [Theory] + [MemberData(nameof(SupportedStreamTokens))] + public void JsonSerializer_round_trips_message_tracker_stream_tokens(StreamSequenceToken token) + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var streamId = StreamId.Create("orders", token.GetType().Name); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(now)); + tracker.ProcessMessage(new StreamCursor(streamId, token), out tracker); + + var json = JsonSerializer.Serialize(tracker); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(tracker, roundTripped); + Assert.Equal(new StreamCursor(streamId, token), roundTripped.LatestStream(streamId)); + } + + [Theory] + [InlineData(1L)] + [InlineData(42L)] + public void JsonSerializer_round_trips_message_tracker_outbox_tokens(long sequenceNumber) + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var sender = GrainId.Create("test/sender", "one"); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(now)); + tracker.ProcessMessage(new OutboxSequenceToken(sequenceNumber, sender, now, now), out tracker); + + var json = JsonSerializer.Serialize(tracker); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(tracker, roundTripped); + Assert.Equal( + new OutboxSequenceToken(sequenceNumber, sender, now, now), + roundTripped.LatestOutbox(sender)); + } + + [Fact] + public void JsonSerializer_throws_for_unsupported_stream_token_type() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var streamId = StreamId.Create("orders", "unsupported"); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(now)); + tracker.ProcessMessage(new StreamCursor(streamId, new UnsupportedSequenceToken(1, 0)), out tracker); + + Assert.Throws(() => JsonSerializer.Serialize(tracker)); + } + + [Fact] + public void JsonSerializer_throws_for_unknown_stream_token_kind() + { + var json = """ + { + "Streams": [ + { + "StreamId": "orders/one", + "LastPosition": { + "StreamId": "orders/one", + "Token": { + "Kind": "unknown-kind", + "SequenceNumber": 1, + "EventIndex": 0 + } + }, + "Received": "2026-05-23T12:30:00+00:00" + } + ], + "Outboxes": [] + } + """; + + Assert.Throws(() => JsonSerializer.Deserialize(json)); + } + + private sealed class UnsupportedSequenceToken(long sequenceNumber, int eventIndex) : StreamSequenceToken + { + public override long SequenceNumber { get; protected set; } = sequenceNumber; + public override int EventIndex { get; protected set; } = eventIndex; + + public override bool Equals(StreamSequenceToken? other) => other is UnsupportedSequenceToken token + && token.SequenceNumber == SequenceNumber + && token.EventIndex == EventIndex; + + public override int CompareTo(StreamSequenceToken? other) + { + if (other is null) + { + return 1; + } + + var sequenceComparison = SequenceNumber.CompareTo(other.SequenceNumber); + if (sequenceComparison != 0) + { + return sequenceComparison; + } + + return EventIndex.CompareTo(other.EventIndex); + } + } +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs new file mode 100644 index 00000000..d2e30013 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs @@ -0,0 +1,231 @@ +using Orleans.Providers.Streams.Common; +using TimeProviderExtensions; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class MessageTrackerTests +{ + [Fact] + public void ProcessMessage_accepts_first_stream_cursor_and_tracks_latest_position() + { + var streamId = StreamId.Create("orders", "one"); + var cursor = new StreamCursor(streamId, new EventSequenceToken(7)); + var time = new ManualTimeProvider(new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero)); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + + var accepted = tracker.ProcessMessage(cursor, out var next); + + Assert.True(accepted); + Assert.Null(tracker.LatestStream(streamId)); + Assert.Equal(cursor, next.LatestStream(streamId)); + Assert.NotSame(tracker, next); + } + + [Fact] + public void ProcessMessage_accepts_first_outbox_token_and_tracks_latest_position() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var sender = GrainId.Create("test/sender", "one"); + var time = new ManualTimeProvider(now); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + var token = new OutboxSequenceToken(1, sender, now, now); + + var accepted = tracker.ProcessMessage(token, out var next); + + Assert.True(accepted); + Assert.Null(tracker.LatestOutbox(sender)); + Assert.Equal(token, next.LatestOutbox(sender)); + } + + [Fact] + public void Evict_removes_stream_and_outbox_entries_at_or_before_cutoff() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var streamId = StreamId.Create("orders", "one"); + var sender = GrainId.Create("test/sender", "one"); + var time = new ManualTimeProvider(now); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + tracker.ProcessMessage(new OutboxSequenceToken(1, sender, now, now), out tracker); + + var evicted = tracker.Evict(now); + + Assert.Null(evicted.LatestStream(streamId)); + Assert.Null(evicted.LatestOutbox(sender)); + } + + [Fact] + public void ProcessMessage_rejects_stream_cursor_when_token_is_not_newer() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var streamId = StreamId.Create("orders", "one"); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + + var accepted = tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out var next); + + Assert.False(accepted); + Assert.Same(tracker, next); + } + + [Fact] + public void ProcessMessage_accepts_stream_cursor_when_token_is_newer() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var streamId = StreamId.Create("orders", "one"); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + + var accepted = tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(8)), out var next); + + Assert.True(accepted); + Assert.Equal(new StreamCursor(streamId, new EventSequenceToken(8)), next.LatestStream(streamId)); + } + + [Fact] + public void ProcessMessage_rejects_outbox_token_when_same_epoch_and_not_newer_sequence() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var sender = GrainId.Create("test/sender", "one"); + var time = new ManualTimeProvider(now); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new OutboxSequenceToken(5, sender, now, now), out tracker); + + var accepted = tracker.ProcessMessage(new OutboxSequenceToken(5, sender, now, now), out var next); + + Assert.False(accepted); + Assert.Same(tracker, next); + } + + [Fact] + public void ProcessMessage_rejects_outbox_token_when_epoch_is_stale() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var sender = GrainId.Create("test/sender", "one"); + var oldEpoch = now.AddMinutes(-10); + var newEpoch = now; + var time = new ManualTimeProvider(now); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new OutboxSequenceToken(1, sender, now, newEpoch), out tracker); + + var accepted = tracker.ProcessMessage(new OutboxSequenceToken(99, sender, now, oldEpoch), out var next); + + Assert.False(accepted); + Assert.Same(tracker, next); + } + + [Fact] + public void ProcessMessage_accepts_outbox_token_when_epoch_is_newer() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var sender = GrainId.Create("test/sender", "one"); + var oldEpoch = now.AddMinutes(-10); + var newEpoch = now; + var time = new ManualTimeProvider(now); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new OutboxSequenceToken(10, sender, now, oldEpoch), out tracker); + + var accepted = tracker.ProcessMessage(new OutboxSequenceToken(1, sender, now, newEpoch), out var next); + + Assert.True(accepted); + Assert.Equal(new OutboxSequenceToken(1, sender, now, newEpoch), next.LatestOutbox(sender)); + } + + [Fact] + public void EvictStreams_only_removes_stream_entries() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var streamId = StreamId.Create("orders", "one"); + var sender = GrainId.Create("test/sender", "one"); + var time = new ManualTimeProvider(now); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + tracker.ProcessMessage(new OutboxSequenceToken(1, sender, now, now), out tracker); + + var evicted = tracker.EvictStreams(now); + + Assert.Null(evicted.LatestStream(streamId)); + Assert.Equal(new OutboxSequenceToken(1, sender, now, now), evicted.LatestOutbox(sender)); + } + + [Fact] + public void EvictOutboxes_only_removes_outbox_entries() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var streamId = StreamId.Create("orders", "one"); + var sender = GrainId.Create("test/sender", "one"); + var time = new ManualTimeProvider(now); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + tracker.ProcessMessage(new OutboxSequenceToken(1, sender, now, now), out tracker); + + var evicted = tracker.EvictOutboxes(now); + + Assert.Equal(new StreamCursor(streamId, new EventSequenceToken(7)), evicted.LatestStream(streamId)); + Assert.Null(evicted.LatestOutbox(sender)); + } + + [Fact] + public void Evict_stream_entry_respects_cutoff_for_targeted_remove() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var streamId = StreamId.Create("orders", "one"); + var time = new ManualTimeProvider(now); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + + var notEvicted = tracker.Evict(streamId, now.AddTicks(-1)); + var evicted = tracker.Evict(streamId, now); + + Assert.Equal(new StreamCursor(streamId, new EventSequenceToken(7)), notEvicted.LatestStream(streamId)); + Assert.Null(evicted.LatestStream(streamId)); + } + + [Fact] + public void Evict_outbox_entry_respects_cutoff_for_targeted_remove() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var sender = GrainId.Create("test/sender", "one"); + var time = new ManualTimeProvider(now); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new OutboxSequenceToken(1, sender, now, now), out tracker); + + var notEvicted = tracker.Evict(sender, now.AddTicks(-1)); + var evicted = tracker.Evict(sender, now); + + Assert.Equal(new OutboxSequenceToken(1, sender, now, now), notEvicted.LatestOutbox(sender)); + Assert.Null(evicted.LatestOutbox(sender)); + } + + [Fact] + public void Equals_returns_false_when_tracked_entries_differ() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var sender = GrainId.Create("test/sender", "one"); + var streamId = StreamId.Create("orders", "one"); + var time = new ManualTimeProvider(now); + var left = new MessageTracker(); + var right = new MessageTracker(); + left.RegisterTimeProvider(time); + right.RegisterTimeProvider(time); + left.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out left); + right.ProcessMessage(new OutboxSequenceToken(1, sender, now, now), out right); + + Assert.NotEqual(left, right); + } + +} From 16107a9163c0b9338fd20bed1f3d77ccfe35a1f9 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 10:27:36 +0000 Subject: [PATCH 06/82] fix(om): restrict StreamCursor JSON tokens to built-in Orleans types Limit StreamCursorJsonConverter to Orleans built-in stream sequence token types and remove custom token adapter serialization paths. Unsupported token kinds now throw a NotSupportedException with an actionable message that explains the stability rationale and points users to the GitHub issues page for feature requests. Update StreamCursorJsonConverterTests to assert unsupported custom tokens and unknown token kinds fail with the new guidance message while keeping built-in token round-trip coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../StreamCursorJsonConverter.cs | 131 +++++++++++++++++- .../StreamCursorJsonConverterTests.cs | 124 +++++++++++++++++ 2 files changed, 249 insertions(+), 6 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs index 3fcee0b9..5e9a8351 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs @@ -1,5 +1,8 @@ using System.Text.Json; using System.Text.Json.Serialization; +using Orleans.Providers.Streams.Common; +using Orleans.Streaming.EventHubs; +using Orleans.Streams; namespace Egil.Orleans.Messaging; @@ -11,12 +14,12 @@ namespace Egil.Orleans.Messaging; /// /// /// wraps a StreamId and an optional -/// StreamSequenceToken. The token is polymorphic — it may be an +/// StreamSequenceToken. The token is polymorphic and this converter +/// supports Orleans built-in sequence token types only: /// , /// EventHubSequenceTokenV2, -/// or another provider-specific subclass. The converter round-trips the -/// concrete type via a discriminator so deserialization restores the -/// original token type. +/// EventHubSequenceToken, EventSequenceTokenV2, and +/// EventSequenceToken. /// /// /// Registered on via [JsonConverter]. @@ -26,15 +29,131 @@ namespace Egil.Orleans.Messaging; /// internal sealed class StreamCursorJsonConverter : JsonConverter { + private const string FeatureRequestUrl = "https://github.com/egil/framework/issues"; + + private const string SupportedKinds = + "EventSequenceToken, EventSequenceTokenV2, EventHubSequenceToken, EventHubSequenceTokenV2, EnrichedEventHubSequenceToken"; + /// public override StreamCursor? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { - throw new NotImplementedException(); + var model = JsonSerializer.Deserialize(ref reader, options); + if (model is null) + { + return null; + } + + return new StreamCursor( + StreamId.Parse(System.Text.Encoding.UTF8.GetBytes(model.StreamId)), + FromJsonModel(model.Token)); } /// public override void Write(Utf8JsonWriter writer, StreamCursor value, JsonSerializerOptions options) { - throw new NotImplementedException(); + JsonSerializer.Serialize( + writer, + new StreamCursorJsonModel( + value.StreamId.ToString(), + ToJsonModel(value.Token)), + options); + } + + private static StreamSequenceToken? FromJsonModel(StreamSequenceTokenJsonModel? model) + { + if (model is null) + { + return null; + } + + return model.Kind switch + { + StreamSequenceTokenKinds.EventSequenceToken => new EventSequenceToken(model.SequenceNumber, model.EventIndex), + StreamSequenceTokenKinds.EventSequenceTokenV2 => new EventSequenceTokenV2(model.SequenceNumber, model.EventIndex), + StreamSequenceTokenKinds.EventHubSequenceToken => new EventHubSequenceToken( + model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), + model.SequenceNumber, + model.EventIndex), + StreamSequenceTokenKinds.EventHubSequenceTokenV2 => new EventHubSequenceTokenV2( + model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), + model.SequenceNumber, + model.EventIndex), + StreamSequenceTokenKinds.EnrichedEventHubSequenceToken => new EnrichedEventHubSequenceToken( + model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), + model.SequenceNumber, + model.EventIndex, + model.EnqueuedTime ?? throw new JsonException("Missing EnqueuedTime."), + model.StreamProviderName ?? throw new JsonException("Missing StreamProviderName."), + model.TraceParent), + _ => throw CreateUnsupportedTokenException(model.Kind ?? "") + }; + } + + private static StreamSequenceTokenJsonModel? ToJsonModel(StreamSequenceToken? value) + { + if (value is null) + { + return null; + } + + return value switch + { + EnrichedEventHubSequenceToken token => new StreamSequenceTokenJsonModel( + Kind: StreamSequenceTokenKinds.EnrichedEventHubSequenceToken, + SequenceNumber: token.SequenceNumber, + EventIndex: token.EventIndex, + EventHubOffset: token.EventHubOffset, + EnqueuedTime: token.EnqueuedTime.UtcDateTime, + StreamProviderName: token.StreamProviderName, + TraceParent: token.TraceParent), + EventHubSequenceTokenV2 token => new StreamSequenceTokenJsonModel( + Kind: StreamSequenceTokenKinds.EventHubSequenceTokenV2, + SequenceNumber: token.SequenceNumber, + EventIndex: token.EventIndex, + EventHubOffset: token.EventHubOffset), + EventHubSequenceToken token => new StreamSequenceTokenJsonModel( + Kind: StreamSequenceTokenKinds.EventHubSequenceToken, + SequenceNumber: token.SequenceNumber, + EventIndex: token.EventIndex, + EventHubOffset: token.EventHubOffset), + EventSequenceTokenV2 token => new StreamSequenceTokenJsonModel( + Kind: StreamSequenceTokenKinds.EventSequenceTokenV2, + SequenceNumber: token.SequenceNumber, + EventIndex: token.EventIndex), + EventSequenceToken token => new StreamSequenceTokenJsonModel( + Kind: StreamSequenceTokenKinds.EventSequenceToken, + SequenceNumber: token.SequenceNumber, + EventIndex: token.EventIndex), + _ => throw CreateUnsupportedTokenException(value.GetType().FullName ?? value.GetType().Name) + }; + } + + private static NotSupportedException CreateUnsupportedTokenException(string tokenIdentifier) => + new( + $"Unsupported stream sequence token '{tokenIdentifier}'. " + + $"This converter supports only built-in Orleans token types: {SupportedKinds}. " + + "Custom token support is intentionally disabled to keep persisted StreamCursor JSON format stable. " + + $"If you need this feature, please request it at {FeatureRequestUrl}."); + + private sealed record StreamCursorJsonModel( + string StreamId, + StreamSequenceTokenJsonModel? Token); + + private sealed record StreamSequenceTokenJsonModel( + string? Kind = null, + long SequenceNumber = 0, + int EventIndex = 0, + string? EventHubOffset = null, + DateTime? EnqueuedTime = null, + string? StreamProviderName = null, + string? TraceParent = null); + + private static class StreamSequenceTokenKinds + { + public const string EventSequenceToken = "event-sequence"; + public const string EventSequenceTokenV2 = "event-sequence-v2"; + public const string EventHubSequenceToken = "event-hub"; + public const string EventHubSequenceTokenV2 = "event-hub-v2"; + public const string EnrichedEventHubSequenceToken = "enriched-event-hub"; } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs new file mode 100644 index 00000000..2acc27bf --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs @@ -0,0 +1,124 @@ +using System.Text.Json; +using System.Text.Json.Serialization; +using Orleans.Providers.Streams.Common; +using Orleans.Streaming.EventHubs; +using Orleans.Streams; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class StreamCursorJsonConverterTests +{ + [Fact] + public void StreamCursor_is_decorated_with_stream_cursor_json_converter() + { + var attribute = typeof(StreamCursor).GetCustomAttributes(typeof(JsonConverterAttribute), inherit: false) + .Cast() + .Single(); + + Assert.Equal(typeof(StreamCursorJsonConverter), attribute.ConverterType); + } + + [Fact] + public void JsonSerializer_round_trips_stream_cursor_with_event_sequence_token() + { + var streamId = StreamId.Create("orders", "one"); + var cursor = new StreamCursor(streamId, new EventSequenceToken(7, 1)); + + var json = JsonSerializer.Serialize(cursor); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(cursor, roundTripped); + } + + [Fact] + public void JsonSerializer_round_trips_stream_cursor_with_event_hub_token_v2() + { + var streamId = StreamId.Create("orders", "eh-v2"); + var cursor = new StreamCursor(streamId, new EventHubSequenceTokenV2("42", 10, 3)); + + var json = JsonSerializer.Serialize(cursor); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(cursor, roundTripped); + } + + [Fact] + public void JsonSerializer_round_trips_stream_cursor_with_enriched_event_hub_token() + { + var streamId = StreamId.Create("orders", "eh-enriched"); + var cursor = new StreamCursor( + streamId, + new EnrichedEventHubSequenceToken( + "43", + 11, + 4, + new DateTimeOffset(2026, 5, 24, 18, 30, 0, TimeSpan.Zero), + "provider-a", + "00-abc-xyz-01")); + + var json = JsonSerializer.Serialize(cursor); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(cursor, roundTripped); + } + + [Fact] + public void JsonSerializer_throws_for_stream_cursor_payload_with_unknown_token_kind() + { + var json = """ + { + "StreamId": "orders/one", + "Token": { + "Kind": "unknown-kind", + "SequenceNumber": 1, + "EventIndex": 0 + } + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); + Assert.Contains("https://github.com/egil/framework/issues", exception.Message); + Assert.Contains("supports only built-in Orleans token types", exception.Message); + } + + [Fact] + public void JsonSerializer_throws_for_custom_token_type() + { + var cursor = new StreamCursor( + StreamId.Create("orders", "custom"), + new UnsupportedCustomSequenceToken(21, 3)); + + var exception = Assert.Throws(() => JsonSerializer.Serialize(cursor)); + Assert.Contains("https://github.com/egil/framework/issues", exception.Message); + Assert.Contains("supports only built-in Orleans token types", exception.Message); + } + + private sealed class UnsupportedCustomSequenceToken(long sequenceNumber, int eventIndex) : StreamSequenceToken + { + public override long SequenceNumber { get; protected set; } = sequenceNumber; + public override int EventIndex { get; protected set; } = eventIndex; + + public override bool Equals(StreamSequenceToken? other) => other is UnsupportedCustomSequenceToken token + && token.SequenceNumber == SequenceNumber + && token.EventIndex == EventIndex; + + public override int CompareTo(StreamSequenceToken? other) + { + if (other is null) + { + return 1; + } + + var sequenceComparison = SequenceNumber.CompareTo(other.SequenceNumber); + if (sequenceComparison != 0) + { + return sequenceComparison; + } + + return EventIndex.CompareTo(other.EventIndex); + } + } +} From 45a27e266120ca63306287df92c985d4d0868b50 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 10:29:06 +0000 Subject: [PATCH 07/82] feat(om): implement OutboxSequenceToken STJ converter Enable JSON converter attribute on OutboxSequenceToken and implement OutboxSequenceTokenJsonConverter read/write paths. The converter now round-trips sequence, sender GrainId shape, timestamp, and epoch without requiring caller-side serializer options. Add focused tests for converter attribute registration, round-trip behavior (including multiple GrainId key shapes), and payload validation for missing sender. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../OutboxSequenceToken.cs | 4 +- .../OutboxSequenceTokenJsonConverter.cs | 34 +++++++++- .../OutboxSequenceTokenJsonConverterTests.cs | 68 +++++++++++++++++++ 3 files changed, 103 insertions(+), 3 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxSequenceTokenJsonConverterTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs index 5bb74d64..98870e21 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs @@ -1,3 +1,5 @@ +using System.Text.Json.Serialization; + namespace Egil.Orleans.Messaging; /// @@ -51,7 +53,7 @@ namespace Egil.Orleans.Messaging; /// [GenerateSerializer] [Alias("egil.orleans.messaging.OutboxSequenceToken")] -// [JsonConverter(typeof(OutboxSequenceTokenJsonConverter))] +[JsonConverter(typeof(OutboxSequenceTokenJsonConverter))] public sealed record OutboxSequenceToken( [property: Id(0)] long SequenceNumber, [property: Id(1)] GrainId Sender, diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs index a15ac735..fe0489ca 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs @@ -21,12 +21,42 @@ internal sealed class OutboxSequenceTokenJsonConverter : JsonConverter public override OutboxSequenceToken? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { - throw new NotImplementedException(); + var model = JsonSerializer.Deserialize(ref reader, options); + if (model is null) + { + return null; + } + + if (model.Sender is null) + { + throw new JsonException("Missing Sender."); + } + + return new OutboxSequenceToken( + model.SequenceNumber, + GrainId.Create(model.Sender.Type, model.Sender.Key), + model.Timestamp, + model.Epoch); } /// public override void Write(Utf8JsonWriter writer, OutboxSequenceToken value, JsonSerializerOptions options) { - throw new NotImplementedException(); + JsonSerializer.Serialize( + writer, + new OutboxSequenceTokenJsonModel( + value.SequenceNumber, + new GrainIdJsonModel(value.Sender.Type.ToString()!, value.Sender.Key.ToString()!), + value.Timestamp, + value.Epoch), + options); } + + private sealed record OutboxSequenceTokenJsonModel( + long SequenceNumber, + GrainIdJsonModel? Sender, + DateTimeOffset Timestamp, + DateTimeOffset Epoch); + + private sealed record GrainIdJsonModel(string Type, string Key); } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxSequenceTokenJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxSequenceTokenJsonConverterTests.cs new file mode 100644 index 00000000..50f11524 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxSequenceTokenJsonConverterTests.cs @@ -0,0 +1,68 @@ +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class OutboxSequenceTokenJsonConverterTests +{ + public static TheoryData GrainIdCases => new() + { + GrainId.Create("test/type", "plain-string"), + GrainId.Create(GrainType.Create("test/type"), GrainIdKeyExtensions.CreateIntegerKey(42)), + GrainId.Create(GrainType.Create("test/type"), GrainIdKeyExtensions.CreateIntegerKey(7, "ext")), + GrainId.Create(GrainType.Create("test/type"), GrainIdKeyExtensions.CreateGuidKey(Guid.Parse("11111111-2222-3333-4444-555555555555"))), + GrainId.Create(GrainType.Create("test/type"), GrainIdKeyExtensions.CreateGuidKey(Guid.Parse("aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"), "ext2")) + }; + + [Fact] + public void OutboxSequenceToken_is_decorated_with_outbox_sequence_token_json_converter() + { + var attribute = typeof(OutboxSequenceToken).GetCustomAttributes(typeof(JsonConverterAttribute), inherit: false) + .Cast() + .Single(); + + Assert.Equal(typeof(OutboxSequenceTokenJsonConverter), attribute.ConverterType); + } + + [Fact] + public void JsonSerializer_round_trips_outbox_sequence_token_without_custom_options() + { + var now = new DateTimeOffset(2026, 5, 24, 16, 0, 0, TimeSpan.Zero); + var sender = GrainId.Create("test/sender", "one"); + var token = new OutboxSequenceToken(7, sender, now, now.AddMinutes(-5)); + + var json = JsonSerializer.Serialize(token); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(token, roundTripped); + } + + [Theory] + [MemberData(nameof(GrainIdCases))] + public void JsonSerializer_round_trips_outbox_sequence_token_for_supported_grain_id_key_shapes(GrainId sender) + { + var now = new DateTimeOffset(2026, 5, 24, 16, 0, 0, TimeSpan.Zero); + var token = new OutboxSequenceToken(9, sender, now, now.AddMinutes(-5)); + + var json = JsonSerializer.Serialize(token); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(token, roundTripped); + } + + [Fact] + public void JsonSerializer_throws_for_outbox_sequence_token_payload_missing_sender() + { + var json = """ + { + "SequenceNumber": 7, + "Timestamp": "2026-05-24T16:00:00+00:00", + "Epoch": "2026-05-24T15:55:00+00:00" + } + """; + + Assert.Throws(() => JsonSerializer.Deserialize(json)); + } +} From 5ca53f611776cbd049236c0720addee68186d00e Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 10:29:36 +0000 Subject: [PATCH 08/82] feat(om): implement OutboxMessageEnvelope STJ converter factory Enable JSON converter factory attribute on OutboxMessageEnvelope and implement OutboxMessageEnvelopeJsonConverterFactory with closed generic converters for read/write round-trip serialization. Add tests for attribute wiring, CanConvert behavior, round-trip serialization, and missing-token payload validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../OutboxMessageEnvelope.cs | 4 +- ...tboxMessageEnvelopeJsonConverterFactory.cs | 45 ++++++++++++- ...essageEnvelopeJsonConverterFactoryTests.cs | 65 +++++++++++++++++++ 3 files changed, 111 insertions(+), 3 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxMessageEnvelopeJsonConverterFactoryTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs index f52b6675..f4df6d7a 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs @@ -1,3 +1,5 @@ +using System.Text.Json.Serialization; + namespace Egil.Orleans.Messaging; /// @@ -39,7 +41,7 @@ namespace Egil.Orleans.Messaging; /// The user-defined message payload. [GenerateSerializer] [Alias("egil.orleans.messaging.OutboxMessageEnvelope`1")] -// [JsonConverter(typeof(OutboxMessageEnvelopeJsonConverterFactory))] +[JsonConverter(typeof(OutboxMessageEnvelopeJsonConverterFactory))] public sealed record OutboxMessageEnvelope( [property: Id(0)] OutboxSequenceToken Token, [property: Id(1)] T Message); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs index 1f4399d4..24237475 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs @@ -19,12 +19,53 @@ internal sealed class OutboxMessageEnvelopeJsonConverterFactory : JsonConverterF /// public override bool CanConvert(Type typeToConvert) { - throw new NotImplementedException(); + return typeToConvert.IsGenericType + && typeToConvert.GetGenericTypeDefinition() == typeof(OutboxMessageEnvelope<>); } /// public override JsonConverter? CreateConverter(Type typeToConvert, JsonSerializerOptions options) { - throw new NotImplementedException(); + var messageType = typeToConvert.GetGenericArguments()[0]; + var converterType = typeof(OutboxMessageEnvelopeJsonConverter<>).MakeGenericType(messageType); + + return (JsonConverter?)Activator.CreateInstance(converterType); + } + + private sealed class OutboxMessageEnvelopeJsonConverter : JsonConverter> + { + public override OutboxMessageEnvelope? Read( + ref Utf8JsonReader reader, + Type typeToConvert, + JsonSerializerOptions options) + { + var model = JsonSerializer.Deserialize>(ref reader, options); + if (model is null) + { + return null; + } + + if (model.Token is null) + { + throw new JsonException("Missing Token."); + } + + return new OutboxMessageEnvelope(model.Token, model.Message); + } + + public override void Write( + Utf8JsonWriter writer, + OutboxMessageEnvelope value, + JsonSerializerOptions options) + { + JsonSerializer.Serialize( + writer, + new OutboxMessageEnvelopeJsonModel(value.Token, value.Message), + options); + } } + + private sealed record OutboxMessageEnvelopeJsonModel( + OutboxSequenceToken? Token, + T Message); } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxMessageEnvelopeJsonConverterFactoryTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxMessageEnvelopeJsonConverterFactoryTests.cs new file mode 100644 index 00000000..c3a79c69 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxMessageEnvelopeJsonConverterFactoryTests.cs @@ -0,0 +1,65 @@ +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class OutboxMessageEnvelopeJsonConverterFactoryTests +{ + [Fact] + public void OutboxMessageEnvelope_is_decorated_with_converter_factory() + { + var attribute = typeof(OutboxMessageEnvelope) + .GetCustomAttributes(typeof(JsonConverterAttribute), inherit: false) + .Cast() + .Single(); + + Assert.Equal(typeof(OutboxMessageEnvelopeJsonConverterFactory), attribute.ConverterType); + } + + [Fact] + public void CanConvert_returns_true_for_outbox_message_envelope_generic_type() + { + var factory = new OutboxMessageEnvelopeJsonConverterFactory(); + + var canConvert = factory.CanConvert(typeof(OutboxMessageEnvelope)); + + Assert.True(canConvert); + } + + [Fact] + public void CanConvert_returns_false_for_non_outbox_message_envelope_type() + { + var factory = new OutboxMessageEnvelopeJsonConverterFactory(); + + var canConvert = factory.CanConvert(typeof(Outbox)); + + Assert.False(canConvert); + } + + [Fact] + public void JsonSerializer_round_trips_outbox_message_envelope_without_custom_options() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 24, 18, 0, 0, TimeSpan.Zero); + var token = new OutboxSequenceToken(11, sender, now, now.AddMinutes(-1)); + var envelope = new OutboxMessageEnvelope(token, "hello"); + + var json = JsonSerializer.Serialize(envelope); + var roundTripped = JsonSerializer.Deserialize>(json); + + Assert.NotNull(roundTripped); + Assert.Equal(envelope, roundTripped); + } + + [Fact] + public void JsonSerializer_throws_for_outbox_message_envelope_payload_missing_token() + { + var json = """ + { + "Message": "hello" + } + """; + + Assert.Throws(() => JsonSerializer.Deserialize>(json)); + } +} From 11bf661699b16f7ae66a1a92fa9c2932026fe9a2 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 10:30:10 +0000 Subject: [PATCH 09/82] feat(om): implement StateManager persistence workflow Implement StateManager state cache, read/write/clear operations, force-write ETag override, and conflict handling that reconciles equivalent persisted state before rethrowing non-equivalent failures. Add focused tests for read refresh, successful writes, write-failure reconciliation behavior, rollback on conflicting writes, rollback when post-failure read fails, and VersionedState version stamping. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Egil.Orleans.Messaging/StateManager.cs | 93 ++++++++- .../StateManagerTests.cs | 186 ++++++++++++++++++ 2 files changed, 271 insertions(+), 8 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs index efc4b5e4..4b308b3d 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs @@ -1,3 +1,5 @@ +using Orleans.Storage; + namespace Egil.Orleans.Messaging; /// @@ -72,6 +74,7 @@ internal sealed class StateManager : IStateManager where T : class, IEquatable { private readonly IPersistentState storage; + private T state; /// /// Creates a new wrapping the given @@ -85,30 +88,87 @@ internal sealed class StateManager : IStateManager internal StateManager(IPersistentState storage) { this.storage = storage; + state = storage.State; } /// public T State { - get => throw new NotImplementedException(); + get => state; } /// - public Task ReadAsync() + public async Task ReadAsync() { - throw new NotImplementedException(); + await storage.ReadStateAsync(); + state = storage.State; } /// - public Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concurrent) + public async Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concurrent) { - throw new NotImplementedException(); + var previousState = state; + var previousEtag = storage.Etag; + + if (newState is VersionedState versioned) + { + versioned.Version = Guid.CreateVersion7(); + } + + if (policy == WritePolicy.Force) + { + TrySetEtag(storage, null); + } + + storage.State = newState; + + try + { + await storage.WriteStateAsync(); + state = newState; + return; + } + catch (Exception ex) + { + try + { + await storage.ReadStateAsync(); + var persisted = storage.State; + + if (ex is InconsistentStateException) + { + storage.State = previousState; + TrySetEtag(storage, previousEtag); + throw; + } + + if (IsEquivalent(persisted, newState)) + { + state = persisted; + return; + } + + storage.State = previousState; + TrySetEtag(storage, previousEtag); + throw; + } + catch when (ex is not InconsistentStateException) + { + // no-op; throw original below + } + + state = previousState; + storage.State = previousState; + TrySetEtag(storage, previousEtag); + throw; + } } /// - public Task ClearAsync() + public async Task ClearAsync() { - throw new NotImplementedException(); + await storage.ClearStateAsync(); + state = storage.State; } /// @@ -119,6 +179,23 @@ public Task ClearAsync() /// private static bool IsEquivalent(T persisted, T attempted) { - throw new NotImplementedException(); + if (persisted is VersionedState persistedVersioned + && attempted is VersionedState attemptedVersioned) + { + return persistedVersioned.Version == attemptedVersioned.Version; + } + + return persisted.Equals(attempted); + } + + private static void TrySetEtag(IPersistentState storage, string? etag) + { + var property = storage.GetType().GetProperty(nameof(IPersistentState.Etag)); + if (property is null || !property.CanWrite) + { + return; + } + + property.SetValue(storage, etag); } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs new file mode 100644 index 00000000..c0438700 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs @@ -0,0 +1,186 @@ +namespace Egil.Orleans.Messaging.Tests; + +public sealed class StateManagerTests +{ + [Fact] + public async Task ReadAsync_refreshes_state_from_storage() + { + var storage = new FakePersistentState(new TestState("initial")); + var manager = new StateManager(storage); + storage.State = new TestState("fresh"); + + await manager.ReadAsync(); + + Assert.Equal(new TestState("fresh"), manager.State); + } + + [Fact] + public async Task WriteAsync_success_updates_committed_state() + { + var storage = new FakePersistentState(new TestState("initial")); + var manager = new StateManager(storage); + var next = new TestState("next"); + + await manager.WriteAsync(next); + + Assert.Equal(next, manager.State); + Assert.Equal(next, storage.State); + } + + [Fact] + public async Task WriteAsync_when_write_fails_but_read_shows_attempted_state_swallows_exception() + { + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = new TimeoutException("write timeout"), + OnRead = state => state.State = new TestState("next") + }; + var manager = new StateManager(storage); + var next = new TestState("next"); + + await manager.WriteAsync(next); + + Assert.Equal(next, manager.State); + } + + [Fact] + public async Task WriteAsync_when_write_fails_and_read_shows_different_state_rethrows_original_exception() + { + var writeException = new TimeoutException("write timeout"); + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = writeException, + OnRead = state => state.State = new TestState("other") + }; + var manager = new StateManager(storage); + + var ex = await Assert.ThrowsAsync(() => manager.WriteAsync(new TestState("next"))); + + Assert.Same(writeException, ex); + Assert.Equal(new TestState("initial"), manager.State); + } + + [Fact] + public async Task WriteAsync_when_write_and_read_fail_reverts_state_and_rethrows_original_exception() + { + var writeException = new TimeoutException("write timeout"); + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = writeException, + ReadException = new InvalidOperationException("read failed") + }; + var manager = new StateManager(storage); + + var ex = await Assert.ThrowsAsync(() => manager.WriteAsync(new TestState("next"))); + + Assert.Same(writeException, ex); + Assert.Equal(new TestState("initial"), manager.State); + } + + [Fact] + public async Task WriteAsync_on_versioned_state_stamps_new_version() + { + var original = new VersionedTestState("initial"); + var storage = new FakePersistentVersionedState(original); + var manager = new StateManager(storage); + var next = new VersionedTestState("next") { Version = Guid.Empty }; + + await manager.WriteAsync(next); + + Assert.NotEqual(Guid.Empty, next.Version); + Assert.Equal(next.Version, manager.State.Version); + } + + private sealed record TestState(string Value); + + private sealed record VersionedTestState(string Value) : VersionedState, IEquatable; + + private sealed class FakePersistentState : IPersistentState + { + public FakePersistentState(TestState state) + { + State = state; + } + + public Exception? ReadException { get; set; } + public Exception? WriteException { get; set; } + public Exception? ClearException { get; set; } + public Action? OnRead { get; set; } + public Action? OnWrite { get; set; } + public Action? OnClear { get; set; } + + public string Etag { get; set; } = "etag-1"; + public bool RecordExists { get; set; } = true; + public TestState State { get; set; } + + public Task ReadStateAsync() + { + OnRead?.Invoke(this); + if (ReadException is not null) + { + throw ReadException; + } + + return Task.CompletedTask; + } + + public Task WriteStateAsync() + { + OnWrite?.Invoke(this); + if (WriteException is not null) + { + throw WriteException; + } + + return Task.CompletedTask; + } + + public Task ClearStateAsync() + { + OnClear?.Invoke(this); + if (ClearException is not null) + { + throw ClearException; + } + + State = null!; + RecordExists = false; + return Task.CompletedTask; + } + + public Task ReadStateAsync(CancellationToken cancellationToken) => ReadStateAsync(); + + public Task WriteStateAsync(CancellationToken cancellationToken) => WriteStateAsync(); + + public Task ClearStateAsync(CancellationToken cancellationToken) => ClearStateAsync(); + } + + private sealed class FakePersistentVersionedState : IPersistentState + { + public FakePersistentVersionedState(VersionedTestState state) + { + State = state; + } + + public string Etag { get; set; } = "etag-1"; + public bool RecordExists { get; set; } = true; + public VersionedTestState State { get; set; } + + public Task ReadStateAsync() => Task.CompletedTask; + + public Task WriteStateAsync() => Task.CompletedTask; + + public Task ClearStateAsync() + { + State = null!; + RecordExists = false; + return Task.CompletedTask; + } + + public Task ReadStateAsync(CancellationToken cancellationToken) => ReadStateAsync(); + + public Task WriteStateAsync(CancellationToken cancellationToken) => WriteStateAsync(); + + public Task ClearStateAsync(CancellationToken cancellationToken) => ClearStateAsync(); + } +} From fc834a03fe1c111f6e491fc38dea6ddaa677dc42 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 10:30:39 +0000 Subject: [PATCH 10/82] feat(om): implement StateManager extension wrapper Implement AsStateManager to validate input and wrap IPersistentState with StateManager for grain-facing usage. Add focused tests for successful wrapping and null-argument validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../StateManagerExtensions.cs | 3 +- .../StateManagerExtensionsTests.cs | 49 +++++++++++++++++++ 2 files changed, 51 insertions(+), 1 deletion(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs index 4df55e39..40f0daeb 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs @@ -53,6 +53,7 @@ public static class StateManagerExtensions public static IStateManager AsStateManager(this IPersistentState storage) where T : class, IEquatable { - throw new NotImplementedException(); + ArgumentNullException.ThrowIfNull(storage); + return new StateManager(storage); } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs new file mode 100644 index 00000000..a4971a50 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs @@ -0,0 +1,49 @@ +namespace Egil.Orleans.Messaging.Tests; + +public sealed class StateManagerExtensionsTests +{ + [Fact] + public void AsStateManager_wraps_persistent_state() + { + var storage = new FakePersistentState(new TestState("initial")); + + var manager = storage.AsStateManager(); + + Assert.NotNull(manager); + Assert.Equal(new TestState("initial"), manager.State); + } + + [Fact] + public void AsStateManager_throws_for_null_storage() + { + var ex = Assert.Throws(() => StateManagerExtensions.AsStateManager(null!)); + + Assert.Equal("storage", ex.ParamName); + } + + private sealed record TestState(string Value); + + private sealed class FakePersistentState(TestState state) : IPersistentState + { + public string Etag { get; set; } = "etag-1"; + public bool RecordExists { get; set; } = true; + public TestState State { get; set; } = state; + + public Task ReadStateAsync() => Task.CompletedTask; + + public Task WriteStateAsync() => Task.CompletedTask; + + public Task ClearStateAsync() + { + State = null!; + RecordExists = false; + return Task.CompletedTask; + } + + public Task ReadStateAsync(CancellationToken cancellationToken) => ReadStateAsync(); + + public Task WriteStateAsync(CancellationToken cancellationToken) => WriteStateAsync(); + + public Task ClearStateAsync(CancellationToken cancellationToken) => ClearStateAsync(); + } +} From 624bf14a8d76c6c5670ceaa48eb6b95026c4f5df Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 10:31:11 +0000 Subject: [PATCH 11/82] feat(om): implement StreamCursor enriched token accessors Enable StreamCursor JSON converter attribute and implement TryGetEnqueuedTime, TryGetStreamProviderName, and TryGetTraceParent to expose enrichment metadata when backed by EnrichedEventHubSequenceToken. Add focused tests for positive and negative cases across enqueued time, stream provider name, and traceparent access. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Egil.Orleans.Messaging/StreamCursor.cs | 30 +++++- .../StreamCursorTests.cs | 101 ++++++++++++++++++ 2 files changed, 127 insertions(+), 4 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs index d1967d0a..f278ddc6 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs @@ -1,3 +1,4 @@ +using System.Text.Json.Serialization; using Orleans.Streams; namespace Egil.Orleans.Messaging; @@ -42,7 +43,7 @@ namespace Egil.Orleans.Messaging; /// [GenerateSerializer] [Alias("egil.orleans.messaging.StreamCursor")] -// [JsonConverter(typeof(StreamCursorJsonConverter))] +[JsonConverter(typeof(StreamCursorJsonConverter))] public sealed record StreamCursor( [property: Id(0)] StreamId StreamId, [property: Id(1)] StreamSequenceToken? Token) @@ -68,7 +69,14 @@ public sealed record StreamCursor( /// public bool TryGetEnqueuedTime(out DateTimeOffset enqueuedTime) { - throw new NotImplementedException(); + if (Token is EnrichedEventHubSequenceToken token) + { + enqueuedTime = token.EnqueuedTime; + return true; + } + + enqueuedTime = default; + return false; } /// @@ -91,7 +99,14 @@ public bool TryGetEnqueuedTime(out DateTimeOffset enqueuedTime) /// public bool TryGetStreamProviderName([NotNullWhen(true)] out string? streamProviderName) { - throw new NotImplementedException(); + if (Token is EnrichedEventHubSequenceToken token) + { + streamProviderName = token.StreamProviderName; + return true; + } + + streamProviderName = null; + return false; } /// @@ -120,6 +135,13 @@ public bool TryGetStreamProviderName([NotNullWhen(true)] out string? streamProvi /// public bool TryGetTraceParent([NotNullWhen(true)] out string? traceParent) { - throw new NotImplementedException(); + if (Token is EnrichedEventHubSequenceToken token && token.TraceParent is not null) + { + traceParent = token.TraceParent; + return true; + } + + traceParent = null; + return false; } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs new file mode 100644 index 00000000..bcc92668 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs @@ -0,0 +1,101 @@ +using Orleans.Providers.Streams.Common; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class StreamCursorTests +{ + [Fact] + public void TryGetEnqueuedTime_returns_true_and_value_for_enriched_event_hub_token() + { + var enqueuedTime = new DateTimeOffset(2026, 5, 24, 12, 30, 0, TimeSpan.Zero); + var cursor = new StreamCursor( + StreamId.Create("orders", "one"), + new EnrichedEventHubSequenceToken("42", 10, 0, enqueuedTime, "provider-a")); + + var found = cursor.TryGetEnqueuedTime(out var actual); + + Assert.True(found); + Assert.Equal(enqueuedTime, actual); + } + + [Fact] + public void TryGetEnqueuedTime_returns_false_for_non_enriched_token() + { + var cursor = new StreamCursor( + StreamId.Create("orders", "one"), + new EventSequenceToken(10)); + + var found = cursor.TryGetEnqueuedTime(out var actual); + + Assert.False(found); + Assert.Equal(default, actual); + } + + [Fact] + public void TryGetStreamProviderName_returns_true_and_value_for_enriched_event_hub_token() + { + var cursor = new StreamCursor( + StreamId.Create("orders", "one"), + new EnrichedEventHubSequenceToken( + "42", + 10, + 0, + new DateTimeOffset(2026, 5, 24, 12, 30, 0, TimeSpan.Zero), + "provider-a")); + + var found = cursor.TryGetStreamProviderName(out var streamProviderName); + + Assert.True(found); + Assert.Equal("provider-a", streamProviderName); + } + + [Fact] + public void TryGetStreamProviderName_returns_false_for_non_enriched_token() + { + var cursor = new StreamCursor( + StreamId.Create("orders", "one"), + new EventSequenceToken(10)); + + var found = cursor.TryGetStreamProviderName(out var streamProviderName); + + Assert.False(found); + Assert.Null(streamProviderName); + } + + [Fact] + public void TryGetTraceParent_returns_true_and_value_when_present_on_enriched_event_hub_token() + { + var cursor = new StreamCursor( + StreamId.Create("orders", "one"), + new EnrichedEventHubSequenceToken( + "42", + 10, + 0, + new DateTimeOffset(2026, 5, 24, 12, 30, 0, TimeSpan.Zero), + "provider-a", + "00-abc-xyz-01")); + + var found = cursor.TryGetTraceParent(out var traceParent); + + Assert.True(found); + Assert.Equal("00-abc-xyz-01", traceParent); + } + + [Fact] + public void TryGetTraceParent_returns_false_when_missing_on_enriched_event_hub_token() + { + var cursor = new StreamCursor( + StreamId.Create("orders", "one"), + new EnrichedEventHubSequenceToken( + "42", + 10, + 0, + new DateTimeOffset(2026, 5, 24, 12, 30, 0, TimeSpan.Zero), + "provider-a")); + + var found = cursor.TryGetTraceParent(out var traceParent); + + Assert.False(found); + Assert.Null(traceParent); + } +} From 84b1eed300cd89ee2029145e5fcc700dc06fd43e Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 10:31:42 +0000 Subject: [PATCH 12/82] feat(om): wire enriched event hub data adapter registration Implement UseEnrichedDataAdapter extension to validate input and register EnrichedEventHubAdapter through the Event Hub configurator using Orleans Serializer from DI. Add focused tests for null-argument validation and configurator callback registration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../EnrichedEventHubAdapterExtensions.cs | 10 ++++- .../EnrichedEventHubAdapterExtensionsTests.cs | 40 +++++++++++++++++++ 2 files changed, 49 insertions(+), 1 deletion(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterExtensionsTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs index 767f9663..8dafa306 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs @@ -1,3 +1,6 @@ +using Microsoft.Extensions.DependencyInjection; +using Orleans.Serialization; + namespace Egil.Orleans.Messaging; /// @@ -58,6 +61,11 @@ public static class EnrichedEventHubAdapterExtensions public static void UseEnrichedDataAdapter( this IEventHubStreamConfigurator configurator) { - throw new NotImplementedException(); + ArgumentNullException.ThrowIfNull(configurator); + + configurator.UseDataAdapter((services, streamProviderName) => + new EnrichedEventHubAdapter( + streamProviderName, + services.GetRequiredService())); } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterExtensionsTests.cs new file mode 100644 index 00000000..68d3f3d1 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterExtensionsTests.cs @@ -0,0 +1,40 @@ +using Microsoft.Extensions.DependencyInjection; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class EnrichedEventHubAdapterExtensionsTests +{ + [Fact] + public void UseEnrichedDataAdapter_throws_for_null_configurator() + { + var ex = Assert.Throws(() => + EnrichedEventHubAdapterExtensions.UseEnrichedDataAdapter(null!)); + + Assert.Equal("configurator", ex.ParamName); + } + + [Fact] + public void UseEnrichedDataAdapter_registers_configuration_callback() + { + var configurator = new FakeEventHubStreamConfigurator("provider-a"); + + configurator.UseEnrichedDataAdapter(); + + Assert.Equal(1, configurator.ConfigureCalls); + Assert.NotNull(configurator.LastConfigureAction); + } + + private sealed class FakeEventHubStreamConfigurator(string name) : IEventHubStreamConfigurator + { + public int ConfigureCalls { get; private set; } + public Action? LastConfigureAction { get; private set; } + + public string Name => name; + + public Action> ConfigureDelegate => configure => + { + ConfigureCalls++; + LastConfigureAction = configure; + }; + } +} From 996e30d22d55fd634a61c905ed63939c9cd5b61e Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 10:32:10 +0000 Subject: [PATCH 13/82] feat(om): implement enriched event hub adapter token stamping Implement EnrichedEventHubAdapter overrides to stamp traceparent on queued messages, create EnrichedEventHubSequenceToken from cached Event Hub messages, and return enriched stream positions that preserve offset/sequence/event index and optional traceparent. Add focused tests for traceparent stamping behavior and enriched token construction from cached message metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../EnrichedEventHubAdapter.cs | 48 ++++++++++++- .../EnrichedEventHubAdapterTests.cs | 69 +++++++++++++++++++ 2 files changed, 114 insertions(+), 3 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs index 584fc178..4c624237 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs @@ -146,7 +146,13 @@ public override Azure.Messaging.EventHubs.EventData ToQueueMessage( StreamSequenceToken token, Dictionary requestContext) { - throw new NotImplementedException(); + var queueMessage = base.ToQueueMessage(streamId, events, token, requestContext); + if (Activity.Current?.Id is { } traceParent) + { + queueMessage.Properties[TraceParentPropertyKey] = traceParent; + } + + return queueMessage; } /// @@ -175,7 +181,23 @@ public override Azure.Messaging.EventHubs.EventData ToQueueMessage( /// public override StreamSequenceToken GetSequenceToken(ref CachedMessage cachedMessage) { - throw new NotImplementedException(); + var offset = cachedMessage.Segment.Array is null + ? string.Empty + : GetOffset(cachedMessage) ?? string.Empty; + + var enqueueTimeUtc = cachedMessage.EnqueueTimeUtc.Kind switch + { + DateTimeKind.Unspecified => DateTime.SpecifyKind(cachedMessage.EnqueueTimeUtc, DateTimeKind.Utc), + DateTimeKind.Local => cachedMessage.EnqueueTimeUtc.ToUniversalTime(), + _ => cachedMessage.EnqueueTimeUtc + }; + + return new EnrichedEventHubSequenceToken( + offset, + cachedMessage.SequenceNumber, + cachedMessage.EventIndex, + new DateTimeOffset(enqueueTimeUtc), + StreamProviderName); } /// @@ -208,6 +230,26 @@ public override StreamPosition GetStreamPosition( string partition, Azure.Messaging.EventHubs.EventData queueMessage) { - throw new NotImplementedException(); + var streamPosition = base.GetStreamPosition(partition, queueMessage); + var traceParent = queueMessage.Properties.TryGetValue(TraceParentPropertyKey, out var value) + ? value as string ?? value?.ToString() + : null; + + var (offset, sequenceNumber, eventIndex) = streamPosition.SequenceToken switch + { + EventHubSequenceTokenV2 token => (token.EventHubOffset, token.SequenceNumber, token.EventIndex), + EventHubSequenceToken token => (token.EventHubOffset, token.SequenceNumber, token.EventIndex), + _ => (queueMessage.OffsetString ?? string.Empty, queueMessage.SequenceNumber, 0) + }; + + var enrichedToken = new EnrichedEventHubSequenceToken( + offset ?? string.Empty, + sequenceNumber, + eventIndex, + queueMessage.EnqueuedTime, + StreamProviderName, + traceParent); + + return new StreamPosition(streamPosition.StreamId, enrichedToken); } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs new file mode 100644 index 00000000..29139e51 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs @@ -0,0 +1,69 @@ +using System.Diagnostics; +using Microsoft.Extensions.DependencyInjection; +using Orleans.Providers.Streams.Common; +using Orleans.Serialization; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class EnrichedEventHubAdapterTests +{ + [Fact] + public void ToQueueMessage_stamps_traceparent_when_activity_is_active() + { + var adapter = CreateAdapter(); + using var activity = new Activity("publish").Start(); + + var message = adapter.ToQueueMessage( + StreamId.Create("orders", "one"), + ["event-1"], + null!, + []); + + var value = Assert.Contains("traceparent", message.Properties); + Assert.Equal(activity.Id, Assert.IsType(value)); + } + + [Fact] + public void ToQueueMessage_does_not_stamp_traceparent_when_no_activity_is_active() + { + var adapter = CreateAdapter(); + + var message = adapter.ToQueueMessage( + StreamId.Create("orders", "one"), + ["event-1"], + null!, + []); + + Assert.DoesNotContain("traceparent", message.Properties); + } + + [Fact] + public void GetSequenceToken_returns_enriched_event_hub_sequence_token() + { + var adapter = CreateAdapter(); + var cachedMessage = new CachedMessage + { + SequenceNumber = 10, + EventIndex = 2, + EnqueueTimeUtc = new DateTime(2026, 5, 24, 19, 0, 0, DateTimeKind.Utc) + }; + + var token = adapter.GetSequenceToken(ref cachedMessage); + + var enriched = Assert.IsType(token); + Assert.Equal(10, enriched.SequenceNumber); + Assert.Equal(2, enriched.EventIndex); + Assert.Equal("provider-a", enriched.StreamProviderName); + Assert.Null(enriched.TraceParent); + } + + private static EnrichedEventHubAdapter CreateAdapter() + { + var services = new ServiceCollection(); + services.AddSerializer(static _ => { }); + var provider = services.BuildServiceProvider(); + var serializer = provider.GetRequiredService(); + + return new EnrichedEventHubAdapter("provider-a", serializer); + } +} From 996c03865f75c70a3258990037d73351d87fbb60 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 12:27:42 +0000 Subject: [PATCH 14/82] chore: update gitignore --- .gitignore | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index e176d031..c26d557b 100644 --- a/.gitignore +++ b/.gitignore @@ -401,4 +401,6 @@ FodyWeavers.xsd # Verify *.received.* -*.received/ \ No newline at end of file +*.received/ +.dotnet/.dotnet/sdk-advertising +.dotnet/.dotnet From ae3aeb2e7decac7e9d0f3153031faf92ec63455d Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 13:56:15 +0000 Subject: [PATCH 15/82] feat(om): add keyed state manager registration and remove write policy Add InitializeStateManager(...) that resolves keyed IStateManagerFactory by storage name, with IServiceCollection and ISiloBuilder helper APIs for keyed singleton registrations. Refactor state management to StateManagerBase/DefaultStateManager, remove WritePolicy and force-write behavior, and drop reflection-based ETag mutation. Update state manager tests and API design docs to match the new factory wiring and write semantics. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/api-design.md | 25 +-- .../Egil.Orleans.Messaging/IStateManager.cs | 10 +- .../IStateManagerFactory.cs | 35 +++++ .../Egil.Orleans.Messaging/StateManager.cs | 107 ++++++++----- .../StateManagerExtensions.cs | 94 +++++++---- .../StateManagerRegistrationExtensions.cs | 146 ++++++++++++++++++ .../src/Egil.Orleans.Messaging/WritePolicy.cs | 24 --- .../StateManagerExtensionsTests.cs | 82 +++++++++- ...StateManagerRegistrationExtensionsTests.cs | 133 ++++++++++++++++ .../StateManagerTests.cs | 12 +- 10 files changed, 532 insertions(+), 136 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManagerFactory.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerRegistrationExtensions.cs delete mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/WritePolicy.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerRegistrationExtensionsTests.cs diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 8fbd9737..49c7d5fa 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -73,24 +73,15 @@ read stale `storage.State` after a failed write. ### Interface ```csharp -public enum WritePolicy { Concurrent, Force } - public interface IStateManager where T : class, IEquatable { T State { get; } Task ReadAsync(); - Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concurrent); + Task WriteAsync(T newState); Task ClearAsync(); } ``` -`WritePolicy.Force` is a rare last-resort escape hatch — nulls -`storage.Etag` before writing so the underlying provider skips its -optimistic-concurrency check. Version stamping and the read-back-on- -failure recovery path still run identically. See §3a for why this lives -at the `IStateManager` layer rather than as a settable `Version` -property on `VersionedState`. - Constraints on `T`: - `class` — atomic reference swap of `State` from `[AlwaysInterleave]` @@ -148,7 +139,7 @@ internal sealed class StateManager(IPersistentState storage) : IStateManag public Task ReadAsync() => storage.ReadStateAsync(); - public async Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concurrent) + public async Task WriteAsync(T newState) { var previous = storage.State; @@ -157,11 +148,6 @@ internal sealed class StateManager(IPersistentState storage) : IStateManag versioned.Version = Guid.CreateVersion7(); } - if (policy == WritePolicy.Force) - { - storage.Etag = null; - } - storage.State = newState; try @@ -543,8 +529,6 @@ and the `[JsonInclude]` + `internal set` fence. `IPersistentState.Etag` is storage concurrency. `Version` is library-internal recovery decoration. Conflating them erases a layer. -Force-write escape hatch lives on `IStateManager` via -`WritePolicy.Force`. --- @@ -1199,8 +1183,7 @@ Matching `Egil.Orleans.Testing` convention: `MessageTracker`, `StateManager`, `VersionedState`, `OutboxProcessor`. - **95% branch coverage** on supporting types: `OutboxSequenceToken`, - `StreamCursor`, `StreamManager`, `OutboxMessageEnvelope`, - `WritePolicy`. + `StreamCursor`, `StreamManager`, `OutboxMessageEnvelope`. ### Test grain shape @@ -1208,7 +1191,7 @@ Purpose-built test grains in the test assembly, each targeting a specific behavior: - **Write recovery grain** — exercises `StateManager.WriteAsync` - failure + recovery path, double failure, `WritePolicy.Force`. + failure + recovery path and double failure. - **Outbox drain grain** — exercises `Outbox` Add/Remove/Clear, epoch reset, `OutboxProcessor` timer/reminder lifecycle, postman dispatch + error callback with attempt count. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs index 0aa1d3ab..37fe0a95 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs @@ -18,9 +18,9 @@ namespace Egil.Orleans.Messaging; /// /// /// Usage: Inject as normal via -/// [PersistentState], then wrap it during OnActivateAsync: +/// [PersistentState], then initialize it during OnActivateAsync: /// -/// stateManager = storage.AsStateManager(); +/// stateManager = this.InitializeStateManager("state", storage); /// /// The raw should not be accessed /// directly after wrapping — doing so bypasses the committed-state fence. @@ -125,11 +125,7 @@ public interface IStateManager /// /// /// The new state value to persist. - /// - /// Concurrency policy. (default) uses - /// the provider's ETag check. bypasses it. - /// - Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concurrent); + Task WriteAsync(T newState); /// /// Clears the persisted state, resetting it to default. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManagerFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManagerFactory.cs new file mode 100644 index 00000000..c52006a0 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManagerFactory.cs @@ -0,0 +1,35 @@ +namespace Egil.Orleans.Messaging; + +/// +/// Factory contract for constructing provider-specific +/// instances from Orleans-managed persistent state. +/// +/// +/// The grain state type. +/// +public interface IStateManagerFactory + where T : class, IEquatable +{ + /// + /// Creates an for the given + /// facet. + /// + IStateManager Create(IPersistentState storage); +} + +/// +/// Default factory that creates instances. +/// +/// +/// The grain state type. +/// +public sealed class DefaultStateManagerFactory : IStateManagerFactory + where T : class, IEquatable +{ + /// + public IStateManager Create(IPersistentState storage) + { + ArgumentNullException.ThrowIfNull(storage); + return new DefaultStateManager(storage); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs index 4b308b3d..f2757270 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs @@ -3,10 +3,26 @@ namespace Egil.Orleans.Messaging; /// -/// Default implementation of . Wraps an -/// and adds committed-state fencing, +/// Classifies a storage write failure for recovery decisions. +/// +public enum WriteFailureKind +{ + /// + /// Unknown outcome. The write may have persisted. Run read-back recovery. + /// + UnknownOutcome, + + /// + /// The write definitely did not persist. Skip read-back and rethrow. + /// + DidNotPersist +} + +/// +/// Base implementation of . Wraps an +/// and provides committed-state fencing, /// version stamping for -derived types, and -/// automatic write recovery. +/// configurable write-failure recovery behavior. /// /// /// @@ -55,11 +71,9 @@ namespace Egil.Orleans.Messaging; /// /// /// -/// Concurrency: (default) uses -/// the storage provider's ETag for optimistic concurrency. Writes against -/// stale ETags fail with InconsistentStateException. -/// nulls the ETag before writing, bypassing -/// the concurrency check — use only for admin repair operations. +/// Concurrency: Uses the storage provider's optimistic concurrency +/// checks (typically ETag-based). Writes against stale versions fail with +/// InconsistentStateException. /// /// /// Thread safety: Relies on Orleans turn-based concurrency. Not safe @@ -70,23 +84,15 @@ namespace Egil.Orleans.Messaging; /// /// /// -internal sealed class StateManager : IStateManager +public abstract class StateManagerBase : IStateManager where T : class, IEquatable { private readonly IPersistentState storage; private T state; - /// - /// Creates a new wrapping the given - /// facet. - /// - /// - /// The Orleans persistent state facet to wrap. The manager takes full - /// ownership — callers must not read or write - /// directly after wrapping. - /// - internal StateManager(IPersistentState storage) + protected StateManagerBase(IPersistentState storage) { + ArgumentNullException.ThrowIfNull(storage); this.storage = storage; state = storage.State; } @@ -105,51 +111,62 @@ public async Task ReadAsync() } /// - public async Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concurrent) + public async Task WriteAsync(T newState) { + ArgumentNullException.ThrowIfNull(newState); + var previousState = state; - var previousEtag = storage.Etag; if (newState is VersionedState versioned) { versioned.Version = Guid.CreateVersion7(); } - if (policy == WritePolicy.Force) - { - TrySetEtag(storage, null); - } - storage.State = newState; try { await storage.WriteStateAsync(); - state = newState; + state = storage.State; return; } catch (Exception ex) { + var failureKind = ClassifyWriteFailure(ex); + if (failureKind is WriteFailureKind.DidNotPersist) + { + // Provider-specific classification says the write never reached durable storage. + // Revert our local fence immediately and rethrow the original write error. + state = previousState; + storage.State = previousState; + throw; + } + try { + // Unknown outcome: write may have persisted despite the exception. + // Read back from storage to distinguish "lost response" from "real failure." await storage.ReadStateAsync(); var persisted = storage.State; if (ex is InconsistentStateException) { + // Concurrency conflicts must be surfaced even if values happen to match. + // A coincidental equality must not hide an optimistic concurrency violation. storage.State = previousState; - TrySetEtag(storage, previousEtag); throw; } if (IsEquivalent(persisted, newState)) { + // Lost-response case: write landed, but acknowledgement failed. + // Advance committed fence to persisted value and swallow write exception. state = persisted; return; } + // Read-back proved write did not land; restore local snapshot and rethrow. storage.State = previousState; - TrySetEtag(storage, previousEtag); throw; } catch when (ex is not InconsistentStateException) @@ -157,9 +174,10 @@ public async Task WriteAsync(T newState, WritePolicy policy = WritePolicy.Concur // no-op; throw original below } + // Write failed and recovery read also failed: keep committed fence coherent by + // reverting to the pre-write snapshot, then surface the original write error. state = previousState; storage.State = previousState; - TrySetEtag(storage, previousEtag); throw; } } @@ -172,11 +190,10 @@ public async Task ClearAsync() } /// - /// Determines whether the persisted state matches the attempted write. - /// For -derived types, compares - /// directly. For plain types, falls - /// back to . + /// Classifies a write failure so the base can decide recovery behavior. /// + protected abstract WriteFailureKind ClassifyWriteFailure(Exception exception); + private static bool IsEquivalent(T persisted, T attempted) { if (persisted is VersionedState persistedVersioned @@ -187,15 +204,23 @@ private static bool IsEquivalent(T persisted, T attempted) return persisted.Equals(attempted); } +} - private static void TrySetEtag(IPersistentState storage, string? etag) +/// +/// Default implementation for general storage providers. +/// +public sealed class DefaultStateManager : StateManagerBase + where T : class, IEquatable +{ + public DefaultStateManager(IPersistentState storage) + : base(storage) { - var property = storage.GetType().GetProperty(nameof(IPersistentState.Etag)); - if (property is null || !property.CanWrite) - { - return; - } + } - property.SetValue(storage, etag); + /// + protected override WriteFailureKind ClassifyWriteFailure(Exception exception) + { + ArgumentNullException.ThrowIfNull(exception); + return WriteFailureKind.UnknownOutcome; } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs index 40f0daeb..da25d4ed 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs @@ -1,59 +1,95 @@ +using Microsoft.Extensions.DependencyInjection; + namespace Egil.Orleans.Messaging; /// -/// Provides the extension method that wraps an -/// in an . +/// Extension methods for wiring into a grain's +/// activation lifecycle. /// public static class StateManagerExtensions { /// - /// Wraps the given in an - /// that provides atomic write recovery and a committed-state fence. - /// See for full behavioral contract, recovery - /// semantics, and deep-immutability requirements. + /// Creates an for the given grain using + /// a keyed registration. /// /// /// /// Call site: Typically called once in OnActivateAsync: /// - /// [PersistentState("state")] IPersistentState<MyState> storage; - /// IStateManager<MyState> stateManager; + /// [PersistentState("state")] private readonly IPersistentState<MyState> storage; + /// private IStateManager<MyState> stateManager = default!; /// /// public override Task OnActivateAsync(CancellationToken ct) /// { - /// stateManager = storage.AsStateManager(); + /// stateManager = this.InitializeStateManager("state", storage); /// // ... /// } /// /// /// - /// Important: After calling this method, the grain should not access - /// directly. Doing so bypasses the committed-state - /// fence. See remarks for details. - /// - /// - /// Provider-specific overrides (future): In v1, this always returns - /// the default StateManager<T>. In a future version, this method - /// will check ActivationServices for a registered - /// IStateManagerFactory to resolve provider-specific implementations - /// that can classify exceptions more precisely (e.g., "definitely did not - /// persist" vs. "unknown outcome") and skip the re-read when safe. + /// Registration: A keyed + /// must be registered for + /// via the provided registration helpers. + /// Missing registrations fail fast with a descriptive + /// . /// /// - /// + /// + /// The grain type. Must implement so activation + /// services can be used to resolve keyed registrations. + /// + /// + /// The grain state type. + /// + /// The grain instance (this). + /// + /// Logical storage name used as the keyed DI registration key. + /// /// - /// The Orleans-managed persistent state facet, typically injected via - /// [PersistentState("name")]. Must already be hydrated (Orleans - /// hydrates it during the SetupState lifecycle stage, before - /// OnActivateAsync). + /// The Orleans-managed persistent state facet. /// /// - /// An wrapping . + /// A keyed instance. /// - public static IStateManager AsStateManager(this IPersistentState storage) - where T : class, IEquatable + public static IStateManager InitializeStateManager( + this TGrain grain, + string storageName, + IPersistentState storage) + where TGrain : IGrainBase + where TState : class, IEquatable { + ArgumentNullException.ThrowIfNull(grain); + ArgumentException.ThrowIfNullOrWhiteSpace(storageName); ArgumentNullException.ThrowIfNull(storage); - return new StateManager(storage); + + return InitializeStateManagerCore( + grain.GrainContext.ActivationServices, + storageName, + storage, + grain.GetType()); + } + + internal static IStateManager InitializeStateManagerCore( + IServiceProvider activationServices, + string storageName, + IPersistentState storage, + Type grainType) + where TState : class, IEquatable + { + ArgumentNullException.ThrowIfNull(activationServices); + ArgumentException.ThrowIfNullOrWhiteSpace(storageName); + ArgumentNullException.ThrowIfNull(storage); + ArgumentNullException.ThrowIfNull(grainType); + + var factory = activationServices.GetKeyedService>(storageName); + if (factory is null) + { + throw new InvalidOperationException( + $"No keyed IStateManagerFactory<{typeof(TState).Name}> registration was found for storage name '{storageName}', " + + $"state type '{typeof(TState).FullName}', grain type '{grainType.FullName}'. " + + "Register one via AddDefaultStateManager(...) or AddStateManagerFactory(...)."); + } + + return factory.Create(storage); } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerRegistrationExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerRegistrationExtensions.cs new file mode 100644 index 00000000..f2663333 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerRegistrationExtensions.cs @@ -0,0 +1,146 @@ +using Microsoft.Extensions.DependencyInjection; + +namespace Egil.Orleans.Messaging; + +/// +/// Registration helpers for keyed +/// services on . +/// +public static class StateManagerRegistrationExtensions +{ + /// + /// Registers the default keyed for the + /// given . + /// + public static IServiceCollection AddDefaultStateManager( + this IServiceCollection services, + string storageName) + { + ArgumentNullException.ThrowIfNull(services); + ValidateStorageName(storageName); + + return services.AddKeyedSingleton( + typeof(IStateManagerFactory<>), + storageName, + typeof(DefaultStateManagerFactory<>)); + } + + /// + /// Registers a custom keyed open-generic + /// implementation for the given . + /// + public static IServiceCollection AddStateManagerFactory( + this IServiceCollection services, + string storageName, + Type openGenericFactoryType) + { + ArgumentNullException.ThrowIfNull(services); + ValidateStorageName(storageName); + ValidateOpenGenericFactoryType(openGenericFactoryType); + + return services.AddKeyedSingleton( + typeof(IStateManagerFactory<>), + storageName, + openGenericFactoryType); + } + + /// + /// Registers a custom keyed factory for a specific state type. + /// + public static IServiceCollection AddStateManagerFactory( + this IServiceCollection services, + string storageName) + where TState : class, IEquatable + where TFactory : class, IStateManagerFactory + { + ArgumentNullException.ThrowIfNull(services); + ValidateStorageName(storageName); + + return services.AddKeyedSingleton, TFactory>(storageName); + } + + /// + /// Registers the default keyed state manager factory on the silo builder. + /// + public static ISiloBuilder AddDefaultStateManager( + this ISiloBuilder builder, + string storageName) + { + ArgumentNullException.ThrowIfNull(builder); + ValidateStorageName(storageName); + + builder.ConfigureServices(services => services.AddDefaultStateManager(storageName)); + return builder; + } + + /// + /// Registers a custom keyed open-generic state manager factory + /// on the silo builder. + /// + public static ISiloBuilder AddStateManagerFactory( + this ISiloBuilder builder, + string storageName, + Type openGenericFactoryType) + { + ArgumentNullException.ThrowIfNull(builder); + ValidateStorageName(storageName); + ValidateOpenGenericFactoryType(openGenericFactoryType); + + builder.ConfigureServices(services => + services.AddStateManagerFactory(storageName, openGenericFactoryType)); + return builder; + } + + /// + /// Registers a custom keyed state manager factory for a specific state type + /// on the silo builder. + /// + public static ISiloBuilder AddStateManagerFactory( + this ISiloBuilder builder, + string storageName) + where TState : class, IEquatable + where TFactory : class, IStateManagerFactory + { + ArgumentNullException.ThrowIfNull(builder); + ValidateStorageName(storageName); + + builder.ConfigureServices(services => + services.AddStateManagerFactory(storageName)); + return builder; + } + + private static void ValidateStorageName(string storageName) + { + ArgumentException.ThrowIfNullOrWhiteSpace(storageName); + } + + private static void ValidateOpenGenericFactoryType(Type openGenericFactoryType) + { + ArgumentNullException.ThrowIfNull(openGenericFactoryType); + + if (!openGenericFactoryType.IsClass || openGenericFactoryType.IsAbstract) + { + throw new ArgumentException( + $"Type '{openGenericFactoryType.FullName}' must be a non-abstract class.", + nameof(openGenericFactoryType)); + } + + if (!openGenericFactoryType.IsGenericTypeDefinition) + { + throw new ArgumentException( + $"Type '{openGenericFactoryType.FullName}' must be an open generic type definition.", + nameof(openGenericFactoryType)); + } + + var implementsFactory = openGenericFactoryType.GetInterfaces().Any( + iface => iface.IsGenericType + && iface.GetGenericTypeDefinition() == typeof(IStateManagerFactory<>)); + + if (!implementsFactory) + { + throw new ArgumentException( + $"Type '{openGenericFactoryType.FullName}' must implement IStateManagerFactory<>.", + nameof(openGenericFactoryType)); + } + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/WritePolicy.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/WritePolicy.cs deleted file mode 100644 index c64c1fd9..00000000 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/WritePolicy.cs +++ /dev/null @@ -1,24 +0,0 @@ -namespace Egil.Orleans.Messaging; - -/// -/// Controls how handles optimistic concurrency. -/// -[GenerateSerializer] -[Alias("egil.orleans.messaging.WritePolicy")] -public enum WritePolicy -{ - /// - /// Default. Uses the storage provider's ETag-based optimistic concurrency check. - /// If another writer changed the state since the last read, the write fails with - /// InconsistentStateException. - /// - Concurrent = 0, - - /// - /// Nulls the ETag before writing, bypassing the provider's concurrency check. - /// Use as a last-resort escape hatch when a grain needs to force-overwrite - /// state regardless of concurrent modifications (e.g., admin repair operations). - /// The recovery path and version stamping still run identically. - /// - Force = 1, -} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs index a4971a50..8fa31814 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs @@ -1,27 +1,81 @@ +using Microsoft.Extensions.DependencyInjection; + namespace Egil.Orleans.Messaging.Tests; public sealed class StateManagerExtensionsTests { [Fact] - public void AsStateManager_wraps_persistent_state() + public void InitializeStateManagerCore_resolves_keyed_factory_and_wraps_persistent_state() { + var storageName = "state"; + var services = new ServiceCollection(); + services.AddDefaultStateManager(storageName); + var provider = services.BuildServiceProvider(); var storage = new FakePersistentState(new TestState("initial")); - var manager = storage.AsStateManager(); + var manager = StateManagerExtensions.InitializeStateManagerCore( + provider, + storageName, + storage, + typeof(StateManagerExtensionsTests)); Assert.NotNull(manager); Assert.Equal(new TestState("initial"), manager.State); } [Fact] - public void AsStateManager_throws_for_null_storage() + public void InitializeStateManagerCore_throws_for_missing_keyed_registration() + { + var provider = new ServiceCollection().BuildServiceProvider(); + var storage = new FakePersistentState(new TestState("initial")); + + var ex = Assert.Throws(() => + StateManagerExtensions.InitializeStateManagerCore( + provider, + "missing", + storage, + typeof(StateManagerExtensionsTests))); + + Assert.Contains("missing", ex.Message); + Assert.Contains(typeof(TestState).FullName!, ex.Message); + Assert.Contains(typeof(StateManagerExtensionsTests).FullName!, ex.Message); + } + + [Fact] + public void InitializeStateManager_throws_for_null_grain() + { + var storage = new FakePersistentState(new TestState("initial")); + + var ex = Assert.Throws(() => + StateManagerExtensions.InitializeStateManager(null!, "state", storage)); + + Assert.Equal("grain", ex.ParamName); + } + + [Fact] + public void InitializeStateManager_throws_for_empty_storage_name() + { + var grain = new FakeGrainBase(); + var storage = new FakePersistentState(new TestState("initial")); + + var ex = Assert.Throws(() => + grain.InitializeStateManager(" ", storage)); + + Assert.Equal("storageName", ex.ParamName); + } + + [Fact] + public void InitializeStateManager_throws_for_null_storage() { - var ex = Assert.Throws(() => StateManagerExtensions.AsStateManager(null!)); + var grain = new FakeGrainBase(); + + var ex = Assert.Throws(() => + grain.InitializeStateManager("state", null!)); Assert.Equal("storage", ex.ParamName); } - private sealed record TestState(string Value); + private sealed record TestState(string Value) : IEquatable; private sealed class FakePersistentState(TestState state) : IPersistentState { @@ -30,7 +84,6 @@ private sealed class FakePersistentState(TestState state) : IPersistentState Task.CompletedTask; - public Task WriteStateAsync() => Task.CompletedTask; public Task ClearStateAsync() @@ -41,9 +94,22 @@ public Task ClearStateAsync() } public Task ReadStateAsync(CancellationToken cancellationToken) => ReadStateAsync(); - public Task WriteStateAsync(CancellationToken cancellationToken) => WriteStateAsync(); - public Task ClearStateAsync(CancellationToken cancellationToken) => ClearStateAsync(); } + + private sealed class FakeGrainBase : TestGrainBase + { + public override IGrainContext GrainContext => + throw new InvalidOperationException("GrainContext should not be used for guard tests."); + } + + private abstract class TestGrainBase : IGrainBase + { + public abstract IGrainContext GrainContext { get; } + + public virtual Task OnActivateAsync(CancellationToken cancellationToken) => Task.CompletedTask; + + public virtual Task OnDeactivateAsync(DeactivationReason reason, CancellationToken cancellationToken) => Task.CompletedTask; + } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerRegistrationExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerRegistrationExtensionsTests.cs new file mode 100644 index 00000000..2aacde3a --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerRegistrationExtensionsTests.cs @@ -0,0 +1,133 @@ +using Microsoft.Extensions.DependencyInjection; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class StateManagerRegistrationExtensionsTests +{ + [Fact] + public void AddDefaultStateManager_registers_keyed_default_factory_as_singleton() + { + var services = new ServiceCollection(); + + services.AddDefaultStateManager("state"); + + var descriptor = services.Single(service => + service.ServiceType == typeof(IStateManagerFactory<>) + && Equals(service.ServiceKey, "state")); + Assert.Equal(ServiceLifetime.Singleton, descriptor.Lifetime); + + var provider = services.BuildServiceProvider(); + var first = provider.GetKeyedService>("state"); + var second = provider.GetKeyedService>("state"); + + Assert.NotNull(first); + Assert.Same(first, second); + Assert.IsType>(first); + } + + [Fact] + public void AddStateManagerFactory_registers_custom_open_generic_factory_as_singleton() + { + var services = new ServiceCollection(); + + services.AddStateManagerFactory("state", typeof(CustomStateManagerFactory<>)); + + var descriptor = services.Single(service => + service.ServiceType == typeof(IStateManagerFactory<>) + && Equals(service.ServiceKey, "state")); + Assert.Equal(ServiceLifetime.Singleton, descriptor.Lifetime); + + var provider = services.BuildServiceProvider(); + var first = provider.GetKeyedService>("state"); + var second = provider.GetKeyedService>("state"); + + Assert.NotNull(first); + Assert.Same(first, second); + Assert.IsType>(first); + } + + [Fact] + public void AddStateManagerFactory_registers_custom_typed_factory_as_singleton() + { + var services = new ServiceCollection(); + + services.AddStateManagerFactory("state"); + + var descriptor = services.Single(service => + service.ServiceType == typeof(IStateManagerFactory) + && Equals(service.ServiceKey, "state")); + Assert.Equal(ServiceLifetime.Singleton, descriptor.Lifetime); + + var provider = services.BuildServiceProvider(); + var first = provider.GetKeyedService>("state"); + var second = provider.GetKeyedService>("state"); + + Assert.NotNull(first); + Assert.Same(first, second); + Assert.IsType(first); + } + + [Fact] + public void AddStateManagerFactory_throws_for_invalid_open_generic_type() + { + var services = new ServiceCollection(); + + var ex = Assert.Throws(() => + services.AddStateManagerFactory("state", typeof(string))); + + Assert.Equal("openGenericFactoryType", ex.ParamName); + } + + [Fact] + public void AddDefaultStateManager_throws_for_empty_storage_name() + { + var services = new ServiceCollection(); + + var ex = Assert.Throws(() => + services.AddDefaultStateManager(" ")); + + Assert.Equal("storageName", ex.ParamName); + } + + [Fact] + public void ISiloBuilder_extensions_delegate_to_service_collection_registration() + { + var builder = new FakeSiloBuilder(); + + builder + .AddDefaultStateManager("default-state") + .AddStateManagerFactory("custom-state", typeof(CustomStateManagerFactory<>)) + .AddStateManagerFactory("typed-state"); + + var provider = builder.Services.BuildServiceProvider(); + Assert.IsType>( + provider.GetKeyedService>("default-state")); + Assert.IsType>( + provider.GetKeyedService>("custom-state")); + Assert.IsType( + provider.GetKeyedService>("typed-state")); + } + + private sealed record TestState(string Value) : IEquatable; + + private sealed class CustomStateManagerFactory : IStateManagerFactory + where TState : class, IEquatable + { + public IStateManager Create(IPersistentState storage) => + new DefaultStateManager(storage); + } + + private sealed class TestStateManagerFactory : IStateManagerFactory + { + public IStateManager Create(IPersistentState storage) => + new DefaultStateManager(storage); + } + + private sealed class FakeSiloBuilder : ISiloBuilder + { + public IServiceCollection Services { get; } = new ServiceCollection(); + + public Microsoft.Extensions.Configuration.IConfiguration Configuration { get; } = + new Microsoft.Extensions.Configuration.ConfigurationManager(); + } +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs index c0438700..0c86e331 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs @@ -6,7 +6,7 @@ public sealed class StateManagerTests public async Task ReadAsync_refreshes_state_from_storage() { var storage = new FakePersistentState(new TestState("initial")); - var manager = new StateManager(storage); + var manager = new DefaultStateManager(storage); storage.State = new TestState("fresh"); await manager.ReadAsync(); @@ -18,7 +18,7 @@ public async Task ReadAsync_refreshes_state_from_storage() public async Task WriteAsync_success_updates_committed_state() { var storage = new FakePersistentState(new TestState("initial")); - var manager = new StateManager(storage); + var manager = new DefaultStateManager(storage); var next = new TestState("next"); await manager.WriteAsync(next); @@ -35,7 +35,7 @@ public async Task WriteAsync_when_write_fails_but_read_shows_attempted_state_swa WriteException = new TimeoutException("write timeout"), OnRead = state => state.State = new TestState("next") }; - var manager = new StateManager(storage); + var manager = new DefaultStateManager(storage); var next = new TestState("next"); await manager.WriteAsync(next); @@ -52,7 +52,7 @@ public async Task WriteAsync_when_write_fails_and_read_shows_different_state_ret WriteException = writeException, OnRead = state => state.State = new TestState("other") }; - var manager = new StateManager(storage); + var manager = new DefaultStateManager(storage); var ex = await Assert.ThrowsAsync(() => manager.WriteAsync(new TestState("next"))); @@ -69,7 +69,7 @@ public async Task WriteAsync_when_write_and_read_fail_reverts_state_and_rethrows WriteException = writeException, ReadException = new InvalidOperationException("read failed") }; - var manager = new StateManager(storage); + var manager = new DefaultStateManager(storage); var ex = await Assert.ThrowsAsync(() => manager.WriteAsync(new TestState("next"))); @@ -82,7 +82,7 @@ public async Task WriteAsync_on_versioned_state_stamps_new_version() { var original = new VersionedTestState("initial"); var storage = new FakePersistentVersionedState(original); - var manager = new StateManager(storage); + var manager = new DefaultStateManager(storage); var next = new VersionedTestState("next") { Version = Guid.Empty }; await manager.WriteAsync(next); From c0b6df3977827a59262f6bafdb668711e6466ba0 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 18:49:53 +0000 Subject: [PATCH 16/82] feat(om): implement stream manager subscriptions StreamManager now separates subscription configuration from activation-time subscription setup via RegisterStreamManager, AddSubscription, and SubscribeAsync. This lets grains await Orleans stream subscription establishment without blocking activation and keeps subscription failures visible. MessageTracker now records receive-lag histograms for accepted enriched stream cursors and outbox tokens, with shared telemetry plumbing kept in MessagingTelemetry. The Messaging test suite adds an in-process Orleans cluster with memory streams covering ValueTask, Task, default error handling, and resume-token behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/api-design.md | 20 +- .../Egil.Orleans.Messaging/MessageTracker.cs | 5 + .../MessagingTelemetry.cs | 86 ++++++ .../Egil.Orleans.Messaging/StreamManager.cs | 253 +++++++++++++++++- .../StreamManagerExtensions.cs | 29 +- .../Egil.Orleans.Messaging.Tests.csproj | 1 + .../MessagingTestClusterFixture.cs | 79 ++++++ .../StreamManagerTests.cs | 194 ++++++++++++++ 8 files changed, 635 insertions(+), 32 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 49c7d5fa..ef99fc48 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -639,7 +639,7 @@ fake-clock tests. Grain-level facade around Orleans implicit subscriptions. Four responsibilities: -1. **Subscribe** to stream namespaces during `OnActivateAsync`. +1. **Configure** stream namespaces during `OnActivateAsync`. 2. **Resume** from last accepted `StreamCursor` per namespace. 3. **Dispatch** with projected `StreamCursor`. 4. **Per-subscription error handling** via the optional `onError` callback. @@ -654,22 +654,24 @@ uses `StreamManager` as a field. ```csharp public sealed class StreamManager { - public StreamManager Subscribe( + public StreamManager AddSubscription( string streamNamespace, Func onNextAsync, Action? onError = default, bool passLatestSequenceTokenOnResume = true); - public StreamManager Subscribe( + public StreamManager AddSubscription( string streamNamespace, Func onNextAsync, Action? onError = default, bool passLatestSequenceTokenOnResume = true); + + public Task SubscribeAsync(CancellationToken cancellationToken = default); } public static class StreamManagerExtensions { - public static StreamManager InitializeStreamManager( + public static StreamManager RegisterStreamManager( this TGrain grain, MessageTracker trackerSnapshot) where TGrain : IGrainBase; @@ -685,9 +687,11 @@ public override async Task OnActivateAsync(CancellationToken ct) var state = await stateManager.ReadAsync(); state.Tracker.RegisterTimeProvider(timeProvider); - streamManager = this.InitializeStreamManager(state.Tracker) - .Subscribe("electricity-prices", HandlePriceTickAsync, LogStreamError) - .Subscribe("tariff-events", HandleTariffChangedAsync); + streamManager = this.RegisterStreamManager(state.Tracker) + .AddSubscription("electricity-prices", HandlePriceTickAsync, LogStreamError) + .AddSubscription("tariff-events", HandleTariffChangedAsync); + + await streamManager.SubscribeAsync(ct); } ``` @@ -697,7 +701,7 @@ group. Users only specify it for inline lambdas or ambiguous method groups. ### Resume semantics - Each subscription reads `trackerSnapshot.LatestStream(streamId)` once - when `Subscribe(...)` activates it. + when `SubscribeAsync(...)` establishes configured subscriptions. - If cursor exists → hydrate `StreamSequenceToken`, pass to `SubscribeAsync`. - If null → subscribe without token (provider default, typically: diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs index 0411013e..eecee2d8 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs @@ -114,6 +114,7 @@ public bool ProcessMessage(StreamCursor cursor, out MessageTracker next) if (!streams.TryGetValue(cursor.StreamId, out var entry)) { + MessagingTelemetry.RecordStreamReceiveLag(cursor, now); next = CreateTracker(streams.Add(cursor.StreamId, new StreamEntry(cursor, now)), outbox); return true; } @@ -124,6 +125,7 @@ public bool ProcessMessage(StreamCursor cursor, out MessageTracker next) return false; } + MessagingTelemetry.RecordStreamReceiveLag(cursor, now); next = CreateTracker(streams.SetItem(cursor.StreamId, new StreamEntry(cursor, now)), outbox); return true; } @@ -145,18 +147,21 @@ public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) if (!outbox.TryGetValue(token.Sender, out var entry)) { + MessagingTelemetry.RecordOutboxReceiveLag(token, now); next = CreateTracker(streams, outbox.Add(token.Sender, new OutboxEntry(token.Epoch, token.SequenceNumber, now))); return true; } if (token.Epoch > entry.Epoch) { + MessagingTelemetry.RecordOutboxReceiveLag(token, now); next = CreateTracker(streams, outbox.SetItem(token.Sender, new OutboxEntry(token.Epoch, token.SequenceNumber, now))); return true; } if (token.Epoch == entry.Epoch && token.SequenceNumber > entry.LastSequenceNumber) { + MessagingTelemetry.RecordOutboxReceiveLag(token, now); next = CreateTracker(streams, outbox.SetItem(token.Sender, new OutboxEntry(entry.Epoch, token.SequenceNumber, now))); return true; } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs new file mode 100644 index 00000000..c713d959 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs @@ -0,0 +1,86 @@ +using System.Diagnostics; +using System.Diagnostics.Metrics; + +namespace Egil.Orleans.Messaging; + +internal static class MessagingTelemetry +{ + private static readonly Meter Meter = new("egil.orleans.messaging"); + private static readonly Counter StreamMessages = Meter.CreateCounter("stream.messages"); + private static readonly Counter StreamSubscriptions = Meter.CreateCounter("stream.subscriptions"); + private static readonly Counter StreamHandlerErrors = Meter.CreateCounter("stream.handler.errors"); + private static readonly Histogram StreamHandlerDuration = Meter.CreateHistogram("stream.handler.duration", "ms"); + private static readonly Histogram StreamReceiveLag = Meter.CreateHistogram( + "stream.message.receive.lag", + "ms", + "Elapsed time between an enriched stream message enqueue time and receiver-side acceptance."); + private static readonly Histogram OutboxReceiveLag = Meter.CreateHistogram( + "outbox.message.receive.lag", + "ms", + "Elapsed time between an outbox message timestamp and receiver-side acceptance."); + + public static readonly ActivitySource ActivitySource = new("egil.orleans.messaging"); + + public static void RecordStreamMessage(string streamNamespace, string status) => + StreamMessages.Add(1, StreamTags(streamNamespace, status)); + + public static void RecordStreamSubscription(string streamNamespace, string status) => + StreamSubscriptions.Add(1, StreamTags(streamNamespace, status)); + + public static void RecordStreamHandlerError(string streamNamespace) => + StreamHandlerErrors.Add(1, StreamTags(streamNamespace, "handler")); + + public static void RecordStreamHandlerDuration(string streamNamespace, string status, double milliseconds) => + StreamHandlerDuration.Record(milliseconds, StreamTags(streamNamespace, status)); + + public static void RecordOutboxReceiveLag(OutboxSequenceToken token, DateTimeOffset receivedAt) + { + var lag = receivedAt - token.Timestamp; + OutboxReceiveLag.Record( + lag.TotalMilliseconds, + new TagList + { + { "messaging.system", "orleans" }, + { "messaging.operation", "receive" }, + { "messaging.source.kind", "outbox" }, + { "status", "accepted" }, + { "sender.grain.type", token.Sender.Type.ToString() } + }); + } + + public static void RecordStreamReceiveLag(StreamCursor cursor, DateTimeOffset receivedAt) + { + if (!cursor.TryGetEnqueuedTime(out var enqueuedTime)) + { + return; + } + + var lag = receivedAt - enqueuedTime; + var tags = new TagList + { + { "messaging.system", "orleans" }, + { "messaging.operation", "receive" }, + { "messaging.source.kind", "stream" }, + { "stream.namespace", cursor.StreamId.GetNamespace() }, + { "status", "accepted" } + }; + + if (cursor.TryGetStreamProviderName(out var streamProviderName)) + { + tags.Add("stream.provider", streamProviderName); + } + + StreamReceiveLag.Record(lag.TotalMilliseconds, tags); + } + + private static TagList StreamTags(string streamNamespace, string status) + { + var tags = new TagList + { + { "stream.namespace", streamNamespace }, + { "status", status } + }; + + return tags; + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs index 31bd7a41..56120712 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs @@ -1,3 +1,6 @@ +using System.Diagnostics; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; using Orleans.Streams; namespace Egil.Orleans.Messaging; @@ -16,9 +19,10 @@ namespace Egil.Orleans.Messaging; /// /// public override async Task OnActivateAsync(CancellationToken ct) /// { -/// streamManager = this.InitializeStreamManager(state.Tracker) -/// .Subscribe("electricity-prices", HandlePriceTickAsync, LogStreamError) -/// .Subscribe("tariff-events", HandleTariffChangedAsync); +/// streamManager = this.RegisterStreamManager(state.Tracker) +/// .AddSubscription("electricity-prices", HandlePriceTickAsync, LogStreamError) +/// .AddSubscription("tariff-events", HandleTariffChangedAsync); +/// await streamManager.SubscribeAsync(ct); /// } /// /// @@ -49,7 +53,27 @@ namespace Egil.Orleans.Messaging; /// public sealed class StreamManager { - private StreamManager() { } + private readonly IGrainBase owner; + private readonly MessageTracker trackerSnapshot; + private readonly Func getStreamProvider; + private readonly Func getStreamId; + private readonly ILogger logger; + private readonly List> subscriptions = []; + private readonly List subscriptionHandles = []; + + private StreamManager( + IGrainBase owner, + MessageTracker trackerSnapshot, + Func getStreamProvider, + Func getStreamId, + ILogger logger) + { + this.owner = owner; + this.trackerSnapshot = trackerSnapshot; + this.getStreamProvider = getStreamProvider; + this.getStreamId = getStreamId; + this.logger = logger; + } /// /// Creates a for the given grain. @@ -65,11 +89,40 @@ internal static StreamManager Create( IGrainBase owner, MessageTracker trackerSnapshot) { - throw new NotImplementedException(); + ArgumentNullException.ThrowIfNull(owner); + ArgumentNullException.ThrowIfNull(trackerSnapshot); + + var services = owner.GrainContext.ActivationServices; + var loggerFactory = services.GetService(); + var logger = loggerFactory?.CreateLogger() + ?? Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance; + + return Create( + owner, + trackerSnapshot, + streamNamespace => services.GetRequiredKeyedService(streamNamespace), + streamNamespace => CreateStreamId(owner, streamNamespace), + logger); + } + + internal static StreamManager Create( + IGrainBase owner, + MessageTracker trackerSnapshot, + Func getStreamProvider, + Func getStreamId, + ILogger logger) + { + ArgumentNullException.ThrowIfNull(owner); + ArgumentNullException.ThrowIfNull(trackerSnapshot); + ArgumentNullException.ThrowIfNull(getStreamProvider); + ArgumentNullException.ThrowIfNull(getStreamId); + ArgumentNullException.ThrowIfNull(logger); + + return new StreamManager(owner, trackerSnapshot, getStreamProvider, getStreamId, logger); } /// - /// Declares a subscription to the given . + /// Adds a subscription configuration for the given . /// /// /// The expected event type on this stream namespace. Must match the type @@ -102,13 +155,25 @@ internal static StreamManager Create( /// /// This instance for fluent subscription chaining. /// - public StreamManager Subscribe( + public StreamManager AddSubscription( string streamNamespace, Func onNextAsync, Action? onError = default, bool passLatestSequenceTokenOnResume = true) { - throw new NotImplementedException(); + ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); + ArgumentNullException.ThrowIfNull(onNextAsync); + + var streamProvider = getStreamProvider(streamNamespace); + var streamId = getStreamId(streamNamespace); + var stream = streamProvider.GetStream(streamId); + var observer = new StreamObserver(this, streamNamespace, streamId, onNextAsync, onError); + var latestCursor = passLatestSequenceTokenOnResume + ? trackerSnapshot.LatestStream(streamId) + : null; + + subscriptions.Add(ct => SubscribeCoreAsync(streamNamespace, stream, observer, latestCursor?.Token, ct)); + return this; } /// @@ -150,12 +215,180 @@ public StreamManager Subscribe( /// /// This instance for fluent subscription chaining. /// - public StreamManager Subscribe( + public StreamManager AddSubscription( string streamNamespace, Func onNextAsync, Action? onError = default, bool passLatestSequenceTokenOnResume = true) { - throw new NotImplementedException(); + ArgumentNullException.ThrowIfNull(onNextAsync); + + return AddSubscription( + streamNamespace, + (item, token) => new ValueTask(onNextAsync(item, token)), + onError, + passLatestSequenceTokenOnResume); + } + + /// + /// Establishes all configured Orleans stream subscriptions. + /// + /// Cancellation token for activation-time subscription setup. + public async Task SubscribeAsync(CancellationToken cancellationToken = default) + { + cancellationToken.ThrowIfCancellationRequested(); + await Task.WhenAll(subscriptions.Select(subscription => subscription(cancellationToken))).ConfigureAwait(false); + } + + private async Task SubscribeCoreAsync( + string streamNamespace, + IAsyncStream stream, + IAsyncObserver observer, + StreamSequenceToken? token, + CancellationToken cancellationToken) + { + try + { + cancellationToken.ThrowIfCancellationRequested(); + + if (token is null) + { + var handle = await stream.SubscribeAsync(observer).ConfigureAwait(false); + subscriptionHandles.Add(handle); + } + else + { + var handle = await stream.SubscribeAsync(observer, token, null!).ConfigureAwait(false); + subscriptionHandles.Add(handle); + } + + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); + } + catch (Exception ex) + { + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "errored"); + logger.LogError(ex, "Stream subscription failed for namespace {StreamNamespace}.", streamNamespace); + throw; + } + } + + private async Task OnNextAsync( + string streamNamespace, + StreamId streamId, + TEvent item, + StreamSequenceToken? token, + Func onNextAsync, + Action? onError) + { + var cursor = new StreamCursor(streamId, token); + var started = Stopwatch.GetTimestamp(); + + using var activity = StartConsumerActivity(streamNamespace, cursor); + try + { + await onNextAsync(item, token!).ConfigureAwait(false); + + MessagingTelemetry.RecordStreamMessage(streamNamespace, "accepted"); + MessagingTelemetry.RecordStreamHandlerDuration(streamNamespace, "accepted", Stopwatch.GetElapsedTime(started).TotalMilliseconds); + } + catch (Exception ex) + { + activity?.SetStatus(ActivityStatusCode.Error, ex.Message); + activity?.AddTag("exception.type", ex.GetType().FullName); + activity?.AddTag("exception.message", ex.Message); + MessagingTelemetry.RecordStreamMessage(streamNamespace, "rejected"); + MessagingTelemetry.RecordStreamHandlerError(streamNamespace); + MessagingTelemetry.RecordStreamHandlerDuration(streamNamespace, "rejected", Stopwatch.GetElapsedTime(started).TotalMilliseconds); + HandleError(streamNamespace, ex, onError); + } + } + + private Task OnErrorAsync(string streamNamespace, Exception ex, Action? onError) + { + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "errored"); + HandleError(streamNamespace, ex, onError); + return Task.CompletedTask; + } + + private void HandleError(string streamNamespace, Exception ex, Action? onError) + { + if (onError is null) + { + logger.LogError(ex, "Stream handler failed for namespace {StreamNamespace}.", streamNamespace); + return; + } + + try + { + onError(streamNamespace, ex); + } + catch (Exception callbackException) + { + logger.LogError( + callbackException, + "Stream error callback failed for namespace {StreamNamespace}. Original error: {OriginalError}", + streamNamespace, + ex.Message); + } + } + + private Activity? StartConsumerActivity(string streamNamespace, StreamCursor cursor) + { + var tags = new KeyValuePair[] + { + new("messaging.system", "orleans"), + new("messaging.operation", "process"), + new("messaging.destination.name", streamNamespace), + new("orleans.grain.id", owner.GrainContext.GrainId.ToString()) + }; + + if (cursor.TryGetTraceParent(out var traceParent) + && ActivityContext.TryParse(traceParent, traceState: null, isRemote: true, out var linkedContext)) + { + return MessagingTelemetry.ActivitySource.StartActivity( + "orleans.stream.process", + ActivityKind.Consumer, + parentContext: default, + tags: tags, + links: [new ActivityLink(linkedContext)]); + } + + return MessagingTelemetry.ActivitySource.StartActivity( + "orleans.stream.process", + ActivityKind.Consumer, + parentContext: default, + tags: tags); + } + + private static StreamId CreateStreamId(IGrainBase owner, string streamNamespace) + { + var grainId = owner.GrainContext.GrainId; + if (GrainIdKeyExtensions.TryGetGuidKey(grainId, out var guidKey, out _)) + { + return StreamId.Create(streamNamespace, guidKey); + } + + if (GrainIdKeyExtensions.TryGetIntegerKey(grainId, out var longKey, out _)) + { + return StreamId.Create(streamNamespace, longKey); + } + + return StreamId.Create(streamNamespace, grainId.Key.ToString()); + } + + private sealed class StreamObserver( + StreamManager manager, + string streamNamespace, + StreamId streamId, + Func onNextAsync, + Action? onError) + : IAsyncObserver + { + public Task OnNextAsync(TEvent item, StreamSequenceToken? token = null) => + manager.OnNextAsync(streamNamespace, streamId, item, token, onNextAsync, onError); + + public Task OnCompletedAsync() => Task.CompletedTask; + + public Task OnErrorAsync(Exception ex) => manager.OnErrorAsync(streamNamespace, ex, onError); } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs index cece3fc3..00fae801 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs @@ -1,29 +1,30 @@ namespace Egil.Orleans.Messaging; /// -/// Extension methods for wiring into a grain's +/// Extension methods for registering into a grain's /// activation lifecycle. /// public static class StreamManagerExtensions { /// - /// Creates a for the grain and returns it for - /// fluent stream subscription configuration. + /// Registers a for the grain and returns it + /// for fluent stream subscription configuration. /// /// /// - /// Call from OnActivateAsync, then chain - /// Subscribe calls on the returned manager. + /// Call from OnActivateAsync, chain AddSubscription calls + /// on the returned manager, then await SubscribeAsync. /// /// /// Usage: /// - /// public override async Task OnActivateAsync(CancellationToken ct) - /// { - /// streamManager = this.InitializeStreamManager(state.Tracker) - /// .Subscribe("electricity-prices", HandlePriceTickAsync); - /// } - /// +/// public override async Task OnActivateAsync(CancellationToken ct) +/// { +/// streamManager = this.RegisterStreamManager(state.Tracker) +/// .AddSubscription("electricity-prices", HandlePriceTickAsync); +/// await streamManager.SubscribeAsync(ct); +/// } +/// /// /// /// @@ -36,10 +37,10 @@ public static class StreamManagerExtensions /// time. Used to look up resume tokens. /// /// - /// The initialized . Store it in a grain field so - /// the subscriptions remain rooted for the activation lifetime. + /// The registered . Store it in a grain field so + /// configured subscriptions remain rooted for the activation lifetime. /// - public static StreamManager InitializeStreamManager( + public static StreamManager RegisterStreamManager( this TGrain grain, MessageTracker trackerSnapshot) where TGrain : IGrainBase diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj index 01209a6d..cd4053db 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj @@ -29,6 +29,7 @@ + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs new file mode 100644 index 00000000..079ddfbe --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs @@ -0,0 +1,79 @@ +using System.Runtime.CompilerServices; +using Egil.Orleans.Testing; +using Orleans.Streams; +using Orleans.TestingHost; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class MessagingTestClusterFixture : IAsyncLifetime, IGrainActivityWaiter +{ + private InProcessTestCluster? cluster; + + public GrainActivityCollector Collector { get; } = new(); + + public InProcessTestCluster Cluster => cluster ?? throw new InvalidOperationException("Test cluster not initialized."); + + public IGrainFactory GrainFactory => Cluster.Client; + + public async ValueTask InitializeAsync() + { + var builder = new InProcessTestClusterBuilder(initialSilosCount: 1); + + builder.ConfigureSilo((_, siloBuilder) => + { + siloBuilder.AddMemoryGrainStorage("Default"); + siloBuilder.AddMemoryGrainStorage("PubSubStore"); + + AddStreamProviders(siloBuilder); + + siloBuilder.AddGrainActivityCollector(Collector) + .CollectStorageActivityFrom("Default"); + }); + + builder.ConfigureClient(AddStreamProviders); + + cluster = builder.Build(); + await cluster.DeployAsync(); + } + + public async ValueTask DisposeAsync() + { + if (cluster is not null) + { + await cluster.DisposeAsync(); + } + } + + public TGrain GetUniqueGrain([CallerMemberName] string memberName = "") + where TGrain : IGrain + => (TGrain)GrainFactory.GetGrain(typeof(TGrain), Guid.NewGuid()); + + public IAsyncStream GetStream(string streamNamespace, Guid key) + { + var provider = Cluster.Client.GetStreamProvider(streamNamespace); + return provider.GetStream(StreamId.Create(streamNamespace, key)); + } + + Task IGrainActivityWaiter.WaitForAssertionAsync( + Func> assertion, + Predicate? filter, + TimeSpan? timeout, + CancellationToken cancellationToken) + => ((IGrainActivityWaiter)Collector).WaitForAssertionAsync(assertion, filter, timeout, cancellationToken); + + private static void AddStreamProviders(ISiloBuilder siloBuilder) + { + siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.ValueTask); + siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Task); + siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Failure); + siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Resume); + } + + private static void AddStreamProviders(IClientBuilder clientBuilder) + { + clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.ValueTask); + clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Task); + clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Failure); + clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Resume); + } +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs new file mode 100644 index 00000000..9b27233b --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs @@ -0,0 +1,194 @@ +using Egil.Orleans.Testing; +using Orleans.Providers.Streams.Common; +using Orleans.Streams; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class StreamManagerTests(MessagingTestClusterFixture fixture) : IClassFixture +{ + [Fact] + public async Task AddSubscription_with_ValueTask_handler_delivers_stream_messages() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.EnsureActiveAsync(); + var stream = fixture.GetStream(StreamManagerTestNamespaces.ValueTask, grainKey); + + await stream.OnNextAsync("value-task"); + + await fixture.WaitForAssertionAsync( + async () => Assert.Equal("value-task", (await grain.GetStateAsync()).ValueTaskValue), + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task AddSubscription_with_Task_handler_delivers_stream_messages() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.EnsureActiveAsync(); + var stream = fixture.GetStream(StreamManagerTestNamespaces.Task, grainKey); + + await stream.OnNextAsync("task"); + + await fixture.WaitForAssertionAsync( + async () => Assert.Equal("task", (await grain.GetStateAsync()).TaskValue), + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task Default_error_handler_does_not_break_subscription_when_handler_throws() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.EnsureActiveAsync(); + var stream = fixture.GetStream(StreamManagerTestNamespaces.Failure, grainKey); + + await stream.OnNextAsync("fail"); + await stream.OnNextAsync("after-failure"); + + await fixture.WaitForAssertionAsync( + async () => Assert.Equal("after-failure", (await grain.GetStateAsync()).FailureValue), + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task SubscribeAsync_uses_tracker_snapshot_when_resume_token_is_configured() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.SeedResumeCursorAsync(7); + await grain.DeactivateAsync(); + + grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.EnsureActiveAsync(); + var stream = fixture.GetStream(StreamManagerTestNamespaces.Resume, grainKey); + + await stream.OnNextAsync("resumed"); + + await fixture.WaitForAssertionAsync( + async () => Assert.Equal("resumed", (await grain.GetStateAsync()).ResumeValue), + ct: TestContext.Current.CancellationToken); + } +} + +internal static class StreamManagerTestNamespaces +{ + public const string ValueTask = "stream-manager-valuetask"; + public const string Task = "stream-manager-task"; + public const string Failure = "stream-manager-failure"; + public const string Resume = "stream-manager-resume"; +} + +public interface IStreamManagerTestGrain : IGrainWithGuidKey +{ + Task EnsureActiveAsync(); + + Task SeedResumeCursorAsync(long sequenceNumber); + + Task DeactivateAsync(); + + Task GetStateAsync(); +} + +[GenerateSerializer] +public sealed class StreamManagerTestState +{ + [Id(0)] + public MessageTracker Tracker { get; set; } = new(); + + [Id(1)] + public string? ValueTaskValue { get; set; } + + [Id(2)] + public string? TaskValue { get; set; } + + [Id(3)] + public string? FailureValue { get; set; } + + [Id(4)] + public string? ResumeValue { get; set; } +} + +public sealed class StreamManagerTestGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IStreamManagerTestGrain +{ + private StreamManager? streamManager; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + streamManager = this.RegisterStreamManager(state.State.Tracker) + .AddSubscription(StreamManagerTestNamespaces.ValueTask, HandleValueTaskAsync) + .AddSubscription(StreamManagerTestNamespaces.Task, HandleTaskAsync) + .AddSubscription(StreamManagerTestNamespaces.Failure, HandleFailureAsync) + .AddSubscription(StreamManagerTestNamespaces.Resume, HandleResumeAsync); + + await streamManager.SubscribeAsync(cancellationToken); + await base.OnActivateAsync(cancellationToken); + } + + public Task EnsureActiveAsync() => Task.CompletedTask; + + public async Task SeedResumeCursorAsync(long sequenceNumber) + { + var cursor = new StreamCursor( + StreamId.Create(StreamManagerTestNamespaces.Resume, this.GetPrimaryKey()), + new EventSequenceToken(sequenceNumber)); + + state.State.Tracker.ProcessMessage(cursor, out var next); + state.State.Tracker = next; + await state.WriteStateAsync(); + } + + public Task DeactivateAsync() + { + DeactivateOnIdle(); + return Task.CompletedTask; + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private async ValueTask HandleValueTaskAsync(string item, StreamSequenceToken token) + { + TrackStreamCursor(StreamManagerTestNamespaces.ValueTask, token); + state.State.ValueTaskValue = item; + await state.WriteStateAsync(); + } + + private async Task HandleTaskAsync(string item, StreamSequenceToken token) + { + TrackStreamCursor(StreamManagerTestNamespaces.Task, token); + state.State.TaskValue = item; + await state.WriteStateAsync(); + } + + private async ValueTask HandleFailureAsync(string item, StreamSequenceToken token) + { + if (item == "fail") + { + throw new InvalidOperationException("Handler failed."); + } + + TrackStreamCursor(StreamManagerTestNamespaces.Failure, token); + state.State.FailureValue = item; + await state.WriteStateAsync(); + } + + private async ValueTask HandleResumeAsync(string item, StreamSequenceToken token) + { + TrackStreamCursor(StreamManagerTestNamespaces.Resume, token); + state.State.ResumeValue = item; + await state.WriteStateAsync(); + } + + private void TrackStreamCursor(string streamNamespace, StreamSequenceToken token) + { + var cursor = new StreamCursor(StreamId.Create(streamNamespace, this.GetPrimaryKey()), token); + if (state.State.Tracker.ProcessMessage(cursor, out var next)) + { + state.State.Tracker = next; + } + + } +} From 57f02370ef2f79bd04beab9cbd5c0c0e1cda8a0a Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 18:58:35 +0000 Subject: [PATCH 17/82] refactor(om): rename state manager activation API [skip notes] Rename InitializeStateManager to RegisterStateManager so state and stream activation helpers use the same registration language before the package ships. Update state manager docs and tests to match the new API name. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/api-design.md | 44 +++++++++++-------- .../Egil.Orleans.Messaging/IStateManager.cs | 2 +- .../StateManagerExtensions.cs | 8 ++-- .../StateManagerExtensionsTests.cs | 20 ++++----- 4 files changed, 41 insertions(+), 33 deletions(-) diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index ef99fc48..1cbd91b3 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -65,10 +65,11 @@ that guarantees the grain's observable `State` is never out of sync with what is durably persisted, even when `WriteStateAsync` fails ambiguously (timeout, network drop, server 5xx, ETag conflict). -Grain code injects `IStateManager`; the raw -`IPersistentState` stays internal to the wrapper. This is -non-negotiable — exposing both is the loophole that lets grain authors -read stale `storage.State` after a failed write. +Grain code injects `IPersistentState` as normal, then registers an +`IStateManager` wrapper during activation. After that point, the raw +`IPersistentState` should stay internal to the wrapper. This is +non-negotiable — exposing both is the loophole that lets grain authors read +stale `storage.State` after a failed write. ### Interface @@ -253,33 +254,40 @@ Extension methods can't provide this fence because the grain still holds ### Grain wiring ```csharp -// User injects IPersistentState as normal, wraps in OnActivateAsync. +// User injects IPersistentState as normal, registers wrapper in activation. [PersistentState("state")] IPersistentState storage; IStateManager stateManager; public override Task OnActivateAsync(CancellationToken ct) { - stateManager = storage.AsStateManager(); + stateManager = this.RegisterStateManager("state", storage); // ... } ``` -`AsStateManager()` is an extension method on `IPersistentState`. -Internally: +`RegisterStateManager()` is an extension method on `IGrainBase`. Internally: ```csharp -public static IStateManager AsStateManager(this IPersistentState storage) - where T : class, IEquatable -{ - // Future: check DI for IStateManagerFactory to resolve - // provider-specific implementations. - return new StateManager(storage); -} +public static IStateManager RegisterStateManager( + this TGrain grain, + string storageName, + IPersistentState storage) + where TGrain : IGrainBase + where TState : class, IEquatable +``` + +The silo must register a keyed `IStateManagerFactory` for each storage +name used by grains: + +```csharp +siloBuilder.AddDefaultStateManager("state"); ``` -No DI registration needed for v1. When provider-specific overrides ship -later, the extension checks DI for a registered `IStateManagerFactory`, -falls back to default `StateManager`. No breaking change. +Provider-specific overrides can be supplied with `AddStateManagerFactory(...)`. +The activation-time `RegisterStateManager(...)` call mirrors +`RegisterStreamManager(...)` and `RegisterGrainTimer(...)`: the grain registers +the runtime helper once during activation and uses the returned manager for +all subsequent state reads and writes. --- diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs index 37fe0a95..6fe45a34 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs @@ -20,7 +20,7 @@ namespace Egil.Orleans.Messaging; /// Usage: Inject as normal via /// [PersistentState], then initialize it during OnActivateAsync: /// -/// stateManager = this.InitializeStateManager("state", storage); +/// stateManager = this.RegisterStateManager("state", storage); /// /// The raw should not be accessed /// directly after wrapping — doing so bypasses the committed-state fence. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs index da25d4ed..9d2a9d45 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs @@ -21,7 +21,7 @@ public static class StateManagerExtensions /// /// public override Task OnActivateAsync(CancellationToken ct) /// { - /// stateManager = this.InitializeStateManager("state", storage); + /// stateManager = this.RegisterStateManager("state", storage); /// // ... /// } /// @@ -51,7 +51,7 @@ public static class StateManagerExtensions /// /// A keyed instance. /// - public static IStateManager InitializeStateManager( + public static IStateManager RegisterStateManager( this TGrain grain, string storageName, IPersistentState storage) @@ -62,14 +62,14 @@ public static IStateManager InitializeStateManager( ArgumentException.ThrowIfNullOrWhiteSpace(storageName); ArgumentNullException.ThrowIfNull(storage); - return InitializeStateManagerCore( + return RegisterStateManagerCore( grain.GrainContext.ActivationServices, storageName, storage, grain.GetType()); } - internal static IStateManager InitializeStateManagerCore( + internal static IStateManager RegisterStateManagerCore( IServiceProvider activationServices, string storageName, IPersistentState storage, diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs index 8fa31814..d4f4038f 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs @@ -5,7 +5,7 @@ namespace Egil.Orleans.Messaging.Tests; public sealed class StateManagerExtensionsTests { [Fact] - public void InitializeStateManagerCore_resolves_keyed_factory_and_wraps_persistent_state() + public void RegisterStateManagerCore_resolves_keyed_factory_and_wraps_persistent_state() { var storageName = "state"; var services = new ServiceCollection(); @@ -13,7 +13,7 @@ public void InitializeStateManagerCore_resolves_keyed_factory_and_wraps_persiste var provider = services.BuildServiceProvider(); var storage = new FakePersistentState(new TestState("initial")); - var manager = StateManagerExtensions.InitializeStateManagerCore( + var manager = StateManagerExtensions.RegisterStateManagerCore( provider, storageName, storage, @@ -24,13 +24,13 @@ public void InitializeStateManagerCore_resolves_keyed_factory_and_wraps_persiste } [Fact] - public void InitializeStateManagerCore_throws_for_missing_keyed_registration() + public void RegisterStateManagerCore_throws_for_missing_keyed_registration() { var provider = new ServiceCollection().BuildServiceProvider(); var storage = new FakePersistentState(new TestState("initial")); var ex = Assert.Throws(() => - StateManagerExtensions.InitializeStateManagerCore( + StateManagerExtensions.RegisterStateManagerCore( provider, "missing", storage, @@ -42,35 +42,35 @@ public void InitializeStateManagerCore_throws_for_missing_keyed_registration() } [Fact] - public void InitializeStateManager_throws_for_null_grain() + public void RegisterStateManager_throws_for_null_grain() { var storage = new FakePersistentState(new TestState("initial")); var ex = Assert.Throws(() => - StateManagerExtensions.InitializeStateManager(null!, "state", storage)); + StateManagerExtensions.RegisterStateManager(null!, "state", storage)); Assert.Equal("grain", ex.ParamName); } [Fact] - public void InitializeStateManager_throws_for_empty_storage_name() + public void RegisterStateManager_throws_for_empty_storage_name() { var grain = new FakeGrainBase(); var storage = new FakePersistentState(new TestState("initial")); var ex = Assert.Throws(() => - grain.InitializeStateManager(" ", storage)); + grain.RegisterStateManager(" ", storage)); Assert.Equal("storageName", ex.ParamName); } [Fact] - public void InitializeStateManager_throws_for_null_storage() + public void RegisterStateManager_throws_for_null_storage() { var grain = new FakeGrainBase(); var ex = Assert.Throws(() => - grain.InitializeStateManager("state", null!)); + grain.RegisterStateManager("state", null!)); Assert.Equal("storage", ex.ParamName); } From 3a5671086180d96c51bc3fcc114595e057b0f002 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 19:31:52 +0000 Subject: [PATCH 18/82] docs(docs): update outbox processor design Capture the current OutboxProcessor design direction, including background posting, timer interleaving options, postman execution modes, and related telemetry guidance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/api-design.md | 129 ++++++++++++++++++++++++--- 1 file changed, 119 insertions(+), 10 deletions(-) diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 1cbd91b3..a8cf25fe 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -877,7 +877,7 @@ Grains using this library should follow a functional-command model: ## 7. `OutboxProcessor` — timer + reminder driven dispatch -**Status:** Settled. +**Status:** Open, current direction captured here. Grain-scoped component that owns the timer, reminder, and postman dispatch lifecycle for draining `Outbox`. Modelled after the @@ -891,9 +891,18 @@ in the `Clever.PricingEngine` codebase. - **GrainTimer** for in-process fast retry while activated. - **Durable Reminder** for cross-activation recovery. Reactivates the grain if it deactivates with pending items. Timer arms on activation. -- **Non-reentrant assumption.** Orleans default turn-based concurrency - serialises calls; the processor adds no internal locks. Not safe on - `[Reentrant]` grains — document and enforce. +- **Single active drain.** At most one send attempt may run per activation. + `PostAsync`, `PostInBackgroundAsync`, timer callbacks, and reminder + callbacks all coalesce through the same drain gate. If a post run is + already active, additional requests mark another run as desired rather + than starting concurrently. +- **Callback interleaving is explicit.** `Interleave` mirrors Orleans' + `GrainTimerCreationOptions.Interleave` and controls whether processor + callbacks run interleaved with other grain turns. It does not by itself + permit overlapping post runs. +- **Keep-alive is explicit.** `KeepAlive` mirrors Orleans' + `GrainTimerCreationOptions.KeepAlive` and controls whether an active + processor timer keeps the grain activation alive. ### Postman dispatch @@ -912,6 +921,57 @@ most specific to least specific (like a `switch`). - `PostAsync` only throws `TimeoutException` (per-run timeout), `OperationCanceledException` (caller token), or callback exceptions. +Postman callbacks can run in one of two execution modes: + +```csharp +public enum OutboxPostmanExecutionMode +{ + /// + /// Executes postman callbacks on the Orleans activation scheduler. + /// + /// + /// This is the default and recommended mode. + /// + /// The Orleans activation scheduler is the per-activation scheduler that + /// provides Orleans' turn-based execution model. Work scheduled on it + /// runs as grain work: it is single-threaded per activation and obeys + /// Orleans request scheduling, reentrancy, and timer interleaving rules. + /// + /// Use this mode for normal async postmen. Well-behaved async libraries + /// should be awaited directly and do not need to be moved to the .NET + /// thread pool. + /// + GrainScheduler, + + /// + /// Executes postman callbacks on the .NET thread pool, outside the + /// Orleans activation scheduler. + /// + /// + /// This is an escape hatch for blocking or legacy postmen which would + /// otherwise block the grain scheduler. It should not be used merely + /// because a postman performs asynchronous I/O. + /// + /// Postmen in this mode must not read or mutate activation-local grain + /// state. They may call other grains through IGrainFactory or use + /// external services which are safe to use from thread-pool code. + /// PendingItems, OnPostedAsync, and OnPostFailedAsync still execute on + /// the Orleans activation scheduler. + /// + ThreadPool +} +``` + +`GrainScheduler` is the safe default. Postmen may close over activation +state because they execute under Orleans activation scheduling. + +`ThreadPool` is for blocking or legacy postmen. In this mode postmen must +not read or mutate activation-local grain state. They should be static or +otherwise state-free and use services such as `IGrainFactory` to deliver the +message. Only the fast grain callbacks (`PendingItems`, `OnPostedAsync`, +`OnPostFailedAsync`) run on the Orleans activation scheduler and respect +`Interleave`. + ### Grain integration pattern ```csharp @@ -952,27 +1012,45 @@ extension(TGrain grain) where TGrain : IOutboxGrain, IGrainBase ```csharp public sealed class OutboxProcessorOptions where TOutbox : notnull { - /// Snapshot of pending items. Called once per post run. - public required Func> GetPending { get; init; } + /// Snapshot of pending items. Called once per post run from a grain turn. + public required Func> PendingItems { get; init; } /// Items successfully posted. Grain must remove from backing collection. public required Func, CancellationToken, ValueTask> - OnPostCompletedAsync { get; init; } + OnPostedAsync { get; init; } /// Failed items with exception and attempt count (in-memory, resets on /// reactivation). Grain decides: leave to retry, or remove to /// dead-letter after N attempts. If null, failed items retry silently. public Func, - CancellationToken, ValueTask>? OnPostErrorAsync { get; init; } + CancellationToken, ValueTask>? OnPostFailedAsync { get; init; } /// Max time per post run. Set below grain's response timeout. public TimeSpan ProcessingTimeout { get; init; } = TimeSpan.FromSeconds(20); /// Timer + reminder period. Orleans reminders fire at most once/minute. public TimeSpan RetryDelay { get; init; } = TimeSpan.FromMinutes(2); + + /// Whether processor timer callbacks may interleave with other grain turns. + /// Mirrors GrainTimerCreationOptions.Interleave. + public bool Interleave { get; init; } = false; + + /// Whether an active processor timer keeps the activation alive. + /// Mirrors GrainTimerCreationOptions.KeepAlive. + public bool KeepAlive { get; init; } = false; + + /// Controls whether postmen run on the Orleans activation scheduler or on + /// the .NET thread pool. + public OutboxPostmanExecutionMode PostmanExecution { get; init; } = + OutboxPostmanExecutionMode.GrainScheduler; } ``` +Naming note: `PendingItems` intentionally names the role of the callback +rather than an imperative method (`GetPending`). `OutboxAccessor` was +considered, but it is less precise because the processor does not need +general outbox access, only a pending-item snapshot. + ### `OutboxProcessor` ```csharp @@ -985,11 +1063,20 @@ public sealed partial class OutboxProcessor : IOutboxComponent Func postman) where TSub : TOutbox; public OutboxProcessor AddPostman( Func postman) where TSub : TOutbox; + public OutboxProcessor AddPostman( + Func postman) + where TSub : TOutbox; /// Posts pending items. Safe to call from grain's task scheduler. /// Arms timer/reminder if items remain, unregisters if empty. public ValueTask PostAsync(CancellationToken cancellationToken = default); + /// Schedules posting through the same timer-backed drain path used for + /// retry. Returns after the run has been scheduled, not after posting + /// completes. Multiple calls coalesce into a single active drain. + public ValueTask PostInBackgroundAsync( + CancellationToken cancellationToken = default); + /// Called by IOutboxGrain DIM. No-ops for unknown reminder names. public ValueTask ReceiveReminderAsync(string reminderName, TickStatus status); } @@ -1028,12 +1115,30 @@ public async Task ReceiveReminder(string name, TickStatus status) - **Callback-based, not DI service.** Grain controls dispatch logic, can pass its own state to the postman. DI service adds indirection without clear benefit. +- **`PendingItems`, not `GetPending`.** The option is a callback consumed + by the processor, not a method users call. The name describes the data + supplied to the processor and avoids implying ad-hoc outbox operations. - **First-registered-wins postman matching.** Simple dispatch model. Order most-specific first. Unmatched items → `NoPostmanRegisteredException` - via `OnPostErrorAsync`. + via `OnPostFailedAsync`. - **`PostAsync` swallows per-item errors.** Grain observes failures via - `OnPostErrorAsync` with attempt count. Processor never drops items + `OnPostFailedAsync` with attempt count. Processor never drops items silently unless the grain explicitly removes them. +- **`PostInBackgroundAsync` uses the same path as retry.** It schedules a + timer-backed drain and returns quickly so a grain command can commit an + outbox item and return to its caller without waiting for external + delivery. +- **No overlapping drains, regardless of `Interleave`.** `Interleave` + controls Orleans callback scheduling only. Internal drain gating still + ensures one send attempt at a time. +- **`ThreadPool` is opt-in.** Blocking or legacy delivery can run outside + the Orleans activation scheduler, but postmen in that mode must not + touch activation-local grain state. Use `IGrainFactory` or injected + services instead. +- **Do not offload normal async libraries.** Orleans guidance is to await + well-behaved async libraries directly. `ThreadPool` exists for code + that blocks synchronously or otherwise must escape the Orleans + scheduler. - **One postman per item.** First-registered-wins, not broadcast. Simpler error semantics, no partial-success ambiguity. - **DIM on `IOutboxGrain`.** Zero ceremony. Grain author never writes @@ -1154,6 +1259,7 @@ metrics for state read/write, activation lifecycle, messaging layer. |---------------------------|-----------|--------------------------------------| | `outbox.post.duration` | Histogram | Post run duration (ms) | | `outbox.post.item.duration` | Histogram | Per-item postman dispatch duration | +| `outbox.post.item.age` | Histogram | Message age at dispatch time (ms), when the item carries a sent/enqueued timestamp | | `outbox.post.items` | Counter | Items successfully dispatched | | `outbox.post.errors` | Counter | Items that failed dispatch | | `outbox.depth` | Gauge | Pending items per grain type | @@ -1162,6 +1268,9 @@ metrics for state read/write, activation lifecycle, messaging layer. - `grain.type` — owning grain type name - `event.type` — outbox item type name - `success` — `true`/`false` on per-item histograms +- `postman.execution` — `grain_scheduler` or `thread_pool` +- `failure.type` — exception type name for failed dispatch +- `postman.type` — registered postman target type or delegate owner, when available **ActivitySource:** `egil.orleans.messaging` for distributed traces. From a39ede6152b5ab9833d5c70937be1d3ceb85abe5 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 19:39:24 +0000 Subject: [PATCH 19/82] fix(om): tighten messaging manager invariants Fix stream subscription initialization so handles are collected without concurrent List mutation, repeated SubscribeAsync calls are rejected, and stream handlers receive nullable sequence tokens instead of relying on null-forgiving calls. Recover StateManager clear failures with a read-back path, clamp negative receive-lag telemetry, validate malformed outbox JSON with JsonException, and document the intentional outbox equality fingerprint invariant. Rename the deferred outbox processor activation API references to RegisterOutboxProcessor to align with the other activation-time helpers while leaving the implementation deferred. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/api-design.md | 56 ++++++++++----- .../Egil.Orleans.Messaging/IOutboxGrain.cs | 4 +- .../MessagingTelemetry.cs | 9 ++- .../src/Egil.Orleans.Messaging/Outbox.cs | 12 +++- .../OutboxJsonConverterFactory.cs | 49 ++++++++++--- .../Egil.Orleans.Messaging/OutboxProcessor.cs | 4 +- .../OutboxProcessorExtensions.cs | 10 +-- .../Egil.Orleans.Messaging/StateManager.cs | 39 ++++++++++- .../Egil.Orleans.Messaging/StreamManager.cs | 69 ++++++++++++++----- .../StreamManagerTests.cs | 10 +-- 10 files changed, 197 insertions(+), 65 deletions(-) diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index a8cf25fe..27867ae4 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -664,13 +664,13 @@ public sealed class StreamManager { public StreamManager AddSubscription( string streamNamespace, - Func onNextAsync, + Func onNextAsync, Action? onError = default, bool passLatestSequenceTokenOnResume = true); public StreamManager AddSubscription( string streamNamespace, - Func onNextAsync, + Func onNextAsync, Action? onError = default, bool passLatestSequenceTokenOnResume = true); @@ -911,7 +911,7 @@ in the `Clever.PricingEngine` codebase. is first-registered-wins against the item's runtime type — order from most specific to least specific (like a `switch`). -- Per-item exceptions are caught and surfaced through `OnPostErrorAsync` +- Per-item exceptions are caught and surfaced through `ReconcileFailedAsync` with attempt count (in-memory, resets on reactivation) — the grain decides: leave item in state to retry, or remove to dead-letter after N attempts. @@ -955,8 +955,8 @@ public enum OutboxPostmanExecutionMode /// Postmen in this mode must not read or mutate activation-local grain /// state. They may call other grains through IGrainFactory or use /// external services which are safe to use from thread-pool code. - /// PendingItems, OnPostedAsync, and OnPostFailedAsync still execute on - /// the Orleans activation scheduler. + /// PendingItems, AcknowledgePostedAsync, and ReconcileFailedAsync still + /// execute on the Orleans activation scheduler. /// ThreadPool } @@ -968,9 +968,9 @@ state because they execute under Orleans activation scheduling. `ThreadPool` is for blocking or legacy postmen. In this mode postmen must not read or mutate activation-local grain state. They should be static or otherwise state-free and use services such as `IGrainFactory` to deliver the -message. Only the fast grain callbacks (`PendingItems`, `OnPostedAsync`, -`OnPostFailedAsync`) run on the Orleans activation scheduler and respect -`Interleave`. +message. Only the fast grain callbacks (`PendingItems`, +`AcknowledgePostedAsync`, `ReconcileFailedAsync`) run on the Orleans +activation scheduler and respect `Interleave`. ### Grain integration pattern @@ -988,10 +988,10 @@ public interface IOutboxGrain : IRemindable } } -// 2. C# 14 extension for InitializeOutboxProcessor. +// 2. C# 14 extension for RegisterOutboxProcessor. extension(TGrain grain) where TGrain : IOutboxGrain, IGrainBase { - public OutboxProcessor InitializeOutboxProcessor( + public OutboxProcessor RegisterOutboxProcessor( OutboxProcessorOptions options) where TOutbox : notnull { var services = grain.GrainContext.ActivationServices; @@ -1015,15 +1015,16 @@ public sealed class OutboxProcessorOptions where TOutbox : notnull /// Snapshot of pending items. Called once per post run from a grain turn. public required Func> PendingItems { get; init; } - /// Items successfully posted. Grain must remove from backing collection. + /// Acknowledges successfully posted items. + /// Expected to remove those items from the durable outbox state. public required Func, CancellationToken, ValueTask> - OnPostedAsync { get; init; } + AcknowledgePostedAsync { get; init; } /// Failed items with exception and attempt count (in-memory, resets on /// reactivation). Grain decides: leave to retry, or remove to /// dead-letter after N attempts. If null, failed items retry silently. public Func, - CancellationToken, ValueTask>? OnPostFailedAsync { get; init; } + CancellationToken, ValueTask>? ReconcileFailedAsync { get; init; } /// Max time per post run. Set below grain's response timeout. public TimeSpan ProcessingTimeout { get; init; } = TimeSpan.FromSeconds(20); @@ -1051,6 +1052,21 @@ rather than an imperative method (`GetPending`). `OutboxAccessor` was considered, but it is less precise because the processor does not need general outbox access, only a pending-item snapshot. +`AcknowledgePostedAsync` and `ReconcileFailedAsync` are reconciliation +callbacks, not passive notifications: + +- `AcknowledgePostedAsync` is expected to remove successfully posted items + from the durable outbox and persist that change. If acknowledged items + still appear in `PendingItems` after the callback returns, the processor + treats them as pending and they may be posted again. +- `ReconcileFailedAsync` is the grain's policy hook for failed items. The + grain may leave them in the outbox for retry, remove them, move them to + dead-letter state, or make any other durable state change. If null, failed + items are left pending and retried silently. +- After either callback returns, the processor reads `PendingItems` again + before scheduling retry/reminder work. The latest pending snapshot is the + source of truth. + ### `OutboxProcessor` ```csharp @@ -1092,7 +1108,7 @@ internal interface IOutboxComponent Two obligations (both compiler-enforced): 1. Implement `IOutboxGrain`. -2. Call `InitializeOutboxProcessor(...)` in `OnActivateAsync`. +2. Call `RegisterOutboxProcessor(...)` in `OnActivateAsync`. No `ReceiveReminder` override needed (DIM handles it). No manual timer/reminder lifecycle. No telemetry wiring. @@ -1120,10 +1136,14 @@ public async Task ReceiveReminder(string name, TickStatus status) supplied to the processor and avoids implying ad-hoc outbox operations. - **First-registered-wins postman matching.** Simple dispatch model. Order most-specific first. Unmatched items → `NoPostmanRegisteredException` - via `OnPostFailedAsync`. + via `ReconcileFailedAsync`. - **`PostAsync` swallows per-item errors.** Grain observes failures via - `OnPostFailedAsync` with attempt count. Processor never drops items + `ReconcileFailedAsync` with attempt count. Processor never drops items silently unless the grain explicitly removes them. +- **Acknowledgement is explicit.** Successfully posted items are not removed + by the processor directly. The grain removes and persists them in + `AcknowledgePostedAsync`, preserving the outbox invariant that all durable + state changes go through the owning grain's state manager. - **`PostInBackgroundAsync` uses the same path as retry.** It schedules a timer-backed drain and returns quickly so a grain command can commit an outbox item and return to its caller without waiting for external @@ -1278,7 +1298,7 @@ metrics for state read/write, activation lifecycle, messaging layer. - `IOutboxGrain` — marker + DIM for `ReceiveReminder`. Kept: DIM saves real boilerplate for the 80% case (grains with no other reminders). - Generic constraint on `InitializeOutboxProcessor` ensures type-safe + Generic constraint on `RegisterOutboxProcessor` ensures type-safe opt-in. - `IOutboxComponent` — internal. Not part of public API. @@ -1321,7 +1341,7 @@ specific behavior: - **Interleaved-read grain** — exercises `[AlwaysInterleave]` reads seeing only committed state while a write is in-flight. - **Stuck postman grain** — exercises `ProcessingTimeout` behavior, - `OnPostErrorAsync` with timeout exception. + `ReconcileFailedAsync` with timeout exception. - **Multi-reminder grain** — exercises `IOutboxGrain` DIM with grain that also has its own reminders (DIM shadowing). diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs index 3842c79b..589ffd6e 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs @@ -9,7 +9,7 @@ namespace Egil.Orleans.Messaging; /// /// Zero ceremony: Implementing this interface is one of two obligations /// for grains using the outbox pattern (the other is calling -/// InitializeOutboxProcessor in OnActivateAsync). No manual +/// RegisterOutboxProcessor in OnActivateAsync). No manual /// ReceiveReminder override is needed — the DIM discovers the /// registered on the grain context and /// forwards the callback. @@ -27,7 +27,7 @@ namespace Egil.Orleans.Messaging; /// /// /// -/// Compiler enforcement: The InitializeOutboxProcessor +/// Compiler enforcement: The RegisterOutboxProcessor /// extension method constrains TGrain : IOutboxGrain, IGrainBase, /// so grains that forget to implement this interface get a compile error. /// diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs index c713d959..8f432ab2 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs @@ -35,7 +35,7 @@ public static void RecordStreamHandlerDuration(string streamNamespace, string st public static void RecordOutboxReceiveLag(OutboxSequenceToken token, DateTimeOffset receivedAt) { - var lag = receivedAt - token.Timestamp; + var lag = ClampLag(receivedAt - token.Timestamp); OutboxReceiveLag.Record( lag.TotalMilliseconds, new TagList @@ -55,7 +55,7 @@ public static void RecordStreamReceiveLag(StreamCursor cursor, DateTimeOffset re return; } - var lag = receivedAt - enqueuedTime; + var lag = ClampLag(receivedAt - enqueuedTime); var tags = new TagList { { "messaging.system", "orleans" }, @@ -73,6 +73,11 @@ public static void RecordStreamReceiveLag(StreamCursor cursor, DateTimeOffset re StreamReceiveLag.Record(lag.TotalMilliseconds, tags); } + private static TimeSpan ClampLag(TimeSpan lag) + { + return lag < TimeSpan.Zero ? TimeSpan.Zero : lag; + } + private static TagList StreamTags(string streamNamespace, string status) { var tags = new TagList diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs index fdeb1694..5f358cdc 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs @@ -303,9 +303,17 @@ public Outbox Clear() } /// - /// O(1) equality over sender, sequence metadata, epoch, count, first - /// sequence number, and last sequence number. + /// O(1) equality over the outbox identity and sequence fingerprint: + /// sender, latest sequence number, epoch, count, first sequence number, + /// and last sequence number. /// + /// + /// This intentionally does not compare message payloads. The outbox owns + /// sequence assignment, removal is FIFO-only, and sequence numbers are never + /// reused within an epoch. Under those invariants the metadata fingerprint + /// uniquely represents the pending outbox contents for equality/recovery + /// purposes while avoiding an O(n) payload walk during state comparisons. + /// public bool Equals(Outbox? other) { return ReferenceEquals(this, other) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs index b9dce726..6852df63 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs @@ -51,16 +51,47 @@ private sealed class OutboxJsonConverter : JsonConverter> return null; } - return new Outbox( - GrainId.Create(model.Sender.Type, model.Sender.Key), - model.LatestSequenceNumber, - [.. model.Items.Select(static item => new OutboxMessageEnvelope( + if (model.Sender is null) + { + throw new JsonException("Missing Sender."); + } + + if (model.Items.IsDefault) + { + throw new JsonException("Missing Items."); + } + + var items = ImmutableArray.CreateBuilder>(model.Items.Length); + foreach (var item in model.Items) + { + if (item is null) + { + throw new JsonException("Missing Item."); + } + + if (item.Token is null) + { + throw new JsonException("Missing Token."); + } + + if (item.Token.Sender is null) + { + throw new JsonException("Missing Token.Sender."); + } + + items.Add(new OutboxMessageEnvelope( new OutboxSequenceToken( item.Token.SequenceNumber, GrainId.Create(item.Token.Sender.Type, item.Token.Sender.Key), item.Token.Timestamp, item.Token.Epoch), - item.Message))], + item.Message)); + } + + return new Outbox( + GrainId.Create(model.Sender.Type, model.Sender.Key), + model.LatestSequenceNumber, + items.ToImmutable(), model.Epoch); } @@ -86,18 +117,18 @@ public override void Write( } private sealed record OutboxJsonModel( - GrainIdJsonModel Sender, + GrainIdJsonModel? Sender, long LatestSequenceNumber, DateTimeOffset? Epoch, - ImmutableArray> Items); + ImmutableArray?> Items); private sealed record OutboxMessageEnvelopeJsonModel( - OutboxSequenceTokenJsonModel Token, + OutboxSequenceTokenJsonModel? Token, T Message); private sealed record OutboxSequenceTokenJsonModel( long SequenceNumber, - GrainIdJsonModel Sender, + GrainIdJsonModel? Sender, DateTimeOffset Timestamp, DateTimeOffset Epoch); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs index 60f17571..ab44a786 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs @@ -48,7 +48,7 @@ namespace Egil.Orleans.Messaging; /// Grain integration: Two obligations (both compiler-enforced): /// /// Implement . -/// Call InitializeOutboxProcessor(...) in OnActivateAsync. +/// Call RegisterOutboxProcessor(...) in OnActivateAsync. /// /// No ReceiveReminder override needed — the /// DIM handles it. No manual timer/reminder lifecycle. No telemetry wiring. @@ -150,7 +150,7 @@ public ValueTask ReceiveReminderAsync(string reminderName, TickStatus status) /// /// Attaches this processor to the grain's as /// a component so the DIM can discover it. - /// Called internally by InitializeOutboxProcessor. + /// Called internally by RegisterOutboxProcessor. /// internal void AttachToGrain() { diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs index ca6d69c4..2641d105 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs @@ -23,15 +23,15 @@ public static class OutboxProcessorExtensions /// /// public override async Task OnActivateAsync(CancellationToken ct) /// { - /// outboxProcessor = this.InitializeOutboxProcessor(new OutboxProcessorOptions<IMyEvent> + /// outboxProcessor = this.RegisterOutboxProcessor(new OutboxProcessorOptions<IMyEvent> /// { - /// GetPending = () => stateManager.State.Outbox + /// PendingItems = () => stateManager.State.Outbox /// .Select(e => e.Message).ToImmutableArray(), - /// OnPostCompletedAsync = async (items, ct) => + /// OnPostedAsync = async (items, ct) => /// { /// // remove delivered items and persist /// }, - /// OnPostErrorAsync = async (failures, ct) => + /// OnPostFailedAsync = async (failures, ct) => /// { /// // log, dead-letter, or leave for retry /// }, @@ -55,7 +55,7 @@ public static class OutboxProcessorExtensions /// in a grain field for later /// calls. /// - public static OutboxProcessor InitializeOutboxProcessor( + public static OutboxProcessor RegisterOutboxProcessor( this TGrain grain, OutboxProcessorOptions options) where TGrain : IGrainBase, IOutboxGrain diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs index f2757270..dcf9bba6 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs @@ -185,8 +185,43 @@ public async Task WriteAsync(T newState) /// public async Task ClearAsync() { - await storage.ClearStateAsync(); - state = storage.State; + var previousState = state; + var recoveryReadSucceeded = false; + + try + { + await storage.ClearStateAsync(); + state = storage.State; + return; + } + catch + { + try + { + await storage.ReadStateAsync(); + recoveryReadSucceeded = true; + + if (!storage.RecordExists) + { + state = storage.State; + return; + } + + state = storage.State; + } + catch + { + // no-op; rethrow original clear error below + } + + if (!recoveryReadSucceeded) + { + state = previousState; + storage.State = previousState; + } + + throw; + } } /// diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs index 56120712..31939984 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs @@ -58,8 +58,9 @@ public sealed class StreamManager private readonly Func getStreamProvider; private readonly Func getStreamId; private readonly ILogger logger; - private readonly List> subscriptions = []; + private readonly List>> subscriptions = []; private readonly List subscriptionHandles = []; + private bool subscribed; private StreamManager( IGrainBase owner, @@ -136,9 +137,11 @@ internal static StreamManager Create( /// runtime enforced). /// /// - /// Receives the deserialized event and the - /// representing this event's position. The handler should update the grain's - /// and persist state before returning. + /// Receives the deserialized event and optional + /// representing this event's position. Some stream providers may deliver a + /// null token. The handler should update the grain's + /// when a token is available and persist state + /// before returning. /// /// /// Optional synchronous error handler. Receives the stream namespace and the @@ -157,12 +160,16 @@ internal static StreamManager Create( /// public StreamManager AddSubscription( string streamNamespace, - Func onNextAsync, + Func onNextAsync, Action? onError = default, bool passLatestSequenceTokenOnResume = true) { ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); ArgumentNullException.ThrowIfNull(onNextAsync); + if (subscribed) + { + throw new InvalidOperationException("Cannot add stream subscriptions after SubscribeAsync has been called."); + } var streamProvider = getStreamProvider(streamNamespace); var streamId = getStreamId(streamNamespace); @@ -196,9 +203,11 @@ public StreamManager AddSubscription( /// runtime enforced). /// /// - /// Receives the deserialized event and the - /// representing this event's position. The handler should update the grain's - /// and persist state before returning. + /// Receives the deserialized event and optional + /// representing this event's position. Some stream providers may deliver a + /// null token. The handler should update the grain's + /// when a token is available and persist state + /// before returning. /// /// /// Optional synchronous error handler. Receives the stream namespace and the @@ -217,7 +226,7 @@ public StreamManager AddSubscription( /// public StreamManager AddSubscription( string streamNamespace, - Func onNextAsync, + Func onNextAsync, Action? onError = default, bool passLatestSequenceTokenOnResume = true) { @@ -236,11 +245,35 @@ public StreamManager AddSubscription( /// Cancellation token for activation-time subscription setup. public async Task SubscribeAsync(CancellationToken cancellationToken = default) { + if (subscribed) + { + throw new InvalidOperationException("StreamManager subscriptions have already been established."); + } + cancellationToken.ThrowIfCancellationRequested(); - await Task.WhenAll(subscriptions.Select(subscription => subscription(cancellationToken))).ConfigureAwait(false); + subscribed = true; + var subscriptionTasks = subscriptions.Select(subscription => subscription(cancellationToken)).ToArray(); + + try + { + var handles = await Task.WhenAll(subscriptionTasks).ConfigureAwait(false); + subscriptionHandles.AddRange(handles); + } + catch + { + foreach (var task in subscriptionTasks) + { + if (task.Status is TaskStatus.RanToCompletion) + { + subscriptionHandles.Add(task.Result); + } + } + + throw; + } } - private async Task SubscribeCoreAsync( + private async Task SubscribeCoreAsync( string streamNamespace, IAsyncStream stream, IAsyncObserver observer, @@ -254,15 +287,15 @@ private async Task SubscribeCoreAsync( if (token is null) { var handle = await stream.SubscribeAsync(observer).ConfigureAwait(false); - subscriptionHandles.Add(handle); + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); + return handle; } else { var handle = await stream.SubscribeAsync(observer, token, null!).ConfigureAwait(false); - subscriptionHandles.Add(handle); + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); + return handle; } - - MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); } catch (Exception ex) { @@ -277,7 +310,7 @@ private async Task OnNextAsync( StreamId streamId, TEvent item, StreamSequenceToken? token, - Func onNextAsync, + Func onNextAsync, Action? onError) { var cursor = new StreamCursor(streamId, token); @@ -286,7 +319,7 @@ private async Task OnNextAsync( using var activity = StartConsumerActivity(streamNamespace, cursor); try { - await onNextAsync(item, token!).ConfigureAwait(false); + await onNextAsync(item, token).ConfigureAwait(false); MessagingTelemetry.RecordStreamMessage(streamNamespace, "accepted"); MessagingTelemetry.RecordStreamHandlerDuration(streamNamespace, "accepted", Stopwatch.GetElapsedTime(started).TotalMilliseconds); @@ -380,7 +413,7 @@ private sealed class StreamObserver( StreamManager manager, string streamNamespace, StreamId streamId, - Func onNextAsync, + Func onNextAsync, Action? onError) : IAsyncObserver { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs index 9b27233b..8b462e9c 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs @@ -149,21 +149,21 @@ public Task DeactivateAsync() public Task GetStateAsync() => Task.FromResult(state.State); - private async ValueTask HandleValueTaskAsync(string item, StreamSequenceToken token) + private async ValueTask HandleValueTaskAsync(string item, StreamSequenceToken? token) { TrackStreamCursor(StreamManagerTestNamespaces.ValueTask, token); state.State.ValueTaskValue = item; await state.WriteStateAsync(); } - private async Task HandleTaskAsync(string item, StreamSequenceToken token) + private async Task HandleTaskAsync(string item, StreamSequenceToken? token) { TrackStreamCursor(StreamManagerTestNamespaces.Task, token); state.State.TaskValue = item; await state.WriteStateAsync(); } - private async ValueTask HandleFailureAsync(string item, StreamSequenceToken token) + private async ValueTask HandleFailureAsync(string item, StreamSequenceToken? token) { if (item == "fail") { @@ -175,14 +175,14 @@ private async ValueTask HandleFailureAsync(string item, StreamSequenceToken toke await state.WriteStateAsync(); } - private async ValueTask HandleResumeAsync(string item, StreamSequenceToken token) + private async ValueTask HandleResumeAsync(string item, StreamSequenceToken? token) { TrackStreamCursor(StreamManagerTestNamespaces.Resume, token); state.State.ResumeValue = item; await state.WriteStateAsync(); } - private void TrackStreamCursor(string streamNamespace, StreamSequenceToken token) + private void TrackStreamCursor(string streamNamespace, StreamSequenceToken? token) { var cursor = new StreamCursor(StreamId.Create(streamNamespace, this.GetPrimaryKey()), token); if (state.State.Tracker.ProcessMessage(cursor, out var next)) From a084feacd42843bd8454d51aea5848bf76b11d23 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 20:16:13 +0000 Subject: [PATCH 20/82] feat(om): implement outbox processor stream flow Implements the outbox processor registration and draining path, including postman dispatch, success acknowledgement, failure reconciliation, retry timer/reminder scheduling, and stream-backed integration coverage. Adds outbox and stream receive telemetry plus focused Orleans in-process tests for stream delivery, error paths, resume behavior, JSON converter branches, and tracker semantics. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Directory.Packages.props | 4 + .../MessagingTelemetry.cs | 78 ++++ .../NoPostmanRegisteredException.cs | 4 +- .../Egil.Orleans.Messaging/OutboxProcessor.cs | 426 +++++++++++++----- .../OutboxProcessorExtensions.cs | 20 +- .../OutboxProcessorOptions.cs | 111 ++--- .../Egil.Orleans.Messaging/StreamManager.cs | 19 +- .../Egil.Orleans.Messaging.Tests.csproj | 11 +- .../EnrichedEventHubAdapterTests.cs | 66 +++ .../MessageTrackerTests.cs | 32 ++ .../MessagingTestClusterFixture.cs | 3 + .../OutboxJsonConverterFactoryTests.cs | 30 ++ .../OutboxProcessorTests.cs | 378 ++++++++++++++++ .../StateManagerTests.cs | 49 ++ .../StreamManagerTests.cs | 54 +++ 15 files changed, 1095 insertions(+), 190 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxProcessorTests.cs diff --git a/Egil.Orleans.Messaging/Directory.Packages.props b/Egil.Orleans.Messaging/Directory.Packages.props index ea7f22ba..14e1472f 100644 --- a/Egil.Orleans.Messaging/Directory.Packages.props +++ b/Egil.Orleans.Messaging/Directory.Packages.props @@ -26,8 +26,12 @@ + + + + diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs index 8f432ab2..f97f5dc1 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs @@ -1,5 +1,6 @@ using System.Diagnostics; using System.Diagnostics.Metrics; +using System.Collections.Concurrent; namespace Egil.Orleans.Messaging; @@ -10,6 +11,15 @@ internal static class MessagingTelemetry private static readonly Counter StreamSubscriptions = Meter.CreateCounter("stream.subscriptions"); private static readonly Counter StreamHandlerErrors = Meter.CreateCounter("stream.handler.errors"); private static readonly Histogram StreamHandlerDuration = Meter.CreateHistogram("stream.handler.duration", "ms"); + private static readonly Counter OutboxPostItems = Meter.CreateCounter("outbox.post.items"); + private static readonly Counter OutboxPostErrors = Meter.CreateCounter("outbox.post.errors"); + private static readonly Histogram OutboxPostDuration = Meter.CreateHistogram("outbox.post.duration", "ms"); + private static readonly Histogram OutboxPostItemDuration = Meter.CreateHistogram("outbox.post.item.duration", "ms"); + private static readonly ConcurrentDictionary OutboxDepthValues = []; + private static readonly ObservableGauge OutboxDepth = Meter.CreateObservableGauge( + "outbox.depth", + ObserveOutboxDepth, + description: "Pending outbox items by grain type."); private static readonly Histogram StreamReceiveLag = Meter.CreateHistogram( "stream.message.receive.lag", "ms", @@ -33,6 +43,55 @@ public static void RecordStreamHandlerError(string streamNamespace) => public static void RecordStreamHandlerDuration(string streamNamespace, string status, double milliseconds) => StreamHandlerDuration.Record(milliseconds, StreamTags(streamNamespace, status)); + public static void RecordOutboxPostDuration( + string grainType, + OutboxPostmanExecutionMode executionMode, + double milliseconds) + { + OutboxPostDuration.Record( + milliseconds, + OutboxTags(grainType, executionMode)); + } + + public static void RecordOutboxPostItem( + string grainType, + string eventType, + string postmanType, + OutboxPostmanExecutionMode executionMode, + bool success, + double milliseconds) + { + var tags = OutboxTags(grainType, executionMode); + tags.Add("event.type", eventType); + tags.Add("postman.type", postmanType); + tags.Add("success", success); + + if (success) + { + OutboxPostItems.Add(1, tags); + } + + OutboxPostItemDuration.Record(milliseconds, tags); + } + + public static void RecordOutboxPostError(string grainType, string eventType, Exception exception) + { + OutboxPostErrors.Add( + 1, + new TagList + { + { "grain.type", grainType }, + { "event.type", eventType }, + { "failure.type", exception.GetType().Name } + }); + } + + public static void RecordOutboxDepth(string grainType, int depth) + { + _ = OutboxDepth; + OutboxDepthValues[grainType] = depth; + } + public static void RecordOutboxReceiveLag(OutboxSequenceToken token, DateTimeOffset receivedAt) { var lag = ClampLag(receivedAt - token.Timestamp); @@ -88,4 +147,23 @@ private static TagList StreamTags(string streamNamespace, string status) return tags; } + + private static TagList OutboxTags(string grainType, OutboxPostmanExecutionMode executionMode) + { + return new TagList + { + { "grain.type", grainType }, + { "postman.execution", executionMode is OutboxPostmanExecutionMode.ThreadPool ? "thread_pool" : "grain_scheduler" } + }; + } + + private static IEnumerable> ObserveOutboxDepth() + { + foreach (var depth in OutboxDepthValues) + { + yield return new Measurement( + depth.Value, + new KeyValuePair("grain.type", depth.Key)); + } + } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs index 3ff3ec6a..3fda919e 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs @@ -1,13 +1,13 @@ namespace Egil.Orleans.Messaging; /// -/// Thrown (internally, via ) +/// Thrown (internally, via ) /// when an outbox item's runtime type does not match any registered postman. /// /// /// /// This exception is never thrown out of . -/// It is surfaced through the +/// It is surfaced through the /// callback as the Error member of the failure tuple, allowing the grain /// to decide how to handle unmatched items (log, dead-letter, remove, etc.). /// diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs index ab44a786..49638a15 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs @@ -1,77 +1,13 @@ +using System.Collections.Immutable; +using System.Diagnostics; +using Microsoft.Extensions.Logging; + namespace Egil.Orleans.Messaging; /// /// Grain-scoped component that owns the timer, reminder, and postman dispatch /// lifecycle for draining an . /// -/// -/// -/// Architecture: Each grain with an outbox gets its own -/// — no external scan, no registry, -/// no second store. -/// -/// GrainTimer for in-process fast retry while activated. -/// Durable Reminder for cross-activation recovery. Reactivates -/// the grain if it deactivates with pending items. -/// -/// -/// -/// Non-reentrant assumption: Relies on Orleans default turn-based -/// concurrency — adds no internal locks. Not safe on -/// [Reentrant] grains. Document and enforce via code review. -/// -/// -/// Postman dispatch: The grain registers one or more postmen via -/// , each handling a -/// subtype of . Matching is -/// first-registered-wins against the item's runtime type — register -/// from most specific to least specific (like a switch). -/// -/// Each item dispatches to exactly one postman. -/// Items whose runtime type matches no postman → reported as failed -/// with via -/// . -/// Per-item exceptions are caught and surfaced through -/// with -/// an in-memory attempt count that resets on grain reactivation. -/// -/// -/// -/// error contract: Only throws -/// (per-run timeout), -/// (caller token), or callback -/// exceptions from -/// / . Per-item -/// postman failures are swallowed and routed to the error callback. -/// -/// -/// Grain integration: Two obligations (both compiler-enforced): -/// -/// Implement . -/// Call RegisterOutboxProcessor(...) in OnActivateAsync. -/// -/// No ReceiveReminder override needed — the -/// DIM handles it. No manual timer/reminder lifecycle. No telemetry wiring. -/// -/// -/// Escape hatch: Grains with their own reminders can forward unknown -/// reminder names to : -/// -/// public async Task ReceiveReminder(string name, TickStatus status) -/// { -/// if (name == MyOwnReminder) { await DoMyWork(); return; } -/// await outboxProcessor.ReceiveReminderAsync(name, status); -/// } -/// -/// -/// -/// Telemetry: Emits to meter egil.orleans.messaging: -/// outbox.post.duration (histogram), outbox.post.item.duration -/// (histogram), outbox.post.items (counter), outbox.post.errors -/// (counter), outbox.depth (gauge). Tags: grain.type, -/// event.type, success. -/// -/// /// /// The base type of items in the outbox. Postmen can handle subtypes via /// . @@ -79,82 +15,366 @@ namespace Egil.Orleans.Messaging; public sealed partial class OutboxProcessor : IOutboxComponent where TOutbox : notnull { - internal OutboxProcessor() { } + private const string ReminderPrefix = "egil.orleans.messaging.outbox."; + + private readonly IGrainBase owner; + private readonly IGrainFactory grainFactory; + private readonly OutboxProcessorOptions options; + private readonly ILogger logger; + private readonly List postmen = []; + private readonly Dictionary attempts = []; + private readonly string grainType; + private readonly string reminderName; + private IGrainTimer? timer; + private IGrainReminder? reminder; + private bool drainActive; + private bool drainRequested; + + internal OutboxProcessor( + IGrainBase owner, + IGrainFactory grainFactory, + OutboxProcessorOptions options, + ILogger logger) + { + ArgumentNullException.ThrowIfNull(owner); + ArgumentNullException.ThrowIfNull(grainFactory); + ArgumentNullException.ThrowIfNull(options); + ArgumentNullException.ThrowIfNull(logger); + + if (options.ProcessingTimeout <= TimeSpan.Zero) + { + throw new ArgumentOutOfRangeException(nameof(options), "ProcessingTimeout must be greater than zero."); + } + + if (options.RetryDelay <= TimeSpan.Zero) + { + throw new ArgumentOutOfRangeException(nameof(options), "RetryDelay must be greater than zero."); + } + + this.owner = owner; + this.grainFactory = grainFactory; + this.options = options; + this.logger = logger; + grainType = owner.GetType().Name; + reminderName = ReminderPrefix + typeof(TOutbox).FullName; + } + + internal string ReminderName => reminderName; /// /// Registers a postman that handles items of type . /// - /// - /// Order matters. Postmen are matched first-registered-wins against - /// the item's runtime type. Register from most specific to least specific. - /// Returns this for fluent chaining during OnActivateAsync. - /// - /// The subtype this postman handles. - /// - /// Async callback that delivers the item. Per-item exceptions are caught - /// and surfaced through . - /// - /// This processor for chaining. public OutboxProcessor AddPostman( Func postman) where TSub : TOutbox { - throw new NotImplementedException(); + ArgumentNullException.ThrowIfNull(postman); + return AddPostmanCore((item, _) => postman(item)); } /// public OutboxProcessor AddPostman( Func postman) where TSub : TOutbox { - throw new NotImplementedException(); + ArgumentNullException.ThrowIfNull(postman); + return AddPostmanCore((item, _) => new ValueTask(postman(item))); } /// - /// - /// Async callback with cancellation support. The token is the per-run - /// timeout from . - /// public OutboxProcessor AddPostman( Func postman) where TSub : TOutbox { - throw new NotImplementedException(); + ArgumentNullException.ThrowIfNull(postman); + return AddPostmanCore((item, cancellationToken) => new ValueTask(postman(item, cancellationToken))); + } + + /// + public OutboxProcessor AddPostman( + Func postman) where TSub : TOutbox + { + ArgumentNullException.ThrowIfNull(postman); + return AddPostmanCore((item, cancellationToken) => postman(item, grainFactory, cancellationToken)); } /// /// Posts all pending items by dispatching each to its matching postman. - /// Arms timer/reminder if items remain; unregisters if the outbox is empty. + /// Arms timer/reminder if items remain; unregisters retry work if empty. /// - /// - /// Safe to call from the grain's task scheduler (turn-based). Does not - /// throw for per-item postman failures — those are routed to - /// . Only - /// throws , - /// , or callback exceptions. - /// - /// Optional cancellation token. - public ValueTask PostAsync(CancellationToken cancellationToken = default) + public async ValueTask PostAsync(CancellationToken cancellationToken = default) { - throw new NotImplementedException(); + if (drainActive) + { + drainRequested = true; + return; + } + + drainActive = true; + try + { + do + { + drainRequested = false; + await DrainOnceAsync(cancellationToken).ConfigureAwait(false); + } + while (drainRequested); + } + finally + { + drainActive = false; + } + } + + /// + /// Schedules a timer-backed post run and returns after scheduling. + /// + public async ValueTask PostInBackgroundAsync(CancellationToken cancellationToken = default) + { + cancellationToken.ThrowIfCancellationRequested(); + if (options.PendingItems().IsDefaultOrEmpty) + { + await DisableRetryAsync().ConfigureAwait(false); + return; + } + + EnsureTimer(TimeSpan.Zero); + await EnsureReminderAsync().ConfigureAwait(false); } /// /// Called by the DIM when a reminder fires. /// No-ops for reminder names not owned by this processor. /// - /// The reminder name from Orleans. - /// The tick status from Orleans. public ValueTask ReceiveReminderAsync(string reminderName, TickStatus status) { - throw new NotImplementedException(); + return string.Equals(reminderName, this.reminderName, StringComparison.Ordinal) + ? PostAsync() + : ValueTask.CompletedTask; } /// /// Attaches this processor to the grain's as /// a component so the DIM can discover it. - /// Called internally by RegisterOutboxProcessor. /// internal void AttachToGrain() { - throw new NotImplementedException(); + owner.GrainContext.SetComponent(this); + } + + private OutboxProcessor AddPostmanCore( + Func postman) where TSub : TOutbox + { + postmen.Add(new PostmanRegistration( + typeof(TSub), + (item, cancellationToken) => postman((TSub)item, cancellationToken))); + + return this; + } + + private async Task DrainOnceAsync(CancellationToken cancellationToken) + { + cancellationToken.ThrowIfCancellationRequested(); + var pending = options.PendingItems(); + MessagingTelemetry.RecordOutboxDepth(grainType, pending.IsDefault ? 0 : pending.Length); + + if (pending.IsDefaultOrEmpty) + { + await DisableRetryAsync().ConfigureAwait(false); + return; + } + + using var timeout = new CancellationTokenSource(options.ProcessingTimeout); + using var linked = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, timeout.Token); + var started = Stopwatch.GetTimestamp(); + var posted = ImmutableArray.CreateBuilder(); + var failed = ImmutableArray.CreateBuilder<(TOutbox Item, Exception Error, int Attempt)>(); + + try + { + foreach (var item in pending) + { + linked.Token.ThrowIfCancellationRequested(); + await DispatchItemAsync(item, posted, failed, linked.Token).ConfigureAwait(false); + } + + if (posted.Count > 0) + { + await options.AcknowledgePostedAsync(posted.ToImmutable(), linked.Token).ConfigureAwait(false); + foreach (var item in posted) + { + attempts.Remove(item); + } + } + + if (failed.Count > 0 && options.ReconcileFailedAsync is { } reconcileFailed) + { + await reconcileFailed(failed.ToImmutable(), linked.Token).ConfigureAwait(false); + } + } + catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested && timeout.IsCancellationRequested) + { + throw new TimeoutException($"Outbox post run exceeded the configured timeout of {options.ProcessingTimeout}."); + } + finally + { + MessagingTelemetry.RecordOutboxPostDuration( + grainType, + options.PostmanExecution, + Stopwatch.GetElapsedTime(started).TotalMilliseconds); + } + + await ReconcileRetryStateAsync().ConfigureAwait(false); + } + + private async Task DispatchItemAsync( + TOutbox item, + ImmutableArray.Builder posted, + ImmutableArray<(TOutbox Item, Exception Error, int Attempt)>.Builder failed, + CancellationToken cancellationToken) + { + var started = Stopwatch.GetTimestamp(); + var itemType = item.GetType(); + var postman = FindPostman(itemType); + if (postman is null) + { + var error = new NoPostmanRegisteredException(itemType); + logger.LogWarning( + error, + "No outbox postman registered for item type {OutboxItemType} on grain {GrainType}.", + itemType.FullName, + grainType); + failed.Add((item, error, IncrementAttempt(item))); + MessagingTelemetry.RecordOutboxPostError(grainType, itemType.Name, error); + return; + } + + try + { + await InvokePostmanAsync(postman, item, cancellationToken).ConfigureAwait(false); + posted.Add(item); + MessagingTelemetry.RecordOutboxPostItem( + grainType, + itemType.Name, + postman.ItemType.Name, + options.PostmanExecution, + success: true, + Stopwatch.GetElapsedTime(started).TotalMilliseconds); + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + logger.LogError( + ex, + "Outbox postman failed for item type {OutboxItemType} on grain {GrainType}.", + itemType.FullName, + grainType); + failed.Add((item, ex, IncrementAttempt(item))); + MessagingTelemetry.RecordOutboxPostError(grainType, itemType.Name, ex); + MessagingTelemetry.RecordOutboxPostItem( + grainType, + itemType.Name, + postman.ItemType.Name, + options.PostmanExecution, + success: false, + Stopwatch.GetElapsedTime(started).TotalMilliseconds); + } + } + + private ValueTask InvokePostmanAsync( + PostmanRegistration postman, + TOutbox item, + CancellationToken cancellationToken) + { + if (options.PostmanExecution is OutboxPostmanExecutionMode.GrainScheduler) + { + return postman.Invoke(item, cancellationToken); + } + + return new ValueTask(Task.Run( + async () => await postman.Invoke(item, cancellationToken).ConfigureAwait(false), + cancellationToken)); + } + + private PostmanRegistration? FindPostman(Type itemType) + { + foreach (var postman in postmen) + { + if (postman.ItemType.IsAssignableFrom(itemType)) + { + return postman; + } + } + + return null; + } + + private int IncrementAttempt(TOutbox item) + { + attempts.TryGetValue(item, out var current); + var next = current + 1; + attempts[item] = next; + return next; + } + + private async Task ReconcileRetryStateAsync() + { + var pending = options.PendingItems(); + MessagingTelemetry.RecordOutboxDepth(grainType, pending.IsDefault ? 0 : pending.Length); + + if (pending.IsDefaultOrEmpty) + { + await DisableRetryAsync().ConfigureAwait(false); + return; + } + + EnsureTimer(options.RetryDelay); + await EnsureReminderAsync().ConfigureAwait(false); + } + + private void EnsureTimer(TimeSpan dueTime) + { + var options = new GrainTimerCreationOptions(dueTime, this.options.RetryDelay) + { + Interleave = this.options.Interleave, + KeepAlive = this.options.KeepAlive + }; + + if (timer is null) + { + timer = owner.RegisterGrainTimer( + static (processor, cancellationToken) => processor.PostAsync(cancellationToken).AsTask(), + this, + options); + return; + } + + timer.Change(dueTime, this.options.RetryDelay); + } + + private async Task EnsureReminderAsync() + { + var period = Max(options.RetryDelay, TimeSpan.FromMinutes(1)); + reminder = await owner.RegisterOrUpdateReminder(reminderName, period, period).ConfigureAwait(false); + } + + private async Task DisableRetryAsync() + { + timer?.Dispose(); + timer = null; + + var activeReminder = reminder ?? await owner.GetReminder(reminderName).ConfigureAwait(false); + if (activeReminder is not null) + { + await owner.UnregisterReminder(activeReminder).ConfigureAwait(false); + reminder = null; + } + } + + private static TimeSpan Max(TimeSpan left, TimeSpan right) => left >= right ? left : right; + + private sealed record PostmanRegistration( + Type ItemType, + Func Postman) + { + public ValueTask Invoke(TOutbox item, CancellationToken cancellationToken) => + Postman(item, cancellationToken); } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs index 2641d105..92162ee7 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs @@ -1,3 +1,6 @@ +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; + namespace Egil.Orleans.Messaging; /// @@ -27,11 +30,11 @@ public static class OutboxProcessorExtensions /// { /// PendingItems = () => stateManager.State.Outbox /// .Select(e => e.Message).ToImmutableArray(), - /// OnPostedAsync = async (items, ct) => + /// AcknowledgePostedAsync = async (items, ct) => /// { /// // remove delivered items and persist /// }, - /// OnPostFailedAsync = async (failures, ct) => + /// ReconcileFailedAsync = async (failures, ct) => /// { /// // log, dead-letter, or leave for retry /// }, @@ -61,6 +64,17 @@ public static OutboxProcessor RegisterOutboxProcessor( where TGrain : IGrainBase, IOutboxGrain where TOutbox : notnull { - throw new NotImplementedException(); + ArgumentNullException.ThrowIfNull(grain); + ArgumentNullException.ThrowIfNull(options); + + var services = grain.GrainContext.ActivationServices; + var processor = new OutboxProcessor( + grain, + services.GetRequiredService(), + options, + services.GetRequiredService().CreateLogger>()); + + processor.AttachToGrain(); + return processor; } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs index 96951a19..562d2354 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs @@ -2,36 +2,27 @@ namespace Egil.Orleans.Messaging; +/// +/// Controls where outbox postman callbacks execute. +/// +public enum OutboxPostmanExecutionMode +{ + /// + /// Executes postman callbacks on the Orleans activation scheduler. + /// + GrainScheduler, + + /// + /// Executes postman callbacks on the .NET thread pool. + /// + ThreadPool +} + /// /// Configuration for . Defines how the -/// processor reads pending items, reports successes/failures, and controls -/// timing. +/// processor reads pending items, reconciles successes/failures, and controls +/// timer/reminder scheduling. /// -/// -/// -/// Callback model: The processor uses callbacks rather than DI services -/// so the grain controls dispatch logic and can pass its own state to the -/// handlers. This keeps the processor grain-scoped with no ambient dependencies. -/// -/// -/// Required callbacks: -/// -/// — snapshot of pending items, called once -/// per post run. -/// — items successfully posted. -/// The grain must remove them from its backing collection and persist. -/// -/// -/// -/// Optional callback: -/// -/// — failed items with exception and -/// attempt count. If null, failed items retry silently on the next -/// run. The grain decides: leave the item in state to retry, or remove it -/// to dead-letter after N attempts. -/// -/// -/// /// /// The base type of outbox items. Must match the type parameter of the /// this options instance configures. @@ -40,59 +31,51 @@ public sealed class OutboxProcessorOptions where TOutbox : notnull { /// - /// Snapshot of pending items. Called once per post run. The processor - /// iterates the returned array and dispatches each item to its matching - /// postman. + /// Snapshot of pending items. Called once before each post run and again + /// after reconciliation to decide whether retry work remains. /// - /// - /// Typically implemented as a lambda reading the grain's current outbox: - /// - /// GetPending = () => stateManager.State.Outbox.Select(e => e.Message).ToImmutableArray() - /// - /// - public required Func> GetPending { get; init; } + public required Func> PendingItems { get; init; } /// - /// Called with the items that were successfully dispatched by their postmen. - /// The grain must remove these items from its backing collection (e.g., - /// ) and persist the updated state. + /// Called with items that were successfully dispatched by their postmen. + /// The grain is expected to remove these items from durable outbox state + /// and persist the update. /// - /// - /// Exceptions thrown from this callback propagate out of - /// — the processor does - /// not catch callback failures. - /// public required Func, CancellationToken, ValueTask> - OnPostCompletedAsync { get; init; } + AcknowledgePostedAsync { get; init; } /// /// Called with items that failed dispatch, along with the exception and - /// the in-memory attempt count (resets on grain reactivation). The grain - /// decides: leave the item in state to retry on the next run, or remove - /// it to dead-letter after N attempts. + /// the in-memory attempt count. The grain decides whether to leave them + /// pending, remove them, or move them to dead-letter state. /// - /// - /// If null, failed items retry silently on the next post run. - /// Exceptions thrown from this callback propagate out of - /// . - /// public Func, - CancellationToken, ValueTask>? OnPostErrorAsync { get; init; } + CancellationToken, ValueTask>? ReconcileFailedAsync { get; init; } /// - /// Maximum time per post run. Set below the grain's response timeout to - /// avoid grain-level timeouts during dispatch. Default: 20 seconds. + /// Maximum time per post run. Default: 20 seconds. /// public TimeSpan ProcessingTimeout { get; init; } = TimeSpan.FromSeconds(20); /// - /// Timer and reminder period. Controls how frequently the processor - /// retries pending items. Default: 2 minutes. + /// Timer/reminder retry period. Reminder period is clamped to at least one + /// minute because Orleans reminders do not support sub-minute precision. /// - /// - /// Orleans reminders fire at most once per minute, so values below 1 - /// minute are effectively timer-only (the reminder still fires at its - /// minimum interval for cross-activation recovery). - /// public TimeSpan RetryDelay { get; init; } = TimeSpan.FromMinutes(2); + + /// + /// Whether timer callbacks may interleave with other grain turns. + /// + public bool Interleave { get; init; } + + /// + /// Whether an active timer keeps the grain activation alive. + /// + public bool KeepAlive { get; init; } + + /// + /// Controls where postman callbacks execute. + /// + public OutboxPostmanExecutionMode PostmanExecution { get; init; } = + OutboxPostmanExecutionMode.GrainScheduler; } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs index 31939984..820400f5 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs @@ -254,23 +254,8 @@ public async Task SubscribeAsync(CancellationToken cancellationToken = default) subscribed = true; var subscriptionTasks = subscriptions.Select(subscription => subscription(cancellationToken)).ToArray(); - try - { - var handles = await Task.WhenAll(subscriptionTasks).ConfigureAwait(false); - subscriptionHandles.AddRange(handles); - } - catch - { - foreach (var task in subscriptionTasks) - { - if (task.Status is TaskStatus.RanToCompletion) - { - subscriptionHandles.Add(task.Result); - } - } - - throw; - } + var handles = await Task.WhenAll(subscriptionTasks).ConfigureAwait(false); + subscriptionHandles.AddRange(handles); } private async Task SubscribeCoreAsync( diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj index cd4053db..c82b60bf 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj @@ -8,6 +8,9 @@ Egil.Orleans.Messaging.Tests net10.0 true + [Egil.Orleans.Messaging]* + [*.Tests]* + **/*Tests.cs @@ -20,12 +23,18 @@ + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + - + + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs index 29139e51..0d7c351f 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs @@ -1,7 +1,9 @@ using System.Diagnostics; +using Azure.Messaging.EventHubs; using Microsoft.Extensions.DependencyInjection; using Orleans.Providers.Streams.Common; using Orleans.Serialization; +using Orleans.Streaming.EventHubs; namespace Egil.Orleans.Messaging.Tests; @@ -57,6 +59,70 @@ public void GetSequenceToken_returns_enriched_event_hub_sequence_token() Assert.Null(enriched.TraceParent); } + [Fact] + public void GetSequenceToken_treats_unspecified_enqueue_time_as_utc() + { + var adapter = CreateAdapter(); + var cachedMessage = new CachedMessage + { + SequenceNumber = 11, + EventIndex = 0, + EnqueueTimeUtc = new DateTime(2026, 5, 24, 19, 0, 0, DateTimeKind.Unspecified) + }; + + var token = adapter.GetSequenceToken(ref cachedMessage); + + var enriched = Assert.IsType(token); + Assert.Equal(TimeSpan.Zero, enriched.EnqueuedTime.Offset); + Assert.Equal(new DateTime(2026, 5, 24, 19, 0, 0, DateTimeKind.Utc), enriched.EnqueuedTime.UtcDateTime); + } + + [Fact] + public void GetSequenceToken_converts_local_enqueue_time_to_utc() + { + var adapter = CreateAdapter(); + var localTime = new DateTime(2026, 5, 24, 19, 0, 0, DateTimeKind.Local); + var cachedMessage = new CachedMessage + { + SequenceNumber = 12, + EventIndex = 0, + EnqueueTimeUtc = localTime + }; + + var token = adapter.GetSequenceToken(ref cachedMessage); + + var enriched = Assert.IsType(token); + Assert.Equal(TimeSpan.Zero, enriched.EnqueuedTime.Offset); + Assert.Equal(localTime.ToUniversalTime(), enriched.EnqueuedTime.UtcDateTime); + } + + [Fact] + public void GetStreamPosition_extracts_traceparent_from_event_data_properties() + { + var adapter = CreateAdapter(); + var traceParent = "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01"; +#pragma warning disable CS0618 // Test-only factory path for broker-owned EventData fields in the installed Event Hubs package. + var message = EventHubsModelFactory.EventData( + BinaryData.FromString("payload"), + new Dictionary + { + ["traceparent"] = traceParent + }, + systemProperties: new Dictionary(), + partitionKey: "one", + sequenceNumber: 42, + offset: 123, + enqueuedTime: new DateTimeOffset(2026, 5, 24, 19, 0, 0, TimeSpan.Zero)); +#pragma warning restore CS0618 + message.SetStreamNamespaceProperty("orders"); + + var position = adapter.GetStreamPosition("0", message); + + var enriched = Assert.IsType(position.SequenceToken); + Assert.Equal(traceParent, enriched.TraceParent); + Assert.Equal("provider-a", enriched.StreamProviderName); + } + private static EnrichedEventHubAdapter CreateAdapter() { var services = new ServiceCollection(); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs index d2e30013..fb338420 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs @@ -89,6 +89,38 @@ public void ProcessMessage_accepts_stream_cursor_when_token_is_newer() Assert.Equal(new StreamCursor(streamId, new EventSequenceToken(8)), next.LatestStream(streamId)); } + [Fact] + public void ProcessMessage_accepts_non_null_stream_token_after_null_initial_position() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var streamId = StreamId.Create("orders", "one"); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new StreamCursor(streamId, null), out tracker); + + var accepted = tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(1)), out var next); + + Assert.True(accepted); + Assert.Equal(new StreamCursor(streamId, new EventSequenceToken(1)), next.LatestStream(streamId)); + } + + [Fact] + public void ProcessMessage_rejects_null_stream_token_after_position_is_tracked() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var time = new ManualTimeProvider(now); + var streamId = StreamId.Create("orders", "one"); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(1)), out tracker); + + var accepted = tracker.ProcessMessage(new StreamCursor(streamId, null), out var next); + + Assert.False(accepted); + Assert.Same(tracker, next); + } + [Fact] public void ProcessMessage_rejects_outbox_token_when_same_epoch_and_not_newer_sequence() { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs index 079ddfbe..3152939d 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs @@ -23,6 +23,7 @@ public async ValueTask InitializeAsync() { siloBuilder.AddMemoryGrainStorage("Default"); siloBuilder.AddMemoryGrainStorage("PubSubStore"); + siloBuilder.UseInMemoryReminderService(); AddStreamProviders(siloBuilder); @@ -67,6 +68,7 @@ private static void AddStreamProviders(ISiloBuilder siloBuilder) siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Task); siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Failure); siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Resume); + siloBuilder.AddMemoryStreams(OutboxProcessorTestNamespaces.Events); } private static void AddStreamProviders(IClientBuilder clientBuilder) @@ -75,5 +77,6 @@ private static void AddStreamProviders(IClientBuilder clientBuilder) clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Task); clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Failure); clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Resume); + clientBuilder.AddMemoryStreams(OutboxProcessorTestNamespaces.Events); } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs index 8e46862e..7c3c4eac 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs @@ -49,6 +49,36 @@ public void JsonSerializer_round_trips_outbox_with_complex_messages() Assert.True(roundTripped[0].Message.Route.IsExpress); } + [Fact] + public void JsonSerializer_throws_json_exception_for_missing_sender() + { + var json = """ + { + "LatestSequenceNumber": 0, + "Epoch": null, + "Items": [] + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize>(json)); + Assert.Equal("Missing Sender.", exception.Message); + } + + [Fact] + public void JsonSerializer_throws_json_exception_for_missing_items() + { + var json = """ + { + "Sender": { "Type": "test/sender", "Key": "one" }, + "LatestSequenceNumber": 0, + "Epoch": null + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize>(json)); + Assert.Equal("Missing Items.", exception.Message); + } + private sealed record ComplexMessage(string OrderId, int Quantity, NestedMessage Route); private sealed record NestedMessage(string Name, bool IsExpress); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxProcessorTests.cs new file mode 100644 index 00000000..31296747 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxProcessorTests.cs @@ -0,0 +1,378 @@ +using System.Collections.Immutable; +using Egil.Orleans.Testing; +using Orleans.Streams; + +namespace Egil.Orleans.Messaging.Tests; + +public sealed class OutboxProcessorTests(MessagingTestClusterFixture fixture) : IClassFixture +{ + [Fact] + public async Task PostInBackgroundAsync_delivers_outbox_envelope_through_stream_and_acknowledges() + { + var grainKey = Guid.NewGuid(); + var sink = fixture.GrainFactory.GetGrain(grainKey); + var source = fixture.GrainFactory.GetGrain(grainKey); + await sink.EnsureActiveAsync(); + + await source.PublishInBackgroundAsync("price-changed"); + + await fixture.WaitForAssertionAsync( + sink, + async () => + { + var sinkState = await sink.GetStateAsync(); + Assert.Contains("price-changed", sinkState.ReceivedValues); + }, + ct: TestContext.Current.CancellationToken); + + await fixture.WaitForAssertionAsync( + source, + async () => + { + var sourceState = await source.GetStateAsync(); + Assert.Equal(1, sourceState.AcknowledgedCount); + Assert.Equal(0, sourceState.Outbox?.Count ?? 0); + }, + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task PostInBackgroundAsync_reports_no_postman_failure_and_reconciles_pending_item() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + await source.PublishInBackgroundAsync("unhandled"); + + await fixture.WaitForAssertionAsync( + source, + async () => + { + var state = await source.GetStateAsync(); + Assert.Equal(nameof(NoPostmanRegisteredException), state.LastFailureType); + Assert.Equal(1, state.FailedCount); + Assert.Equal(0, state.Outbox?.Count ?? 0); + }, + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task PostInBackgroundAsync_reports_postman_failure_and_reconciles_pending_item() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + await source.PublishInBackgroundAsync("boom"); + + await fixture.WaitForAssertionAsync( + source, + async () => + { + var state = await source.GetStateAsync(); + Assert.Equal(nameof(InvalidOperationException), state.LastFailureType); + Assert.Equal(1, state.FailedCount); + Assert.Equal(0, state.AcknowledgedCount); + Assert.Equal(0, state.Outbox?.Count ?? 0); + }, + ct: TestContext.Current.CancellationToken); + } +} + +internal static class OutboxProcessorTestNamespaces +{ + public const string Events = "outbox-processor-events"; +} + +public interface IOutboxProcessorSourceGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorNoPostmanGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorFailingPostmanGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorSinkGrain : IGrainWithGuidKey +{ + Task EnsureActiveAsync(); + + Task GetStateAsync(); +} + +[GenerateSerializer] +public sealed class OutboxProcessorSourceState +{ + [Id(0)] + public Outbox? Outbox { get; set; } + + [Id(1)] + public int AcknowledgedCount { get; set; } + + [Id(2)] + public int FailedCount { get; set; } + + [Id(3)] + public string? LastFailureType { get; set; } +} + +[GenerateSerializer] +public sealed class OutboxProcessorSinkState +{ + [Id(0)] + public MessageTracker Tracker { get; set; } = new(); + + [Id(1)] + public ImmutableArray ReceivedValues { get; set; } = []; +} + +[GenerateSerializer] +public sealed record OutboxProcessorTestEvent([property: Id(0)] string Value); + +public sealed class OutboxProcessorSourceGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorSourceGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>(PublishEnvelopeAsync); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishInBackgroundAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private Task PublishEnvelopeAsync( + OutboxMessageEnvelope envelope, + CancellationToken cancellationToken) + { + var stream = this.GetStreamProvider(OutboxProcessorTestNamespaces.Events) + .GetStream>( + StreamId.Create(OutboxProcessorTestNamespaces.Events, this.GetPrimaryKey())); + + _ = stream.OnNextAsync(envelope); + return Task.CompletedTask; + } + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } + + private async ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var failure in failures) + { + outbox = outbox.Remove(failure.Item.Token); + state.State.LastFailureType = failure.Error.GetType().Name; + } + + state.State.Outbox = outbox; + state.State.FailedCount += failures.Length; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() + { + return state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); + } +} + +public sealed class OutboxProcessorNoPostmanGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorNoPostmanGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishInBackgroundAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private async ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var failure in failures) + { + outbox = outbox.Remove(failure.Item.Token); + state.State.LastFailureType = failure.Error.GetType().Name; + } + + state.State.Outbox = outbox; + state.State.FailedCount += failures.Length; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() + { + return state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); + } +} + +public sealed class OutboxProcessorFailingPostmanGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorFailingPostmanGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>(FailAsync); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishInBackgroundAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private static ValueTask FailAsync(OutboxMessageEnvelope envelope) + { + throw new InvalidOperationException($"Cannot post {envelope.Message.Value}."); + } + + private ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private async ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var failure in failures) + { + outbox = outbox.Remove(failure.Item.Token); + state.State.LastFailureType = failure.Error.GetType().Name; + } + + state.State.Outbox = outbox; + state.State.FailedCount += failures.Length; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() + { + return state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); + } +} + +public sealed class OutboxProcessorSinkGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorSinkGrain +{ + private StreamManager? streamManager; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + streamManager = this.RegisterStreamManager(state.State.Tracker) + .AddSubscription>( + OutboxProcessorTestNamespaces.Events, + HandleEnvelopeAsync); + + await streamManager.SubscribeAsync(cancellationToken); + await base.OnActivateAsync(cancellationToken); + } + + public Task EnsureActiveAsync() => Task.CompletedTask; + + public Task GetStateAsync() => Task.FromResult(state.State); + + private async ValueTask HandleEnvelopeAsync( + OutboxMessageEnvelope envelope, + StreamSequenceToken? token) + { + if (!state.State.Tracker.ProcessMessage(envelope.Token, out var next)) + { + return; + } + + state.State.Tracker = next; + state.State.ReceivedValues = state.State.ReceivedValues.Add(envelope.Message.Value); + await state.WriteStateAsync(); + } +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs index 0c86e331..fb35c72c 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs @@ -91,6 +91,55 @@ public async Task WriteAsync_on_versioned_state_stamps_new_version() Assert.Equal(next.Version, manager.State.Version); } + [Fact] + public async Task ClearAsync_success_updates_committed_state() + { + var storage = new FakePersistentState(new TestState("initial")); + var manager = new DefaultStateManager(storage); + + await manager.ClearAsync(); + + Assert.Null(manager.State); + Assert.False(storage.RecordExists); + } + + [Fact] + public async Task ClearAsync_when_clear_fails_but_read_shows_missing_record_swallows_exception() + { + var storage = new FakePersistentState(new TestState("initial")) + { + ClearException = new TimeoutException("clear timeout"), + OnRead = state => + { + state.State = null!; + state.RecordExists = false; + } + }; + var manager = new DefaultStateManager(storage); + + await manager.ClearAsync(); + + Assert.Null(manager.State); + Assert.False(storage.RecordExists); + } + + [Fact] + public async Task ClearAsync_when_clear_and_read_fail_reverts_state_and_rethrows_original_exception() + { + var clearException = new TimeoutException("clear timeout"); + var storage = new FakePersistentState(new TestState("initial")) + { + ClearException = clearException, + ReadException = new InvalidOperationException("read failed") + }; + var manager = new DefaultStateManager(storage); + + var ex = await Assert.ThrowsAsync(() => manager.ClearAsync()); + + Assert.Same(clearException, ex); + Assert.Equal(new TestState("initial"), manager.State); + } + private sealed record TestState(string Value); private sealed record VersionedTestState(string Value) : VersionedState, IEquatable; diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs index 8b462e9c..038828d9 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs @@ -70,6 +70,28 @@ await fixture.WaitForAssertionAsync( async () => Assert.Equal("resumed", (await grain.GetStateAsync()).ResumeValue), ct: TestContext.Current.CancellationToken); } + + [Fact] + public async Task SubscribeAsync_rejects_duplicate_subscription_initialization() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + await grain.EnsureActiveAsync(); + + var exceptionType = await grain.SubscribeAgainAsync(); + + Assert.Equal(nameof(InvalidOperationException), exceptionType); + } + + [Fact] + public async Task AddSubscription_rejects_configuration_after_subscribe() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + await grain.EnsureActiveAsync(); + + var exceptionType = await grain.AddSubscriptionAfterSubscribeAsync(); + + Assert.Equal(nameof(InvalidOperationException), exceptionType); + } } internal static class StreamManagerTestNamespaces @@ -88,6 +110,10 @@ public interface IStreamManagerTestGrain : IGrainWithGuidKey Task DeactivateAsync(); + Task SubscribeAgainAsync(); + + Task AddSubscriptionAfterSubscribeAsync(); + Task GetStateAsync(); } @@ -149,6 +175,34 @@ public Task DeactivateAsync() public Task GetStateAsync() => Task.FromResult(state.State); + public async Task SubscribeAgainAsync() + { + try + { + await streamManager!.SubscribeAsync(); + return null; + } + catch (Exception ex) + { + return ex.GetType().Name; + } + } + + public Task AddSubscriptionAfterSubscribeAsync() + { + try + { + streamManager!.AddSubscription( + "not-registered", + static (_, _) => ValueTask.CompletedTask); + return Task.FromResult(null); + } + catch (Exception ex) + { + return Task.FromResult(ex.GetType().Name); + } + } + private async ValueTask HandleValueTaskAsync(string item, StreamSequenceToken? token) { TrackStreamCursor(StreamManagerTestNamespaces.ValueTask, token); From 0ea404c6c2aedbd1f4e714473a1430695819148f Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sun, 24 May 2026 21:03:01 +0000 Subject: [PATCH 21/82] test(om): cover messaging converter edge cases Adds targeted tests for high-risk converter and tracker branches, including stream cursor token variants, malformed persisted JSON, and same-count MessageTracker equality mismatches. These tests raise branch coverage around runtime serialization and dedup semantics without padding low-value argument validation paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Directory.Packages.props | 11 +- .../Egil.Orleans.Messaging.Tests.csproj | 1 - .../MessageTrackerJsonConverterTests.cs | 133 ++++++++++++++++++ .../MessageTrackerTests.cs | 68 +++++++++ .../OutboxJsonConverterFactoryTests.cs | 63 +++++++++ .../StreamCursorJsonConverterTests.cs | 103 ++++++++++++++ 6 files changed, 370 insertions(+), 9 deletions(-) diff --git a/Egil.Orleans.Messaging/Directory.Packages.props b/Egil.Orleans.Messaging/Directory.Packages.props index 14e1472f..f0a1e559 100644 --- a/Egil.Orleans.Messaging/Directory.Packages.props +++ b/Egil.Orleans.Messaging/Directory.Packages.props @@ -4,17 +4,14 @@ false $(NoWarn);NU1507 - - - @@ -24,14 +21,12 @@ - - - - + + - + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj index c82b60bf..e7add5c6 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj @@ -31,7 +31,6 @@ - diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs index eb9f6a3e..2784bea1 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs @@ -74,6 +74,49 @@ public void JsonSerializer_throws_for_unsupported_stream_token_type() Assert.Throws(() => JsonSerializer.Serialize(tracker)); } + [Fact] + public void JsonSerializer_deserializes_null_stream_and_outbox_arrays_as_empty_tracker() + { + var json = """ + { + "Streams": null, + "Outboxes": null + } + """; + + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Null(roundTripped.LatestStream(StreamId.Create("orders", "one"))); + Assert.Null(roundTripped.LatestOutbox(GrainId.Create("test/sender", "one"))); + } + + [Fact] + public void JsonSerializer_deserializes_tracked_stream_without_sequence_token() + { + var streamId = StreamId.Create("orders", "no-token"); + var json = """ + { + "Streams": [ + { + "StreamId": "orders/no-token", + "LastPosition": { + "StreamId": "orders/no-token", + "Token": null + }, + "Received": "2026-05-23T12:30:00+00:00" + } + ], + "Outboxes": [] + } + """; + + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(new StreamCursor(streamId, null), roundTripped.LatestStream(streamId)); + } + [Fact] public void JsonSerializer_throws_for_unknown_stream_token_kind() { @@ -100,6 +143,96 @@ public void JsonSerializer_throws_for_unknown_stream_token_kind() Assert.Throws(() => JsonSerializer.Deserialize(json)); } + [Theory] + [InlineData("event-hub")] + [InlineData("event-hub-v2")] + [InlineData("enriched-event-hub")] + public void JsonSerializer_throws_for_event_hub_stream_token_without_offset(string kind) + { + var json = $$""" + { + "Streams": [ + { + "StreamId": "orders/one", + "LastPosition": { + "StreamId": "orders/one", + "Token": { + "Kind": "{{kind}}", + "SequenceNumber": 1, + "EventIndex": 0, + "EnqueuedTime": "2026-05-23T12:30:00+00:00", + "StreamProviderName": "provider-a" + } + }, + "Received": "2026-05-23T12:30:00+00:00" + } + ], + "Outboxes": [] + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); + Assert.Equal("Missing EventHubOffset.", exception.Message); + } + + [Fact] + public void JsonSerializer_throws_for_enriched_stream_token_without_enqueued_time() + { + var json = """ + { + "Streams": [ + { + "StreamId": "orders/one", + "LastPosition": { + "StreamId": "orders/one", + "Token": { + "Kind": "enriched-event-hub", + "SequenceNumber": 1, + "EventIndex": 0, + "EventHubOffset": "42", + "StreamProviderName": "provider-a" + } + }, + "Received": "2026-05-23T12:30:00+00:00" + } + ], + "Outboxes": [] + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); + Assert.Equal("Missing EnqueuedTime.", exception.Message); + } + + [Fact] + public void JsonSerializer_throws_for_enriched_stream_token_without_stream_provider_name() + { + var json = """ + { + "Streams": [ + { + "StreamId": "orders/one", + "LastPosition": { + "StreamId": "orders/one", + "Token": { + "Kind": "enriched-event-hub", + "SequenceNumber": 1, + "EventIndex": 0, + "EventHubOffset": "42", + "EnqueuedTime": "2026-05-23T12:30:00+00:00" + } + }, + "Received": "2026-05-23T12:30:00+00:00" + } + ], + "Outboxes": [] + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); + Assert.Equal("Missing StreamProviderName.", exception.Message); + } + private sealed class UnsupportedSequenceToken(long sequenceNumber, int eventIndex) : StreamSequenceToken { public override long SequenceNumber { get; protected set; } = sequenceNumber; diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs index fb338420..4285f7b6 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs @@ -260,4 +260,72 @@ public void Equals_returns_false_when_tracked_entries_differ() Assert.NotEqual(left, right); } + [Fact] + public void Equals_returns_false_when_stream_keys_differ_but_counts_match() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var left = CreateTracker( + stream: new StreamCursor(StreamId.Create("orders", "one"), new EventSequenceToken(7)), + received: now); + var right = CreateTracker( + stream: new StreamCursor(StreamId.Create("orders", "two"), new EventSequenceToken(7)), + received: now); + + Assert.NotEqual(left, right); + } + + [Fact] + public void Equals_returns_false_when_stream_values_differ_but_keys_match() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var streamId = StreamId.Create("orders", "one"); + var left = CreateTracker(new StreamCursor(streamId, new EventSequenceToken(7)), now); + var right = CreateTracker(new StreamCursor(streamId, new EventSequenceToken(8)), now); + + Assert.NotEqual(left, right); + } + + [Fact] + public void Equals_returns_false_when_outbox_keys_differ_but_counts_match() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var left = CreateTracker( + outbox: new OutboxSequenceToken(1, GrainId.Create("test/sender", "one"), now, now), + received: now); + var right = CreateTracker( + outbox: new OutboxSequenceToken(1, GrainId.Create("test/sender", "two"), now, now), + received: now); + + Assert.NotEqual(left, right); + } + + [Fact] + public void Equals_returns_false_when_outbox_values_differ_but_keys_match() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var sender = GrainId.Create("test/sender", "one"); + var left = CreateTracker(new OutboxSequenceToken(1, sender, now, now), now); + var right = CreateTracker(new OutboxSequenceToken(2, sender, now, now), now); + + Assert.NotEqual(left, right); + } + + private static MessageTracker CreateTracker(StreamCursor stream, DateTimeOffset received) + { + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(received)); + tracker.ProcessMessage(stream, out tracker); + + return tracker; + } + + private static MessageTracker CreateTracker(OutboxSequenceToken outbox, DateTimeOffset received) + { + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(received)); + tracker.ProcessMessage(outbox, out tracker); + + return tracker; + } + } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs index 7c3c4eac..21d5d344 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs @@ -79,6 +79,69 @@ public void JsonSerializer_throws_json_exception_for_missing_items() Assert.Equal("Missing Items.", exception.Message); } + [Fact] + public void JsonSerializer_throws_json_exception_for_missing_item() + { + var json = """ + { + "Sender": { "Type": "test/sender", "Key": "one" }, + "LatestSequenceNumber": 1, + "Epoch": "2026-05-23T12:30:00+00:00", + "Items": [ null ] + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize>(json)); + Assert.Equal("Missing Item.", exception.Message); + } + + [Fact] + public void JsonSerializer_throws_json_exception_for_missing_item_token() + { + var json = """ + { + "Sender": { "Type": "test/sender", "Key": "one" }, + "LatestSequenceNumber": 1, + "Epoch": "2026-05-23T12:30:00+00:00", + "Items": [ + { + "Token": null, + "Message": "order-17" + } + ] + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize>(json)); + Assert.Equal("Missing Token.", exception.Message); + } + + [Fact] + public void JsonSerializer_throws_json_exception_for_missing_item_token_sender() + { + var json = """ + { + "Sender": { "Type": "test/sender", "Key": "one" }, + "LatestSequenceNumber": 1, + "Epoch": "2026-05-23T12:30:00+00:00", + "Items": [ + { + "Token": { + "SequenceNumber": 1, + "Sender": null, + "Timestamp": "2026-05-23T12:30:00+00:00", + "Epoch": "2026-05-23T12:30:00+00:00" + }, + "Message": "order-17" + } + ] + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize>(json)); + Assert.Equal("Missing Token.Sender.", exception.Message); + } + private sealed record ComplexMessage(string OrderId, int Quantity, NestedMessage Route); private sealed record NestedMessage(string Name, bool IsExpress); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs index 2acc27bf..5d51efd9 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs @@ -31,6 +31,46 @@ public void JsonSerializer_round_trips_stream_cursor_with_event_sequence_token() Assert.Equal(cursor, roundTripped); } + [Fact] + public void JsonSerializer_round_trips_stream_cursor_without_token() + { + var streamId = StreamId.Create("orders", "no-token"); + var cursor = new StreamCursor(streamId, null); + + var json = JsonSerializer.Serialize(cursor); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(cursor, roundTripped); + Assert.Null(roundTripped.Token); + } + + [Fact] + public void JsonSerializer_round_trips_stream_cursor_with_event_sequence_token_v2() + { + var streamId = StreamId.Create("orders", "event-v2"); + var cursor = new StreamCursor(streamId, new EventSequenceTokenV2(8, 2)); + + var json = JsonSerializer.Serialize(cursor); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(cursor, roundTripped); + } + + [Fact] + public void JsonSerializer_round_trips_stream_cursor_with_event_hub_token() + { + var streamId = StreamId.Create("orders", "eh"); + var cursor = new StreamCursor(streamId, new EventHubSequenceToken("41", 9, 2)); + + var json = JsonSerializer.Serialize(cursor); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.NotNull(roundTripped); + Assert.Equal(cursor, roundTripped); + } + [Fact] public void JsonSerializer_round_trips_stream_cursor_with_event_hub_token_v2() { @@ -96,6 +136,69 @@ public void JsonSerializer_throws_for_custom_token_type() Assert.Contains("supports only built-in Orleans token types", exception.Message); } + [Theory] + [InlineData("event-hub")] + [InlineData("event-hub-v2")] + [InlineData("enriched-event-hub")] + public void JsonSerializer_throws_for_event_hub_payload_without_offset(string kind) + { + var json = $$""" + { + "StreamId": "orders/one", + "Token": { + "Kind": "{{kind}}", + "SequenceNumber": 1, + "EventIndex": 0, + "EnqueuedTime": "2026-05-23T12:30:00Z", + "StreamProviderName": "provider-a" + } + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); + Assert.Equal("Missing EventHubOffset.", exception.Message); + } + + [Fact] + public void JsonSerializer_throws_for_enriched_event_hub_payload_without_enqueued_time() + { + var json = """ + { + "StreamId": "orders/one", + "Token": { + "Kind": "enriched-event-hub", + "SequenceNumber": 1, + "EventIndex": 0, + "EventHubOffset": "42", + "StreamProviderName": "provider-a" + } + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); + Assert.Equal("Missing EnqueuedTime.", exception.Message); + } + + [Fact] + public void JsonSerializer_throws_for_enriched_event_hub_payload_without_stream_provider_name() + { + var json = """ + { + "StreamId": "orders/one", + "Token": { + "Kind": "enriched-event-hub", + "SequenceNumber": 1, + "EventIndex": 0, + "EventHubOffset": "42", + "EnqueuedTime": "2026-05-23T12:30:00Z" + } + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); + Assert.Equal("Missing StreamProviderName.", exception.Message); + } + private sealed class UnsupportedCustomSequenceToken(long sequenceNumber, int eventIndex) : StreamSequenceToken { public override long SequenceNumber { get; protected set; } = sequenceNumber; From ff89072cdd0217b496a0c1f763bb45f036fb16ae Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 13:12:38 +0000 Subject: [PATCH 22/82] refactor(om): track streams by namespace and provider Stream cursors now carry namespace and provider metadata instead of using the Orleans stream id as the persisted tracking identity. Provider names are populated from subscription handles, with enriched Event Hub tokens taking precedence when present. This keeps tracking grain-local while still supporting provider-aware deduplication for non-conventional multi-provider namespace usage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/api-design.md | 4 +- .../EnrichedEventHubAdapter.cs | 20 ++-- .../EnrichedEventHubAdapterExtensions.cs | 6 +- .../EnrichedEventHubSequenceToken.cs | 18 ++-- .../Egil.Orleans.Messaging/MessageTracker.cs | 101 +++++++++++++++--- .../MessageTrackerJsonConverter.cs | 37 +++---- .../MessagingTelemetry.cs | 10 +- .../Egil.Orleans.Messaging/OutboxProcessor.cs | 51 ++++----- .../Egil.Orleans.Messaging/StreamCursor.cs | 42 ++++++-- .../StreamCursorJsonConverter.cs | 24 ++--- .../Egil.Orleans.Messaging/StreamManager.cs | 41 ++++--- .../EnrichedEventHubAdapterTests.cs | 4 +- .../MessageTrackerJsonConverterTests.cs | 29 ++--- .../MessageTrackerTests.cs | 2 +- .../StreamCursorJsonConverterTests.cs | 15 +-- .../StreamCursorTests.cs | 12 +-- 16 files changed, 255 insertions(+), 161 deletions(-) diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 27867ae4..4c787938 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -750,7 +750,7 @@ The library ships `EnrichedEventHubAdapter`, a public unsealed `EnrichedEventHubSequenceToken`, which carries: - `EnqueuedTime` — broker-side enqueue time for lag measurement. -- `StreamProviderName` — provider identity for dedup and multi-provider +- `ProviderName` — provider identity for dedup and multi-provider edge cases. - `TraceParent` — W3C traceparent captured from the producer-side `Activity.Current?.Id`. @@ -774,7 +774,7 @@ usually need extra services/options. `StreamManager` is unaware of Event Hubs specifically — enrichment surfaces through `StreamCursor.TryGetEnqueuedTime(...)`, -`StreamCursor.TryGetStreamProviderName(...)`, and +`StreamCursor.TryGetProviderName(...)`, and `StreamCursor.TryGetTraceParent(...)`. ### OpenTelemetry trace correlation diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs index 4c624237..1133bb9e 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs @@ -66,8 +66,8 @@ namespace Egil.Orleans.Messaging; /// /// public class MyAdapter : EnrichedEventHubAdapter /// { -/// public MyAdapter(string streamProviderName, Serializer serializer) -/// : base(streamProviderName, serializer) { } +/// public MyAdapter(string providerName, Serializer serializer) +/// : base(providerName, serializer) { } /// /// public override string GetPartitionKey(StreamId streamId) /// => streamId.GetNamespace(); @@ -89,13 +89,13 @@ public class EnrichedEventHubAdapter : EventHubDataAdapter /// The name of the Orleans stream provider. Accessible to subclasses for /// custom token construction or diagnostics. /// - protected string StreamProviderName { get; } + protected string ProviderName { get; } /// /// Creates a new adapter that enriches tokens with the given - /// . + /// . /// - /// + /// /// The name of the Orleans stream provider, passed through from the /// UseDataAdapter factory delegate. Stored in every /// produced by this adapter. @@ -105,10 +105,10 @@ public class EnrichedEventHubAdapter : EventHubDataAdapter /// for batch container /// serialization/deserialization. /// - public EnrichedEventHubAdapter(string streamProviderName, Serializer serializer) + public EnrichedEventHubAdapter(string providerName, Serializer serializer) : base(serializer) { - StreamProviderName = streamProviderName; + ProviderName = providerName; } /// @@ -197,7 +197,7 @@ public override StreamSequenceToken GetSequenceToken(ref CachedMessage cachedMes cachedMessage.SequenceNumber, cachedMessage.EventIndex, new DateTimeOffset(enqueueTimeUtc), - StreamProviderName); + ProviderName); } /// @@ -209,7 +209,7 @@ public override StreamSequenceToken GetSequenceToken(ref CachedMessage cachedMes /// /// Called once per incoming EventData before it enters the cache. /// The EventData.EnqueuedTime property carries the broker-stamped - /// UTC time; the is captured at adapter + /// UTC time; the is captured at adapter /// construction time. /// /// @@ -247,7 +247,7 @@ public override StreamPosition GetStreamPosition( sequenceNumber, eventIndex, queueMessage.EnqueuedTime, - StreamProviderName, + ProviderName, traceParent); return new StreamPosition(streamPosition.StreamId, enrichedToken); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs index 8dafa306..2dde19f6 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs @@ -32,7 +32,7 @@ namespace Egil.Orleans.Messaging; /// /// Downstream access: Once registered, the enrichment is transparently /// available on every stream token. Use -/// for lag measurement and +/// for lag measurement and /// for provider-aware dedup and diagnostics. /// /// @@ -63,9 +63,9 @@ public static void UseEnrichedDataAdapter( { ArgumentNullException.ThrowIfNull(configurator); - configurator.UseDataAdapter((services, streamProviderName) => + configurator.UseDataAdapter((services, providerName) => new EnrichedEventHubAdapter( - streamProviderName, + providerName, services.GetRequiredService())); } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs index a2d8dda0..28671740 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs @@ -4,7 +4,7 @@ namespace Egil.Orleans.Messaging; /// /// An subclass that carries the -/// broker-side and the , +/// broker-side and the , /// enabling end-to-end lag measurement and provider-aware dedup in /// and . /// @@ -21,11 +21,11 @@ namespace Egil.Orleans.Messaging; /// Transparent to StreamManager: The is /// unaware of the adapter. The enrichment surfaces through /// and -/// in the user's +/// in the user's /// OnNext handler. /// /// -/// Dedup and edge cases: The embedded +/// Dedup and edge cases: The embedded /// in the token enables to distinguish messages /// arriving from different stream providers on the same stream namespace, /// supporting multi-provider topologies and provider-specific eviction. @@ -44,7 +44,7 @@ namespace Egil.Orleans.Messaging; /// . The library's STJ converter for /// recognizes this subtype via a $kind /// discriminator and round-trips , -/// , and fields. +/// , and fields. /// /// [GenerateSerializer] @@ -64,7 +64,7 @@ public class EnrichedEventHubSequenceToken : EventHubSequenceTokenV2 /// Enables provider-aware dedup and diagnostics in . /// [Id(1)] - public string StreamProviderName { get; } + public string ProviderName { get; } /// /// The W3C traceparent header value from the producer-side @@ -95,7 +95,7 @@ public class EnrichedEventHubSequenceToken : EventHubSequenceTokenV2 /// The when the event was enqueued at the /// Event Hub broker. /// - /// + /// /// The name of the Orleans stream provider. Passed through from the /// EnrichedEventHubAdapter at construction time. /// @@ -108,12 +108,12 @@ public EnrichedEventHubSequenceToken( long sequenceNumber, int eventIndex, DateTimeOffset enqueuedTime, - string streamProviderName, + string providerName, string? traceParent = null) : base(offset, sequenceNumber, eventIndex) { EnqueuedTime = enqueuedTime; - StreamProviderName = streamProviderName; + ProviderName = providerName; TraceParent = traceParent; } @@ -122,6 +122,6 @@ public EnrichedEventHubSequenceToken( /// public EnrichedEventHubSequenceToken() { - StreamProviderName = string.Empty; + ProviderName = string.Empty; } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs index eecee2d8..74652c6c 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs @@ -12,8 +12,9 @@ namespace Egil.Orleans.Messaging; /// /// Two source kinds: /// -/// Orleans streams — keyed by ; position -/// is a wrapping (StreamId, StreamSequenceToken). +/// Orleans streams — keyed by stream namespace within the grain, +/// plus stream provider name when the received token exposes one; position is +/// a wrapping (streamNamespace, StreamSequenceToken). /// Outbox messages — keyed by sender ; /// position is an . /// @@ -64,7 +65,7 @@ namespace Egil.Orleans.Messaging; [JsonConverter(typeof(MessageTrackerJsonConverter))] public sealed class MessageTracker : IEquatable { - [Id(0)] private readonly ImmutableDictionary streams; + [Id(0)] private readonly ImmutableDictionary streams; [Id(1)] private readonly ImmutableDictionary outbox; /// @@ -80,12 +81,12 @@ public sealed class MessageTracker : IEquatable /// public MessageTracker() { - streams = ImmutableDictionary.Empty; + streams = ImmutableDictionary.Empty; outbox = ImmutableDictionary.Empty; } internal MessageTracker( - ImmutableDictionary streams, + ImmutableDictionary streams, ImmutableDictionary outbox) { this.streams = streams; @@ -112,10 +113,11 @@ public bool ProcessMessage(StreamCursor cursor, out MessageTracker next) { var now = time.GetUtcNow(); - if (!streams.TryGetValue(cursor.StreamId, out var entry)) + var source = StreamSource.From(cursor); + if (!streams.TryGetValue(source, out var entry)) { MessagingTelemetry.RecordStreamReceiveLag(cursor, now); - next = CreateTracker(streams.Add(cursor.StreamId, new StreamEntry(cursor, now)), outbox); + next = CreateTracker(streams.Add(source, new StreamEntry(cursor, now)), outbox); return true; } @@ -126,7 +128,7 @@ public bool ProcessMessage(StreamCursor cursor, out MessageTracker next) } MessagingTelemetry.RecordStreamReceiveLag(cursor, now); - next = CreateTracker(streams.SetItem(cursor.StreamId, new StreamEntry(cursor, now)), outbox); + next = CreateTracker(streams.SetItem(source, new StreamEntry(cursor, now)), outbox); return true; } @@ -172,14 +174,44 @@ public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) /// /// Returns the last accepted for the given - /// , or null if no messages from that - /// stream have been tracked. + /// , or null if no messages from + /// that namespace have been tracked by this grain. /// - public StreamCursor? LatestStream(StreamId stream) + public StreamCursor? LatestStream(string streamNamespace) { - return streams.TryGetValue(stream, out var entry) ? entry.LastPosition : null; + ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); + var conventionSource = new StreamSource(streamNamespace, null); + if (streams.TryGetValue(conventionSource, out var entry)) + { + return entry.LastPosition; + } + + StreamCursor? result = null; + foreach (var item in streams) + { + if (!string.Equals(item.Key.StreamNamespace, streamNamespace, StringComparison.Ordinal)) + { + continue; + } + + if (result is not null) + { + return null; + } + + result = item.Value.LastPosition; + } + + return result; } + /// + /// Returns the last accepted for the namespace + /// in the given . + /// + public StreamCursor? LatestStream(StreamId stream) => + LatestStream(stream.GetNamespace() ?? throw new ArgumentException("StreamId must have a namespace.", nameof(stream))); + /// /// Returns the last accepted for the /// given , or null if no outbox messages @@ -229,20 +261,40 @@ public MessageTracker EvictOutboxes(DateTimeOffset olderThan) } /// - /// Removes the entry for the given if + /// Removes the entry for the given if /// entry.Received <= . /// Use DateTimeOffset.MaxValue to unconditionally remove. /// - public MessageTracker Evict(StreamId stream, DateTimeOffset olderThan) + public MessageTracker Evict(string streamNamespace, DateTimeOffset olderThan) { - if (!streams.TryGetValue(stream, out var entry) || entry.Received > olderThan) + ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); + var builder = streams.ToBuilder(); + var changed = false; + foreach (var item in streams) + { + if (string.Equals(item.Key.StreamNamespace, streamNamespace, StringComparison.Ordinal) + && item.Value.Received <= olderThan) + { + builder.Remove(item.Key); + changed = true; + } + } + + if (!changed) { return this; } - return CreateTracker(streams.Remove(stream), outbox); + return CreateTracker(builder.ToImmutable(), outbox); } + /// + /// Removes the entry for the namespace in the given + /// if entry.Received <= . + /// + public MessageTracker Evict(StreamId stream, DateTimeOffset olderThan) => + Evict(stream.GetNamespace() ?? throw new ArgumentException("StreamId must have a namespace.", nameof(stream)), olderThan); + /// /// Removes the entry for the given outbox if /// entry.Received <= . @@ -304,7 +356,7 @@ public override int GetHashCode() } private MessageTracker CreateTracker( - ImmutableDictionary streams, + ImmutableDictionary streams, ImmutableDictionary outbox) { var next = new MessageTracker(streams, outbox) @@ -367,6 +419,21 @@ private static int AggregateHash(ImmutableDictionary return hash; } + /// + /// Internal entry tracking a stream source's last known position and + /// the wall-clock time it was received. + /// + [GenerateSerializer] + internal readonly record struct StreamSource( + [property: Id(0)] string StreamNamespace, + [property: Id(1)] string? ProviderName) + { + public static StreamSource From(StreamCursor cursor) => + new( + cursor.StreamNamespace, + cursor.TryGetProviderName(out var providerName) ? providerName : null); + } + /// /// Internal entry tracking a stream source's last known position and /// the wall-clock time it was received. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs index b1c09a3f..35806551 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs @@ -1,6 +1,5 @@ using System.Collections.Immutable; using System.Reflection; -using System.Text; using System.Text.Json; using System.Text.Json.Serialization; using Orleans.Providers.Streams.Common; @@ -47,14 +46,13 @@ internal sealed class MessageTrackerJsonConverter : JsonConverter(); + var streams = ImmutableDictionary.CreateBuilder(); foreach (var item in model.Streams ?? Array.Empty()) { + var lastPosition = FromJsonModel(item.LastPosition); streams.Add( - ParseStreamId(item.StreamId), - new MessageTracker.StreamEntry( - FromJsonModel(item.LastPosition), - item.Received)); + MessageTracker.StreamSource.From(lastPosition), + new MessageTracker.StreamEntry(lastPosition, item.Received)); } var outbox = ImmutableDictionary.CreateBuilder(); @@ -82,7 +80,6 @@ public override void Write(Utf8JsonWriter writer, MessageTracker value, JsonSeri foreach (var item in streams) { streamModels[streamIndex++] = new StreamEntryJsonModel( - item.Key.ToString(), ToJsonModel(item.Value.LastPosition), item.Value.Received); } @@ -104,20 +101,17 @@ public override void Write(Utf8JsonWriter writer, MessageTracker value, JsonSeri options); } - private static ImmutableDictionary GetStreams(MessageTracker value) => - (ImmutableDictionary)StreamsField.GetValue(value)!; + private static ImmutableDictionary GetStreams(MessageTracker value) => + (ImmutableDictionary)StreamsField.GetValue(value)!; private static ImmutableDictionary GetOutbox(MessageTracker value) => (ImmutableDictionary)OutboxField.GetValue(value)!; - private static StreamId ParseStreamId(string value) => - StreamId.Parse(Encoding.UTF8.GetBytes(value)); - private static GrainId ParseGrainId(GrainIdJsonModel value) => GrainId.Create(value.Type, value.Key); private static StreamCursor FromJsonModel(StreamCursorJsonModel model) => - new(ParseStreamId(model.StreamId), FromJsonModel(model.Token)); + new(model.StreamNamespace, FromJsonModel(model.Token), model.ProviderName); private static StreamSequenceToken? FromJsonModel(StreamSequenceTokenJsonModel? model) { @@ -143,7 +137,7 @@ private static StreamCursor FromJsonModel(StreamCursorJsonModel model) => model.SequenceNumber, model.EventIndex, model.EnqueuedTime ?? throw new JsonException("Missing EnqueuedTime."), - model.StreamProviderName ?? throw new JsonException("Missing StreamProviderName."), + model.ProviderName ?? throw new JsonException("Missing ProviderName."), model.TraceParent), _ => throw new JsonException($"Unsupported stream sequence token kind '{model.Kind}'.") }; @@ -151,8 +145,9 @@ private static StreamCursor FromJsonModel(StreamCursorJsonModel model) => private static StreamCursorJsonModel ToJsonModel(StreamCursor value) => new( - value.StreamId.ToString(), - ToJsonModel(value.Token)); + value.StreamNamespace, + ToJsonModel(value.Token), + value.ProviderName); private static StreamSequenceTokenJsonModel? ToJsonModel(StreamSequenceToken? value) { @@ -169,7 +164,7 @@ private static StreamCursorJsonModel ToJsonModel(StreamCursor value) => token.EventIndex, token.EventHubOffset, token.EnqueuedTime, - token.StreamProviderName, + token.ProviderName, token.TraceParent), EventHubSequenceTokenV2 token => new StreamSequenceTokenJsonModel( StreamSequenceTokenKinds.EventHubSequenceTokenV2, @@ -216,7 +211,6 @@ private sealed record MessageTrackerJsonModel( OutboxEntryJsonModel[]? Outboxes); private sealed record StreamEntryJsonModel( - string StreamId, StreamCursorJsonModel LastPosition, DateTimeOffset Received); @@ -227,8 +221,9 @@ private sealed record OutboxEntryJsonModel( DateTimeOffset Received); private sealed record StreamCursorJsonModel( - string StreamId, - StreamSequenceTokenJsonModel? Token); + string StreamNamespace, + StreamSequenceTokenJsonModel? Token, + string? ProviderName); private sealed record StreamSequenceTokenJsonModel( string Kind, @@ -236,7 +231,7 @@ private sealed record StreamSequenceTokenJsonModel( int EventIndex, string? EventHubOffset, DateTimeOffset? EnqueuedTime, - string? StreamProviderName, + string? ProviderName, string? TraceParent); private sealed record GrainIdJsonModel( diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs index f97f5dc1..bfb2d5d2 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs @@ -1,6 +1,6 @@ +using System.Collections.Concurrent; using System.Diagnostics; using System.Diagnostics.Metrics; -using System.Collections.Concurrent; namespace Egil.Orleans.Messaging; @@ -55,14 +55,12 @@ public static void RecordOutboxPostDuration( public static void RecordOutboxPostItem( string grainType, - string eventType, string postmanType, OutboxPostmanExecutionMode executionMode, bool success, double milliseconds) { var tags = OutboxTags(grainType, executionMode); - tags.Add("event.type", eventType); tags.Add("postman.type", postmanType); tags.Add("success", success); @@ -120,13 +118,13 @@ public static void RecordStreamReceiveLag(StreamCursor cursor, DateTimeOffset re { "messaging.system", "orleans" }, { "messaging.operation", "receive" }, { "messaging.source.kind", "stream" }, - { "stream.namespace", cursor.StreamId.GetNamespace() }, + { "stream.namespace", cursor.StreamNamespace }, { "status", "accepted" } }; - if (cursor.TryGetStreamProviderName(out var streamProviderName)) + if (cursor.TryGetProviderName(out var providerName)) { - tags.Add("stream.provider", streamProviderName); + tags.Add("stream.provider", providerName); } StreamReceiveLag.Record(lag.TotalMilliseconds, tags); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs index 49638a15..4d00f8d0 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs @@ -113,7 +113,7 @@ public async ValueTask PostAsync(CancellationToken cancellationToken = default) do { drainRequested = false; - await DrainOnceAsync(cancellationToken).ConfigureAwait(false); + await DrainOnceAsync(cancellationToken); } while (drainRequested); } @@ -131,12 +131,12 @@ public async ValueTask PostInBackgroundAsync(CancellationToken cancellationToken cancellationToken.ThrowIfCancellationRequested(); if (options.PendingItems().IsDefaultOrEmpty) { - await DisableRetryAsync().ConfigureAwait(false); + await DisableRetryAsync(); return; } EnsureTimer(TimeSpan.Zero); - await EnsureReminderAsync().ConfigureAwait(false); + await EnsureReminderAsync(); } /// @@ -163,8 +163,9 @@ private OutboxProcessor AddPostmanCore( Func postman) where TSub : TOutbox { postmen.Add(new PostmanRegistration( - typeof(TSub), - (item, cancellationToken) => postman((TSub)item, cancellationToken))); + item => item is TSub, + (item, cancellationToken) => postman((TSub)item, cancellationToken), + typeof(TSub))); return this; } @@ -177,7 +178,7 @@ private async Task DrainOnceAsync(CancellationToken cancellationToken) if (pending.IsDefaultOrEmpty) { - await DisableRetryAsync().ConfigureAwait(false); + await DisableRetryAsync(); return; } @@ -192,12 +193,12 @@ private async Task DrainOnceAsync(CancellationToken cancellationToken) foreach (var item in pending) { linked.Token.ThrowIfCancellationRequested(); - await DispatchItemAsync(item, posted, failed, linked.Token).ConfigureAwait(false); + await DispatchItemAsync(item, posted, failed, linked.Token); } if (posted.Count > 0) { - await options.AcknowledgePostedAsync(posted.ToImmutable(), linked.Token).ConfigureAwait(false); + await options.AcknowledgePostedAsync(posted.ToImmutable(), linked.Token); foreach (var item in posted) { attempts.Remove(item); @@ -206,7 +207,7 @@ private async Task DrainOnceAsync(CancellationToken cancellationToken) if (failed.Count > 0 && options.ReconcileFailedAsync is { } reconcileFailed) { - await reconcileFailed(failed.ToImmutable(), linked.Token).ConfigureAwait(false); + await reconcileFailed(failed.ToImmutable(), linked.Token); } } catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested && timeout.IsCancellationRequested) @@ -221,7 +222,7 @@ private async Task DrainOnceAsync(CancellationToken cancellationToken) Stopwatch.GetElapsedTime(started).TotalMilliseconds); } - await ReconcileRetryStateAsync().ConfigureAwait(false); + await ReconcileRetryStateAsync(); } private async Task DispatchItemAsync( @@ -231,10 +232,10 @@ private async Task DispatchItemAsync( CancellationToken cancellationToken) { var started = Stopwatch.GetTimestamp(); - var itemType = item.GetType(); - var postman = FindPostman(itemType); + var postman = FindPostman(item); if (postman is null) { + var itemType = item.GetType(); var error = new NoPostmanRegisteredException(itemType); logger.LogWarning( error, @@ -248,11 +249,10 @@ private async Task DispatchItemAsync( try { - await InvokePostmanAsync(postman, item, cancellationToken).ConfigureAwait(false); + await InvokePostmanAsync(postman, item, cancellationToken); posted.Add(item); MessagingTelemetry.RecordOutboxPostItem( grainType, - itemType.Name, postman.ItemType.Name, options.PostmanExecution, success: true, @@ -260,6 +260,7 @@ private async Task DispatchItemAsync( } catch (Exception ex) when (ex is not OperationCanceledException) { + var itemType = item.GetType(); logger.LogError( ex, "Outbox postman failed for item type {OutboxItemType} on grain {GrainType}.", @@ -269,7 +270,6 @@ private async Task DispatchItemAsync( MessagingTelemetry.RecordOutboxPostError(grainType, itemType.Name, ex); MessagingTelemetry.RecordOutboxPostItem( grainType, - itemType.Name, postman.ItemType.Name, options.PostmanExecution, success: false, @@ -288,15 +288,15 @@ private ValueTask InvokePostmanAsync( } return new ValueTask(Task.Run( - async () => await postman.Invoke(item, cancellationToken).ConfigureAwait(false), + async () => await postman.Invoke(item, cancellationToken), cancellationToken)); } - private PostmanRegistration? FindPostman(Type itemType) + private PostmanRegistration? FindPostman(TOutbox item) { foreach (var postman in postmen) { - if (postman.ItemType.IsAssignableFrom(itemType)) + if (postman.ItemFilter(item)) { return postman; } @@ -320,12 +320,12 @@ private async Task ReconcileRetryStateAsync() if (pending.IsDefaultOrEmpty) { - await DisableRetryAsync().ConfigureAwait(false); + await DisableRetryAsync(); return; } EnsureTimer(options.RetryDelay); - await EnsureReminderAsync().ConfigureAwait(false); + await EnsureReminderAsync(); } private void EnsureTimer(TimeSpan dueTime) @@ -351,7 +351,7 @@ private void EnsureTimer(TimeSpan dueTime) private async Task EnsureReminderAsync() { var period = Max(options.RetryDelay, TimeSpan.FromMinutes(1)); - reminder = await owner.RegisterOrUpdateReminder(reminderName, period, period).ConfigureAwait(false); + reminder = await owner.RegisterOrUpdateReminder(reminderName, period, period); } private async Task DisableRetryAsync() @@ -359,10 +359,10 @@ private async Task DisableRetryAsync() timer?.Dispose(); timer = null; - var activeReminder = reminder ?? await owner.GetReminder(reminderName).ConfigureAwait(false); + var activeReminder = reminder ?? await owner.GetReminder(reminderName); if (activeReminder is not null) { - await owner.UnregisterReminder(activeReminder).ConfigureAwait(false); + await owner.UnregisterReminder(activeReminder); reminder = null; } } @@ -370,8 +370,9 @@ private async Task DisableRetryAsync() private static TimeSpan Max(TimeSpan left, TimeSpan right) => left >= right ? left : right; private sealed record PostmanRegistration( - Type ItemType, - Func Postman) + Func ItemFilter, + Func Postman, + Type ItemType) { public ValueTask Invoke(TOutbox item, CancellationToken cancellationToken) => Postman(item, cancellationToken); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs index f278ddc6..8c76b695 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs @@ -4,7 +4,7 @@ namespace Egil.Orleans.Messaging; /// -/// Wraps a and its associated +/// Wraps a stream namespace and associated /// into a single value that uses for dedup and /// uses for resume. /// @@ -36,18 +36,34 @@ namespace Egil.Orleans.Messaging; /// the polymorphic via the discriminator. /// /// -/// The Orleans stream identity. +/// The Orleans stream namespace within the grain. /// /// The opaque sequence token for resumption. May be null when no prior /// position exists (subscribe from provider default). /// +/// +/// Optional provider name observed at runtime. +/// populates this from +/// when available. +/// [GenerateSerializer] [Alias("egil.orleans.messaging.StreamCursor")] [JsonConverter(typeof(StreamCursorJsonConverter))] public sealed record StreamCursor( - [property: Id(0)] StreamId StreamId, - [property: Id(1)] StreamSequenceToken? Token) + [property: Id(0)] string StreamNamespace, + [property: Id(1)] StreamSequenceToken? Token, + [property: Id(2)] string? ProviderName = null) { + /// + /// Creates a cursor from an Orleans stream identity by using only its + /// namespace. Message tracking is scoped to one grain, so the stream key + /// portion of is redundant for dedup. + /// + public StreamCursor(StreamId streamId, StreamSequenceToken? token) + : this(streamId.GetNamespace() ?? throw new ArgumentException("StreamId must have a namespace.", nameof(streamId)), token) + { + } + /// /// Attempts to extract the broker-side enqueue time from the underlying /// . @@ -88,24 +104,30 @@ public bool TryGetEnqueuedTime(out DateTimeOffset enqueuedTime) /// . Enables provider-aware /// dedup and diagnostics without coupling to a specific streaming provider. /// - /// + /// /// The name of the stream provider that delivered this event, or /// null if the token type does not carry this information. /// /// /// true if is an - /// and the name was extracted; - /// false otherwise. + /// or + /// is populated; false otherwise. /// - public bool TryGetStreamProviderName([NotNullWhen(true)] out string? streamProviderName) + public bool TryGetProviderName([NotNullWhen(true)] out string? providerName) { if (Token is EnrichedEventHubSequenceToken token) { - streamProviderName = token.StreamProviderName; + providerName = token.ProviderName; + return true; + } + + if (!string.IsNullOrWhiteSpace(ProviderName)) + { + providerName = ProviderName; return true; } - streamProviderName = null; + providerName = null; return false; } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs index 5e9a8351..772faee7 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs @@ -8,12 +8,12 @@ namespace Egil.Orleans.Messaging; /// /// STJ converter for . Serializes and -/// deserializes the cursor's and +/// deserializes the cursor's and /// sequence token. /// /// /// -/// wraps a StreamId and an optional +/// wraps a stream namespace and an optional /// StreamSequenceToken. The token is polymorphic and this converter /// supports Orleans built-in sequence token types only: /// , @@ -43,9 +43,7 @@ internal sealed class StreamCursorJsonConverter : JsonConverter return null; } - return new StreamCursor( - StreamId.Parse(System.Text.Encoding.UTF8.GetBytes(model.StreamId)), - FromJsonModel(model.Token)); + return new StreamCursor(model.StreamNamespace, FromJsonModel(model.Token), model.ProviderName); } /// @@ -54,8 +52,9 @@ public override void Write(Utf8JsonWriter writer, StreamCursor value, JsonSerial JsonSerializer.Serialize( writer, new StreamCursorJsonModel( - value.StreamId.ToString(), - ToJsonModel(value.Token)), + value.StreamNamespace, + ToJsonModel(value.Token), + value.ProviderName), options); } @@ -83,7 +82,7 @@ public override void Write(Utf8JsonWriter writer, StreamCursor value, JsonSerial model.SequenceNumber, model.EventIndex, model.EnqueuedTime ?? throw new JsonException("Missing EnqueuedTime."), - model.StreamProviderName ?? throw new JsonException("Missing StreamProviderName."), + model.ProviderName ?? throw new JsonException("Missing ProviderName."), model.TraceParent), _ => throw CreateUnsupportedTokenException(model.Kind ?? "") }; @@ -104,7 +103,7 @@ public override void Write(Utf8JsonWriter writer, StreamCursor value, JsonSerial EventIndex: token.EventIndex, EventHubOffset: token.EventHubOffset, EnqueuedTime: token.EnqueuedTime.UtcDateTime, - StreamProviderName: token.StreamProviderName, + ProviderName: token.ProviderName, TraceParent: token.TraceParent), EventHubSequenceTokenV2 token => new StreamSequenceTokenJsonModel( Kind: StreamSequenceTokenKinds.EventHubSequenceTokenV2, @@ -136,8 +135,9 @@ private static NotSupportedException CreateUnsupportedTokenException(string toke $"If you need this feature, please request it at {FeatureRequestUrl}."); private sealed record StreamCursorJsonModel( - string StreamId, - StreamSequenceTokenJsonModel? Token); + string StreamNamespace, + StreamSequenceTokenJsonModel? Token, + string? ProviderName = null); private sealed record StreamSequenceTokenJsonModel( string? Kind = null, @@ -145,7 +145,7 @@ private sealed record StreamSequenceTokenJsonModel( int EventIndex = 0, string? EventHubOffset = null, DateTime? EnqueuedTime = null, - string? StreamProviderName = null, + string? ProviderName = null, string? TraceParent = null); private static class StreamSequenceTokenKinds diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs index 820400f5..446f66a9 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs @@ -1,6 +1,7 @@ using System.Diagnostics; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; using Orleans.Streams; namespace Egil.Orleans.Messaging; @@ -28,7 +29,7 @@ namespace Egil.Orleans.Messaging; /// /// /// Resume semantics: each subscription reads -/// trackerSnapshot.LatestStream(streamId) once per subscription. +/// trackerSnapshot.LatestStream(streamNamespace) once per subscription. /// If a cursor exists, the is passed to /// SubscribeAsync. If null, subscribes without a token /// (provider default, typically: start from current). The tracker is read @@ -96,7 +97,7 @@ internal static StreamManager Create( var services = owner.GrainContext.ActivationServices; var loggerFactory = services.GetService(); var logger = loggerFactory?.CreateLogger() - ?? Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance; + ?? NullLogger.Instance; return Create( owner, @@ -174,12 +175,12 @@ public StreamManager AddSubscription( var streamProvider = getStreamProvider(streamNamespace); var streamId = getStreamId(streamNamespace); var stream = streamProvider.GetStream(streamId); - var observer = new StreamObserver(this, streamNamespace, streamId, onNextAsync, onError); + var observer = new StreamObserver(this, streamNamespace, onNextAsync, onError); var latestCursor = passLatestSequenceTokenOnResume - ? trackerSnapshot.LatestStream(streamId) + ? trackerSnapshot.LatestStream(streamNamespace) : null; - subscriptions.Add(ct => SubscribeCoreAsync(streamNamespace, stream, observer, latestCursor?.Token, ct)); + subscriptions.Add(async ct => await SubscribeCoreAsync(streamNamespace, stream, observer, latestCursor?.Token, ct)); return this; } @@ -252,16 +253,16 @@ public async Task SubscribeAsync(CancellationToken cancellationToken = default) cancellationToken.ThrowIfCancellationRequested(); subscribed = true; - var subscriptionTasks = subscriptions.Select(subscription => subscription(cancellationToken)).ToArray(); + var subscriptionTasks = subscriptions.Select(subscription => subscription(cancellationToken)); - var handles = await Task.WhenAll(subscriptionTasks).ConfigureAwait(false); + var handles = await Task.WhenAll(subscriptionTasks); subscriptionHandles.AddRange(handles); } - private async Task SubscribeCoreAsync( + private async Task> SubscribeCoreAsync( string streamNamespace, IAsyncStream stream, - IAsyncObserver observer, + StreamObserver observer, StreamSequenceToken? token, CancellationToken cancellationToken) { @@ -271,13 +272,15 @@ private async Task SubscribeCoreAsync( if (token is null) { - var handle = await stream.SubscribeAsync(observer).ConfigureAwait(false); + var handle = await stream.SubscribeAsync(observer); + observer.SetProviderName(handle.ProviderName); MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); return handle; } else { - var handle = await stream.SubscribeAsync(observer, token, null!).ConfigureAwait(false); + var handle = await stream.SubscribeAsync(observer, token, null!); + observer.SetProviderName(handle.ProviderName); MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); return handle; } @@ -292,19 +295,19 @@ private async Task SubscribeCoreAsync( private async Task OnNextAsync( string streamNamespace, - StreamId streamId, + string? providerName, TEvent item, StreamSequenceToken? token, Func onNextAsync, Action? onError) { - var cursor = new StreamCursor(streamId, token); + var cursor = new StreamCursor(streamNamespace, token, providerName); var started = Stopwatch.GetTimestamp(); using var activity = StartConsumerActivity(streamNamespace, cursor); try { - await onNextAsync(item, token).ConfigureAwait(false); + await onNextAsync(item, token); MessagingTelemetry.RecordStreamMessage(streamNamespace, "accepted"); MessagingTelemetry.RecordStreamHandlerDuration(streamNamespace, "accepted", Stopwatch.GetElapsedTime(started).TotalMilliseconds); @@ -397,13 +400,19 @@ private static StreamId CreateStreamId(IGrainBase owner, string streamNamespace) private sealed class StreamObserver( StreamManager manager, string streamNamespace, - StreamId streamId, Func onNextAsync, Action? onError) : IAsyncObserver { + private string? providerName; + + public void SetProviderName(string providerName) + { + this.providerName = providerName; + } + public Task OnNextAsync(TEvent item, StreamSequenceToken? token = null) => - manager.OnNextAsync(streamNamespace, streamId, item, token, onNextAsync, onError); + manager.OnNextAsync(streamNamespace, providerName, item, token, onNextAsync, onError); public Task OnCompletedAsync() => Task.CompletedTask; diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs index 0d7c351f..2cce475e 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs @@ -55,7 +55,7 @@ public void GetSequenceToken_returns_enriched_event_hub_sequence_token() var enriched = Assert.IsType(token); Assert.Equal(10, enriched.SequenceNumber); Assert.Equal(2, enriched.EventIndex); - Assert.Equal("provider-a", enriched.StreamProviderName); + Assert.Equal("provider-a", enriched.ProviderName); Assert.Null(enriched.TraceParent); } @@ -120,7 +120,7 @@ public void GetStreamPosition_extracts_traceparent_from_event_data_properties() var enriched = Assert.IsType(position.SequenceToken); Assert.Equal(traceParent, enriched.TraceParent); - Assert.Equal("provider-a", enriched.StreamProviderName); + Assert.Equal("provider-a", enriched.ProviderName); } private static EnrichedEventHubAdapter CreateAdapter() diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs index 2784bea1..64ab0397 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs @@ -19,7 +19,7 @@ public sealed class MessageTrackerJsonConverterTests sequenceNumber: 12, eventIndex: 5, enqueuedTime: new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero), - streamProviderName: "provider-a", + providerName: "provider-a", traceParent: "00-abc-xyz-01") }; @@ -99,9 +99,9 @@ public void JsonSerializer_deserializes_tracked_stream_without_sequence_token() { "Streams": [ { - "StreamId": "orders/no-token", + "StreamNamespace": "orders", "LastPosition": { - "StreamId": "orders/no-token", + "StreamNamespace": "orders", "Token": null }, "Received": "2026-05-23T12:30:00+00:00" @@ -124,9 +124,9 @@ public void JsonSerializer_throws_for_unknown_stream_token_kind() { "Streams": [ { - "StreamId": "orders/one", + "StreamNamespace": "orders", "LastPosition": { - "StreamId": "orders/one", + "StreamNamespace": "orders", "Token": { "Kind": "unknown-kind", "SequenceNumber": 1, @@ -153,15 +153,15 @@ public void JsonSerializer_throws_for_event_hub_stream_token_without_offset(stri { "Streams": [ { - "StreamId": "orders/one", + "StreamNamespace": "orders", "LastPosition": { - "StreamId": "orders/one", + "StreamNamespace": "orders", "Token": { "Kind": "{{kind}}", "SequenceNumber": 1, "EventIndex": 0, "EnqueuedTime": "2026-05-23T12:30:00+00:00", - "StreamProviderName": "provider-a" + "ProviderName": "provider-a" } }, "Received": "2026-05-23T12:30:00+00:00" @@ -182,15 +182,15 @@ public void JsonSerializer_throws_for_enriched_stream_token_without_enqueued_tim { "Streams": [ { - "StreamId": "orders/one", + "StreamNamespace": "orders", "LastPosition": { - "StreamId": "orders/one", + "StreamNamespace": "orders", "Token": { "Kind": "enriched-event-hub", "SequenceNumber": 1, "EventIndex": 0, "EventHubOffset": "42", - "StreamProviderName": "provider-a" + "ProviderName": "provider-a" } }, "Received": "2026-05-23T12:30:00+00:00" @@ -211,9 +211,9 @@ public void JsonSerializer_throws_for_enriched_stream_token_without_stream_provi { "Streams": [ { - "StreamId": "orders/one", + "StreamNamespace": "orders", "LastPosition": { - "StreamId": "orders/one", + "StreamNamespace": "orders", "Token": { "Kind": "enriched-event-hub", "SequenceNumber": 1, @@ -230,7 +230,7 @@ public void JsonSerializer_throws_for_enriched_stream_token_without_stream_provi """; var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); - Assert.Equal("Missing StreamProviderName.", exception.Message); + Assert.Equal("Missing ProviderName.", exception.Message); } private sealed class UnsupportedSequenceToken(long sequenceNumber, int eventIndex) : StreamSequenceToken @@ -259,3 +259,4 @@ public override int CompareTo(StreamSequenceToken? other) } } } + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs index 4285f7b6..c96f04be 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs @@ -268,7 +268,7 @@ public void Equals_returns_false_when_stream_keys_differ_but_counts_match() stream: new StreamCursor(StreamId.Create("orders", "one"), new EventSequenceToken(7)), received: now); var right = CreateTracker( - stream: new StreamCursor(StreamId.Create("orders", "two"), new EventSequenceToken(7)), + stream: new StreamCursor(StreamId.Create("invoices", "one"), new EventSequenceToken(7)), received: now); Assert.NotEqual(left, right); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs index 5d51efd9..40187005 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs @@ -110,7 +110,7 @@ public void JsonSerializer_throws_for_stream_cursor_payload_with_unknown_token_k { var json = """ { - "StreamId": "orders/one", + "StreamNamespace": "orders", "Token": { "Kind": "unknown-kind", "SequenceNumber": 1, @@ -144,13 +144,13 @@ public void JsonSerializer_throws_for_event_hub_payload_without_offset(string ki { var json = $$""" { - "StreamId": "orders/one", + "StreamNamespace": "orders", "Token": { "Kind": "{{kind}}", "SequenceNumber": 1, "EventIndex": 0, "EnqueuedTime": "2026-05-23T12:30:00Z", - "StreamProviderName": "provider-a" + "ProviderName": "provider-a" } } """; @@ -164,13 +164,13 @@ public void JsonSerializer_throws_for_enriched_event_hub_payload_without_enqueue { var json = """ { - "StreamId": "orders/one", + "StreamNamespace": "orders", "Token": { "Kind": "enriched-event-hub", "SequenceNumber": 1, "EventIndex": 0, "EventHubOffset": "42", - "StreamProviderName": "provider-a" + "ProviderName": "provider-a" } } """; @@ -184,7 +184,7 @@ public void JsonSerializer_throws_for_enriched_event_hub_payload_without_stream_ { var json = """ { - "StreamId": "orders/one", + "StreamNamespace": "orders", "Token": { "Kind": "enriched-event-hub", "SequenceNumber": 1, @@ -196,7 +196,7 @@ public void JsonSerializer_throws_for_enriched_event_hub_payload_without_stream_ """; var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); - Assert.Equal("Missing StreamProviderName.", exception.Message); + Assert.Equal("Missing ProviderName.", exception.Message); } private sealed class UnsupportedCustomSequenceToken(long sequenceNumber, int eventIndex) : StreamSequenceToken @@ -225,3 +225,4 @@ public override int CompareTo(StreamSequenceToken? other) } } } + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs index bcc92668..27bed0f9 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs @@ -32,7 +32,7 @@ public void TryGetEnqueuedTime_returns_false_for_non_enriched_token() } [Fact] - public void TryGetStreamProviderName_returns_true_and_value_for_enriched_event_hub_token() + public void TryGetProviderName_returns_true_and_value_for_enriched_event_hub_token() { var cursor = new StreamCursor( StreamId.Create("orders", "one"), @@ -43,23 +43,23 @@ public void TryGetStreamProviderName_returns_true_and_value_for_enriched_event_h new DateTimeOffset(2026, 5, 24, 12, 30, 0, TimeSpan.Zero), "provider-a")); - var found = cursor.TryGetStreamProviderName(out var streamProviderName); + var found = cursor.TryGetProviderName(out var providerName); Assert.True(found); - Assert.Equal("provider-a", streamProviderName); + Assert.Equal("provider-a", providerName); } [Fact] - public void TryGetStreamProviderName_returns_false_for_non_enriched_token() + public void TryGetProviderName_returns_false_for_non_enriched_token() { var cursor = new StreamCursor( StreamId.Create("orders", "one"), new EventSequenceToken(10)); - var found = cursor.TryGetStreamProviderName(out var streamProviderName); + var found = cursor.TryGetProviderName(out var providerName); Assert.False(found); - Assert.Null(streamProviderName); + Assert.Null(providerName); } [Fact] From 5d67c9464cf7d830ded82a915c6d79b725555536 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 13:28:10 +0000 Subject: [PATCH 23/82] docs(om): add messaging package documentation Document the Messaging package entry points, add it to the repository package list, and register its conventional commit scope. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- AGENTS.md | 1 + Egil.Orleans.Messaging/README.md | 149 +++++++++++++++++++++++++++++++ README.md | 1 + 3 files changed, 151 insertions(+) create mode 100644 Egil.Orleans.Messaging/README.md diff --git a/AGENTS.md b/AGENTS.md index af96f7f8..5c409387 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,6 +9,7 @@ Each top-level project ships as its own NuGet package and has its own CI workflo | Project directory | Conventional Commit scope | |----------------------------------|---------------------------| | `Egil.SystemTextJson.Migration/` | `stjm` | +| `Egil.Orleans.Messaging/` | `om` | | `Egil.Orleans.Testing/` | `ot` | | `Egil.StronglyTypedPrimitives/` | `stp` | diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md new file mode 100644 index 00000000..09049ae4 --- /dev/null +++ b/Egil.Orleans.Messaging/README.md @@ -0,0 +1,149 @@ +# Egil.Orleans.Messaging + +Composable messaging infrastructure for Microsoft Orleans grains. + +`Egil.Orleans.Messaging` provides building blocks for grains that need durable state changes and durable message handoff to move together: + +- `IStateManager` wraps `IPersistentState` so a grain does not keep observing uncommitted state after ambiguous write failures. +- `Outbox` stores messages alongside grain state and assigns durable sender sequence tokens. +- `OutboxProcessor` dispatches pending outbox items through registered postmen, with retry, reminder forwarding, failure reconciliation, and telemetry. +- `MessageTracker` records receiver-side high-water marks for outbox messages and Orleans streams. +- `StreamManager` gives grains a fluent subscription facade with resume-token and handler-error support. + +## Install + +```shell +dotnet add package Egil.Orleans.Messaging +``` + +## State Manager + +Register the default state manager factory on the silo: + +```csharp +siloBuilder.Services.AddDefaultStateManager("state"); +``` + +Then wrap the Orleans persistent state facet during activation: + +```csharp +public sealed class OrderGrain( + [PersistentState("state", "Default")] IPersistentState storage) + : Grain, IOrderGrain +{ + private IStateManager state = default!; + + public override Task OnActivateAsync(CancellationToken cancellationToken) + { + state = this.RegisterStateManager("state", storage); + return Task.CompletedTask; + } + + public async Task RenameAsync(string name) + { + await state.WriteAsync(state.State with { Name = name }); + } +} +``` + +State types must be reference types and implement `IEquatable`. For non-trivial state graphs, inherit from `VersionedState` so the recovery path compares a library-stamped version rather than relying on structural collection equality. + +## Outbox + +Store an `Outbox` on the grain state and commit messages with the business state change: + +```csharp +[GenerateSerializer] +public sealed record OrderState : VersionedState +{ + [Id(0)] public string? Name { get; init; } + + [Id(1)] public Outbox Outbox { get; init; } = + Outbox.Create(GrainId.Create("order", "example")); +} + +public async Task SubmitAsync() +{ + var next = state.State with + { + Outbox = state.State.Outbox.Add(new OrderSubmitted()) + }; + + await state.WriteAsync(next); + await outboxProcessor.PostInBackgroundAsync(); +} +``` + +Use `OutboxProcessor` to dispatch pending items and acknowledge only the items that were posted successfully: + +```csharp +public sealed class OrderGrain : Grain, IOutboxGrain +{ + private OutboxProcessor outboxProcessor = default!; + + public override Task OnActivateAsync(CancellationToken cancellationToken) + { + outboxProcessor = this.RegisterOutboxProcessor(new OutboxProcessorOptions + { + PendingItems = () => state.State.Outbox.Select(item => item.Message).ToImmutableArray(), + AcknowledgePostedAsync = async (items, ct) => + { + var posted = state.State.Outbox + .Take(items.Length) + .Select(item => item.Token); + + await state.WriteAsync(state.State with + { + Outbox = state.State.Outbox.RemoveRange(posted) + }); + }, + ReconcileFailedAsync = (_, _) => ValueTask.CompletedTask, + }) + .AddPostman(PublishSubmittedAsync); + + return Task.CompletedTask; + } +} +``` + +`IOutboxGrain` forwards reminder ticks to the attached processor. The grain remains responsible for its own message contracts, posting target, and dead-letter policy. + +## Receiver Dedup + +`MessageTracker` accepts a message only when its stream cursor or outbox token advances the stored high-water mark: + +```csharp +if (!state.State.Tracker.ProcessMessage(token, out var tracker)) +{ + return; +} + +await state.WriteAsync(state.State with { Tracker = tracker }); +``` + +The tracker can also evict old sender or stream entries when your retention policy allows it. + +## Streams + +Use `StreamManager` to configure stream subscriptions from `OnActivateAsync` and resume from the tracker snapshot: + +```csharp +streamManager = this.RegisterStreamManager(state.State.Tracker) + .AddSubscription( + "prices", + async (message, cursor, ct) => + { + if (!state.State.Tracker.ProcessMessage(cursor, out var tracker)) + { + return; + } + + await state.WriteAsync(state.State with { Tracker = tracker }); + }); + +await streamManager.SubscribeAsync(cancellationToken); +``` + +## Scope + +This package is messaging infrastructure, not an event-sourcing or CQRS framework. It wraps Orleans state, outbox dispatch, receiver deduplication, and stream subscription management while leaving domain modeling, read models, transport targets, and operational policy to the application. diff --git a/README.md b/README.md index ce7327bc..c6273cc6 100644 --- a/README.md +++ b/README.md @@ -3,5 +3,6 @@ A collection of libraries! - **[Strongly Typed Primitives](https://github.com/egil/framework/tree/main/Egil.StronglyTypedPrimitives):** A source generator for creating strongly-typed primitive types that makes it easy to avoid the primitive obsession anti pattern. Add a [StronglyTyped] attribute to a partial record struct to get started. +- **[Orleans Messaging](https://github.com/egil/framework/tree/main/Egil.Orleans.Messaging):** Composable messaging infrastructure for Microsoft Orleans grains, including atomic state writes, durable outboxes, receiver-side deduplication, and stream subscription management. - **[Orleans Testing](https://github.com/egil/framework/tree/main/Egil.Orleans.Testing):** Deterministic async assertion helpers for Microsoft Orleans integration tests. Provides a `GrainActivityCollector` that monitors grain calls and storage operations so you can write reliable, signal-driven assertions instead of `Task.Delay` waits. - **[System.Text.Json Migration](https://github.com/egil/framework/tree/main/Egil.SystemTextJson.Migration):** A migration-focused extension for System.Text.Json that supports explicit migrator registration, optional scoped assembly discovery, and AOT-friendly serialization workflows. From b4f3d9ad77c960bfc9337038c5f344542de022fa Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 13:29:12 +0000 Subject: [PATCH 24/82] build(om): wire messaging package CI Add the Messaging package workflow, include the package README in the NuGet artifact, and make Messaging tests consume Egil.Orleans.Testing as a package dependency instead of a sibling project reference. [skip notes] Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../workflows/egil-orleans-messaging-ci.yml | 163 ++++++++++++++++++ .../Directory.Packages.props | 1 + .../Egil.Orleans.Messaging.csproj | 6 +- .../Egil.Orleans.Messaging.Tests.csproj | 2 +- 4 files changed, 170 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/egil-orleans-messaging-ci.yml diff --git a/.github/workflows/egil-orleans-messaging-ci.yml b/.github/workflows/egil-orleans-messaging-ci.yml new file mode 100644 index 00000000..f8ab40a4 --- /dev/null +++ b/.github/workflows/egil-orleans-messaging-ci.yml @@ -0,0 +1,163 @@ +# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json +name: egil-orleans-messaging +on: + workflow_dispatch: + push: + branches: + - 'main' + - 'release/egil-orleans-messaging/v*' + paths: + - 'Egil.Orleans.Messaging/**' + - '.github/workflows/egil-orleans-messaging-ci.yml' + pull_request: + branches: + - 'main' + paths: + - 'Egil.Orleans.Messaging/**' + - '.github/workflows/egil-orleans-messaging-ci.yml' + +concurrency: + group: ${{ github.workflow }}-${{ github.ref == 'refs/heads/main' && github.run_id || github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +env: + DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1 + DOTNET_NOLOGO: true + NuGetDirectory: ${{ github.workspace }}/nuget + TestResultsDirectory: ${{ github.workspace }}/TestResults + BRANCH_NAME: ${{ github.head_ref || github.ref_name }} + Project: Egil.Orleans.Messaging + +permissions: + contents: read + +jobs: + create-nuget: + runs-on: ubuntu-latest + if: github.event.inputs.prepare-release != 'true' + permissions: + contents: read + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Setup .NET + uses: actions/setup-dotnet@v4 + with: + dotnet-version: 10.x + + - name: Build and pack + run: | + dotnet build ./Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj -c Release -f net10.0 -p:ContinuousIntegrationBuild=true + dotnet pack ./Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj -c Release --no-build -p:ContinuousIntegrationBuild=true --output ${{ env.NuGetDirectory }} + + - uses: actions/upload-artifact@v4 + with: + name: nuget + if-no-files-found: error + retention-days: 7 + path: ${{ env.NuGetDirectory }}/*.nupkg + + validate-nuget: + runs-on: ubuntu-latest + if: github.event.inputs.prepare-release != 'true' + needs: [ create-nuget ] + permissions: + contents: read + steps: + - uses: actions/checkout@v4 + + - name: Setup .NET + uses: actions/setup-dotnet@v4 + with: + dotnet-version: 10.x + + - uses: actions/download-artifact@v4 + with: + name: nuget + path: ${{ env.NuGetDirectory }} + + - name: Install nuget validator + run: dotnet tool restore + + - name: Validate package + shell: pwsh + run: dotnet meziantou.validate-nuget-package (Get-ChildItem "${{ env.NuGetDirectory }}/*.nupkg") --excluded-rules IconMustBeSet + + run-test: + runs-on: ubuntu-latest + if: github.event.inputs.prepare-release != 'true' + timeout-minutes: 30 + env: + TestResultsDirectory: ${{ github.workspace }}/TestResults + permissions: + contents: read + checks: write + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Setup .NET + uses: actions/setup-dotnet@v4 + with: + dotnet-version: 10.x + + - name: Run tests + working-directory: Egil.Orleans.Messaging + run: | + export GITHUB_ENV=/dev/null + dotnet test --solution ./Egil.Orleans.Messaging.slnx --configuration Release + + release: + needs: [validate-nuget, run-test] + if: github.event_name == 'workflow_dispatch' || startsWith(github.ref, 'refs/heads/release/egil-orleans-messaging/v') + runs-on: ubuntu-latest + permissions: + contents: write + id-token: write + steps: + - uses: actions/download-artifact@v4 + with: + name: nuget + path: ${{ env.NuGetDirectory }} + + - name: Setup .NET + uses: actions/setup-dotnet@v4 + with: + dotnet-version: 10.x + + - name: NuGet login (trusted publishing) + uses: NuGet/login@v1 + id: login + with: + user: ${{ secrets.NUGET_USER }} + + - name: Publish NuGet package + shell: pwsh + run: | + foreach($file in (Get-ChildItem "${{ env.NuGetDirectory }}" -Recurse -Include *.nupkg)) { + dotnet nuget push $file --api-key ${{ steps.login.outputs.NUGET_API_KEY }} --source https://api.nuget.org/v3/index.json --skip-duplicate + } + + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Configure GIT + run: | + git config user.name "${{ github.actor }}" + git config user.email "${{ github.actor }}@users.noreply.github.com" + + - name: Add git tag using dotnet nbgv + run: | + cd ${{ env.Project }} + dotnet tool restore + dotnet nbgv tag + git push origin --tags + + - name: Delete release branch on origin + if: startsWith(github.ref, 'refs/heads/release/egil-orleans-messaging/v') + run: git push origin --delete "${{ env.BRANCH_NAME }}" diff --git a/Egil.Orleans.Messaging/Directory.Packages.props b/Egil.Orleans.Messaging/Directory.Packages.props index f0a1e559..ea96da40 100644 --- a/Egil.Orleans.Messaging/Directory.Packages.props +++ b/Egil.Orleans.Messaging/Directory.Packages.props @@ -23,6 +23,7 @@ + diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj index 6cffc4ca..cdd51962 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj @@ -19,7 +19,7 @@ stream subscription management. Pick what you need, leave the rest. README.md - orleans,messaging,outbox,dedup,cqrs,grain,egil + orleans,messaging,outbox,deduplication,streams,grain,egil Egil Hansen https://github.com/egil/framework https://github.com/egil/framework @@ -52,6 +52,10 @@ True \ + + True + \ + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj index e7add5c6..e9db0fd0 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Egil.Orleans.Messaging.Tests.csproj @@ -27,6 +27,7 @@ all runtime; build; native; contentfiles; analyzers; buildtransitive + @@ -37,7 +38,6 @@ - From 3b8ea25d48c8c6cbe5345decf7f01647fcaa2d73 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 14:44:31 +0000 Subject: [PATCH 25/82] docs(om): document messaging API direction Capture the StreamManager implicit/explicit subscription split and the capability namespace layout before the refactor continues. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/api-design.md | 246 +++++++++++++++++++++++---- 1 file changed, 212 insertions(+), 34 deletions(-) diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 4c787938..5889cba9 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -29,8 +29,8 @@ A set of composable building blocks for Orleans grains that need: with retry, telemetry, and failure callbacks. 4. **Receiver-side dedup** — `MessageTracker` that tracks high-water positions from both outbox senders and Orleans streams. -5. **Stream subscription management** — fluent subscribe/resume/error - facade over Orleans implicit subscriptions. +5. **Stream subscription management** — fluent configure/resume/error + facade over Orleans implicit and explicit stream subscriptions. ### What this library is NOT @@ -45,6 +45,31 @@ One NuGet package for now. Internal boundaries (state management vs messaging vs streaming) are kept clean so a future split is mechanical. Split only when dependency weight becomes a concrete problem. +### Capability namespaces and folders + +Source and test files are grouped by library tool. Tests mirror production +folders so behavior is easy to find from the type under test. + +| Capability | Production namespace | Test folder | +| ---------- | -------------------- | ----------- | +| State management | `Egil.Orleans.Messaging.State` | `State/` | +| Outbox | `Egil.Orleans.Messaging.Outboxes` | `Outboxes/` | +| Receiver tracking | `Egil.Orleans.Messaging.Tracking` | `Tracking/` | +| Streams | `Egil.Orleans.Messaging.Streams` | `Streams/` | +| Event Hub stream enrichment | `Egil.Orleans.Messaging.Streams.EventHubs` | `Streams/EventHubs/` | + +Extension entry points live in the namespace of the type they extend so they +are discoverable from normal Orleans, hosting, and DI imports: + +- Grain registration methods: `namespace Orleans`. +- `IServiceCollection` registration methods: + `namespace Microsoft.Extensions.DependencyInjection`. +- `ISiloBuilder` and `IEventHubStreamConfigurator` registration methods: + `namespace Orleans.Hosting`. + +C# 14 extension blocks are the preferred shape for new extension entry +points. + ### Name **`Egil.Orleans.Messaging`** — the outbox, postman, dedup, and stream @@ -640,41 +665,106 @@ fake-clock tests. --- -## 5. `StreamManager` — fluent subscribe/resume/error facade +## 5. `StreamManager` — stream subscription handler facade **Status:** Settled. -Grain-level facade around Orleans implicit subscriptions. Four -responsibilities: +Grain-level facade around Orleans stream subscription handler attachment. +The API must preserve Orleans' distinction between implicit and explicit +subscriptions: + +- **Implicit subscriptions** are declared by Orleans attributes such as + `[ImplicitStreamSubscription("namespace")]`. Orleans owns activation and + subscription creation. The library only attaches handler logic when Orleans + calls `IStreamSubscriptionObserver.OnSubscribed(...)`. +- **Explicit subscriptions** are created by the grain via Orleans' + `SubscribeAsync(...)`. Orleans persists the subscription handle, and the + grain must resume existing handles after reactivation using + `GetAllSubscriptionHandles()` + `ResumeAsync(...)`. + +Four responsibilities: 1. **Configure** stream namespaces during `OnActivateAsync`. -2. **Resume** from last accepted `StreamCursor` per namespace. -3. **Dispatch** with projected `StreamCursor`. -4. **Per-subscription error handling** via the optional `onError` callback. +2. **Attach** handlers to implicit subscription handles provided by Orleans. +3. **Resume or ensure** durable explicit subscription handles. +4. **Dispatch** with projected `StreamCursor` and per-subscription error + handling via the optional `onError` callback. ### Why a facade, not a base class Extension/composition over inheritance. Grain inherits from `Grain`, -uses `StreamManager` as a field. +registers a `StreamManager`, and optionally implements marker interfaces for +runtime callbacks. + +For implicit streams, the grain implements `IImplicitStreamGrain`; it does +not expose a `StreamManager` property. `StreamManager` follows the same +component pattern as `OutboxProcessor.AttachToGrain()`: + +1. `RegisterStreamManager(...)` creates the manager. +2. The manager attaches itself to `grain.GrainContext` as an internal + stream component. +3. The `IImplicitStreamGrain` default interface method receives Orleans' + `OnSubscribed(...)` callback. +4. The default method resolves the attached component and forwards the + callback to `StreamManager`. + +```mermaid +sequenceDiagram + participant Producer + participant Orleans + participant Grain as "Grain : IImplicitStreamGrain" + participant Component as "IStreamManagerComponent" + participant Manager as "StreamManager" + + Grain->>Manager: RegisterStreamManager(...).ConfigureImplicitSubscription(...) + Manager->>Component: AttachToGrain() + Producer->>Orleans: OnNext(stream namespace, grain key) + Orleans->>Grain: activate grain when needed + Orleans->>Grain: IStreamSubscriptionObserver.OnSubscribed(factory) + Grain->>Component: forward OnSubscribed(factory) + Component->>Manager: resolve configured namespace handler + Manager->>Orleans: factory.Create().ResumeAsync(observer) + Orleans->>Manager: observer.OnNext(event, token) +``` ### Shape ```csharp +public interface IImplicitStreamGrain : IStreamSubscriptionObserver +{ + Task IStreamSubscriptionObserver.OnSubscribed( + IStreamSubscriptionHandleFactory handleFactory); +} + public sealed class StreamManager { - public StreamManager AddSubscription( + public StreamManager ConfigureImplicitSubscription( + string streamNamespace, + Func onNextAsync, + Action? onError = default); + + public StreamManager ConfigureImplicitSubscription( + string streamNamespace, + Func onNextAsync, + Action? onError = default); + + public StreamManager ConfigureExplicitSubscription( + string streamProviderName, string streamNamespace, - Func onNextAsync, - Action? onError = default, - bool passLatestSequenceTokenOnResume = true); + Func onNextAsync, + Action? onError = default); - public StreamManager AddSubscription( + public StreamManager ConfigureExplicitSubscription( + string streamProviderName, string streamNamespace, - Func onNextAsync, - Action? onError = default, - bool passLatestSequenceTokenOnResume = true); + Func onNextAsync, + Action? onError = default); + + public Task ResumeExplicitSubscriptionsAsync( + CancellationToken cancellationToken = default); - public Task SubscribeAsync(CancellationToken cancellationToken = default); + public Task EnsureExplicitSubscriptionsAsync( + CancellationToken cancellationToken = default); } public static class StreamManagerExtensions @@ -687,7 +777,42 @@ public static class StreamManagerExtensions ``` -### Typical wiring +`ConfigureImplicitSubscription` intentionally has no provider-name +parameter. Orleans provides the concrete provider and stream id through +`IStreamSubscriptionHandleFactory` when the implicit subscription fires. + +`ConfigureExplicitSubscription` requires a provider name because the library +must ask Orleans for the stream and durable subscription handles itself. + +### Typical implicit wiring + +```csharp +public override async Task OnActivateAsync(CancellationToken ct) +{ + var state = await stateManager.ReadAsync(); + state.Tracker.RegisterTimeProvider(timeProvider); + + this.RegisterStreamManager(state.Tracker) + .ConfigureImplicitSubscription("electricity-prices", HandlePriceTickAsync, LogStreamError) + .ConfigureImplicitSubscription("tariff-events", HandleTariffChangedAsync); +} +``` + +The grain must also be attributed and implement `IImplicitStreamGrain`: + +```csharp +[ImplicitStreamSubscription("electricity-prices")] +public sealed class PriceProjectionGrain : Grain, IImplicitStreamGrain +{ + // OnSubscribed is supplied by IImplicitStreamGrain's default method. +} +``` + +No `SubscribeAsync` call belongs in implicit activation. Orleans activates +the grain and calls `OnSubscribed(...)` when an event targets the implicit +subscription. + +### Typical explicit wiring ```csharp public override async Task OnActivateAsync(CancellationToken ct) @@ -696,26 +821,78 @@ public override async Task OnActivateAsync(CancellationToken ct) state.Tracker.RegisterTimeProvider(timeProvider); streamManager = this.RegisterStreamManager(state.Tracker) - .AddSubscription("electricity-prices", HandlePriceTickAsync, LogStreamError) - .AddSubscription("tariff-events", HandleTariffChangedAsync); + .ConfigureExplicitSubscription("StreamProvider", "tariff-events", HandleTariffChangedAsync); - await streamManager.SubscribeAsync(ct); + await streamManager.EnsureExplicitSubscriptionsAsync(ct); } ``` The `TEvent` generic argument is usually inferred from the handler method group. Users only specify it for inline lambdas or ambiguous method groups. -### Resume semantics - -- Each subscription reads `trackerSnapshot.LatestStream(streamId)` once - when `SubscribeAsync(...)` establishes configured subscriptions. -- If cursor exists → hydrate `StreamSequenceToken`, pass to - `SubscribeAsync`. -- If null → subscribe without token (provider default, typically: - start from current). -- Tracker read once at subscription time, never again. Handler is the - only path that mutates the tracker. +### Implementation changes from the previous design + +- Rename `AddSubscription(...)` to the more explicit + `ConfigureImplicitSubscription(...)` and + `ConfigureExplicitSubscription(...)`. +- Replace `SubscribeAsync(...)` with explicit-only + `ResumeExplicitSubscriptionsAsync(...)` and + `EnsureExplicitSubscriptionsAsync(...)`. +- Add `IImplicitStreamGrain` as a public convenience interface with a default + `IStreamSubscriptionObserver.OnSubscribed(...)` implementation. +- Add an internal `IStreamManagerComponent`, attach `StreamManager` to + `IGrainContext`, and let `IImplicitStreamGrain` forward through that + component. +- Change stream handlers to receive `StreamCursor` instead of raw + `StreamSequenceToken?`. +- Keep explicit subscription unsubscribe orchestration out of the initial API. + The first version only resumes or ensures explicit subscriptions. + +### Implicit subscription semantics + +- `ConfigureImplicitSubscription(...)` records the handler shape only. +- It does not call `SubscribeAsync(...)`, inspect existing explicit handles, + or read a resume token from `MessageTracker`. +- When Orleans calls `OnSubscribed(factory)`, `StreamManager` matches + `factory.StreamId.GetNamespace()` to a configured implicit subscription, + creates the typed handle with `factory.Create()`, and calls + `ResumeAsync(observer)` to attach the handler. +- Orleans controls activation and target stream identity. If the grain is not + active, a matching stream event can activate it. +- Implicit subscriptions do not show up as explicit handles from + `GetAllSubscriptionHandles()` and cannot be removed by `UnsubscribeAsync()` + on an explicit handle. + +### Explicit subscription semantics + +- `ConfigureExplicitSubscription(...)` records the provider, namespace, event + type, handler, and error callback. +- `ResumeExplicitSubscriptionsAsync(...)` resumes all existing durable + handles for each configured explicit stream. It never creates a new + subscription. +- `EnsureExplicitSubscriptionsAsync(...)` resumes existing handles when + present. If none exist for a configured explicit stream, it creates exactly + one explicit subscription with `SubscribeAsync(observer)`. +- Repeated calls to `EnsureExplicitSubscriptionsAsync(...)` are idempotent: + once a durable explicit handle exists, later calls resume it instead of + creating duplicates. +- Explicit handles persist across grain deactivation until Orleans removes + them via `UnsubscribeAsync()`. The initial version does not expose + unsubscribe orchestration; consumers can use Orleans handles directly if + they need to intentionally detach. + +### Cursor semantics + +- Stream handlers receive a `StreamCursor`, not a raw + `StreamSequenceToken?`. +- The cursor includes the stream namespace, provider name when known, and the + delivered Orleans sequence token. +- `MessageTracker` is not used to drive implicit subscription start position. + For implicit streams, Orleans owns subscription creation and the manager + only resumes handler logic for the current delivery path. +- For explicit streams, the durable Orleans subscription handle is the primary + resume mechanism. `MessageTracker` remains the application-level dedup and + high-water marker; handlers update it after accepting events. ### Per-subscription `OnError` @@ -724,8 +901,9 @@ namespace. Default when omitted: log + emit counter, do NOT rethrow. ### One-provider-per-namespace -Accepted as a code-review convention. Orleans's own constraint — the -library doesn't make it worse. +Accepted as a code-review convention for implicit subscriptions. Explicit +subscriptions carry a provider name in configuration, so provider/namespace +ambiguity is visible in code. ### `StreamCursor` projection constraint From 2d8c55791d83c4ea95cf697f9cfd9bc408b8899f Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 15:04:54 +0000 Subject: [PATCH 26/82] refactor(om)!: split messaging tools by capability Move messaging tools into capability namespaces and matching test folders, with Outboxes used to avoid a namespace/type name collision. Replace StreamManager's AddSubscription/SubscribeAsync surface with explicit implicit/explicit configuration APIs and IImplicitStreamGrain forwarding through an attached grain component. BREAKING CHANGE: Messaging types moved from Egil.Orleans.Messaging into capability namespaces such as Egil.Orleans.Messaging.State, Egil.Orleans.Messaging.Outboxes, Egil.Orleans.Messaging.Tracking, and Egil.Orleans.Messaging.Streams. StreamManager now uses ConfigureImplicitSubscription, ConfigureExplicitSubscription, ResumeExplicitSubscriptionsAsync, and EnsureExplicitSubscriptionsAsync instead of AddSubscription and SubscribeAsync. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 28 +- .../MessagingTelemetry.cs | 2 + .../{ => Outboxes}/IOutboxGrain.cs | 2 +- .../NoPostmanRegisteredException.cs | 2 +- .../{ => Outboxes}/Outbox.cs | 2 +- .../OutboxJsonConverterFactory.cs | 2 +- .../{ => Outboxes}/OutboxMessageEnvelope.cs | 2 +- ...tboxMessageEnvelopeJsonConverterFactory.cs | 2 +- .../{ => Outboxes}/OutboxProcessor.cs | 2 +- .../OutboxProcessorExtensions.cs | 18 +- .../{ => Outboxes}/OutboxProcessorOptions.cs | 2 +- .../{ => Outboxes}/OutboxSequenceToken.cs | 3 +- .../OutboxSequenceTokenJsonConverter.cs | 2 +- .../{ => State}/IStateManager.cs | 2 +- .../{ => State}/IStateManagerFactory.cs | 2 +- .../{ => State}/StateManager.cs | 2 +- .../{ => State}/StateManagerExtensions.cs | 16 +- .../StateManagerRegistrationExtensions.cs | 70 +-- .../StateManagerSiloBuilderExtensions.cs | 57 ++ .../{ => State}/VersionedState.cs | 2 +- .../Egil.Orleans.Messaging/StreamManager.cs | 421 --------------- .../StreamManagerExtensions.cs | 50 -- .../EventHubs}/EnrichedEventHubAdapter.cs | 2 +- .../EnrichedEventHubAdapterExtensions.cs | 14 +- .../EnrichedEventHubSequenceToken.cs | 3 +- .../Streams/IImplicitStreamGrain.cs | 19 + .../Streams/IStreamManagerComponent.cs | 8 + .../{ => Streams}/StreamCursor.cs | 4 +- .../StreamCursorJsonConverter.cs | 3 +- .../Streams/StreamManager.cs | 495 ++++++++++++++++++ .../Streams/StreamManagerExtensions.cs | 72 +++ .../{ => Tracking}/MessageTracker.cs | 4 +- .../MessageTrackerJsonConverter.cs | 4 +- .../GlobalUsings.cs | 8 + .../MessagingTestClusterFixture.cs | 10 + .../OutboxJsonConverterFactoryTests.cs | 2 +- ...essageEnvelopeJsonConverterFactoryTests.cs | 2 +- .../{ => Outboxes}/OutboxProcessorTests.cs | 10 +- .../OutboxSequenceTokenJsonConverterTests.cs | 2 +- .../{ => Outboxes}/OutboxTests.cs | 2 +- .../StateManagerExtensionsTests.cs | 4 +- ...StateManagerRegistrationExtensionsTests.cs | 4 +- .../{ => State}/StateManagerTests.cs | 2 +- .../StreamManagerTests.cs | 248 --------- .../EnrichedEventHubAdapterExtensionsTests.cs | 4 +- .../EnrichedEventHubAdapterTests.cs | 3 +- .../StreamCursorJsonConverterTests.cs | 2 +- .../{ => Streams}/StreamCursorTests.cs | 2 +- .../Streams/StreamManagerTests.cs | 366 +++++++++++++ .../MessageTrackerJsonConverterTests.cs | 2 +- .../{ => Tracking}/MessageTrackerTests.cs | 2 +- 51 files changed, 1138 insertions(+), 856 deletions(-) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/IOutboxGrain.cs (98%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/NoPostmanRegisteredException.cs (97%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/Outbox.cs (99%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/OutboxJsonConverterFactory.cs (99%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/OutboxMessageEnvelope.cs (97%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/OutboxMessageEnvelopeJsonConverterFactory.cs (98%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/OutboxProcessor.cs (99%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/OutboxProcessorExtensions.cs (83%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/OutboxProcessorOptions.cs (98%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/OutboxSequenceToken.cs (97%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Outboxes}/OutboxSequenceTokenJsonConverter.cs (97%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => State}/IStateManager.cs (99%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => State}/IStateManagerFactory.cs (96%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => State}/StateManager.cs (99%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => State}/StateManagerExtensions.cs (88%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => State}/StateManagerRegistrationExtensions.cs (54%) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => State}/VersionedState.cs (98%) delete mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs delete mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Streams/EventHubs}/EnrichedEventHubAdapter.cs (99%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Streams/EventHubs}/EnrichedEventHubAdapterExtensions.cs (88%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Streams/EventHubs}/EnrichedEventHubSequenceToken.cs (98%) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamManagerComponent.cs rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Streams}/StreamCursor.cs (98%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Streams}/StreamCursorJsonConverter.cs (98%) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Tracking}/MessageTracker.cs (99%) rename Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/{ => Tracking}/MessageTrackerJsonConverter.cs (98%) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Outboxes}/OutboxJsonConverterFactoryTests.cs (99%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Outboxes}/OutboxMessageEnvelopeJsonConverterFactoryTests.cs (97%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Outboxes}/OutboxProcessorTests.cs (97%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Outboxes}/OutboxSequenceTokenJsonConverterTests.cs (98%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Outboxes}/OutboxTests.cs (99%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => State}/StateManagerExtensionsTests.cs (97%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => State}/StateManagerRegistrationExtensionsTests.cs (98%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => State}/StateManagerTests.cs (99%) delete mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Streams/EventHubs}/EnrichedEventHubAdapterExtensionsTests.cs (92%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Streams/EventHubs}/EnrichedEventHubAdapterTests.cs (98%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Streams}/StreamCursorJsonConverterTests.cs (99%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Streams}/StreamCursorTests.cs (98%) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Tracking}/MessageTrackerJsonConverterTests.cs (99%) rename Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/{ => Tracking}/MessageTrackerTests.cs (99%) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index 09049ae4..6607c666 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -16,12 +16,31 @@ Composable messaging infrastructure for Microsoft Orleans grains. dotnet add package Egil.Orleans.Messaging ``` +Use the capability namespaces for the tools you need: + +```csharp +using Egil.Orleans.Messaging.Outboxes; +using Egil.Orleans.Messaging.State; +using Egil.Orleans.Messaging.Streams; +using Egil.Orleans.Messaging.Streams.EventHubs; +using Egil.Orleans.Messaging.Tracking; +``` + +Registration extension members live with the Orleans, hosting, and DI types +they extend: + +```csharp +using Microsoft.Extensions.DependencyInjection; +using Orleans; +using Orleans.Hosting; +``` + ## State Manager Register the default state manager factory on the silo: ```csharp -siloBuilder.Services.AddDefaultStateManager("state"); +siloBuilder.AddDefaultStateManager("state"); ``` Then wrap the Orleans persistent state facet during activation: @@ -129,9 +148,10 @@ Use `StreamManager` to configure stream subscriptions from `OnActivateAsync` and ```csharp streamManager = this.RegisterStreamManager(state.State.Tracker) - .AddSubscription( + .ConfigureExplicitSubscription( + "StreamProvider", "prices", - async (message, cursor, ct) => + async (message, cursor) => { if (!state.State.Tracker.ProcessMessage(cursor, out var tracker)) { @@ -141,7 +161,7 @@ streamManager = this.RegisterStreamManager(state.State.Tracker) await state.WriteAsync(state.State with { Tracker = tracker }); }); -await streamManager.SubscribeAsync(cancellationToken); +await streamManager.EnsureExplicitSubscriptionsAsync(cancellationToken); ``` ## Scope diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs index bfb2d5d2..5cd04056 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs @@ -1,6 +1,8 @@ using System.Collections.Concurrent; using System.Diagnostics; using System.Diagnostics.Metrics; +using Egil.Orleans.Messaging.Outboxes; +using Egil.Orleans.Messaging.Streams; namespace Egil.Orleans.Messaging; diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs similarity index 98% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs index 589ffd6e..7bf0af6e 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IOutboxGrain.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs @@ -1,4 +1,4 @@ -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Outboxes; /// /// Marker interface for grains that use . diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/NoPostmanRegisteredException.cs similarity index 97% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/NoPostmanRegisteredException.cs index 3fda919e..532c6afa 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/NoPostmanRegisteredException.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/NoPostmanRegisteredException.cs @@ -1,4 +1,4 @@ -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Outboxes; /// /// Thrown (internally, via ) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs similarity index 99% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs index 5f358cdc..52e6a29c 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs @@ -2,7 +2,7 @@ using System.Collections.Immutable; using System.Text.Json.Serialization; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Outboxes; /// /// A per-grain durable buffer of messages that have been announced diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs similarity index 99% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs index 6852df63..0911f587 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxJsonConverterFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs @@ -2,7 +2,7 @@ using System.Text.Json; using System.Text.Json.Serialization; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Outboxes; /// /// STJ converter factory that creates closed diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs similarity index 97% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs index f4df6d7a..9774a446 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelope.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs @@ -1,6 +1,6 @@ using System.Text.Json.Serialization; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Outboxes; /// /// Wraps a user-defined message with its diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs similarity index 98% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs index 24237475..9b51d43f 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxMessageEnvelopeJsonConverterFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs @@ -1,7 +1,7 @@ using System.Text.Json; using System.Text.Json.Serialization; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Outboxes; /// /// STJ converter factory that creates closed diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs similarity index 99% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index 4d00f8d0..8052689a 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -2,7 +2,7 @@ using System.Diagnostics; using Microsoft.Extensions.Logging; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Outboxes; /// /// Grain-scoped component that owns the timer, reminder, and postman dispatch diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs similarity index 83% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs index 92162ee7..7cf507b0 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs @@ -1,7 +1,8 @@ using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; +using Egil.Orleans.Messaging.Outboxes; -namespace Egil.Orleans.Messaging; +namespace Orleans; /// /// Extension methods for wiring into @@ -9,6 +10,9 @@ namespace Egil.Orleans.Messaging; /// public static class OutboxProcessorExtensions { + extension(TGrain grain) + where TGrain : IGrainBase, IOutboxGrain + { /// /// Creates and attaches an to the /// grain. Call in OnActivateAsync and chain @@ -45,23 +49,14 @@ public static class OutboxProcessorExtensions /// /// /// - /// - /// The grain type. Must implement (for reminder - /// DIM) and (for grain context access). Both - /// constraints are compiler-enforced. - /// /// The base type of outbox items. - /// The grain instance (this). /// Processor configuration. /// /// The processor instance. Chain AddPostman calls on it, then store /// in a grain field for later /// calls. /// - public static OutboxProcessor RegisterOutboxProcessor( - this TGrain grain, - OutboxProcessorOptions options) - where TGrain : IGrainBase, IOutboxGrain + public OutboxProcessor RegisterOutboxProcessor(OutboxProcessorOptions options) where TOutbox : notnull { ArgumentNullException.ThrowIfNull(grain); @@ -77,4 +72,5 @@ public static OutboxProcessor RegisterOutboxProcessor( processor.AttachToGrain(); return processor; } + } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs similarity index 98% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs index 562d2354..769ae3e9 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxProcessorOptions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs @@ -1,6 +1,6 @@ using System.Collections.Immutable; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Outboxes; /// /// Controls where outbox postman callbacks execute. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs similarity index 97% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs index 98870e21..0ce4892b 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs @@ -1,6 +1,7 @@ using System.Text.Json.Serialization; +using Egil.Orleans.Messaging.Tracking; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Outboxes; /// /// Identifies a specific message within an . Combines a diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs similarity index 97% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs index fe0489ca..64a2878e 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/OutboxSequenceTokenJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs @@ -1,7 +1,7 @@ using System.Text.Json; using System.Text.Json.Serialization; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Outboxes; /// /// STJ converter for . Serializes and diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs similarity index 99% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs index 6fe45a34..728ee6d2 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs @@ -1,4 +1,4 @@ -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.State; /// /// A thin wrapper around that guarantees diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManagerFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs similarity index 96% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManagerFactory.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs index c52006a0..440bcd2a 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/IStateManagerFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs @@ -1,4 +1,4 @@ -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.State; /// /// Factory contract for constructing provider-specific diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs similarity index 99% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs index dcf9bba6..df72b083 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs @@ -1,6 +1,6 @@ using Orleans.Storage; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.State; /// /// Classifies a storage write failure for recovery decisions. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs similarity index 88% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs index 9d2a9d45..c42a429b 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs @@ -1,6 +1,7 @@ using Microsoft.Extensions.DependencyInjection; +using Egil.Orleans.Messaging.State; -namespace Egil.Orleans.Messaging; +namespace Orleans; /// /// Extension methods for wiring into a grain's @@ -8,6 +9,9 @@ namespace Egil.Orleans.Messaging; /// public static class StateManagerExtensions { + extension(TGrain grain) + where TGrain : IGrainBase + { /// /// Creates an for the given grain using /// a keyed registration. @@ -34,14 +38,9 @@ public static class StateManagerExtensions /// . /// /// - /// - /// The grain type. Must implement so activation - /// services can be used to resolve keyed registrations. - /// /// /// The grain state type. /// - /// The grain instance (this). /// /// Logical storage name used as the keyed DI registration key. /// @@ -51,11 +50,9 @@ public static class StateManagerExtensions /// /// A keyed instance. /// - public static IStateManager RegisterStateManager( - this TGrain grain, + public IStateManager RegisterStateManager( string storageName, IPersistentState storage) - where TGrain : IGrainBase where TState : class, IEquatable { ArgumentNullException.ThrowIfNull(grain); @@ -68,6 +65,7 @@ public static IStateManager RegisterStateManager( storage, grain.GetType()); } + } internal static IStateManager RegisterStateManagerCore( IServiceProvider activationServices, diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerRegistrationExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs similarity index 54% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerRegistrationExtensions.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs index f2663333..357ef3aa 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StateManagerRegistrationExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs @@ -1,6 +1,6 @@ -using Microsoft.Extensions.DependencyInjection; +using Egil.Orleans.Messaging.State; -namespace Egil.Orleans.Messaging; +namespace Microsoft.Extensions.DependencyInjection; /// /// Registration helpers for keyed @@ -8,13 +8,13 @@ namespace Egil.Orleans.Messaging; /// public static class StateManagerRegistrationExtensions { + extension(IServiceCollection services) + { /// /// Registers the default keyed for the /// given . /// - public static IServiceCollection AddDefaultStateManager( - this IServiceCollection services, - string storageName) + public IServiceCollection AddDefaultStateManager(string storageName) { ArgumentNullException.ThrowIfNull(services); ValidateStorageName(storageName); @@ -29,8 +29,7 @@ public static IServiceCollection AddDefaultStateManager( /// Registers a custom keyed open-generic /// implementation for the given . /// - public static IServiceCollection AddStateManagerFactory( - this IServiceCollection services, + public IServiceCollection AddStateManagerFactory( string storageName, Type openGenericFactoryType) { @@ -47,9 +46,7 @@ public static IServiceCollection AddStateManagerFactory( /// /// Registers a custom keyed factory for a specific state type. /// - public static IServiceCollection AddStateManagerFactory( - this IServiceCollection services, - string storageName) + public IServiceCollection AddStateManagerFactory(string storageName) where TState : class, IEquatable where TFactory : class, IStateManagerFactory { @@ -58,63 +55,14 @@ public static IServiceCollection AddStateManagerFactory( return services.AddKeyedSingleton, TFactory>(storageName); } - - /// - /// Registers the default keyed state manager factory on the silo builder. - /// - public static ISiloBuilder AddDefaultStateManager( - this ISiloBuilder builder, - string storageName) - { - ArgumentNullException.ThrowIfNull(builder); - ValidateStorageName(storageName); - - builder.ConfigureServices(services => services.AddDefaultStateManager(storageName)); - return builder; - } - - /// - /// Registers a custom keyed open-generic state manager factory - /// on the silo builder. - /// - public static ISiloBuilder AddStateManagerFactory( - this ISiloBuilder builder, - string storageName, - Type openGenericFactoryType) - { - ArgumentNullException.ThrowIfNull(builder); - ValidateStorageName(storageName); - ValidateOpenGenericFactoryType(openGenericFactoryType); - - builder.ConfigureServices(services => - services.AddStateManagerFactory(storageName, openGenericFactoryType)); - return builder; - } - - /// - /// Registers a custom keyed state manager factory for a specific state type - /// on the silo builder. - /// - public static ISiloBuilder AddStateManagerFactory( - this ISiloBuilder builder, - string storageName) - where TState : class, IEquatable - where TFactory : class, IStateManagerFactory - { - ArgumentNullException.ThrowIfNull(builder); - ValidateStorageName(storageName); - - builder.ConfigureServices(services => - services.AddStateManagerFactory(storageName)); - return builder; } - private static void ValidateStorageName(string storageName) + internal static void ValidateStorageName(string storageName) { ArgumentException.ThrowIfNullOrWhiteSpace(storageName); } - private static void ValidateOpenGenericFactoryType(Type openGenericFactoryType) + internal static void ValidateOpenGenericFactoryType(Type openGenericFactoryType) { ArgumentNullException.ThrowIfNull(openGenericFactoryType); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs new file mode 100644 index 00000000..1c9dc4cc --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs @@ -0,0 +1,57 @@ +using Microsoft.Extensions.DependencyInjection; +using Egil.Orleans.Messaging.State; + +namespace Orleans.Hosting; + +/// +/// Registration helpers for keyed +/// services on . +/// +public static class StateManagerSiloBuilderExtensions +{ + extension(ISiloBuilder builder) + { + /// + /// Registers the default keyed state manager factory on the silo builder. + /// + public ISiloBuilder AddDefaultStateManager(string storageName) + { + ArgumentNullException.ThrowIfNull(builder); + StateManagerRegistrationExtensions.ValidateStorageName(storageName); + + builder.ConfigureServices(services => services.AddDefaultStateManager(storageName)); + return builder; + } + + /// + /// Registers a custom keyed open-generic state manager factory on the silo builder. + /// + public ISiloBuilder AddStateManagerFactory( + string storageName, + Type openGenericFactoryType) + { + ArgumentNullException.ThrowIfNull(builder); + StateManagerRegistrationExtensions.ValidateStorageName(storageName); + StateManagerRegistrationExtensions.ValidateOpenGenericFactoryType(openGenericFactoryType); + + builder.ConfigureServices(services => + services.AddStateManagerFactory(storageName, openGenericFactoryType)); + return builder; + } + + /// + /// Registers a custom keyed state manager factory for a specific state type on the silo builder. + /// + public ISiloBuilder AddStateManagerFactory(string storageName) + where TState : class, IEquatable + where TFactory : class, IStateManagerFactory + { + ArgumentNullException.ThrowIfNull(builder); + StateManagerRegistrationExtensions.ValidateStorageName(storageName); + + builder.ConfigureServices(services => + services.AddStateManagerFactory(storageName)); + return builder; + } + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/VersionedState.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/VersionedState.cs similarity index 98% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/VersionedState.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/VersionedState.cs index 2ad14842..3b8b9d93 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/VersionedState.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/VersionedState.cs @@ -1,6 +1,6 @@ using System.Text.Json.Serialization; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.State; /// /// Abstract base for version-stamped grain state. Provides the diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs deleted file mode 100644 index 446f66a9..00000000 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManager.cs +++ /dev/null @@ -1,421 +0,0 @@ -using System.Diagnostics; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using Microsoft.Extensions.Logging.Abstractions; -using Orleans.Streams; - -namespace Egil.Orleans.Messaging; - -/// -/// Grain-level facade around Orleans implicit subscriptions. Manages subscribe, -/// resume, dispatch, and per-subscription error handling. -/// -/// -/// -/// Composition over inheritance: The grain inherits from Grain -/// and uses as a field — no base-class coupling. -/// -/// -/// Typical wiring: -/// -/// public override async Task OnActivateAsync(CancellationToken ct) -/// { -/// streamManager = this.RegisterStreamManager(state.Tracker) -/// .AddSubscription("electricity-prices", HandlePriceTickAsync, LogStreamError) -/// .AddSubscription("tariff-events", HandleTariffChangedAsync); -/// await streamManager.SubscribeAsync(ct); -/// } -/// -/// -/// -/// Resume semantics: each subscription reads -/// trackerSnapshot.LatestStream(streamNamespace) once per subscription. -/// If a cursor exists, the is passed to -/// SubscribeAsync. If null, subscribes without a token -/// (provider default, typically: start from current). The tracker is read -/// once at subscription time, never again — the handler is the only path that -/// mutates the tracker. -/// -/// -/// OTel trace correlation: The OnNext wrapper reads -/// when the stream -/// token is enriched, parses it into an ActivityContext, and starts -/// the handler span with ActivityKind.Consumer and an -/// ActivityLink back to the producer. This produces separate traces -/// per delivery without collapsing weeks of traffic into one distributed trace. -/// -/// -/// Telemetry: Emits counters for messages delivered per -/// (streamNamespace, accepted|rejected), subscriptions -/// established/torn-down/errored, histograms for handler latency, and -/// (when is available) -/// end-to-end lag. -/// -/// -public sealed class StreamManager -{ - private readonly IGrainBase owner; - private readonly MessageTracker trackerSnapshot; - private readonly Func getStreamProvider; - private readonly Func getStreamId; - private readonly ILogger logger; - private readonly List>> subscriptions = []; - private readonly List subscriptionHandles = []; - private bool subscribed; - - private StreamManager( - IGrainBase owner, - MessageTracker trackerSnapshot, - Func getStreamProvider, - Func getStreamId, - ILogger logger) - { - this.owner = owner; - this.trackerSnapshot = trackerSnapshot; - this.getStreamProvider = getStreamProvider; - this.getStreamId = getStreamId; - this.logger = logger; - } - - /// - /// Creates a for the given grain. - /// - /// The grain that owns this stream manager. - /// - /// A snapshot of the grain's at activation - /// time. Used to look up resume tokens. Read once per subscription — not - /// held afterward. - /// - /// A for fluent subscription configuration. - internal static StreamManager Create( - IGrainBase owner, - MessageTracker trackerSnapshot) - { - ArgumentNullException.ThrowIfNull(owner); - ArgumentNullException.ThrowIfNull(trackerSnapshot); - - var services = owner.GrainContext.ActivationServices; - var loggerFactory = services.GetService(); - var logger = loggerFactory?.CreateLogger() - ?? NullLogger.Instance; - - return Create( - owner, - trackerSnapshot, - streamNamespace => services.GetRequiredKeyedService(streamNamespace), - streamNamespace => CreateStreamId(owner, streamNamespace), - logger); - } - - internal static StreamManager Create( - IGrainBase owner, - MessageTracker trackerSnapshot, - Func getStreamProvider, - Func getStreamId, - ILogger logger) - { - ArgumentNullException.ThrowIfNull(owner); - ArgumentNullException.ThrowIfNull(trackerSnapshot); - ArgumentNullException.ThrowIfNull(getStreamProvider); - ArgumentNullException.ThrowIfNull(getStreamId); - ArgumentNullException.ThrowIfNull(logger); - - return new StreamManager(owner, trackerSnapshot, getStreamProvider, getStreamId, logger); - } - - /// - /// Adds a subscription configuration for the given . - /// - /// - /// The expected event type on this stream namespace. Must match the type - /// published by the producer. Usually inferred from - /// ; specify explicitly only for inline - /// lambdas or ambiguous method groups. - /// - /// - /// The Orleans stream namespace to subscribe to. Must follow the - /// one-provider-per-namespace convention (code-review enforced, not - /// runtime enforced). - /// - /// - /// Receives the deserialized event and optional - /// representing this event's position. Some stream providers may deliver a - /// null token. The handler should update the grain's - /// when a token is available and persist state - /// before returning. - /// - /// - /// Optional synchronous error handler. Receives the stream namespace and the - /// exception. If omitted, the default behavior is to log and emit a counter - /// without rethrowing. - /// - /// - /// When true (default), the subscription looks up the last accepted - /// from the snapshot - /// and passes its token to SubscribeAsync for resumption. When - /// false, subscribes from the provider's default position (typically: - /// current/latest). - /// - /// - /// This instance for fluent subscription chaining. - /// - public StreamManager AddSubscription( - string streamNamespace, - Func onNextAsync, - Action? onError = default, - bool passLatestSequenceTokenOnResume = true) - { - ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); - ArgumentNullException.ThrowIfNull(onNextAsync); - if (subscribed) - { - throw new InvalidOperationException("Cannot add stream subscriptions after SubscribeAsync has been called."); - } - - var streamProvider = getStreamProvider(streamNamespace); - var streamId = getStreamId(streamNamespace); - var stream = streamProvider.GetStream(streamId); - var observer = new StreamObserver(this, streamNamespace, onNextAsync, onError); - var latestCursor = passLatestSequenceTokenOnResume - ? trackerSnapshot.LatestStream(streamNamespace) - : null; - - subscriptions.Add(async ct => await SubscribeCoreAsync(streamNamespace, stream, observer, latestCursor?.Token, ct)); - return this; - } - - /// - /// Declares a subscription to the given - /// using a -returning event handler. - /// - /// - /// Convenience overload for handlers that already return . - /// Implementations should adapt this to the overload. - /// - /// - /// The expected event type on this stream namespace. Must match the type - /// published by the producer. Usually inferred from - /// ; specify explicitly only for inline - /// lambdas or ambiguous method groups. - /// - /// - /// The Orleans stream namespace to subscribe to. Must follow the - /// one-provider-per-namespace convention (code-review enforced, not - /// runtime enforced). - /// - /// - /// Receives the deserialized event and optional - /// representing this event's position. Some stream providers may deliver a - /// null token. The handler should update the grain's - /// when a token is available and persist state - /// before returning. - /// - /// - /// Optional synchronous error handler. Receives the stream namespace and the - /// exception. If omitted, the default behavior is to log and emit a counter - /// without rethrowing. - /// - /// - /// When true (default), the subscription looks up the last accepted - /// from the snapshot - /// and passes its token to SubscribeAsync for resumption. When - /// false, subscribes from the provider's default position (typically: - /// current/latest). - /// - /// - /// This instance for fluent subscription chaining. - /// - public StreamManager AddSubscription( - string streamNamespace, - Func onNextAsync, - Action? onError = default, - bool passLatestSequenceTokenOnResume = true) - { - ArgumentNullException.ThrowIfNull(onNextAsync); - - return AddSubscription( - streamNamespace, - (item, token) => new ValueTask(onNextAsync(item, token)), - onError, - passLatestSequenceTokenOnResume); - } - - /// - /// Establishes all configured Orleans stream subscriptions. - /// - /// Cancellation token for activation-time subscription setup. - public async Task SubscribeAsync(CancellationToken cancellationToken = default) - { - if (subscribed) - { - throw new InvalidOperationException("StreamManager subscriptions have already been established."); - } - - cancellationToken.ThrowIfCancellationRequested(); - subscribed = true; - var subscriptionTasks = subscriptions.Select(subscription => subscription(cancellationToken)); - - var handles = await Task.WhenAll(subscriptionTasks); - subscriptionHandles.AddRange(handles); - } - - private async Task> SubscribeCoreAsync( - string streamNamespace, - IAsyncStream stream, - StreamObserver observer, - StreamSequenceToken? token, - CancellationToken cancellationToken) - { - try - { - cancellationToken.ThrowIfCancellationRequested(); - - if (token is null) - { - var handle = await stream.SubscribeAsync(observer); - observer.SetProviderName(handle.ProviderName); - MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); - return handle; - } - else - { - var handle = await stream.SubscribeAsync(observer, token, null!); - observer.SetProviderName(handle.ProviderName); - MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); - return handle; - } - } - catch (Exception ex) - { - MessagingTelemetry.RecordStreamSubscription(streamNamespace, "errored"); - logger.LogError(ex, "Stream subscription failed for namespace {StreamNamespace}.", streamNamespace); - throw; - } - } - - private async Task OnNextAsync( - string streamNamespace, - string? providerName, - TEvent item, - StreamSequenceToken? token, - Func onNextAsync, - Action? onError) - { - var cursor = new StreamCursor(streamNamespace, token, providerName); - var started = Stopwatch.GetTimestamp(); - - using var activity = StartConsumerActivity(streamNamespace, cursor); - try - { - await onNextAsync(item, token); - - MessagingTelemetry.RecordStreamMessage(streamNamespace, "accepted"); - MessagingTelemetry.RecordStreamHandlerDuration(streamNamespace, "accepted", Stopwatch.GetElapsedTime(started).TotalMilliseconds); - } - catch (Exception ex) - { - activity?.SetStatus(ActivityStatusCode.Error, ex.Message); - activity?.AddTag("exception.type", ex.GetType().FullName); - activity?.AddTag("exception.message", ex.Message); - MessagingTelemetry.RecordStreamMessage(streamNamespace, "rejected"); - MessagingTelemetry.RecordStreamHandlerError(streamNamespace); - MessagingTelemetry.RecordStreamHandlerDuration(streamNamespace, "rejected", Stopwatch.GetElapsedTime(started).TotalMilliseconds); - HandleError(streamNamespace, ex, onError); - } - } - - private Task OnErrorAsync(string streamNamespace, Exception ex, Action? onError) - { - MessagingTelemetry.RecordStreamSubscription(streamNamespace, "errored"); - HandleError(streamNamespace, ex, onError); - return Task.CompletedTask; - } - - private void HandleError(string streamNamespace, Exception ex, Action? onError) - { - if (onError is null) - { - logger.LogError(ex, "Stream handler failed for namespace {StreamNamespace}.", streamNamespace); - return; - } - - try - { - onError(streamNamespace, ex); - } - catch (Exception callbackException) - { - logger.LogError( - callbackException, - "Stream error callback failed for namespace {StreamNamespace}. Original error: {OriginalError}", - streamNamespace, - ex.Message); - } - } - - private Activity? StartConsumerActivity(string streamNamespace, StreamCursor cursor) - { - var tags = new KeyValuePair[] - { - new("messaging.system", "orleans"), - new("messaging.operation", "process"), - new("messaging.destination.name", streamNamespace), - new("orleans.grain.id", owner.GrainContext.GrainId.ToString()) - }; - - if (cursor.TryGetTraceParent(out var traceParent) - && ActivityContext.TryParse(traceParent, traceState: null, isRemote: true, out var linkedContext)) - { - return MessagingTelemetry.ActivitySource.StartActivity( - "orleans.stream.process", - ActivityKind.Consumer, - parentContext: default, - tags: tags, - links: [new ActivityLink(linkedContext)]); - } - - return MessagingTelemetry.ActivitySource.StartActivity( - "orleans.stream.process", - ActivityKind.Consumer, - parentContext: default, - tags: tags); - } - - private static StreamId CreateStreamId(IGrainBase owner, string streamNamespace) - { - var grainId = owner.GrainContext.GrainId; - if (GrainIdKeyExtensions.TryGetGuidKey(grainId, out var guidKey, out _)) - { - return StreamId.Create(streamNamespace, guidKey); - } - - if (GrainIdKeyExtensions.TryGetIntegerKey(grainId, out var longKey, out _)) - { - return StreamId.Create(streamNamespace, longKey); - } - - return StreamId.Create(streamNamespace, grainId.Key.ToString()); - } - - private sealed class StreamObserver( - StreamManager manager, - string streamNamespace, - Func onNextAsync, - Action? onError) - : IAsyncObserver - { - private string? providerName; - - public void SetProviderName(string providerName) - { - this.providerName = providerName; - } - - public Task OnNextAsync(TEvent item, StreamSequenceToken? token = null) => - manager.OnNextAsync(streamNamespace, providerName, item, token, onNextAsync, onError); - - public Task OnCompletedAsync() => Task.CompletedTask; - - public Task OnErrorAsync(Exception ex) => manager.OnErrorAsync(streamNamespace, ex, onError); - } -} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs deleted file mode 100644 index 00fae801..00000000 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamManagerExtensions.cs +++ /dev/null @@ -1,50 +0,0 @@ -namespace Egil.Orleans.Messaging; - -/// -/// Extension methods for registering into a grain's -/// activation lifecycle. -/// -public static class StreamManagerExtensions -{ - /// - /// Registers a for the grain and returns it - /// for fluent stream subscription configuration. - /// - /// - /// - /// Call from OnActivateAsync, chain AddSubscription calls - /// on the returned manager, then await SubscribeAsync. - /// - /// - /// Usage: - /// -/// public override async Task OnActivateAsync(CancellationToken ct) -/// { -/// streamManager = this.RegisterStreamManager(state.Tracker) -/// .AddSubscription("electricity-prices", HandlePriceTickAsync); -/// await streamManager.SubscribeAsync(ct); -/// } -/// - /// - /// - /// - /// The grain type. Must implement so stream providers - /// and telemetry infrastructure can be resolved through activation services. - /// - /// The grain instance (this). - /// - /// A snapshot of the grain's at activation - /// time. Used to look up resume tokens. - /// - /// - /// The registered . Store it in a grain field so - /// configured subscriptions remain rooted for the activation lifetime. - /// - public static StreamManager RegisterStreamManager( - this TGrain grain, - MessageTracker trackerSnapshot) - where TGrain : IGrainBase - { - return StreamManager.Create(grain, trackerSnapshot); - } -} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapter.cs similarity index 99% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapter.cs index 1133bb9e..55d4773b 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapter.cs @@ -4,7 +4,7 @@ using Orleans.Streaming.EventHubs; using Orleans.Streams; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Streams.EventHubs; /// /// An subclass that produces diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapterExtensions.cs similarity index 88% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapterExtensions.cs index 2dde19f6..2e283d74 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubAdapterExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapterExtensions.cs @@ -1,7 +1,9 @@ using Microsoft.Extensions.DependencyInjection; +using Egil.Orleans.Messaging.Streams; +using Egil.Orleans.Messaging.Streams.EventHubs; using Orleans.Serialization; -namespace Egil.Orleans.Messaging; +namespace Orleans.Hosting; /// /// Extension methods for registering the -producing @@ -38,6 +40,8 @@ namespace Egil.Orleans.Messaging; /// public static class EnrichedEventHubAdapterExtensions { + extension(IEventHubStreamConfigurator configurator) + { /// /// Registers the library's -producing /// data adapter on the given Event Hub stream configurator. @@ -54,12 +58,7 @@ public static class EnrichedEventHubAdapterExtensions /// name is captured from the configurator's factory delegate. /// /// - /// - /// The Event Hub stream configurator, obtained from - /// ISiloBuilder.AddEventHubStreams. - /// - public static void UseEnrichedDataAdapter( - this IEventHubStreamConfigurator configurator) + public void UseEnrichedDataAdapter() { ArgumentNullException.ThrowIfNull(configurator); @@ -68,4 +67,5 @@ public static void UseEnrichedDataAdapter( providerName, services.GetRequiredService())); } + } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubSequenceToken.cs similarity index 98% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubSequenceToken.cs index 28671740..b8dd7d90 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/EnrichedEventHubSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubSequenceToken.cs @@ -1,6 +1,7 @@ using Orleans.Streaming.EventHubs; +using Egil.Orleans.Messaging.Tracking; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Streams.EventHubs; /// /// An subclass that carries the diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs new file mode 100644 index 00000000..4e002ca0 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs @@ -0,0 +1,19 @@ +using Orleans.Streams.Core; + +namespace Egil.Orleans.Messaging.Streams; + +/// +/// Orleans grain marker interface that forwards implicit stream subscription +/// callbacks to the grain's attached . +/// +public interface IImplicitStreamGrain : IStreamSubscriptionObserver +{ + Task IStreamSubscriptionObserver.OnSubscribed(IStreamSubscriptionHandleFactory handleFactory) + { + var grainBase = (IGrainBase)this; + var component = grainBase.GrainContext.GetComponent(); + return component is null + ? Task.CompletedTask + : component.OnSubscribedAsync(handleFactory); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamManagerComponent.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamManagerComponent.cs new file mode 100644 index 00000000..65b418b6 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamManagerComponent.cs @@ -0,0 +1,8 @@ +using Orleans.Streams.Core; + +namespace Egil.Orleans.Messaging.Streams; + +internal interface IStreamManagerComponent +{ + Task OnSubscribedAsync(IStreamSubscriptionHandleFactory handleFactory); +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs similarity index 98% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs index 8c76b695..ce09b10b 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs @@ -1,7 +1,9 @@ using System.Text.Json.Serialization; +using Egil.Orleans.Messaging.Streams.EventHubs; +using Egil.Orleans.Messaging.Tracking; using Orleans.Streams; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Streams; /// /// Wraps a stream namespace and associated diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs similarity index 98% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs index 772faee7..d219d206 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/StreamCursorJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs @@ -1,10 +1,11 @@ using System.Text.Json; using System.Text.Json.Serialization; +using Egil.Orleans.Messaging.Streams.EventHubs; using Orleans.Providers.Streams.Common; using Orleans.Streaming.EventHubs; using Orleans.Streams; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Streams; /// /// STJ converter for . Serializes and diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs new file mode 100644 index 00000000..a62e6bc2 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs @@ -0,0 +1,495 @@ +using System.Diagnostics; +using Egil.Orleans.Messaging.Tracking; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; +using Orleans.Streams; +using Orleans.Streams.Core; + +namespace Egil.Orleans.Messaging.Streams; + +/// +/// Grain-level facade around Orleans stream handler attachment. Supports +/// Orleans implicit subscriptions and durable explicit subscription handles. +/// +public sealed class StreamManager : IStreamManagerComponent +{ + private readonly IGrainBase owner; + private readonly Func getStreamProvider; + private readonly Func getStreamId; + private readonly ILogger logger; + private readonly Dictionary implicitSubscriptions = new(StringComparer.Ordinal); + private readonly List explicitSubscriptions = []; + private readonly List subscriptionHandles = []; + private bool configurationLocked; + + private StreamManager( + IGrainBase owner, + Func getStreamProvider, + Func getStreamId, + ILogger logger) + { + this.owner = owner; + this.getStreamProvider = getStreamProvider; + this.getStreamId = getStreamId; + this.logger = logger; + } + + /// + /// Creates a for the given grain. + /// + internal static StreamManager Create( + IGrainBase owner, + MessageTracker trackerSnapshot) + => Create(owner, trackerSnapshot, streamProviderName: null); + + internal static StreamManager Create( + IGrainBase owner, + MessageTracker trackerSnapshot, + string? streamProviderName) + { + ArgumentNullException.ThrowIfNull(owner); + ArgumentNullException.ThrowIfNull(trackerSnapshot); + + var services = owner.GrainContext.ActivationServices; + var loggerFactory = services.GetService(); + var logger = loggerFactory?.CreateLogger() + ?? NullLogger.Instance; + + var manager = Create( + owner, + streamNamespace => services.GetRequiredKeyedService(streamProviderName ?? streamNamespace), + streamNamespace => CreateStreamId(owner, streamNamespace), + logger); + + manager.AttachToGrain(); + return manager; + } + + internal static StreamManager Create( + IGrainBase owner, + Func getStreamProvider, + Func getStreamId, + ILogger logger) + { + ArgumentNullException.ThrowIfNull(owner); + ArgumentNullException.ThrowIfNull(getStreamProvider); + ArgumentNullException.ThrowIfNull(getStreamId); + ArgumentNullException.ThrowIfNull(logger); + + return new StreamManager(owner, getStreamProvider, getStreamId, logger); + } + + /// + /// Configures handler attachment for an Orleans implicit stream subscription. + /// + public StreamManager ConfigureImplicitSubscription( + string streamNamespace, + Func onNextAsync, + Action? onError = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); + ArgumentNullException.ThrowIfNull(onNextAsync); + EnsureCanConfigure(); + + if (!implicitSubscriptions.TryAdd( + streamNamespace, + new ImplicitSubscription(streamNamespace, onNextAsync, onError))) + { + throw new InvalidOperationException( + $"An implicit stream subscription for namespace '{streamNamespace}' has already been configured."); + } + + return this; + } + + /// + public StreamManager ConfigureImplicitSubscription( + string streamNamespace, + Func onNextAsync, + Action? onError = default) + { + ArgumentNullException.ThrowIfNull(onNextAsync); + + return ConfigureImplicitSubscription( + streamNamespace, + (item, cursor) => new ValueTask(onNextAsync(item, cursor)), + onError); + } + + /// + /// Configures a durable Orleans explicit stream subscription. + /// + public StreamManager ConfigureExplicitSubscription( + string streamProviderName, + string streamNamespace, + Func onNextAsync, + Action? onError = default) + { + ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); + ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); + ArgumentNullException.ThrowIfNull(onNextAsync); + EnsureCanConfigure(); + + if (explicitSubscriptions.Any(subscription => + subscription.StreamProviderName == streamProviderName + && subscription.StreamNamespace == streamNamespace)) + { + throw new InvalidOperationException( + $"An explicit stream subscription for provider '{streamProviderName}' and namespace '{streamNamespace}' has already been configured."); + } + + explicitSubscriptions.Add(new ExplicitSubscription( + streamProviderName, + streamNamespace, + onNextAsync, + onError)); + + return this; + } + + /// + public StreamManager ConfigureExplicitSubscription( + string streamProviderName, + string streamNamespace, + Func onNextAsync, + Action? onError = default) + { + ArgumentNullException.ThrowIfNull(onNextAsync); + + return ConfigureExplicitSubscription( + streamProviderName, + streamNamespace, + (item, cursor) => new ValueTask(onNextAsync(item, cursor)), + onError); + } + + /// + /// Resumes existing durable explicit subscription handles without creating + /// new subscriptions. + /// + public async Task ResumeExplicitSubscriptionsAsync(CancellationToken cancellationToken = default) + { + configurationLocked = true; + foreach (var subscription in explicitSubscriptions) + { + cancellationToken.ThrowIfCancellationRequested(); + await subscription.ResumeExistingAsync(this, cancellationToken); + } + } + + /// + /// Resumes existing durable explicit handles, or creates exactly one + /// subscription when none exists for a configured explicit stream. + /// + public async Task EnsureExplicitSubscriptionsAsync(CancellationToken cancellationToken = default) + { + configurationLocked = true; + foreach (var subscription in explicitSubscriptions) + { + cancellationToken.ThrowIfCancellationRequested(); + await subscription.EnsureAsync(this, cancellationToken); + } + } + + internal void AttachToGrain() + { + owner.GrainContext.SetComponent(this); + } + + async Task IStreamManagerComponent.OnSubscribedAsync(IStreamSubscriptionHandleFactory handleFactory) + { + ArgumentNullException.ThrowIfNull(handleFactory); + + configurationLocked = true; + var streamNamespace = handleFactory.StreamId.GetNamespace(); + if (streamNamespace is null + || !implicitSubscriptions.TryGetValue(streamNamespace, out var subscription)) + { + logger.LogWarning( + "No implicit stream subscription handler is configured for namespace {StreamNamespace}.", + streamNamespace); + return; + } + + await subscription.ResumeAsync(this, handleFactory); + } + + private async Task> ResumeImplicitAsync( + string streamNamespace, + Func onNextAsync, + Action? onError, + IStreamSubscriptionHandleFactory handleFactory) + { + try + { + var observer = new StreamObserver( + this, + streamNamespace, + handleFactory.ProviderName, + onNextAsync, + onError); + var handle = await handleFactory.Create().ResumeAsync(observer); + + subscriptionHandles.Add(handle); + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); + return handle; + } + catch (Exception ex) + { + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "errored"); + logger.LogError(ex, "Implicit stream subscription attach failed for namespace {StreamNamespace}.", streamNamespace); + throw; + } + } + + private async Task ResumeExplicitAsync( + string streamProviderName, + string streamNamespace, + Func onNextAsync, + Action? onError, + bool createIfMissing, + CancellationToken cancellationToken) + { + try + { + cancellationToken.ThrowIfCancellationRequested(); + + var stream = GetExplicitStream(streamProviderName, streamNamespace); + var handles = await stream.GetAllSubscriptionHandles(); + if (handles.Count == 0 && createIfMissing) + { + var observer = new StreamObserver( + this, + streamNamespace, + streamProviderName, + onNextAsync, + onError); + var handle = await stream.SubscribeAsync(observer); + + subscriptionHandles.Add(handle); + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); + return; + } + + foreach (var handle in handles) + { + cancellationToken.ThrowIfCancellationRequested(); + + var observer = new StreamObserver( + this, + streamNamespace, + handle.ProviderName, + onNextAsync, + onError); + var resumedHandle = await handle.ResumeAsync(observer); + + subscriptionHandles.Add(resumedHandle); + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); + } + } + catch (Exception ex) + { + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "errored"); + logger.LogError( + ex, + "Explicit stream subscription resume failed for provider {StreamProviderName} and namespace {StreamNamespace}.", + streamProviderName, + streamNamespace); + throw; + } + } + + private IAsyncStream GetExplicitStream( + string streamProviderName, + string streamNamespace) + { + var streamProvider = getStreamProvider(streamProviderName); + var streamId = getStreamId(streamNamespace); + return streamProvider.GetStream(streamId); + } + + private async Task OnNextAsync( + string streamNamespace, + string? providerName, + TEvent item, + StreamSequenceToken? token, + Func onNextAsync, + Action? onError) + { + var cursor = new StreamCursor(streamNamespace, token, providerName); + var started = Stopwatch.GetTimestamp(); + + using var activity = StartConsumerActivity(streamNamespace, cursor); + try + { + await onNextAsync(item, cursor); + + MessagingTelemetry.RecordStreamMessage(streamNamespace, "accepted"); + MessagingTelemetry.RecordStreamHandlerDuration(streamNamespace, "accepted", Stopwatch.GetElapsedTime(started).TotalMilliseconds); + } + catch (Exception ex) + { + activity?.SetStatus(ActivityStatusCode.Error, ex.Message); + activity?.AddTag("exception.type", ex.GetType().FullName); + activity?.AddTag("exception.message", ex.Message); + MessagingTelemetry.RecordStreamMessage(streamNamespace, "rejected"); + MessagingTelemetry.RecordStreamHandlerError(streamNamespace); + MessagingTelemetry.RecordStreamHandlerDuration(streamNamespace, "rejected", Stopwatch.GetElapsedTime(started).TotalMilliseconds); + HandleError(streamNamespace, ex, onError); + } + } + + private Task OnErrorAsync(string streamNamespace, Exception ex, Action? onError) + { + MessagingTelemetry.RecordStreamSubscription(streamNamespace, "errored"); + HandleError(streamNamespace, ex, onError); + return Task.CompletedTask; + } + + private void HandleError(string streamNamespace, Exception ex, Action? onError) + { + if (onError is null) + { + logger.LogError(ex, "Stream handler failed for namespace {StreamNamespace}.", streamNamespace); + return; + } + + try + { + onError(streamNamespace, ex); + } + catch (Exception callbackException) + { + logger.LogError( + callbackException, + "Stream error callback failed for namespace {StreamNamespace}. Original error: {OriginalError}", + streamNamespace, + ex.Message); + } + } + + private Activity? StartConsumerActivity(string streamNamespace, StreamCursor cursor) + { + var tags = new KeyValuePair[] + { + new("messaging.system", "orleans"), + new("messaging.operation", "process"), + new("messaging.destination.name", streamNamespace), + new("orleans.grain.id", owner.GrainContext.GrainId.ToString()) + }; + + if (cursor.TryGetTraceParent(out var traceParent) + && ActivityContext.TryParse(traceParent, traceState: null, isRemote: true, out var linkedContext)) + { + return MessagingTelemetry.ActivitySource.StartActivity( + "orleans.stream.process", + ActivityKind.Consumer, + parentContext: default, + tags: tags, + links: [new ActivityLink(linkedContext)]); + } + + return MessagingTelemetry.ActivitySource.StartActivity( + "orleans.stream.process", + ActivityKind.Consumer, + parentContext: default, + tags: tags); + } + + private void EnsureCanConfigure() + { + if (configurationLocked) + { + throw new InvalidOperationException("Cannot configure stream subscriptions after handler attachment or explicit resume has started."); + } + } + + private static StreamId CreateStreamId(IGrainBase owner, string streamNamespace) + { + var grainId = owner.GrainContext.GrainId; + if (GrainIdKeyExtensions.TryGetGuidKey(grainId, out var guidKey, out _)) + { + return StreamId.Create(streamNamespace, guidKey); + } + + if (GrainIdKeyExtensions.TryGetIntegerKey(grainId, out var longKey, out _)) + { + return StreamId.Create(streamNamespace, longKey); + } + + return StreamId.Create(streamNamespace, grainId.Key.ToString()); + } + + private interface IImplicitSubscription + { + Task ResumeAsync(StreamManager manager, IStreamSubscriptionHandleFactory handleFactory); + } + + private interface IExplicitSubscription + { + string StreamProviderName { get; } + + string StreamNamespace { get; } + + Task ResumeExistingAsync(StreamManager manager, CancellationToken cancellationToken); + + Task EnsureAsync(StreamManager manager, CancellationToken cancellationToken); + } + + private sealed class ImplicitSubscription( + string streamNamespace, + Func onNextAsync, + Action? onError) + : IImplicitSubscription + { + public Task ResumeAsync(StreamManager manager, IStreamSubscriptionHandleFactory handleFactory) => + manager.ResumeImplicitAsync(streamNamespace, onNextAsync, onError, handleFactory); + } + + private sealed class ExplicitSubscription( + string streamProviderName, + string streamNamespace, + Func onNextAsync, + Action? onError) + : IExplicitSubscription + { + public string StreamProviderName => streamProviderName; + + public string StreamNamespace => streamNamespace; + + public Task ResumeExistingAsync(StreamManager manager, CancellationToken cancellationToken) => + manager.ResumeExplicitAsync( + streamProviderName, + streamNamespace, + onNextAsync, + onError, + createIfMissing: false, + cancellationToken); + + public Task EnsureAsync(StreamManager manager, CancellationToken cancellationToken) => + manager.ResumeExplicitAsync( + streamProviderName, + streamNamespace, + onNextAsync, + onError, + createIfMissing: true, + cancellationToken); + } + + private sealed class StreamObserver( + StreamManager manager, + string streamNamespace, + string? providerName, + Func onNextAsync, + Action? onError) + : IAsyncObserver + { + public Task OnNextAsync(TEvent item, StreamSequenceToken? token = null) => + manager.OnNextAsync(streamNamespace, providerName, item, token, onNextAsync, onError); + + public Task OnCompletedAsync() => Task.CompletedTask; + + public Task OnErrorAsync(Exception ex) => manager.OnErrorAsync(streamNamespace, ex, onError); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs new file mode 100644 index 00000000..cee9f2fc --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs @@ -0,0 +1,72 @@ +using Egil.Orleans.Messaging.Streams; +using Egil.Orleans.Messaging.Tracking; + +namespace Orleans; + +/// +/// Extension methods for registering into a grain's +/// activation lifecycle. +/// +public static class StreamManagerExtensions +{ + extension(TGrain grain) + where TGrain : IGrainBase + { + /// + /// Registers a for the grain and returns it + /// for fluent stream subscription configuration. + /// + /// + /// + /// Call from OnActivateAsync, then chain + /// ConfigureImplicitSubscription or + /// ConfigureExplicitSubscription calls on the returned manager. + /// + /// + /// Usage: + /// +/// public override async Task OnActivateAsync(CancellationToken ct) +/// { +/// streamManager = this.RegisterStreamManager(state.Tracker) +/// .ConfigureExplicitSubscription("StreamProvider", "electricity-prices", HandlePriceTickAsync); +/// await streamManager.EnsureExplicitSubscriptionsAsync(ct); +/// } +/// + /// + /// + /// + /// A snapshot of the grain's at activation + /// time. Used to look up resume tokens. + /// + /// + /// The registered . Store it in a grain field so + /// configured subscriptions remain rooted for the activation lifetime. + /// + public StreamManager RegisterStreamManager(MessageTracker trackerSnapshot) + { + return StreamManager.Create(grain, trackerSnapshot); + } + + /// + /// Registers a for a grain whose configured + /// subscriptions all use the same Orleans stream provider. + /// + /// + /// Prefer this overload when provider name and stream namespace are + /// different. This is the usual shape for grains using + /// , because the attribute + /// describes namespaces while the provider name is runtime configuration. + /// + /// A snapshot of the grain's tracker at activation time. + /// The Orleans stream provider name. + /// The registered . + public StreamManager RegisterStreamManager( + MessageTracker trackerSnapshot, + string streamProviderName) + { + ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); + + return StreamManager.Create(grain, trackerSnapshot, streamProviderName); + } + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs similarity index 99% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs index 74652c6c..4fa9e1a5 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTracker.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs @@ -1,8 +1,10 @@ using System.Collections.Immutable; using System.Text.Json.Serialization; +using Egil.Orleans.Messaging.Outboxes; +using Egil.Orleans.Messaging.Streams; using Orleans.Streams; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Tracking; /// /// Receiver-side, persisted dedup state. Tracks the high-water position from diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs similarity index 98% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs index 35806551..0f4cd644 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessageTrackerJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs @@ -2,11 +2,13 @@ using System.Reflection; using System.Text.Json; using System.Text.Json.Serialization; +using Egil.Orleans.Messaging.Streams; +using Egil.Orleans.Messaging.Streams.EventHubs; using Orleans.Providers.Streams.Common; using Orleans.Streaming.EventHubs; using Orleans.Streams; -namespace Egil.Orleans.Messaging; +namespace Egil.Orleans.Messaging.Tracking; /// /// STJ converter for . Serializes and diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs new file mode 100644 index 00000000..42ca71ec --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs @@ -0,0 +1,8 @@ +global using Egil.Orleans.Messaging.Outboxes; +global using Egil.Orleans.Messaging.State; +global using Egil.Orleans.Messaging.Streams; +global using Egil.Orleans.Messaging.Streams.EventHubs; +global using Egil.Orleans.Messaging.Tracking; +global using Microsoft.Extensions.DependencyInjection; +global using Orleans; +global using Orleans.Hosting; diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs index 3152939d..8bd8d9ea 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs @@ -1,5 +1,7 @@ using System.Runtime.CompilerServices; using Egil.Orleans.Testing; +using Egil.Orleans.Messaging.Tests.Outboxes; +using Egil.Orleans.Messaging.Tests.Streams; using Orleans.Streams; using Orleans.TestingHost; @@ -55,6 +57,12 @@ public IAsyncStream GetStream(string streamNamespace, Guid key) return provider.GetStream(StreamId.Create(streamNamespace, key)); } + public IAsyncStream GetStream(string providerName, string streamNamespace, Guid key) + { + var provider = Cluster.Client.GetStreamProvider(providerName); + return provider.GetStream(StreamId.Create(streamNamespace, key)); + } + Task IGrainActivityWaiter.WaitForAssertionAsync( Func> assertion, Predicate? filter, @@ -68,6 +76,7 @@ private static void AddStreamProviders(ISiloBuilder siloBuilder) siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Task); siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Failure); siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Resume); + siloBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Implicit); siloBuilder.AddMemoryStreams(OutboxProcessorTestNamespaces.Events); } @@ -77,6 +86,7 @@ private static void AddStreamProviders(IClientBuilder clientBuilder) clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Task); clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Failure); clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Resume); + clientBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Implicit); clientBuilder.AddMemoryStreams(OutboxProcessorTestNamespaces.Events); } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs similarity index 99% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs index 21d5d344..ac334c21 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxJsonConverterFactoryTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs @@ -1,7 +1,7 @@ using System.Text.Json; using TimeProviderExtensions; -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Outboxes; public sealed class OutboxJsonConverterFactoryTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxMessageEnvelopeJsonConverterFactoryTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxMessageEnvelopeJsonConverterFactoryTests.cs similarity index 97% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxMessageEnvelopeJsonConverterFactoryTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxMessageEnvelopeJsonConverterFactoryTests.cs index c3a79c69..2d8d211f 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxMessageEnvelopeJsonConverterFactoryTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxMessageEnvelopeJsonConverterFactoryTests.cs @@ -1,7 +1,7 @@ using System.Text.Json; using System.Text.Json.Serialization; -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Outboxes; public sealed class OutboxMessageEnvelopeJsonConverterFactoryTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs similarity index 97% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxProcessorTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index 31296747..72e68171 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -1,8 +1,7 @@ using System.Collections.Immutable; using Egil.Orleans.Testing; -using Orleans.Streams; -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Outboxes; public sealed class OutboxProcessorTests(MessagingTestClusterFixture fixture) : IClassFixture { @@ -350,11 +349,12 @@ public sealed class OutboxProcessorSinkGrain( public override async Task OnActivateAsync(CancellationToken cancellationToken) { streamManager = this.RegisterStreamManager(state.State.Tracker) - .AddSubscription>( + .ConfigureExplicitSubscription>( + OutboxProcessorTestNamespaces.Events, OutboxProcessorTestNamespaces.Events, HandleEnvelopeAsync); - await streamManager.SubscribeAsync(cancellationToken); + await streamManager.EnsureExplicitSubscriptionsAsync(cancellationToken); await base.OnActivateAsync(cancellationToken); } @@ -364,7 +364,7 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) private async ValueTask HandleEnvelopeAsync( OutboxMessageEnvelope envelope, - StreamSequenceToken? token) + StreamCursor cursor) { if (!state.State.Tracker.ProcessMessage(envelope.Token, out var next)) { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxSequenceTokenJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs similarity index 98% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxSequenceTokenJsonConverterTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs index 50f11524..709dfa24 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxSequenceTokenJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs @@ -1,7 +1,7 @@ using System.Text.Json; using System.Text.Json.Serialization; -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Outboxes; public sealed class OutboxSequenceTokenJsonConverterTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs similarity index 99% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs index 0584cf7c..2eee4c6d 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/OutboxTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs @@ -1,6 +1,6 @@ using TimeProviderExtensions; -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Outboxes; public sealed class OutboxTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerExtensionsTests.cs similarity index 97% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerExtensionsTests.cs index d4f4038f..f0ea11c2 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerExtensionsTests.cs @@ -1,6 +1,4 @@ -using Microsoft.Extensions.DependencyInjection; - -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.State; public sealed class StateManagerExtensionsTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerRegistrationExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs similarity index 98% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerRegistrationExtensionsTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs index 2aacde3a..aa539bba 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerRegistrationExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs @@ -1,6 +1,4 @@ -using Microsoft.Extensions.DependencyInjection; - -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.State; public sealed class StateManagerRegistrationExtensionsTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs similarity index 99% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs index fb35c72c..15bdab2f 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StateManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs @@ -1,4 +1,4 @@ -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.State; public sealed class StateManagerTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs deleted file mode 100644 index 038828d9..00000000 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamManagerTests.cs +++ /dev/null @@ -1,248 +0,0 @@ -using Egil.Orleans.Testing; -using Orleans.Providers.Streams.Common; -using Orleans.Streams; - -namespace Egil.Orleans.Messaging.Tests; - -public sealed class StreamManagerTests(MessagingTestClusterFixture fixture) : IClassFixture -{ - [Fact] - public async Task AddSubscription_with_ValueTask_handler_delivers_stream_messages() - { - var grainKey = Guid.NewGuid(); - var grain = fixture.GrainFactory.GetGrain(grainKey); - await grain.EnsureActiveAsync(); - var stream = fixture.GetStream(StreamManagerTestNamespaces.ValueTask, grainKey); - - await stream.OnNextAsync("value-task"); - - await fixture.WaitForAssertionAsync( - async () => Assert.Equal("value-task", (await grain.GetStateAsync()).ValueTaskValue), - ct: TestContext.Current.CancellationToken); - } - - [Fact] - public async Task AddSubscription_with_Task_handler_delivers_stream_messages() - { - var grainKey = Guid.NewGuid(); - var grain = fixture.GrainFactory.GetGrain(grainKey); - await grain.EnsureActiveAsync(); - var stream = fixture.GetStream(StreamManagerTestNamespaces.Task, grainKey); - - await stream.OnNextAsync("task"); - - await fixture.WaitForAssertionAsync( - async () => Assert.Equal("task", (await grain.GetStateAsync()).TaskValue), - ct: TestContext.Current.CancellationToken); - } - - [Fact] - public async Task Default_error_handler_does_not_break_subscription_when_handler_throws() - { - var grainKey = Guid.NewGuid(); - var grain = fixture.GrainFactory.GetGrain(grainKey); - await grain.EnsureActiveAsync(); - var stream = fixture.GetStream(StreamManagerTestNamespaces.Failure, grainKey); - - await stream.OnNextAsync("fail"); - await stream.OnNextAsync("after-failure"); - - await fixture.WaitForAssertionAsync( - async () => Assert.Equal("after-failure", (await grain.GetStateAsync()).FailureValue), - ct: TestContext.Current.CancellationToken); - } - - [Fact] - public async Task SubscribeAsync_uses_tracker_snapshot_when_resume_token_is_configured() - { - var grainKey = Guid.NewGuid(); - var grain = fixture.GrainFactory.GetGrain(grainKey); - await grain.SeedResumeCursorAsync(7); - await grain.DeactivateAsync(); - - grain = fixture.GrainFactory.GetGrain(grainKey); - await grain.EnsureActiveAsync(); - var stream = fixture.GetStream(StreamManagerTestNamespaces.Resume, grainKey); - - await stream.OnNextAsync("resumed"); - - await fixture.WaitForAssertionAsync( - async () => Assert.Equal("resumed", (await grain.GetStateAsync()).ResumeValue), - ct: TestContext.Current.CancellationToken); - } - - [Fact] - public async Task SubscribeAsync_rejects_duplicate_subscription_initialization() - { - var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); - await grain.EnsureActiveAsync(); - - var exceptionType = await grain.SubscribeAgainAsync(); - - Assert.Equal(nameof(InvalidOperationException), exceptionType); - } - - [Fact] - public async Task AddSubscription_rejects_configuration_after_subscribe() - { - var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); - await grain.EnsureActiveAsync(); - - var exceptionType = await grain.AddSubscriptionAfterSubscribeAsync(); - - Assert.Equal(nameof(InvalidOperationException), exceptionType); - } -} - -internal static class StreamManagerTestNamespaces -{ - public const string ValueTask = "stream-manager-valuetask"; - public const string Task = "stream-manager-task"; - public const string Failure = "stream-manager-failure"; - public const string Resume = "stream-manager-resume"; -} - -public interface IStreamManagerTestGrain : IGrainWithGuidKey -{ - Task EnsureActiveAsync(); - - Task SeedResumeCursorAsync(long sequenceNumber); - - Task DeactivateAsync(); - - Task SubscribeAgainAsync(); - - Task AddSubscriptionAfterSubscribeAsync(); - - Task GetStateAsync(); -} - -[GenerateSerializer] -public sealed class StreamManagerTestState -{ - [Id(0)] - public MessageTracker Tracker { get; set; } = new(); - - [Id(1)] - public string? ValueTaskValue { get; set; } - - [Id(2)] - public string? TaskValue { get; set; } - - [Id(3)] - public string? FailureValue { get; set; } - - [Id(4)] - public string? ResumeValue { get; set; } -} - -public sealed class StreamManagerTestGrain( - [PersistentState("state", "Default")] IPersistentState state) - : Grain, IStreamManagerTestGrain -{ - private StreamManager? streamManager; - - public override async Task OnActivateAsync(CancellationToken cancellationToken) - { - streamManager = this.RegisterStreamManager(state.State.Tracker) - .AddSubscription(StreamManagerTestNamespaces.ValueTask, HandleValueTaskAsync) - .AddSubscription(StreamManagerTestNamespaces.Task, HandleTaskAsync) - .AddSubscription(StreamManagerTestNamespaces.Failure, HandleFailureAsync) - .AddSubscription(StreamManagerTestNamespaces.Resume, HandleResumeAsync); - - await streamManager.SubscribeAsync(cancellationToken); - await base.OnActivateAsync(cancellationToken); - } - - public Task EnsureActiveAsync() => Task.CompletedTask; - - public async Task SeedResumeCursorAsync(long sequenceNumber) - { - var cursor = new StreamCursor( - StreamId.Create(StreamManagerTestNamespaces.Resume, this.GetPrimaryKey()), - new EventSequenceToken(sequenceNumber)); - - state.State.Tracker.ProcessMessage(cursor, out var next); - state.State.Tracker = next; - await state.WriteStateAsync(); - } - - public Task DeactivateAsync() - { - DeactivateOnIdle(); - return Task.CompletedTask; - } - - public Task GetStateAsync() => Task.FromResult(state.State); - - public async Task SubscribeAgainAsync() - { - try - { - await streamManager!.SubscribeAsync(); - return null; - } - catch (Exception ex) - { - return ex.GetType().Name; - } - } - - public Task AddSubscriptionAfterSubscribeAsync() - { - try - { - streamManager!.AddSubscription( - "not-registered", - static (_, _) => ValueTask.CompletedTask); - return Task.FromResult(null); - } - catch (Exception ex) - { - return Task.FromResult(ex.GetType().Name); - } - } - - private async ValueTask HandleValueTaskAsync(string item, StreamSequenceToken? token) - { - TrackStreamCursor(StreamManagerTestNamespaces.ValueTask, token); - state.State.ValueTaskValue = item; - await state.WriteStateAsync(); - } - - private async Task HandleTaskAsync(string item, StreamSequenceToken? token) - { - TrackStreamCursor(StreamManagerTestNamespaces.Task, token); - state.State.TaskValue = item; - await state.WriteStateAsync(); - } - - private async ValueTask HandleFailureAsync(string item, StreamSequenceToken? token) - { - if (item == "fail") - { - throw new InvalidOperationException("Handler failed."); - } - - TrackStreamCursor(StreamManagerTestNamespaces.Failure, token); - state.State.FailureValue = item; - await state.WriteStateAsync(); - } - - private async ValueTask HandleResumeAsync(string item, StreamSequenceToken? token) - { - TrackStreamCursor(StreamManagerTestNamespaces.Resume, token); - state.State.ResumeValue = item; - await state.WriteStateAsync(); - } - - private void TrackStreamCursor(string streamNamespace, StreamSequenceToken? token) - { - var cursor = new StreamCursor(StreamId.Create(streamNamespace, this.GetPrimaryKey()), token); - if (state.State.Tracker.ProcessMessage(cursor, out var next)) - { - state.State.Tracker = next; - } - - } -} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs similarity index 92% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterExtensionsTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs index 68d3f3d1..4850061a 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs @@ -1,6 +1,4 @@ -using Microsoft.Extensions.DependencyInjection; - -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Streams.EventHubs; public sealed class EnrichedEventHubAdapterExtensionsTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterTests.cs similarity index 98% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterTests.cs index 2cce475e..a706fbc8 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/EnrichedEventHubAdapterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterTests.cs @@ -1,11 +1,10 @@ using System.Diagnostics; using Azure.Messaging.EventHubs; -using Microsoft.Extensions.DependencyInjection; using Orleans.Providers.Streams.Common; using Orleans.Serialization; using Orleans.Streaming.EventHubs; -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Streams.EventHubs; public sealed class EnrichedEventHubAdapterTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs similarity index 99% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs index 40187005..e5a35380 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs @@ -4,7 +4,7 @@ using Orleans.Streaming.EventHubs; using Orleans.Streams; -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Streams; public sealed class StreamCursorJsonConverterTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs similarity index 98% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs index 27bed0f9..d272f716 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/StreamCursorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs @@ -1,6 +1,6 @@ using Orleans.Providers.Streams.Common; -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Streams; public sealed class StreamCursorTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs new file mode 100644 index 00000000..1384a7de --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs @@ -0,0 +1,366 @@ +using Egil.Orleans.Testing; + +namespace Egil.Orleans.Messaging.Tests.Streams; + +public sealed class StreamManagerTests(MessagingTestClusterFixture fixture) : IClassFixture +{ + [Fact] + public async Task ConfigureExplicitSubscription_with_ValueTask_handler_delivers_stream_messages() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.EnsureActiveAsync(); + var stream = fixture.GetStream(StreamManagerTestNamespaces.ValueTask, grainKey); + + await stream.OnNextAsync("value-task"); + + await fixture.WaitForAssertionAsync( + async () => Assert.Equal("value-task", (await grain.GetStateAsync()).ValueTaskValue), + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task Implicit_subscription_activates_grain_and_delivers_stream_message() + { + var grainKey = Guid.NewGuid(); + var stream = fixture.GetStream( + StreamManagerTestProviderNames.Implicit, + StreamManagerTestNamespaces.Implicit, + grainKey); + + await stream.OnNextAsync("implicit"); + + var grain = fixture.GrainFactory.GetGrain(grainKey); + await fixture.WaitForAssertionAsync( + async () => Assert.Equal("implicit", await grain.GetValueAsync()), + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task Implicit_subscription_does_not_duplicate_after_reactivation() + { + var grainKey = Guid.NewGuid(); + var stream = fixture.GetStream( + StreamManagerTestProviderNames.Implicit, + StreamManagerTestNamespaces.Implicit, + grainKey); + + await stream.OnNextAsync("first"); + + var grain = fixture.GrainFactory.GetGrain(grainKey); + await fixture.WaitForAssertionAsync( + async () => Assert.Equal(1, await grain.GetDeliveryCountAsync()), + ct: TestContext.Current.CancellationToken); + await grain.DeactivateAsync(); + + await stream.OnNextAsync("second"); + + await fixture.WaitForAssertionAsync( + async () => Assert.Equal(2, await grain.GetDeliveryCountAsync()), + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task Implicit_subscription_has_no_explicit_subscription_handles() + { + var grainKey = Guid.NewGuid(); + var stream = fixture.GetStream( + StreamManagerTestProviderNames.Implicit, + StreamManagerTestNamespaces.Implicit, + grainKey); + + await stream.OnNextAsync("implicit"); + + var grain = fixture.GrainFactory.GetGrain(grainKey); + await fixture.WaitForAssertionAsync( + async () => Assert.Equal("implicit", await grain.GetValueAsync()), + ct: TestContext.Current.CancellationToken); + + Assert.Empty(await stream.GetAllSubscriptionHandles()); + } + + [Fact] + public async Task Explicit_subscription_does_not_duplicate_after_reactivation() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.EnsureActiveAsync(); + await grain.DeactivateAsync(); + + grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.EnsureActiveAsync(); + var stream = fixture.GetStream(StreamManagerTestNamespaces.ValueTask, grainKey); + + await stream.OnNextAsync("after-reactivation"); + + await fixture.WaitForAssertionAsync( + async () => Assert.Equal(1, (await grain.GetStateAsync()).ValueTaskDeliveryCount), + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task ConfigureExplicitSubscription_with_Task_handler_delivers_stream_messages() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.EnsureActiveAsync(); + var stream = fixture.GetStream(StreamManagerTestNamespaces.Task, grainKey); + + await stream.OnNextAsync("task"); + + await fixture.WaitForAssertionAsync( + async () => Assert.Equal("task", (await grain.GetStateAsync()).TaskValue), + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task Default_error_handler_does_not_break_subscription_when_handler_throws() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.EnsureActiveAsync(); + var stream = fixture.GetStream(StreamManagerTestNamespaces.Failure, grainKey); + + await stream.OnNextAsync("fail"); + await stream.OnNextAsync("after-failure"); + + await fixture.WaitForAssertionAsync( + async () => Assert.Equal("after-failure", (await grain.GetStateAsync()).FailureValue), + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task EnsureExplicitSubscriptionsAsync_is_idempotent() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + await grain.EnsureActiveAsync(); + var stream = fixture.GetStream(StreamManagerTestNamespaces.ValueTask, grainKey); + + var exceptionType = await grain.EnsureExplicitSubscriptionsAgainAsync(); + + Assert.Null(exceptionType); + + await stream.OnNextAsync("idempotent"); + + await fixture.WaitForAssertionAsync( + async () => Assert.Equal(1, (await grain.GetStateAsync()).ValueTaskDeliveryCount), + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task ConfigureExplicitSubscription_rejects_configuration_after_resume() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + await grain.EnsureActiveAsync(); + + var exceptionType = await grain.ConfigureExplicitSubscriptionAfterResumeAsync(); + + Assert.Equal(nameof(InvalidOperationException), exceptionType); + } +} + +internal static class StreamManagerTestNamespaces +{ + public const string ValueTask = "stream-manager-valuetask"; + public const string Task = "stream-manager-task"; + public const string Failure = "stream-manager-failure"; + public const string Resume = "stream-manager-resume"; + public const string Implicit = "stream-manager-implicit"; +} + +internal static class StreamManagerTestProviderNames +{ + public const string Implicit = "stream-manager-provider"; +} + +public interface IStreamManagerTestGrain : IGrainWithGuidKey +{ + Task EnsureActiveAsync(); + + Task DeactivateAsync(); + + Task EnsureExplicitSubscriptionsAgainAsync(); + + Task ConfigureExplicitSubscriptionAfterResumeAsync(); + + Task GetStateAsync(); +} + +[GenerateSerializer] +public sealed class StreamManagerTestState +{ + [Id(0)] + public MessageTracker Tracker { get; set; } = new(); + + [Id(1)] + public string? ValueTaskValue { get; set; } + + [Id(2)] + public string? TaskValue { get; set; } + + [Id(3)] + public string? FailureValue { get; set; } + + [Id(4)] + public string? ResumeValue { get; set; } + + [Id(5)] + public int ValueTaskDeliveryCount { get; set; } +} + +public sealed class StreamManagerTestGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IStreamManagerTestGrain +{ + private StreamManager? streamManager; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + streamManager = this.RegisterStreamManager(state.State.Tracker) + .ConfigureExplicitSubscription(StreamManagerTestNamespaces.ValueTask, StreamManagerTestNamespaces.ValueTask, HandleValueTaskAsync) + .ConfigureExplicitSubscription(StreamManagerTestNamespaces.Task, StreamManagerTestNamespaces.Task, HandleTaskAsync) + .ConfigureExplicitSubscription(StreamManagerTestNamespaces.Failure, StreamManagerTestNamespaces.Failure, HandleFailureAsync) + .ConfigureExplicitSubscription(StreamManagerTestNamespaces.Resume, StreamManagerTestNamespaces.Resume, HandleResumeAsync); + + await streamManager.EnsureExplicitSubscriptionsAsync(cancellationToken); + await base.OnActivateAsync(cancellationToken); + } + + public Task EnsureActiveAsync() => Task.CompletedTask; + + public Task DeactivateAsync() + { + DeactivateOnIdle(); + return Task.CompletedTask; + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + public async Task EnsureExplicitSubscriptionsAgainAsync() + { + try + { + await streamManager!.EnsureExplicitSubscriptionsAsync(); + return null; + } + catch (Exception ex) + { + return ex.GetType().Name; + } + } + + public Task ConfigureExplicitSubscriptionAfterResumeAsync() + { + try + { + streamManager!.ConfigureExplicitSubscription( + "not-registered", + "not-registered", + static (_, _) => ValueTask.CompletedTask); + return Task.FromResult(null); + } + catch (Exception ex) + { + return Task.FromResult(ex.GetType().Name); + } + } + + private async ValueTask HandleValueTaskAsync(string item, StreamCursor cursor) + { + TrackStreamCursor(cursor); + state.State.ValueTaskValue = item; + state.State.ValueTaskDeliveryCount++; + await state.WriteStateAsync(); + } + + private async Task HandleTaskAsync(string item, StreamCursor cursor) + { + TrackStreamCursor(cursor); + state.State.TaskValue = item; + await state.WriteStateAsync(); + } + + private async ValueTask HandleFailureAsync(string item, StreamCursor cursor) + { + if (item == "fail") + { + throw new InvalidOperationException("Handler failed."); + } + + TrackStreamCursor(cursor); + state.State.FailureValue = item; + await state.WriteStateAsync(); + } + + private async ValueTask HandleResumeAsync(string item, StreamCursor cursor) + { + TrackStreamCursor(cursor); + state.State.ResumeValue = item; + await state.WriteStateAsync(); + } + + private void TrackStreamCursor(StreamCursor cursor) + { + if (state.State.Tracker.ProcessMessage(cursor, out var next)) + { + state.State.Tracker = next; + } + + } +} + +public interface IImplicitStreamManagerTestGrain : IGrainWithGuidKey +{ + Task GetValueAsync(); + + Task GetDeliveryCountAsync(); + + Task DeactivateAsync(); +} + +[ImplicitStreamSubscription(StreamManagerTestNamespaces.Implicit)] +public sealed class ImplicitStreamManagerTestGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IImplicitStreamManagerTestGrain, IImplicitStreamGrain +{ + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + this.RegisterStreamManager(state.State.Tracker) + .ConfigureImplicitSubscription(StreamManagerTestNamespaces.Implicit, HandleImplicitAsync); + + await base.OnActivateAsync(cancellationToken); + } + + public Task GetValueAsync() => Task.FromResult(state.State.Value); + + public Task GetDeliveryCountAsync() => Task.FromResult(state.State.DeliveryCount); + + public Task DeactivateAsync() + { + DeactivateOnIdle(); + return Task.CompletedTask; + } + + private async ValueTask HandleImplicitAsync(string item, StreamCursor cursor) + { + state.State.Tracker.ProcessMessage(cursor, out var tracker); + state.State.Tracker = tracker; + state.State.Value = item; + state.State.DeliveryCount++; + await state.WriteStateAsync(); + } +} + +[GenerateSerializer] +public sealed class ImplicitStreamManagerTestState +{ + [Id(0)] + public MessageTracker Tracker { get; set; } = new(); + + [Id(1)] + public string? Value { get; set; } + + [Id(2)] + public int DeliveryCount { get; set; } +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs similarity index 99% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs index 64ab0397..6fbd1e2c 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs @@ -4,7 +4,7 @@ using Orleans.Streams; using TimeProviderExtensions; -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Tracking; public sealed class MessageTrackerJsonConverterTests { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs similarity index 99% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs index c96f04be..a49d440d 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessageTrackerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs @@ -1,7 +1,7 @@ using Orleans.Providers.Streams.Common; using TimeProviderExtensions; -namespace Egil.Orleans.Messaging.Tests; +namespace Egil.Orleans.Messaging.Tests.Tracking; public sealed class MessageTrackerTests { From 999408fd5cb37903fdbac1dbc4ee1c95dfd04b8d Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 15:17:24 +0000 Subject: [PATCH 27/82] fix(om): resume streams from tracked cursors Use the activation-time MessageTracker snapshot to provide provider-aware resume tokens when StreamManager attaches implicit handles, resumes explicit handles, or creates missing explicit subscriptions. Add focused tests for provider-specific cursor lookup and the Orleans stream APIs that receive those tokens. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/api-design.md | 24 +- .../Streams/StreamManager.cs | 30 ++- .../Tracking/MessageTracker.cs | 27 ++ .../Streams/StreamManagerResumeTests.cs | 250 ++++++++++++++++++ .../Tracking/MessageTrackerTests.cs | 29 ++ 5 files changed, 346 insertions(+), 14 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 5889cba9..1933b59f 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -852,11 +852,13 @@ group. Users only specify it for inline lambdas or ambiguous method groups. - `ConfigureImplicitSubscription(...)` records the handler shape only. - It does not call `SubscribeAsync(...)`, inspect existing explicit handles, - or read a resume token from `MessageTracker`. + or create Orleans subscription state. - When Orleans calls `OnSubscribed(factory)`, `StreamManager` matches `factory.StreamId.GetNamespace()` to a configured implicit subscription, creates the typed handle with `factory.Create()`, and calls - `ResumeAsync(observer)` to attach the handler. + `ResumeAsync(observer, token)` to attach the handler. The token comes from + the activation-time `MessageTracker` snapshot when a cursor exists for the + same provider and namespace. - Orleans controls activation and target stream identity. If the grain is not active, a matching stream event can activate it. - Implicit subscriptions do not show up as explicit handles from @@ -868,11 +870,13 @@ group. Users only specify it for inline lambdas or ambiguous method groups. - `ConfigureExplicitSubscription(...)` records the provider, namespace, event type, handler, and error callback. - `ResumeExplicitSubscriptionsAsync(...)` resumes all existing durable - handles for each configured explicit stream. It never creates a new + handles for each configured explicit stream from the activation-time + `MessageTracker` cursor when one exists. It never creates a new subscription. - `EnsureExplicitSubscriptionsAsync(...)` resumes existing handles when present. If none exist for a configured explicit stream, it creates exactly - one explicit subscription with `SubscribeAsync(observer)`. + one explicit subscription with `SubscribeAsync(observer, token)`, using the + tracked cursor token when available. - Repeated calls to `EnsureExplicitSubscriptionsAsync(...)` are idempotent: once a durable explicit handle exists, later calls resume it instead of creating duplicates. @@ -887,12 +891,12 @@ group. Users only specify it for inline lambdas or ambiguous method groups. `StreamSequenceToken?`. - The cursor includes the stream namespace, provider name when known, and the delivered Orleans sequence token. -- `MessageTracker` is not used to drive implicit subscription start position. - For implicit streams, Orleans owns subscription creation and the manager - only resumes handler logic for the current delivery path. -- For explicit streams, the durable Orleans subscription handle is the primary - resume mechanism. `MessageTracker` remains the application-level dedup and - high-water marker; handlers update it after accepting events. +- `MessageTracker` provides provider-aware resume tokens for both implicit + handle resume and explicit subscribe/resume. +- For explicit streams, the durable Orleans subscription handle is still the + primary subscription identity. `MessageTracker` remains the + application-level dedup and high-water marker; handlers update it after + accepting events. ### Per-subscription `OnError` diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs index a62e6bc2..90d234c7 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs @@ -15,6 +15,7 @@ namespace Egil.Orleans.Messaging.Streams; public sealed class StreamManager : IStreamManagerComponent { private readonly IGrainBase owner; + private readonly MessageTracker trackerSnapshot; private readonly Func getStreamProvider; private readonly Func getStreamId; private readonly ILogger logger; @@ -25,11 +26,13 @@ public sealed class StreamManager : IStreamManagerComponent private StreamManager( IGrainBase owner, + MessageTracker trackerSnapshot, Func getStreamProvider, Func getStreamId, ILogger logger) { this.owner = owner; + this.trackerSnapshot = trackerSnapshot; this.getStreamProvider = getStreamProvider; this.getStreamId = getStreamId; this.logger = logger; @@ -58,6 +61,7 @@ internal static StreamManager Create( var manager = Create( owner, + trackerSnapshot, streamNamespace => services.GetRequiredKeyedService(streamProviderName ?? streamNamespace), streamNamespace => CreateStreamId(owner, streamNamespace), logger); @@ -68,16 +72,18 @@ internal static StreamManager Create( internal static StreamManager Create( IGrainBase owner, + MessageTracker trackerSnapshot, Func getStreamProvider, Func getStreamId, ILogger logger) { ArgumentNullException.ThrowIfNull(owner); + ArgumentNullException.ThrowIfNull(trackerSnapshot); ArgumentNullException.ThrowIfNull(getStreamProvider); ArgumentNullException.ThrowIfNull(getStreamId); ArgumentNullException.ThrowIfNull(logger); - return new StreamManager(owner, getStreamProvider, getStreamId, logger); + return new StreamManager(owner, trackerSnapshot, getStreamProvider, getStreamId, logger); } /// @@ -229,7 +235,9 @@ private async Task> ResumeImplicitAsync handleFactory.ProviderName, onNextAsync, onError); - var handle = await handleFactory.Create().ResumeAsync(observer); + var handle = await handleFactory.Create().ResumeAsync( + observer, + GetResumeToken(handleFactory.ProviderName, streamNamespace)); subscriptionHandles.Add(handle); MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); @@ -265,7 +273,10 @@ private async Task ResumeExplicitAsync( streamProviderName, onNextAsync, onError); - var handle = await stream.SubscribeAsync(observer); + var handle = await stream.SubscribeAsync( + observer, + GetResumeToken(streamProviderName, streamNamespace), + filterData: null); subscriptionHandles.Add(handle); MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); @@ -282,7 +293,9 @@ private async Task ResumeExplicitAsync( handle.ProviderName, onNextAsync, onError); - var resumedHandle = await handle.ResumeAsync(observer); + var resumedHandle = await handle.ResumeAsync( + observer, + GetResumeToken(handle.ProviderName, streamNamespace)); subscriptionHandles.Add(resumedHandle); MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); @@ -309,6 +322,15 @@ private IAsyncStream GetExplicitStream( return streamProvider.GetStream(streamId); } + private StreamSequenceToken? GetResumeToken(string? streamProviderName, string streamNamespace) + { + var cursor = string.IsNullOrWhiteSpace(streamProviderName) + ? trackerSnapshot.LatestStream(streamNamespace) + : trackerSnapshot.LatestStream(streamProviderName, streamNamespace); + + return cursor?.Token; + } + private async Task OnNextAsync( string streamNamespace, string? providerName, diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs index 4fa9e1a5..84a9b6af 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs @@ -207,6 +207,33 @@ public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) return result; } + /// + /// Returns the last accepted for the given + /// and + /// , or null if no matching + /// stream position has been tracked. + /// + /// + /// Cursors without a provider name are treated as convention-compatible + /// with the requested provider. This preserves positions produced before + /// provider-aware stream tracking was available, while avoiding accidental + /// fallback to a different provider's cursor. + /// + public StreamCursor? LatestStream(string streamProviderName, string streamNamespace) + { + ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); + ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); + + if (streams.TryGetValue(new StreamSource(streamNamespace, streamProviderName), out var providerEntry)) + { + return providerEntry.LastPosition; + } + + return streams.TryGetValue(new StreamSource(streamNamespace, null), out var conventionEntry) + ? conventionEntry.LastPosition + : null; + } + /// /// Returns the last accepted for the namespace /// in the given . diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs new file mode 100644 index 00000000..f3a2af7d --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs @@ -0,0 +1,250 @@ +using Microsoft.Extensions.Logging.Abstractions; +using Orleans.Providers.Streams.Common; +using Orleans.Streams; +using Orleans.Streams.Core; + +namespace Egil.Orleans.Messaging.Tests.Streams; + +public sealed class StreamManagerResumeTests +{ + [Fact] + public async Task EnsureExplicitSubscriptionsAsync_uses_tracker_resume_token_when_creating_subscription() + { + var stream = new FakeStream("provider-a", StreamId.Create("orders", "one")); + var tracker = CreateTracker("provider-a", "orders", sequenceNumber: 7); + var manager = CreateManager(tracker, stream); + + await manager + .ConfigureExplicitSubscription("provider-a", "orders", static (_, _) => ValueTask.CompletedTask) + .EnsureExplicitSubscriptionsAsync(TestContext.Current.CancellationToken); + + Assert.Equal(new EventSequenceToken(7), stream.SubscribeToken); + Assert.Equal(1, stream.SubscribeCount); + } + + [Fact] + public async Task ResumeExplicitSubscriptionsAsync_uses_tracker_resume_token_when_resuming_existing_handle() + { + var handle = new FakeSubscriptionHandle("provider-a", StreamId.Create("orders", "one")); + var stream = new FakeStream("provider-a", StreamId.Create("orders", "one")); + stream.Handles.Add(handle); + var tracker = CreateTracker("provider-a", "orders", sequenceNumber: 7); + var manager = CreateManager(tracker, stream); + + await manager + .ConfigureExplicitSubscription("provider-a", "orders", static (_, _) => ValueTask.CompletedTask) + .ResumeExplicitSubscriptionsAsync(TestContext.Current.CancellationToken); + + Assert.Equal(new EventSequenceToken(7), handle.ResumeToken); + Assert.Equal(1, handle.ResumeCount); + Assert.Equal(0, stream.SubscribeCount); + } + + [Fact] + public async Task Implicit_subscription_resume_uses_tracker_resume_token() + { + var streamId = StreamId.Create("orders", "one"); + var tracker = CreateTracker("provider-a", "orders", sequenceNumber: 7); + var handleFactory = new FakeStreamSubscriptionHandleFactory( + "provider-a", + streamId, + new FakeSubscriptionHandle("provider-a", streamId)); + var manager = CreateManager(tracker, new FakeStream("provider-a", streamId)); + + await ((IStreamManagerComponent)manager + .ConfigureImplicitSubscription("orders", static (_, _) => ValueTask.CompletedTask)) + .OnSubscribedAsync(handleFactory); + + Assert.Equal(new EventSequenceToken(7), handleFactory.StringHandle.ResumeToken); + Assert.Equal(1, handleFactory.StringHandle.ResumeCount); + } + + private static StreamManager CreateManager( + MessageTracker tracker, + FakeStream stream) + { + var owner = new FakeGrainBase(); + return StreamManager.Create( + owner, + tracker, + _ => new FakeStreamProvider(stream), + streamNamespace => StreamId.Create(streamNamespace, "one"), + NullLogger.Instance); + } + + private static MessageTracker CreateTracker( + string providerName, + string streamNamespace, + long sequenceNumber) + { + var tracker = new MessageTracker(); + tracker.ProcessMessage( + new StreamCursor( + streamNamespace, + new EventSequenceToken(sequenceNumber), + providerName), + out tracker); + + return tracker; + } + + private sealed class FakeStreamProvider(FakeStream stream) : IStreamProvider + { + public string Name => stream.ProviderName; + + public bool IsRewindable => stream.IsRewindable; + + public IAsyncStream GetStream(StreamId streamId) + { + if (stream is IAsyncStream typedStream) + { + return typedStream; + } + + throw new InvalidOperationException($"Unexpected stream type '{typeof(T).FullName}'."); + } + } + + private sealed class FakeStream( + string providerName, + StreamId streamId) : IAsyncStream + { + public List> Handles { get; } = []; + + public int SubscribeCount { get; private set; } + + public StreamSequenceToken? SubscribeToken { get; private set; } + + public bool IsRewindable => true; + + public string ProviderName => providerName; + + public StreamId StreamId => streamId; + + public Task>> GetAllSubscriptionHandles() => + Task.FromResult>>(Handles); + + public Task> SubscribeAsync(IAsyncObserver observer) + { + return SubscribeAsync(observer, token: null, filterData: null); + } + + public Task> SubscribeAsync( + IAsyncObserver observer, + StreamSequenceToken? token, + string? filterData = null) + { + SubscribeCount++; + SubscribeToken = token; + var handle = new FakeSubscriptionHandle(providerName, streamId); + Handles.Add(handle); + + return Task.FromResult>(handle); + } + + public Task OnNextAsync(T item, StreamSequenceToken? token = null) => Task.CompletedTask; + + public Task OnNextBatchAsync(IEnumerable batch, StreamSequenceToken? token = null) => Task.CompletedTask; + + public Task OnCompletedAsync() => Task.CompletedTask; + + public Task OnErrorAsync(Exception ex) => Task.CompletedTask; + + public Task> SubscribeAsync(IAsyncBatchObserver observer) + { + return SubscribeAsync(observer, token: null); + } + + public Task> SubscribeAsync( + IAsyncBatchObserver observer, + StreamSequenceToken? token) + { + SubscribeCount++; + SubscribeToken = token; + var handle = new FakeSubscriptionHandle(providerName, streamId); + Handles.Add(handle); + + return Task.FromResult>(handle); + } + + public int CompareTo(IAsyncStream? other) => + string.Compare(StreamId.ToString(), other?.StreamId.ToString(), StringComparison.Ordinal); + + public bool Equals(IAsyncStream? other) => + other is not null + && string.Equals(ProviderName, other.ProviderName, StringComparison.Ordinal) + && StreamId.Equals(other.StreamId); + } + + private sealed class FakeSubscriptionHandle( + string providerName, + StreamId streamId) : StreamSubscriptionHandle + { + public int ResumeCount { get; private set; } + + public StreamSequenceToken? ResumeToken { get; private set; } + + public override StreamId StreamId => streamId; + + public override string ProviderName => providerName; + + public override Guid HandleId { get; } = Guid.NewGuid(); + + public override Task UnsubscribeAsync() => Task.CompletedTask; + + public override Task> ResumeAsync( + IAsyncObserver observer, + StreamSequenceToken? token) + { + ResumeCount++; + ResumeToken = token; + return Task.FromResult>(this); + } + + public override Task> ResumeAsync( + IAsyncBatchObserver observer, + StreamSequenceToken? token) + { + ResumeCount++; + ResumeToken = token; + return Task.FromResult>(this); + } + + public override bool Equals(StreamSubscriptionHandle? other) => + other?.HandleId == HandleId; + } + + private sealed class FakeStreamSubscriptionHandleFactory( + string providerName, + StreamId streamId, + FakeSubscriptionHandle stringHandle) : IStreamSubscriptionHandleFactory + { + public FakeSubscriptionHandle StringHandle => stringHandle; + + public StreamId StreamId => streamId; + + public string ProviderName => providerName; + + public GuidId SubscriptionId => GuidId.GetGuidId(Guid.NewGuid()); + + public StreamSubscriptionHandle Create() + { + if (stringHandle is StreamSubscriptionHandle typedHandle) + { + return typedHandle; + } + + throw new InvalidOperationException($"Unexpected stream type '{typeof(T).FullName}'."); + } + } + + private sealed class FakeGrainBase : IGrainBase + { + public IGrainContext GrainContext => + throw new InvalidOperationException("The fake StreamManager tests do not use the grain context."); + + public Task OnActivateAsync(CancellationToken cancellationToken) => Task.CompletedTask; + + public Task OnDeactivateAsync(DeactivationReason reason, CancellationToken cancellationToken) => Task.CompletedTask; + } +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs index a49d440d..9ba54cb0 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs @@ -89,6 +89,35 @@ public void ProcessMessage_accepts_stream_cursor_when_token_is_newer() Assert.Equal(new StreamCursor(streamId, new EventSequenceToken(8)), next.LatestStream(streamId)); } + [Fact] + public void LatestStream_with_provider_returns_matching_provider_cursor() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var streamId = StreamId.Create("orders", "one"); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(now)); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7), "provider-a"), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(9), "provider-b"), out tracker); + + var latest = tracker.LatestStream("provider-a", "orders"); + + Assert.Equal(new StreamCursor("orders", new EventSequenceToken(7), "provider-a"), latest); + } + + [Fact] + public void LatestStream_with_provider_does_not_fall_back_to_different_provider() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var streamId = StreamId.Create("orders", "one"); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(now)); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7), "provider-a"), out tracker); + + var latest = tracker.LatestStream("provider-b", "orders"); + + Assert.Null(latest); + } + [Fact] public void ProcessMessage_accepts_non_null_stream_token_after_null_initial_position() { From 4abc03f4e6cab0dfa4a103b920e48d5128e659b0 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 15:25:32 +0000 Subject: [PATCH 28/82] fix(om): fail fast for missing grain components Throw clear configuration errors when implicit stream and outbox marker interfaces receive callbacks before their corresponding components have been attached. This avoids silently swallowing stream subscription or reminder callbacks when a grain implements the marker interface but forgets to call the registration helper during activation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/api-design.md | 10 +++++++--- .../Outboxes/IOutboxGrain.cs | 17 +++++++++++------ .../Streams/IImplicitStreamGrain.cs | 10 +++++++--- 3 files changed, 25 insertions(+), 12 deletions(-) diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 1933b59f..23f9b352 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -1164,9 +1164,13 @@ public interface IOutboxGrain : IRemindable { var grainBase = (IGrainBase)this; var component = grainBase.GrainContext.GetComponent(); - return component is null - ? Task.CompletedTask - : component.ReceiveReminderAsync(reminderName, status).AsTask(); + if (component is null) + { + throw new InvalidOperationException( + "No OutboxProcessor is attached to the grain context."); + } + + return component.ReceiveReminderAsync(reminderName, status).AsTask(); } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs index 7bf0af6e..e0763398 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs @@ -36,16 +36,21 @@ public interface IOutboxGrain : IRemindable { /// /// Default implementation that discovers the - /// on the grain context and forwards the reminder callback. Returns - /// if no processor is attached (safe - /// no-op for reminders that fire before activation completes). + /// on the grain context and forwards the reminder callback. Throws if no + /// processor is attached, since that means the grain implemented + /// but did not call + /// RegisterOutboxProcessor during activation. /// Task IRemindable.ReceiveReminder(string reminderName, TickStatus status) { var grainBase = (IGrainBase)this; var component = grainBase.GrainContext.GetComponent(); - return component is null - ? Task.CompletedTask - : component.ReceiveReminderAsync(reminderName, status).AsTask(); + if (component is null) + { + throw new InvalidOperationException( + $"No {nameof(OutboxProcessor)} is attached to the grain context. Call RegisterOutboxProcessor(...) from OnActivateAsync before outbox reminders are received."); + } + + return component.ReceiveReminderAsync(reminderName, status).AsTask(); } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs index 4e002ca0..2270eeda 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs @@ -12,8 +12,12 @@ Task IStreamSubscriptionObserver.OnSubscribed(IStreamSubscriptionHandleFactory h { var grainBase = (IGrainBase)this; var component = grainBase.GrainContext.GetComponent(); - return component is null - ? Task.CompletedTask - : component.OnSubscribedAsync(handleFactory); + if (component is null) + { + throw new InvalidOperationException( + $"No {nameof(StreamManager)} is attached to the grain context. Call RegisterStreamManager(...) from OnActivateAsync before implicit stream subscriptions are resumed."); + } + + return component.OnSubscribedAsync(handleFactory); } } From 5702801307a5d09b6a0d839556f73a2c99f9441b Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 15:32:33 +0000 Subject: [PATCH 29/82] feat(om): make stream tracking optional Allow grains to register StreamManager without providing a MessageTracker when they do not persist stream high-water marks. The tracked overloads remain source-compatible and still supply resume tokens when a tracker snapshot is provided. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 9 +++++- Egil.Orleans.Messaging/api-design.md | 26 ++++++++++++++- .../Streams/StreamManagerExtensions.cs | 32 +++++++++++++++---- .../Streams/StreamManagerTests.cs | 2 +- 4 files changed, 59 insertions(+), 10 deletions(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index 6607c666..c7b60456 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -144,7 +144,7 @@ The tracker can also evict old sender or stream entries when your retention poli ## Streams -Use `StreamManager` to configure stream subscriptions from `OnActivateAsync` and resume from the tracker snapshot: +Use `StreamManager` to configure stream subscriptions from `OnActivateAsync`. Pass a tracker snapshot when you want persisted resume tokens, or omit it when the grain does not track stream positions: ```csharp streamManager = this.RegisterStreamManager(state.State.Tracker) @@ -164,6 +164,13 @@ streamManager = this.RegisterStreamManager(state.State.Tracker) await streamManager.EnsureExplicitSubscriptionsAsync(cancellationToken); ``` +```csharp +this.RegisterStreamManager() + .ConfigureImplicitSubscription( + "prices", + async (message, cursor) => await UpdateProjectionAsync(message)); +``` + ## Scope This package is messaging infrastructure, not an event-sourcing or CQRS framework. It wraps Orleans state, outbox dispatch, receiver deduplication, and stream subscription management while leaving domain modeling, read models, transport targets, and operational policy to the application. diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 23f9b352..eba8126b 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -771,7 +771,18 @@ public static class StreamManagerExtensions { public static StreamManager RegisterStreamManager( this TGrain grain, - MessageTracker trackerSnapshot) + MessageTracker? trackerSnapshot = null) + where TGrain : IGrainBase; + + public static StreamManager RegisterStreamManager( + this TGrain grain, + string streamProviderName) + where TGrain : IGrainBase; + + public static StreamManager RegisterStreamManager( + this TGrain grain, + MessageTracker? trackerSnapshot, + string streamProviderName) where TGrain : IGrainBase; } @@ -784,6 +795,12 @@ parameter. Orleans provides the concrete provider and stream id through `ConfigureExplicitSubscription` requires a provider name because the library must ask Orleans for the stream and durable subscription handles itself. +`RegisterStreamManager()` can be called without a `MessageTracker` when the +grain does not persist stream high-water marks. In that mode the manager +attaches handlers without supplying resume tokens. Pass the activation-time +tracker snapshot only when the grain wants `StreamManager` to resume from +persisted cursors. + ### Typical implicit wiring ```csharp @@ -812,6 +829,13 @@ No `SubscribeAsync` call belongs in implicit activation. Orleans activates the grain and calls `OnSubscribed(...)` when an event targets the implicit subscription. +For grains that do not track stream positions, omit the tracker: + +```csharp +this.RegisterStreamManager() + .ConfigureImplicitSubscription("electricity-prices", HandlePriceTickAsync); +``` + ### Typical explicit wiring ```csharp diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs index cee9f2fc..885cdee4 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs @@ -35,16 +35,17 @@ public static class StreamManagerExtensions /// /// /// - /// A snapshot of the grain's at activation - /// time. Used to look up resume tokens. + /// Optional snapshot of the grain's at + /// activation time. When provided, it is used to look up resume tokens. + /// Omit it when the grain does not persist stream tracking state. /// /// /// The registered . Store it in a grain field so /// configured subscriptions remain rooted for the activation lifetime. /// - public StreamManager RegisterStreamManager(MessageTracker trackerSnapshot) + public StreamManager RegisterStreamManager(MessageTracker? trackerSnapshot = null) { - return StreamManager.Create(grain, trackerSnapshot); + return StreamManager.Create(grain, trackerSnapshot ?? new MessageTracker()); } /// @@ -57,16 +58,33 @@ public StreamManager RegisterStreamManager(MessageTracker trackerSnapshot) /// , because the attribute /// describes namespaces while the provider name is runtime configuration. /// - /// A snapshot of the grain's tracker at activation time. + /// + /// Optional snapshot of the grain's tracker at activation time. Omit it + /// when the grain does not persist stream tracking state. + /// /// The Orleans stream provider name. /// The registered . public StreamManager RegisterStreamManager( - MessageTracker trackerSnapshot, + MessageTracker? trackerSnapshot, string streamProviderName) { ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); - return StreamManager.Create(grain, trackerSnapshot, streamProviderName); + return StreamManager.Create(grain, trackerSnapshot ?? new MessageTracker(), streamProviderName); + } + + /// + /// Registers a for a grain whose configured + /// subscriptions all use the same Orleans stream provider and which does + /// not provide persisted stream tracking state. + /// + /// The Orleans stream provider name. + /// The registered . + public StreamManager RegisterStreamManager(string streamProviderName) + { + ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); + + return StreamManager.Create(grain, new MessageTracker(), streamProviderName); } } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs index 1384a7de..636ea6e1 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs @@ -326,7 +326,7 @@ public sealed class ImplicitStreamManagerTestGrain( { public override async Task OnActivateAsync(CancellationToken cancellationToken) { - this.RegisterStreamManager(state.State.Tracker) + this.RegisterStreamManager() .ConfigureImplicitSubscription(StreamManagerTestNamespaces.Implicit, HandleImplicitAsync); await base.OnActivateAsync(cancellationToken); From 052169dfdf9f1459ca3814cfd508c9f3e435ff86 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 15:37:02 +0000 Subject: [PATCH 30/82] refactor(om): remove registration-level stream provider Keep provider selection at the subscription boundary: implicit subscriptions get the provider from Orleans handle factories, and explicit subscriptions specify it when configured. This removes redundant RegisterStreamManager provider overloads and avoids carrying a provider default through StreamManager registration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/api-design.md | 11 ------ .../Streams/StreamManager.cs | 8 +--- .../Streams/StreamManagerExtensions.cs | 39 ------------------- 3 files changed, 1 insertion(+), 57 deletions(-) diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index eba8126b..7786d5bc 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -773,17 +773,6 @@ public static class StreamManagerExtensions this TGrain grain, MessageTracker? trackerSnapshot = null) where TGrain : IGrainBase; - - public static StreamManager RegisterStreamManager( - this TGrain grain, - string streamProviderName) - where TGrain : IGrainBase; - - public static StreamManager RegisterStreamManager( - this TGrain grain, - MessageTracker? trackerSnapshot, - string streamProviderName) - where TGrain : IGrainBase; } ``` diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs index 90d234c7..2cd113ab 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs @@ -44,12 +44,6 @@ private StreamManager( internal static StreamManager Create( IGrainBase owner, MessageTracker trackerSnapshot) - => Create(owner, trackerSnapshot, streamProviderName: null); - - internal static StreamManager Create( - IGrainBase owner, - MessageTracker trackerSnapshot, - string? streamProviderName) { ArgumentNullException.ThrowIfNull(owner); ArgumentNullException.ThrowIfNull(trackerSnapshot); @@ -62,7 +56,7 @@ internal static StreamManager Create( var manager = Create( owner, trackerSnapshot, - streamNamespace => services.GetRequiredKeyedService(streamProviderName ?? streamNamespace), + streamProviderName => services.GetRequiredKeyedService(streamProviderName), streamNamespace => CreateStreamId(owner, streamNamespace), logger); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs index 885cdee4..8f06f03d 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs @@ -47,44 +47,5 @@ public StreamManager RegisterStreamManager(MessageTracker? trackerSnapshot = nul { return StreamManager.Create(grain, trackerSnapshot ?? new MessageTracker()); } - - /// - /// Registers a for a grain whose configured - /// subscriptions all use the same Orleans stream provider. - /// - /// - /// Prefer this overload when provider name and stream namespace are - /// different. This is the usual shape for grains using - /// , because the attribute - /// describes namespaces while the provider name is runtime configuration. - /// - /// - /// Optional snapshot of the grain's tracker at activation time. Omit it - /// when the grain does not persist stream tracking state. - /// - /// The Orleans stream provider name. - /// The registered . - public StreamManager RegisterStreamManager( - MessageTracker? trackerSnapshot, - string streamProviderName) - { - ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); - - return StreamManager.Create(grain, trackerSnapshot ?? new MessageTracker(), streamProviderName); - } - - /// - /// Registers a for a grain whose configured - /// subscriptions all use the same Orleans stream provider and which does - /// not provide persisted stream tracking state. - /// - /// The Orleans stream provider name. - /// The registered . - public StreamManager RegisterStreamManager(string streamProviderName) - { - ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); - - return StreamManager.Create(grain, new MessageTracker(), streamProviderName); - } } } From f3fdc97c2a66dec17c94f856e3c56a85201b6fec Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 15:39:10 +0000 Subject: [PATCH 31/82] style(om): indent stream manager extension docs [skip notes] Align XML documentation indentation inside the C# extension block. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Streams/StreamManagerExtensions.cs | 70 +++++++++---------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs index 8f06f03d..60218365 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs @@ -12,40 +12,40 @@ public static class StreamManagerExtensions extension(TGrain grain) where TGrain : IGrainBase { - /// - /// Registers a for the grain and returns it - /// for fluent stream subscription configuration. - /// - /// - /// - /// Call from OnActivateAsync, then chain - /// ConfigureImplicitSubscription or - /// ConfigureExplicitSubscription calls on the returned manager. - /// - /// - /// Usage: - /// -/// public override async Task OnActivateAsync(CancellationToken ct) -/// { -/// streamManager = this.RegisterStreamManager(state.Tracker) -/// .ConfigureExplicitSubscription("StreamProvider", "electricity-prices", HandlePriceTickAsync); -/// await streamManager.EnsureExplicitSubscriptionsAsync(ct); -/// } -/// - /// - /// - /// - /// Optional snapshot of the grain's at - /// activation time. When provided, it is used to look up resume tokens. - /// Omit it when the grain does not persist stream tracking state. - /// - /// - /// The registered . Store it in a grain field so - /// configured subscriptions remain rooted for the activation lifetime. - /// - public StreamManager RegisterStreamManager(MessageTracker? trackerSnapshot = null) - { - return StreamManager.Create(grain, trackerSnapshot ?? new MessageTracker()); - } + /// + /// Registers a for the grain and returns it + /// for fluent stream subscription configuration. + /// + /// + /// + /// Call from OnActivateAsync, then chain + /// ConfigureImplicitSubscription or + /// ConfigureExplicitSubscription calls on the returned manager. + /// + /// + /// Usage: + /// + /// public override async Task OnActivateAsync(CancellationToken ct) + /// { + /// streamManager = this.RegisterStreamManager(state.Tracker) + /// .ConfigureExplicitSubscription("StreamProvider", "electricity-prices", HandlePriceTickAsync); + /// await streamManager.EnsureExplicitSubscriptionsAsync(ct); + /// } + /// + /// + /// + /// + /// Optional snapshot of the grain's at + /// activation time. When provided, it is used to look up resume tokens. + /// Omit it when the grain does not persist stream tracking state. + /// + /// + /// The registered . Store it in a grain field so + /// configured subscriptions remain rooted for the activation lifetime. + /// + public StreamManager RegisterStreamManager(MessageTracker? trackerSnapshot = null) + { + return StreamManager.Create(grain, trackerSnapshot ?? new MessageTracker()); + } } } From bc729a0b4da7798243f6feb5626fb6d794fab163 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 15:42:38 +0000 Subject: [PATCH 32/82] style(om): apply dotnet format [skip notes] Run dotnet format against the messaging solution after the capability namespace refactor. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../MessagingTelemetry.cs | 2 +- .../Outboxes/IOutboxGrain.cs | 2 +- .../Outboxes/NoPostmanRegisteredException.cs | 2 +- .../Egil.Orleans.Messaging/Outboxes/Outbox.cs | 2 +- .../Outboxes/OutboxJsonConverterFactory.cs | 2 +- .../Outboxes/OutboxMessageEnvelope.cs | 2 +- ...tboxMessageEnvelopeJsonConverterFactory.cs | 2 +- .../Outboxes/OutboxProcessor.cs | 2 +- .../Outboxes/OutboxProcessorExtensions.cs | 116 +++++++++--------- .../Outboxes/OutboxProcessorOptions.cs | 8 +- .../Outboxes/OutboxSequenceToken.cs | 2 +- .../OutboxSequenceTokenJsonConverter.cs | 2 +- .../State/IStateManager.cs | 2 +- .../State/IStateManagerFactory.cs | 2 +- .../State/StateManager.cs | 2 +- .../State/StateManagerExtensions.cs | 106 ++++++++-------- .../StateManagerRegistrationExtensions.cs | 82 ++++++------- .../StateManagerSiloBuilderExtensions.cs | 2 +- .../State/VersionedState.cs | 2 +- .../EventHubs/EnrichedEventHubAdapter.cs | 2 +- .../EnrichedEventHubAdapterExtensions.cs | 50 ++++---- .../EnrichedEventHubSequenceToken.cs | 2 +- .../Streams/IImplicitStreamGrain.cs | 2 +- .../Streams/IStreamManagerComponent.cs | 2 +- .../Streams/StreamCursor.cs | 2 +- .../Streams/StreamCursorJsonConverter.cs | 2 +- .../Streams/StreamManager.cs | 2 +- .../Streams/StreamManagerExtensions.cs | 2 +- .../Tracking/MessageTracker.cs | 2 +- .../Tracking/MessageTrackerJsonConverter.cs | 2 +- .../GlobalUsings.cs | 2 +- .../MessagingTestClusterFixture.cs | 2 +- .../OutboxJsonConverterFactoryTests.cs | 2 +- ...essageEnvelopeJsonConverterFactoryTests.cs | 2 +- .../Outboxes/OutboxProcessorTests.cs | 2 +- .../OutboxSequenceTokenJsonConverterTests.cs | 2 +- .../Outboxes/OutboxTests.cs | 2 +- .../State/StateManagerExtensionsTests.cs | 2 +- ...StateManagerRegistrationExtensionsTests.cs | 2 +- .../State/StateManagerTests.cs | 2 +- .../EnrichedEventHubAdapterExtensionsTests.cs | 2 +- .../EventHubs/EnrichedEventHubAdapterTests.cs | 2 +- .../Streams/StreamCursorJsonConverterTests.cs | 3 +- .../Streams/StreamCursorTests.cs | 2 +- .../Streams/StreamManagerResumeTests.cs | 2 +- .../Streams/StreamManagerTests.cs | 2 +- .../MessageTrackerJsonConverterTests.cs | 3 +- .../Tracking/MessageTrackerTests.cs | 2 +- 48 files changed, 225 insertions(+), 225 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs index 5cd04056..f6005b0b 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs @@ -166,4 +166,4 @@ private static IEnumerable> ObserveOutboxDepth() new KeyValuePair("grain.type", depth.Key)); } } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs index e0763398..df6e4533 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs @@ -53,4 +53,4 @@ Task IRemindable.ReceiveReminder(string reminderName, TickStatus status) return component.ReceiveReminderAsync(reminderName, status).AsTask(); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/NoPostmanRegisteredException.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/NoPostmanRegisteredException.cs index 532c6afa..1aba1168 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/NoPostmanRegisteredException.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/NoPostmanRegisteredException.cs @@ -42,4 +42,4 @@ public NoPostmanRegisteredException(Type itemType) /// registered postman. /// public Type ItemType { get; } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs index 52e6a29c..acfc3621 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs @@ -344,4 +344,4 @@ public override int GetHashCode() private long LastSequenceNumber => items.IsDefaultOrEmpty ? 0 : items[^1].Token.SequenceNumber; -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs index 0911f587..096c6816 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs @@ -136,4 +136,4 @@ private sealed record GrainIdJsonModel(string Type, string Key); private static GrainIdJsonModel ToJsonModel(GrainId grainId) => new(grainId.Type.ToString()!, grainId.Key.ToString()!); -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs index 9774a446..53e56e9e 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs @@ -44,4 +44,4 @@ namespace Egil.Orleans.Messaging.Outboxes; [JsonConverter(typeof(OutboxMessageEnvelopeJsonConverterFactory))] public sealed record OutboxMessageEnvelope( [property: Id(0)] OutboxSequenceToken Token, - [property: Id(1)] T Message); + [property: Id(1)] T Message); \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs index 9b51d43f..c28f0946 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs @@ -68,4 +68,4 @@ public override void Write( private sealed record OutboxMessageEnvelopeJsonModel( OutboxSequenceToken? Token, T Message); -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index 8052689a..59b123e9 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -388,4 +388,4 @@ internal interface IOutboxComponent { /// Forwards a reminder callback to the outbox processor. ValueTask ReceiveReminderAsync(string reminderName, TickStatus status); -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs index 7cf507b0..d51a9ce1 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs @@ -13,64 +13,64 @@ public static class OutboxProcessorExtensions extension(TGrain grain) where TGrain : IGrainBase, IOutboxGrain { - /// - /// Creates and attaches an to the - /// grain. Call in OnActivateAsync and chain - /// - /// calls on the result. - /// - /// - /// - /// Registers the processor as an on the - /// grain context so the DIM can discover it - /// for reminder forwarding. - /// - /// - /// Usage: - /// - /// public override async Task OnActivateAsync(CancellationToken ct) - /// { - /// outboxProcessor = this.RegisterOutboxProcessor(new OutboxProcessorOptions<IMyEvent> - /// { - /// PendingItems = () => stateManager.State.Outbox - /// .Select(e => e.Message).ToImmutableArray(), - /// AcknowledgePostedAsync = async (items, ct) => - /// { - /// // remove delivered items and persist - /// }, - /// ReconcileFailedAsync = async (failures, ct) => - /// { - /// // log, dead-letter, or leave for retry - /// }, - /// }) - /// .AddPostman<PriceCalculated>(PublishPriceCalculatedAsync) - /// .AddPostman<InvoiceReady>(SendInvoiceReadyAsync); - /// } - /// - /// - /// - /// The base type of outbox items. - /// Processor configuration. - /// - /// The processor instance. Chain AddPostman calls on it, then store - /// in a grain field for later - /// calls. - /// - public OutboxProcessor RegisterOutboxProcessor(OutboxProcessorOptions options) - where TOutbox : notnull - { - ArgumentNullException.ThrowIfNull(grain); - ArgumentNullException.ThrowIfNull(options); + /// + /// Creates and attaches an to the + /// grain. Call in OnActivateAsync and chain + /// + /// calls on the result. + /// + /// + /// + /// Registers the processor as an on the + /// grain context so the DIM can discover it + /// for reminder forwarding. + /// + /// + /// Usage: + /// + /// public override async Task OnActivateAsync(CancellationToken ct) + /// { + /// outboxProcessor = this.RegisterOutboxProcessor(new OutboxProcessorOptions<IMyEvent> + /// { + /// PendingItems = () => stateManager.State.Outbox + /// .Select(e => e.Message).ToImmutableArray(), + /// AcknowledgePostedAsync = async (items, ct) => + /// { + /// // remove delivered items and persist + /// }, + /// ReconcileFailedAsync = async (failures, ct) => + /// { + /// // log, dead-letter, or leave for retry + /// }, + /// }) + /// .AddPostman<PriceCalculated>(PublishPriceCalculatedAsync) + /// .AddPostman<InvoiceReady>(SendInvoiceReadyAsync); + /// } + /// + /// + /// + /// The base type of outbox items. + /// Processor configuration. + /// + /// The processor instance. Chain AddPostman calls on it, then store + /// in a grain field for later + /// calls. + /// + public OutboxProcessor RegisterOutboxProcessor(OutboxProcessorOptions options) + where TOutbox : notnull + { + ArgumentNullException.ThrowIfNull(grain); + ArgumentNullException.ThrowIfNull(options); - var services = grain.GrainContext.ActivationServices; - var processor = new OutboxProcessor( - grain, - services.GetRequiredService(), - options, - services.GetRequiredService().CreateLogger>()); + var services = grain.GrainContext.ActivationServices; + var processor = new OutboxProcessor( + grain, + services.GetRequiredService(), + options, + services.GetRequiredService().CreateLogger>()); - processor.AttachToGrain(); - return processor; - } + processor.AttachToGrain(); + return processor; + } } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs index 769ae3e9..df2a6eab 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs @@ -42,7 +42,8 @@ public sealed class OutboxProcessorOptions /// and persist the update. /// public required Func, CancellationToken, ValueTask> - AcknowledgePostedAsync { get; init; } + AcknowledgePostedAsync + { get; init; } /// /// Called with items that failed dispatch, along with the exception and @@ -50,7 +51,8 @@ public required Func, CancellationToken, ValueTask> /// pending, remove them, or move them to dead-letter state. /// public Func, - CancellationToken, ValueTask>? ReconcileFailedAsync { get; init; } + CancellationToken, ValueTask>? ReconcileFailedAsync + { get; init; } /// /// Maximum time per post run. Default: 20 seconds. @@ -78,4 +80,4 @@ public required Func, CancellationToken, ValueTask> /// public OutboxPostmanExecutionMode PostmanExecution { get; init; } = OutboxPostmanExecutionMode.GrainScheduler; -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs index 0ce4892b..788865f6 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs @@ -59,4 +59,4 @@ public sealed record OutboxSequenceToken( [property: Id(0)] long SequenceNumber, [property: Id(1)] GrainId Sender, [property: Id(2)] DateTimeOffset Timestamp, - [property: Id(3)] DateTimeOffset Epoch); + [property: Id(3)] DateTimeOffset Epoch); \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs index 64a2878e..7f4c9607 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs @@ -59,4 +59,4 @@ private sealed record OutboxSequenceTokenJsonModel( DateTimeOffset Epoch); private sealed record GrainIdJsonModel(string Type, string Key); -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs index 728ee6d2..054dd7e8 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs @@ -131,4 +131,4 @@ public interface IStateManager /// Clears the persisted state, resetting it to default. /// Task ClearAsync(); -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs index 440bcd2a..cc88e4aa 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs @@ -32,4 +32,4 @@ public IStateManager Create(IPersistentState storage) ArgumentNullException.ThrowIfNull(storage); return new DefaultStateManager(storage); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs index df72b083..753c50e9 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs @@ -258,4 +258,4 @@ protected override WriteFailureKind ClassifyWriteFailure(Exception exception) ArgumentNullException.ThrowIfNull(exception); return WriteFailureKind.UnknownOutcome; } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs index c42a429b..408cc41c 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs @@ -12,59 +12,59 @@ public static class StateManagerExtensions extension(TGrain grain) where TGrain : IGrainBase { - /// - /// Creates an for the given grain using - /// a keyed registration. - /// - /// - /// - /// Call site: Typically called once in OnActivateAsync: - /// - /// [PersistentState("state")] private readonly IPersistentState<MyState> storage; - /// private IStateManager<MyState> stateManager = default!; - /// - /// public override Task OnActivateAsync(CancellationToken ct) - /// { - /// stateManager = this.RegisterStateManager("state", storage); - /// // ... - /// } - /// - /// - /// - /// Registration: A keyed - /// must be registered for - /// via the provided registration helpers. - /// Missing registrations fail fast with a descriptive - /// . - /// - /// - /// - /// The grain state type. - /// - /// - /// Logical storage name used as the keyed DI registration key. - /// - /// - /// The Orleans-managed persistent state facet. - /// - /// - /// A keyed instance. - /// - public IStateManager RegisterStateManager( - string storageName, - IPersistentState storage) - where TState : class, IEquatable - { - ArgumentNullException.ThrowIfNull(grain); - ArgumentException.ThrowIfNullOrWhiteSpace(storageName); - ArgumentNullException.ThrowIfNull(storage); + /// + /// Creates an for the given grain using + /// a keyed registration. + /// + /// + /// + /// Call site: Typically called once in OnActivateAsync: + /// + /// [PersistentState("state")] private readonly IPersistentState<MyState> storage; + /// private IStateManager<MyState> stateManager = default!; + /// + /// public override Task OnActivateAsync(CancellationToken ct) + /// { + /// stateManager = this.RegisterStateManager("state", storage); + /// // ... + /// } + /// + /// + /// + /// Registration: A keyed + /// must be registered for + /// via the provided registration helpers. + /// Missing registrations fail fast with a descriptive + /// . + /// + /// + /// + /// The grain state type. + /// + /// + /// Logical storage name used as the keyed DI registration key. + /// + /// + /// The Orleans-managed persistent state facet. + /// + /// + /// A keyed instance. + /// + public IStateManager RegisterStateManager( + string storageName, + IPersistentState storage) + where TState : class, IEquatable + { + ArgumentNullException.ThrowIfNull(grain); + ArgumentException.ThrowIfNullOrWhiteSpace(storageName); + ArgumentNullException.ThrowIfNull(storage); - return RegisterStateManagerCore( - grain.GrainContext.ActivationServices, - storageName, - storage, - grain.GetType()); - } + return RegisterStateManagerCore( + grain.GrainContext.ActivationServices, + storageName, + storage, + grain.GetType()); + } } internal static IStateManager RegisterStateManagerCore( @@ -90,4 +90,4 @@ internal static IStateManager RegisterStateManagerCore( return factory.Create(storage); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs index 357ef3aa..a8080957 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs @@ -10,51 +10,51 @@ public static class StateManagerRegistrationExtensions { extension(IServiceCollection services) { - /// - /// Registers the default keyed for the - /// given . - /// - public IServiceCollection AddDefaultStateManager(string storageName) - { - ArgumentNullException.ThrowIfNull(services); - ValidateStorageName(storageName); + /// + /// Registers the default keyed for the + /// given . + /// + public IServiceCollection AddDefaultStateManager(string storageName) + { + ArgumentNullException.ThrowIfNull(services); + ValidateStorageName(storageName); - return services.AddKeyedSingleton( - typeof(IStateManagerFactory<>), - storageName, - typeof(DefaultStateManagerFactory<>)); - } + return services.AddKeyedSingleton( + typeof(IStateManagerFactory<>), + storageName, + typeof(DefaultStateManagerFactory<>)); + } - /// - /// Registers a custom keyed open-generic - /// implementation for the given . - /// - public IServiceCollection AddStateManagerFactory( - string storageName, - Type openGenericFactoryType) - { - ArgumentNullException.ThrowIfNull(services); - ValidateStorageName(storageName); - ValidateOpenGenericFactoryType(openGenericFactoryType); + /// + /// Registers a custom keyed open-generic + /// implementation for the given . + /// + public IServiceCollection AddStateManagerFactory( + string storageName, + Type openGenericFactoryType) + { + ArgumentNullException.ThrowIfNull(services); + ValidateStorageName(storageName); + ValidateOpenGenericFactoryType(openGenericFactoryType); - return services.AddKeyedSingleton( - typeof(IStateManagerFactory<>), - storageName, - openGenericFactoryType); - } + return services.AddKeyedSingleton( + typeof(IStateManagerFactory<>), + storageName, + openGenericFactoryType); + } - /// - /// Registers a custom keyed factory for a specific state type. - /// - public IServiceCollection AddStateManagerFactory(string storageName) - where TState : class, IEquatable - where TFactory : class, IStateManagerFactory - { - ArgumentNullException.ThrowIfNull(services); - ValidateStorageName(storageName); + /// + /// Registers a custom keyed factory for a specific state type. + /// + public IServiceCollection AddStateManagerFactory(string storageName) + where TState : class, IEquatable + where TFactory : class, IStateManagerFactory + { + ArgumentNullException.ThrowIfNull(services); + ValidateStorageName(storageName); - return services.AddKeyedSingleton, TFactory>(storageName); - } + return services.AddKeyedSingleton, TFactory>(storageName); + } } internal static void ValidateStorageName(string storageName) @@ -91,4 +91,4 @@ internal static void ValidateOpenGenericFactoryType(Type openGenericFactoryType) nameof(openGenericFactoryType)); } } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs index 1c9dc4cc..14714de1 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs @@ -54,4 +54,4 @@ public ISiloBuilder AddStateManagerFactory(string storageName) return builder; } } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/VersionedState.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/VersionedState.cs index 3b8b9d93..45ccfdea 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/VersionedState.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/VersionedState.cs @@ -61,4 +61,4 @@ public abstract record VersionedState [Id(0)] [JsonInclude] public Guid Version { get; internal set; } = Guid.CreateVersion7(); -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapter.cs index 55d4773b..5a205177 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapter.cs @@ -252,4 +252,4 @@ public override StreamPosition GetStreamPosition( return new StreamPosition(streamPosition.StreamId, enrichedToken); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapterExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapterExtensions.cs index 2e283d74..566c0154 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapterExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapterExtensions.cs @@ -42,30 +42,30 @@ public static class EnrichedEventHubAdapterExtensions { extension(IEventHubStreamConfigurator configurator) { - /// - /// Registers the library's -producing - /// data adapter on the given Event Hub stream configurator. - /// - /// - /// - /// Replaces the default EventHubDataAdapter. Only one data adapter - /// can be active per stream provider — calling this after a prior - /// UseDataAdapter replaces the previous registration. - /// - /// - /// The adapter is resolved per-provider: the Serializer is obtained - /// from the silo's , and the stream provider - /// name is captured from the configurator's factory delegate. - /// - /// - public void UseEnrichedDataAdapter() - { - ArgumentNullException.ThrowIfNull(configurator); + /// + /// Registers the library's -producing + /// data adapter on the given Event Hub stream configurator. + /// + /// + /// + /// Replaces the default EventHubDataAdapter. Only one data adapter + /// can be active per stream provider — calling this after a prior + /// UseDataAdapter replaces the previous registration. + /// + /// + /// The adapter is resolved per-provider: the Serializer is obtained + /// from the silo's , and the stream provider + /// name is captured from the configurator's factory delegate. + /// + /// + public void UseEnrichedDataAdapter() + { + ArgumentNullException.ThrowIfNull(configurator); - configurator.UseDataAdapter((services, providerName) => - new EnrichedEventHubAdapter( - providerName, - services.GetRequiredService())); - } + configurator.UseDataAdapter((services, providerName) => + new EnrichedEventHubAdapter( + providerName, + services.GetRequiredService())); + } } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubSequenceToken.cs index b8dd7d90..b64b6e05 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubSequenceToken.cs @@ -125,4 +125,4 @@ public EnrichedEventHubSequenceToken() { ProviderName = string.Empty; } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs index 2270eeda..2bf8323c 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IImplicitStreamGrain.cs @@ -20,4 +20,4 @@ Task IStreamSubscriptionObserver.OnSubscribed(IStreamSubscriptionHandleFactory h return component.OnSubscribedAsync(handleFactory); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamManagerComponent.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamManagerComponent.cs index 65b418b6..901fb9e5 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamManagerComponent.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamManagerComponent.cs @@ -5,4 +5,4 @@ namespace Egil.Orleans.Messaging.Streams; internal interface IStreamManagerComponent { Task OnSubscribedAsync(IStreamSubscriptionHandleFactory handleFactory); -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs index ce09b10b..9ae773c5 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs @@ -168,4 +168,4 @@ public bool TryGetTraceParent([NotNullWhen(true)] out string? traceParent) traceParent = null; return false; } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs index d219d206..7c24dbba 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs @@ -157,4 +157,4 @@ private static class StreamSequenceTokenKinds public const string EventHubSequenceTokenV2 = "event-hub-v2"; public const string EnrichedEventHubSequenceToken = "enriched-event-hub"; } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs index 2cd113ab..6c1970b1 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs @@ -508,4 +508,4 @@ public Task OnNextAsync(TEvent item, StreamSequenceToken? token = null) => public Task OnErrorAsync(Exception ex) => manager.OnErrorAsync(streamNamespace, ex, onError); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs index 60218365..887f466a 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs @@ -48,4 +48,4 @@ public StreamManager RegisterStreamManager(MessageTracker? trackerSnapshot = nul return StreamManager.Create(grain, trackerSnapshot ?? new MessageTracker()); } } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs index 84a9b6af..f085628f 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs @@ -481,4 +481,4 @@ internal readonly record struct OutboxEntry( [property: Id(0)] DateTimeOffset Epoch, [property: Id(1)] long LastSequenceNumber, [property: Id(2)] DateTimeOffset Received); -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs index 0f4cd644..9ae56029 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs @@ -248,4 +248,4 @@ private static class StreamSequenceTokenKinds public const string EventHubSequenceTokenV2 = "event-hub-v2"; public const string EnrichedEventHubSequenceToken = "enriched-event-hub"; } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs index 42ca71ec..32bd2312 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs @@ -5,4 +5,4 @@ global using Egil.Orleans.Messaging.Tracking; global using Microsoft.Extensions.DependencyInjection; global using Orleans; -global using Orleans.Hosting; +global using Orleans.Hosting; \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs index 8bd8d9ea..df5fbb6a 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs @@ -89,4 +89,4 @@ private static void AddStreamProviders(IClientBuilder clientBuilder) clientBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Implicit); clientBuilder.AddMemoryStreams(OutboxProcessorTestNamespaces.Events); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs index ac334c21..1cedd5ea 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs @@ -145,4 +145,4 @@ public void JsonSerializer_throws_json_exception_for_missing_item_token_sender() private sealed record ComplexMessage(string OrderId, int Quantity, NestedMessage Route); private sealed record NestedMessage(string Name, bool IsExpress); -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxMessageEnvelopeJsonConverterFactoryTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxMessageEnvelopeJsonConverterFactoryTests.cs index 2d8d211f..1736f256 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxMessageEnvelopeJsonConverterFactoryTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxMessageEnvelopeJsonConverterFactoryTests.cs @@ -62,4 +62,4 @@ public void JsonSerializer_throws_for_outbox_message_envelope_payload_missing_to Assert.Throws(() => JsonSerializer.Deserialize>(json)); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index 72e68171..592b60b6 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -375,4 +375,4 @@ private async ValueTask HandleEnvelopeAsync( state.State.ReceivedValues = state.State.ReceivedValues.Add(envelope.Message.Value); await state.WriteStateAsync(); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs index 709dfa24..fc0b33fd 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs @@ -65,4 +65,4 @@ public void JsonSerializer_throws_for_outbox_sequence_token_payload_missing_send Assert.Throws(() => JsonSerializer.Deserialize(json)); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs index 2eee4c6d..c972d37c 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs @@ -208,4 +208,4 @@ public void Equals_returns_false_when_sequence_metadata_differs() Assert.NotEqual(left, right); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerExtensionsTests.cs index f0ea11c2..cd3c925f 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerExtensionsTests.cs @@ -110,4 +110,4 @@ private abstract class TestGrainBase : IGrainBase public virtual Task OnDeactivateAsync(DeactivationReason reason, CancellationToken cancellationToken) => Task.CompletedTask; } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs index aa539bba..19ef800a 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs @@ -128,4 +128,4 @@ private sealed class FakeSiloBuilder : ISiloBuilder public Microsoft.Extensions.Configuration.IConfiguration Configuration { get; } = new Microsoft.Extensions.Configuration.ConfigurationManager(); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs index 15bdab2f..fabce5ce 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs @@ -232,4 +232,4 @@ public Task ClearStateAsync() public Task ClearStateAsync(CancellationToken cancellationToken) => ClearStateAsync(); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs index 4850061a..8b41edfd 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs @@ -35,4 +35,4 @@ private sealed class FakeEventHubStreamConfigurator(string name) : IEventHubStre LastConfigureAction = configure; }; } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterTests.cs index a706fbc8..776a16ba 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterTests.cs @@ -131,4 +131,4 @@ private static EnrichedEventHubAdapter CreateAdapter() return new EnrichedEventHubAdapter("provider-a", serializer); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs index e5a35380..69b9da5f 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs @@ -224,5 +224,4 @@ public override int CompareTo(StreamSequenceToken? other) return EventIndex.CompareTo(other.EventIndex); } } -} - +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs index d272f716..34809e87 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs @@ -98,4 +98,4 @@ public void TryGetTraceParent_returns_false_when_missing_on_enriched_event_hub_t Assert.False(found); Assert.Null(traceParent); } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs index f3a2af7d..4a67c81b 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs @@ -247,4 +247,4 @@ private sealed class FakeGrainBase : IGrainBase public Task OnDeactivateAsync(DeactivationReason reason, CancellationToken cancellationToken) => Task.CompletedTask; } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs index 636ea6e1..aa83391e 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs @@ -363,4 +363,4 @@ public sealed class ImplicitStreamManagerTestState [Id(2)] public int DeliveryCount { get; set; } -} +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs index 6fbd1e2c..6296cd73 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs @@ -258,5 +258,4 @@ public override int CompareTo(StreamSequenceToken? other) return EventIndex.CompareTo(other.EventIndex); } } -} - +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs index 9ba54cb0..ccb2a5d3 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs @@ -357,4 +357,4 @@ private static MessageTracker CreateTracker(OutboxSequenceToken outbox, DateTime return tracker; } -} +} \ No newline at end of file From c842279a28f0f40958c6dcb9854ad213c801e609 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 16:02:24 +0000 Subject: [PATCH 33/82] feat(om)!: make stream resume tokens configurable Make MessageTracker fully optional for StreamManager registration and let each implicit or explicit subscription opt out of passing the tracked resume token to Orleans. Remove the StreamCursor constructor that accepted StreamId so cursors are built from the stream namespace directly. Add tests and docs for the resume-token policy and include the outbox postman API plan. BREAKING CHANGE: StreamCursor no longer exposes a constructor that accepts StreamId; construct it with the stream namespace string instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 13 ++ Egil.Orleans.Messaging/api-design.md | 36 +++- .../outbox-postman-api-plan.md | 202 ++++++++++++++++++ .../Streams/StreamCursor.cs | 10 - .../Streams/StreamManager.cs | 71 +++--- .../Streams/StreamManagerExtensions.cs | 2 +- .../Streams/StreamCursorJsonConverterTests.cs | 20 +- .../Streams/StreamCursorTests.cs | 12 +- .../Streams/StreamManagerResumeTests.cs | 79 ++++++- .../MessageTrackerJsonConverterTests.cs | 8 +- .../Tracking/MessageTrackerTests.cs | 48 ++--- 11 files changed, 403 insertions(+), 98 deletions(-) create mode 100644 Egil.Orleans.Messaging/outbox-postman-api-plan.md diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index c7b60456..706a8246 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -164,6 +164,19 @@ streamManager = this.RegisterStreamManager(state.State.Tracker) await streamManager.EnsureExplicitSubscriptionsAsync(cancellationToken); ``` +Tracked resume tokens are a per-subscription choice. The default is to pass +the previous token when a tracker snapshot is supplied. Opt out when a +subscription should attach without a resume token: + +```csharp +streamManager = this.RegisterStreamManager(state.State.Tracker) + .ConfigureExplicitSubscription( + "StreamProvider", + "prices", + HandlePriceChangedAsync, + useTrackedResumeToken: false); +``` + ```csharp this.RegisterStreamManager() .ConfigureImplicitSubscription( diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 7786d5bc..c40d54c3 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -573,7 +573,7 @@ Receiver-side, persisted dedup state. Tracks high-water position from each upstream source. Two source kinds: - **Orleans streams** — keyed by `StreamId`; position is a - `StreamCursor` wrapping `(StreamId, StreamSequenceToken)`. + `StreamCursor` wrapping `(stream namespace, StreamSequenceToken)`. - **Outbox messages** — keyed by sender `GrainId`; position is an `OutboxSequenceToken`. @@ -741,24 +741,28 @@ public sealed class StreamManager public StreamManager ConfigureImplicitSubscription( string streamNamespace, Func onNextAsync, - Action? onError = default); + Action? onError = default, + bool useTrackedResumeToken = true); public StreamManager ConfigureImplicitSubscription( string streamNamespace, Func onNextAsync, - Action? onError = default); + Action? onError = default, + bool useTrackedResumeToken = true); public StreamManager ConfigureExplicitSubscription( string streamProviderName, string streamNamespace, Func onNextAsync, - Action? onError = default); + Action? onError = default, + bool useTrackedResumeToken = true); public StreamManager ConfigureExplicitSubscription( string streamProviderName, string streamNamespace, Func onNextAsync, - Action? onError = default); + Action? onError = default, + bool useTrackedResumeToken = true); public Task ResumeExplicitSubscriptionsAsync( CancellationToken cancellationToken = default); @@ -790,6 +794,14 @@ attaches handlers without supplying resume tokens. Pass the activation-time tracker snapshot only when the grain wants `StreamManager` to resume from persisted cursors. +Each configured subscription independently decides whether `StreamManager` +passes the tracked cursor token into Orleans resume/subscribe APIs. The +default is `useTrackedResumeToken: true` for compatibility: if a tracker +snapshot has a cursor for the stream, the previous token is supplied. Set +`useTrackedResumeToken: false` when the grain wants Orleans to attach the +handler without a resume token for that subscription, even if a tracker +snapshot is available. + ### Typical implicit wiring ```csharp @@ -800,7 +812,7 @@ public override async Task OnActivateAsync(CancellationToken ct) this.RegisterStreamManager(state.Tracker) .ConfigureImplicitSubscription("electricity-prices", HandlePriceTickAsync, LogStreamError) - .ConfigureImplicitSubscription("tariff-events", HandleTariffChangedAsync); + .ConfigureImplicitSubscription("tariff-events", HandleTariffChangedAsync, useTrackedResumeToken: false); } ``` @@ -871,7 +883,8 @@ group. Users only specify it for inline lambdas or ambiguous method groups. creates the typed handle with `factory.Create()`, and calls `ResumeAsync(observer, token)` to attach the handler. The token comes from the activation-time `MessageTracker` snapshot when a cursor exists for the - same provider and namespace. + same provider and namespace and that subscription has + `useTrackedResumeToken: true`. - Orleans controls activation and target stream identity. If the grain is not active, a matching stream event can activate it. - Implicit subscriptions do not show up as explicit handles from @@ -884,12 +897,12 @@ group. Users only specify it for inline lambdas or ambiguous method groups. type, handler, and error callback. - `ResumeExplicitSubscriptionsAsync(...)` resumes all existing durable handles for each configured explicit stream from the activation-time - `MessageTracker` cursor when one exists. It never creates a new - subscription. + `MessageTracker` cursor when one exists and that subscription has + `useTrackedResumeToken: true`. It never creates a new subscription. - `EnsureExplicitSubscriptionsAsync(...)` resumes existing handles when present. If none exist for a configured explicit stream, it creates exactly one explicit subscription with `SubscribeAsync(observer, token)`, using the - tracked cursor token when available. + tracked cursor token when available and enabled for that subscription. - Repeated calls to `EnsureExplicitSubscriptionsAsync(...)` are idempotent: once a durable explicit handle exists, later calls resume it instead of creating duplicates. @@ -905,7 +918,8 @@ group. Users only specify it for inline lambdas or ambiguous method groups. - The cursor includes the stream namespace, provider name when known, and the delivered Orleans sequence token. - `MessageTracker` provides provider-aware resume tokens for both implicit - handle resume and explicit subscribe/resume. + handle resume and explicit subscribe/resume when a subscription opts into + tracked resume tokens. - For explicit streams, the durable Orleans subscription handle is still the primary subscription identity. `MessageTracker` remains the application-level dedup and high-water marker; handlers update it after diff --git a/Egil.Orleans.Messaging/outbox-postman-api-plan.md b/Egil.Orleans.Messaging/outbox-postman-api-plan.md new file mode 100644 index 00000000..49f982d7 --- /dev/null +++ b/Egil.Orleans.Messaging/outbox-postman-api-plan.md @@ -0,0 +1,202 @@ +# Outbox Postman API Plan + +## Goal + +Make `OutboxProcessor` less dependent on inline grain lambdas by +introducing reusable postman services. Grains should declare which postmen +handle which outbox message types, while delivery implementation lives in DI +services that can be tested outside Orleans. + +## Core Decisions + +- Add `IPostman` as the delivery abstraction. +- Keep current `AddPostman(...)` lambda overloads for local/simple cases. +- Add `OutboxProcessor.AddPostman(string postmanName)` to + resolve keyed `IPostman` from the grain activation service + provider. +- Use the grain-scoped service provider. `OutboxProcessor` must not create or + dispose child scopes. +- Register postmen through explicit outbox APIs named `AddOutboxPostman`, not + generic `AddPostman` service-registration APIs. +- Allow one concrete postman to implement multiple `IPostman` + interfaces and register all of them with one call. +- Prefer attribute-based postman names for discovery and one-line + registration. +- Keep `OutboxProcessor` responsible for orchestration, timeout/cancellation, + retry scheduling, acknowledgements, and telemetry. +- Keep `IPostman` responsible only for delivery work under the + processor-owned cancellation budget. + +## Proposed Public API + +```csharp +public interface IPostman + where TMessage : notnull +{ + ValueTask PostAsync( + TMessage message, + CancellationToken cancellationToken); +} + +[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = false)] +public sealed class OutboxPostmanAttribute(string name) : Attribute +{ + public string Name { get; } = name; +} +``` + +```csharp +public sealed partial class OutboxProcessor + where TOutbox : notnull +{ + public OutboxProcessor AddPostman( + string postmanName) + where TMessage : TOutbox; +} +``` + +```csharp +namespace Microsoft.Extensions.DependencyInjection; + +public static class OutboxPostmanServiceCollectionExtensions +{ + extension(IServiceCollection services) + { + public IServiceCollection AddOutboxPostman() + where TPostman : class; + + public IServiceCollection AddOutboxPostman( + string postmanName) + where TPostman : class; + + public IServiceCollection AddOutboxPostman( + string postmanName) + where TMessage : notnull + where TPostman : class, IPostman; + } +} +``` + +## Registration Behavior + +`AddOutboxPostman()`: + +- Reads `[OutboxPostman("name")]` from `TPostman`. +- Finds all closed `IPostman` interfaces implemented by `TPostman`. +- Registers `TPostman` as scoped. +- Registers every discovered `IPostman` as keyed scoped using the + attribute name. +- Throws if `TPostman` does not implement any closed `IPostman`. +- Throws if no attribute exists and no explicit name was provided. +- Rejects open generic postman types in v1. + +Important lifetime detail: + +```csharp +services.AddScoped(); + +services.AddKeyedScoped( + typeof(IPostman<>).MakeGenericType(messageType), + postmanName, + (sp, _) => sp.GetRequiredService()); +``` + +This lets one concrete scoped postman instance satisfy multiple message +contracts during one grain activation. + +## Usage Examples + +```csharp +[OutboxPostman("eventhub")] +public sealed class EventHubOrderPostman : + IPostman, + IPostman +{ + public ValueTask PostAsync(OrderPlaced message, CancellationToken ct) + { + // Send to Event Hubs. + } + + public ValueTask PostAsync(OrderCancelled message, CancellationToken ct) + { + // Send to Event Hubs. + } +} +``` + +```csharp +services.AddOutboxPostman(); +``` + +```csharp +outboxProcessor = this.RegisterOutboxProcessor(options) + .AddPostman("eventhub") + .AddPostman("eventhub"); +``` + +## Postman Execution Contract + +`OutboxProcessor` owns delivery budget. A postman may retry internally, use SDK +retry policies, or perform target-specific backoff, but it must: + +- Pass the cancellation token to async SDK/API calls. +- Stop promptly when cancellation is requested. +- Let `OperationCanceledException` escape. +- Avoid acknowledging or removing outbox items. +- Avoid mutating grain state. +- Be idempotent when the external target can receive duplicate attempts. + +Processor behavior: + +- Successful `PostAsync` call marks the item as posted. +- Exceptions are treated as failed delivery and reconciled through existing + retry behavior. +- Cancellation/timeout remains processor-owned and leaves the item pending. +- Processor telemetry measures total postman call duration. + +## Built-In Postman Helpers + +Add later as convenience APIs, implemented on top of `AddPostmanCore`. + +Stream postman: + +```csharp +public OutboxProcessor AddStreamPostman( + string streamProviderName, + Func streamId) + where TMessage : TOutbox; + +public OutboxProcessor AddStreamPostman( + string streamProviderName, + Func streamId, + Func project) + where TMessage : TOutbox; +``` + +Grain postman: + +```csharp +public OutboxProcessor AddGrainPostman( + Func resolveGrain, + Func call) + where TMessage : TOutbox + where TGrain : IGrain; +``` + +These helpers are optional and should not block the initial `IPostman` +design. + +## Tests + +- `AddOutboxPostman()` registers all implemented + `IPostman` contracts. +- Multiple `IPostman` contracts on one concrete postman resolve to + one scoped concrete instance. +- Missing `[OutboxPostman]` without explicit name throws. +- Concrete type with no `IPostman` contracts throws. +- `OutboxProcessor.AddPostman("name")` resolves keyed postman from + grain activation services. +- Successful keyed postman call acknowledges item. +- Throwing keyed postman goes through existing retry/reconcile path. +- Cancellation from processor propagates to postman and leaves item pending. +- Existing lambda-based `AddPostman(...)` behavior remains unchanged. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs index 9ae773c5..ade12aaf 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs @@ -56,16 +56,6 @@ public sealed record StreamCursor( [property: Id(1)] StreamSequenceToken? Token, [property: Id(2)] string? ProviderName = null) { - /// - /// Creates a cursor from an Orleans stream identity by using only its - /// namespace. Message tracking is scoped to one grain, so the stream key - /// portion of is redundant for dedup. - /// - public StreamCursor(StreamId streamId, StreamSequenceToken? token) - : this(streamId.GetNamespace() ?? throw new ArgumentException("StreamId must have a namespace.", nameof(streamId)), token) - { - } - /// /// Attempts to extract the broker-side enqueue time from the underlying /// . diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs index 6c1970b1..b7219089 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs @@ -15,7 +15,7 @@ namespace Egil.Orleans.Messaging.Streams; public sealed class StreamManager : IStreamManagerComponent { private readonly IGrainBase owner; - private readonly MessageTracker trackerSnapshot; + private readonly MessageTracker? trackerSnapshot; private readonly Func getStreamProvider; private readonly Func getStreamId; private readonly ILogger logger; @@ -26,7 +26,7 @@ public sealed class StreamManager : IStreamManagerComponent private StreamManager( IGrainBase owner, - MessageTracker trackerSnapshot, + MessageTracker? trackerSnapshot, Func getStreamProvider, Func getStreamId, ILogger logger) @@ -43,10 +43,9 @@ private StreamManager( /// internal static StreamManager Create( IGrainBase owner, - MessageTracker trackerSnapshot) + MessageTracker? trackerSnapshot) { ArgumentNullException.ThrowIfNull(owner); - ArgumentNullException.ThrowIfNull(trackerSnapshot); var services = owner.GrainContext.ActivationServices; var loggerFactory = services.GetService(); @@ -56,7 +55,7 @@ internal static StreamManager Create( var manager = Create( owner, trackerSnapshot, - streamProviderName => services.GetRequiredKeyedService(streamProviderName), + services.GetRequiredKeyedService, streamNamespace => CreateStreamId(owner, streamNamespace), logger); @@ -66,13 +65,12 @@ internal static StreamManager Create( internal static StreamManager Create( IGrainBase owner, - MessageTracker trackerSnapshot, + MessageTracker? trackerSnapshot, Func getStreamProvider, Func getStreamId, ILogger logger) { ArgumentNullException.ThrowIfNull(owner); - ArgumentNullException.ThrowIfNull(trackerSnapshot); ArgumentNullException.ThrowIfNull(getStreamProvider); ArgumentNullException.ThrowIfNull(getStreamId); ArgumentNullException.ThrowIfNull(logger); @@ -86,7 +84,8 @@ internal static StreamManager Create( public StreamManager ConfigureImplicitSubscription( string streamNamespace, Func onNextAsync, - Action? onError = default) + Action? onError = default, + bool useTrackedResumeToken = true) { ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); ArgumentNullException.ThrowIfNull(onNextAsync); @@ -94,7 +93,7 @@ public StreamManager ConfigureImplicitSubscription( if (!implicitSubscriptions.TryAdd( streamNamespace, - new ImplicitSubscription(streamNamespace, onNextAsync, onError))) + new ImplicitSubscription(streamNamespace, onNextAsync, onError, useTrackedResumeToken))) { throw new InvalidOperationException( $"An implicit stream subscription for namespace '{streamNamespace}' has already been configured."); @@ -103,18 +102,20 @@ public StreamManager ConfigureImplicitSubscription( return this; } - /// + /// public StreamManager ConfigureImplicitSubscription( string streamNamespace, Func onNextAsync, - Action? onError = default) + Action? onError = default, + bool useTrackedResumeToken = true) { ArgumentNullException.ThrowIfNull(onNextAsync); return ConfigureImplicitSubscription( streamNamespace, (item, cursor) => new ValueTask(onNextAsync(item, cursor)), - onError); + onError, + useTrackedResumeToken); } /// @@ -124,7 +125,8 @@ public StreamManager ConfigureExplicitSubscription( string streamProviderName, string streamNamespace, Func onNextAsync, - Action? onError = default) + Action? onError = default, + bool useTrackedResumeToken = true) { ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); @@ -143,17 +145,19 @@ public StreamManager ConfigureExplicitSubscription( streamProviderName, streamNamespace, onNextAsync, - onError)); + onError, + useTrackedResumeToken)); return this; } - /// + /// public StreamManager ConfigureExplicitSubscription( string streamProviderName, string streamNamespace, Func onNextAsync, - Action? onError = default) + Action? onError = default, + bool useTrackedResumeToken = true) { ArgumentNullException.ThrowIfNull(onNextAsync); @@ -161,7 +165,8 @@ public StreamManager ConfigureExplicitSubscription( streamProviderName, streamNamespace, (item, cursor) => new ValueTask(onNextAsync(item, cursor)), - onError); + onError, + useTrackedResumeToken); } /// @@ -219,6 +224,7 @@ private async Task> ResumeImplicitAsync string streamNamespace, Func onNextAsync, Action? onError, + bool useTrackedResumeToken, IStreamSubscriptionHandleFactory handleFactory) { try @@ -231,7 +237,7 @@ private async Task> ResumeImplicitAsync onError); var handle = await handleFactory.Create().ResumeAsync( observer, - GetResumeToken(handleFactory.ProviderName, streamNamespace)); + GetResumeToken(handleFactory.ProviderName, streamNamespace, useTrackedResumeToken)); subscriptionHandles.Add(handle); MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); @@ -250,6 +256,7 @@ private async Task ResumeExplicitAsync( string streamNamespace, Func onNextAsync, Action? onError, + bool useTrackedResumeToken, bool createIfMissing, CancellationToken cancellationToken) { @@ -269,7 +276,7 @@ private async Task ResumeExplicitAsync( onError); var handle = await stream.SubscribeAsync( observer, - GetResumeToken(streamProviderName, streamNamespace), + GetResumeToken(streamProviderName, streamNamespace, useTrackedResumeToken), filterData: null); subscriptionHandles.Add(handle); @@ -289,7 +296,7 @@ private async Task ResumeExplicitAsync( onError); var resumedHandle = await handle.ResumeAsync( observer, - GetResumeToken(handle.ProviderName, streamNamespace)); + GetResumeToken(handle.ProviderName, streamNamespace, useTrackedResumeToken)); subscriptionHandles.Add(resumedHandle); MessagingTelemetry.RecordStreamSubscription(streamNamespace, "established"); @@ -316,11 +323,19 @@ private IAsyncStream GetExplicitStream( return streamProvider.GetStream(streamId); } - private StreamSequenceToken? GetResumeToken(string? streamProviderName, string streamNamespace) + private StreamSequenceToken? GetResumeToken( + string? streamProviderName, + string streamNamespace, + bool useTrackedResumeToken) { + if (!useTrackedResumeToken) + { + return null; + } + var cursor = string.IsNullOrWhiteSpace(streamProviderName) - ? trackerSnapshot.LatestStream(streamNamespace) - : trackerSnapshot.LatestStream(streamProviderName, streamNamespace); + ? trackerSnapshot?.LatestStream(streamNamespace) + : trackerSnapshot?.LatestStream(streamProviderName, streamNamespace); return cursor?.Token; } @@ -456,18 +471,20 @@ private interface IExplicitSubscription private sealed class ImplicitSubscription( string streamNamespace, Func onNextAsync, - Action? onError) + Action? onError, + bool useTrackedResumeToken) : IImplicitSubscription { public Task ResumeAsync(StreamManager manager, IStreamSubscriptionHandleFactory handleFactory) => - manager.ResumeImplicitAsync(streamNamespace, onNextAsync, onError, handleFactory); + manager.ResumeImplicitAsync(streamNamespace, onNextAsync, onError, useTrackedResumeToken, handleFactory); } private sealed class ExplicitSubscription( string streamProviderName, string streamNamespace, Func onNextAsync, - Action? onError) + Action? onError, + bool useTrackedResumeToken) : IExplicitSubscription { public string StreamProviderName => streamProviderName; @@ -480,6 +497,7 @@ public Task ResumeExistingAsync(StreamManager manager, CancellationToken cancell streamNamespace, onNextAsync, onError, + useTrackedResumeToken, createIfMissing: false, cancellationToken); @@ -489,6 +507,7 @@ public Task EnsureAsync(StreamManager manager, CancellationToken cancellationTok streamNamespace, onNextAsync, onError, + useTrackedResumeToken, createIfMissing: true, cancellationToken); } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs index 887f466a..58f1c976 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManagerExtensions.cs @@ -45,7 +45,7 @@ public static class StreamManagerExtensions /// public StreamManager RegisterStreamManager(MessageTracker? trackerSnapshot = null) { - return StreamManager.Create(grain, trackerSnapshot ?? new MessageTracker()); + return StreamManager.Create(grain, trackerSnapshot); } } } \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs index 69b9da5f..082bbcc5 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs @@ -21,8 +21,7 @@ public void StreamCursor_is_decorated_with_stream_cursor_json_converter() [Fact] public void JsonSerializer_round_trips_stream_cursor_with_event_sequence_token() { - var streamId = StreamId.Create("orders", "one"); - var cursor = new StreamCursor(streamId, new EventSequenceToken(7, 1)); + var cursor = new StreamCursor("orders", new EventSequenceToken(7, 1)); var json = JsonSerializer.Serialize(cursor); var roundTripped = JsonSerializer.Deserialize(json); @@ -34,8 +33,7 @@ public void JsonSerializer_round_trips_stream_cursor_with_event_sequence_token() [Fact] public void JsonSerializer_round_trips_stream_cursor_without_token() { - var streamId = StreamId.Create("orders", "no-token"); - var cursor = new StreamCursor(streamId, null); + var cursor = new StreamCursor("orders", null); var json = JsonSerializer.Serialize(cursor); var roundTripped = JsonSerializer.Deserialize(json); @@ -48,8 +46,7 @@ public void JsonSerializer_round_trips_stream_cursor_without_token() [Fact] public void JsonSerializer_round_trips_stream_cursor_with_event_sequence_token_v2() { - var streamId = StreamId.Create("orders", "event-v2"); - var cursor = new StreamCursor(streamId, new EventSequenceTokenV2(8, 2)); + var cursor = new StreamCursor("orders", new EventSequenceTokenV2(8, 2)); var json = JsonSerializer.Serialize(cursor); var roundTripped = JsonSerializer.Deserialize(json); @@ -61,8 +58,7 @@ public void JsonSerializer_round_trips_stream_cursor_with_event_sequence_token_v [Fact] public void JsonSerializer_round_trips_stream_cursor_with_event_hub_token() { - var streamId = StreamId.Create("orders", "eh"); - var cursor = new StreamCursor(streamId, new EventHubSequenceToken("41", 9, 2)); + var cursor = new StreamCursor("orders", new EventHubSequenceToken("41", 9, 2)); var json = JsonSerializer.Serialize(cursor); var roundTripped = JsonSerializer.Deserialize(json); @@ -74,8 +70,7 @@ public void JsonSerializer_round_trips_stream_cursor_with_event_hub_token() [Fact] public void JsonSerializer_round_trips_stream_cursor_with_event_hub_token_v2() { - var streamId = StreamId.Create("orders", "eh-v2"); - var cursor = new StreamCursor(streamId, new EventHubSequenceTokenV2("42", 10, 3)); + var cursor = new StreamCursor("orders", new EventHubSequenceTokenV2("42", 10, 3)); var json = JsonSerializer.Serialize(cursor); var roundTripped = JsonSerializer.Deserialize(json); @@ -87,9 +82,8 @@ public void JsonSerializer_round_trips_stream_cursor_with_event_hub_token_v2() [Fact] public void JsonSerializer_round_trips_stream_cursor_with_enriched_event_hub_token() { - var streamId = StreamId.Create("orders", "eh-enriched"); var cursor = new StreamCursor( - streamId, + "orders", new EnrichedEventHubSequenceToken( "43", 11, @@ -128,7 +122,7 @@ public void JsonSerializer_throws_for_stream_cursor_payload_with_unknown_token_k public void JsonSerializer_throws_for_custom_token_type() { var cursor = new StreamCursor( - StreamId.Create("orders", "custom"), + "orders", new UnsupportedCustomSequenceToken(21, 3)); var exception = Assert.Throws(() => JsonSerializer.Serialize(cursor)); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs index 34809e87..9d4ae8af 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs @@ -9,7 +9,7 @@ public void TryGetEnqueuedTime_returns_true_and_value_for_enriched_event_hub_tok { var enqueuedTime = new DateTimeOffset(2026, 5, 24, 12, 30, 0, TimeSpan.Zero); var cursor = new StreamCursor( - StreamId.Create("orders", "one"), + "orders", new EnrichedEventHubSequenceToken("42", 10, 0, enqueuedTime, "provider-a")); var found = cursor.TryGetEnqueuedTime(out var actual); @@ -22,7 +22,7 @@ public void TryGetEnqueuedTime_returns_true_and_value_for_enriched_event_hub_tok public void TryGetEnqueuedTime_returns_false_for_non_enriched_token() { var cursor = new StreamCursor( - StreamId.Create("orders", "one"), + "orders", new EventSequenceToken(10)); var found = cursor.TryGetEnqueuedTime(out var actual); @@ -35,7 +35,7 @@ public void TryGetEnqueuedTime_returns_false_for_non_enriched_token() public void TryGetProviderName_returns_true_and_value_for_enriched_event_hub_token() { var cursor = new StreamCursor( - StreamId.Create("orders", "one"), + "orders", new EnrichedEventHubSequenceToken( "42", 10, @@ -53,7 +53,7 @@ public void TryGetProviderName_returns_true_and_value_for_enriched_event_hub_tok public void TryGetProviderName_returns_false_for_non_enriched_token() { var cursor = new StreamCursor( - StreamId.Create("orders", "one"), + "orders", new EventSequenceToken(10)); var found = cursor.TryGetProviderName(out var providerName); @@ -66,7 +66,7 @@ public void TryGetProviderName_returns_false_for_non_enriched_token() public void TryGetTraceParent_returns_true_and_value_when_present_on_enriched_event_hub_token() { var cursor = new StreamCursor( - StreamId.Create("orders", "one"), + "orders", new EnrichedEventHubSequenceToken( "42", 10, @@ -85,7 +85,7 @@ public void TryGetTraceParent_returns_true_and_value_when_present_on_enriched_ev public void TryGetTraceParent_returns_false_when_missing_on_enriched_event_hub_token() { var cursor = new StreamCursor( - StreamId.Create("orders", "one"), + "orders", new EnrichedEventHubSequenceToken( "42", 10, diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs index 4a67c81b..6685340c 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs @@ -22,6 +22,39 @@ await manager Assert.Equal(1, stream.SubscribeCount); } + [Fact] + public async Task EnsureExplicitSubscriptionsAsync_uses_null_resume_token_when_tracker_is_omitted() + { + var stream = new FakeStream("provider-a", StreamId.Create("orders", "one")); + var manager = CreateManager(null, stream); + + await manager + .ConfigureExplicitSubscription("provider-a", "orders", static (_, _) => ValueTask.CompletedTask) + .EnsureExplicitSubscriptionsAsync(TestContext.Current.CancellationToken); + + Assert.Null(stream.SubscribeToken); + Assert.Equal(1, stream.SubscribeCount); + } + + [Fact] + public async Task EnsureExplicitSubscriptionsAsync_uses_null_resume_token_when_subscription_opts_out() + { + var stream = new FakeStream("provider-a", StreamId.Create("orders", "one")); + var tracker = CreateTracker("provider-a", "orders", sequenceNumber: 7); + var manager = CreateManager(tracker, stream); + + await manager + .ConfigureExplicitSubscription( + "provider-a", + "orders", + static (_, _) => ValueTask.CompletedTask, + useTrackedResumeToken: false) + .EnsureExplicitSubscriptionsAsync(TestContext.Current.CancellationToken); + + Assert.Null(stream.SubscribeToken); + Assert.Equal(1, stream.SubscribeCount); + } + [Fact] public async Task ResumeExplicitSubscriptionsAsync_uses_tracker_resume_token_when_resuming_existing_handle() { @@ -40,6 +73,28 @@ await manager Assert.Equal(0, stream.SubscribeCount); } + [Fact] + public async Task ResumeExplicitSubscriptionsAsync_uses_null_resume_token_when_subscription_opts_out() + { + var handle = new FakeSubscriptionHandle("provider-a", StreamId.Create("orders", "one")); + var stream = new FakeStream("provider-a", StreamId.Create("orders", "one")); + stream.Handles.Add(handle); + var tracker = CreateTracker("provider-a", "orders", sequenceNumber: 7); + var manager = CreateManager(tracker, stream); + + await manager + .ConfigureExplicitSubscription( + "provider-a", + "orders", + static (_, _) => ValueTask.CompletedTask, + useTrackedResumeToken: false) + .ResumeExplicitSubscriptionsAsync(TestContext.Current.CancellationToken); + + Assert.Null(handle.ResumeToken); + Assert.Equal(1, handle.ResumeCount); + Assert.Equal(0, stream.SubscribeCount); + } + [Fact] public async Task Implicit_subscription_resume_uses_tracker_resume_token() { @@ -59,8 +114,30 @@ public async Task Implicit_subscription_resume_uses_tracker_resume_token() Assert.Equal(1, handleFactory.StringHandle.ResumeCount); } + [Fact] + public async Task Implicit_subscription_resume_uses_null_resume_token_when_subscription_opts_out() + { + var streamId = StreamId.Create("orders", "one"); + var tracker = CreateTracker("provider-a", "orders", sequenceNumber: 7); + var handleFactory = new FakeStreamSubscriptionHandleFactory( + "provider-a", + streamId, + new FakeSubscriptionHandle("provider-a", streamId)); + var manager = CreateManager(tracker, new FakeStream("provider-a", streamId)); + + await ((IStreamManagerComponent)manager + .ConfigureImplicitSubscription( + "orders", + static (_, _) => ValueTask.CompletedTask, + useTrackedResumeToken: false)) + .OnSubscribedAsync(handleFactory); + + Assert.Null(handleFactory.StringHandle.ResumeToken); + Assert.Equal(1, handleFactory.StringHandle.ResumeCount); + } + private static StreamManager CreateManager( - MessageTracker tracker, + MessageTracker? tracker, FakeStream stream) { var owner = new FakeGrainBase(); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs index 6296cd73..a9fafab8 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs @@ -31,14 +31,14 @@ public void JsonSerializer_round_trips_message_tracker_stream_tokens(StreamSeque var streamId = StreamId.Create("orders", token.GetType().Name); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(new ManualTimeProvider(now)); - tracker.ProcessMessage(new StreamCursor(streamId, token), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", token), out tracker); var json = JsonSerializer.Serialize(tracker); var roundTripped = JsonSerializer.Deserialize(json); Assert.NotNull(roundTripped); Assert.Equal(tracker, roundTripped); - Assert.Equal(new StreamCursor(streamId, token), roundTripped.LatestStream(streamId)); + Assert.Equal(new StreamCursor("orders", token), roundTripped.LatestStream(streamId)); } [Theory] @@ -69,7 +69,7 @@ public void JsonSerializer_throws_for_unsupported_stream_token_type() var streamId = StreamId.Create("orders", "unsupported"); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(new ManualTimeProvider(now)); - tracker.ProcessMessage(new StreamCursor(streamId, new UnsupportedSequenceToken(1, 0)), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", new UnsupportedSequenceToken(1, 0)), out tracker); Assert.Throws(() => JsonSerializer.Serialize(tracker)); } @@ -114,7 +114,7 @@ public void JsonSerializer_deserializes_tracked_stream_without_sequence_token() var roundTripped = JsonSerializer.Deserialize(json); Assert.NotNull(roundTripped); - Assert.Equal(new StreamCursor(streamId, null), roundTripped.LatestStream(streamId)); + Assert.Equal(new StreamCursor("orders", null), roundTripped.LatestStream(streamId)); } [Fact] diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs index ccb2a5d3..4179dbb8 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs @@ -9,7 +9,7 @@ public sealed class MessageTrackerTests public void ProcessMessage_accepts_first_stream_cursor_and_tracks_latest_position() { var streamId = StreamId.Create("orders", "one"); - var cursor = new StreamCursor(streamId, new EventSequenceToken(7)); + var cursor = new StreamCursor("orders", new EventSequenceToken(7)); var time = new ManualTimeProvider(new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero)); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(time); @@ -48,7 +48,7 @@ public void Evict_removes_stream_and_outbox_entries_at_or_before_cutoff() var time = new ManualTimeProvider(now); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(time); - tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7)), out tracker); tracker.ProcessMessage(new OutboxSequenceToken(1, sender, now, now), out tracker); var evicted = tracker.Evict(now); @@ -65,9 +65,9 @@ public void ProcessMessage_rejects_stream_cursor_when_token_is_not_newer() var streamId = StreamId.Create("orders", "one"); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(time); - tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7)), out tracker); - var accepted = tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out var next); + var accepted = tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7)), out var next); Assert.False(accepted); Assert.Same(tracker, next); @@ -81,19 +81,18 @@ public void ProcessMessage_accepts_stream_cursor_when_token_is_newer() var streamId = StreamId.Create("orders", "one"); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(time); - tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7)), out tracker); - var accepted = tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(8)), out var next); + var accepted = tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(8)), out var next); Assert.True(accepted); - Assert.Equal(new StreamCursor(streamId, new EventSequenceToken(8)), next.LatestStream(streamId)); + Assert.Equal(new StreamCursor("orders", new EventSequenceToken(8)), next.LatestStream(streamId)); } [Fact] public void LatestStream_with_provider_returns_matching_provider_cursor() { var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var streamId = StreamId.Create("orders", "one"); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(new ManualTimeProvider(now)); tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7), "provider-a"), out tracker); @@ -108,7 +107,6 @@ public void LatestStream_with_provider_returns_matching_provider_cursor() public void LatestStream_with_provider_does_not_fall_back_to_different_provider() { var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var streamId = StreamId.Create("orders", "one"); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(new ManualTimeProvider(now)); tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7), "provider-a"), out tracker); @@ -126,12 +124,12 @@ public void ProcessMessage_accepts_non_null_stream_token_after_null_initial_posi var streamId = StreamId.Create("orders", "one"); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(time); - tracker.ProcessMessage(new StreamCursor(streamId, null), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", null), out tracker); - var accepted = tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(1)), out var next); + var accepted = tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(1)), out var next); Assert.True(accepted); - Assert.Equal(new StreamCursor(streamId, new EventSequenceToken(1)), next.LatestStream(streamId)); + Assert.Equal(new StreamCursor("orders", new EventSequenceToken(1)), next.LatestStream(streamId)); } [Fact] @@ -142,9 +140,9 @@ public void ProcessMessage_rejects_null_stream_token_after_position_is_tracked() var streamId = StreamId.Create("orders", "one"); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(time); - tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(1)), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(1)), out tracker); - var accepted = tracker.ProcessMessage(new StreamCursor(streamId, null), out var next); + var accepted = tracker.ProcessMessage(new StreamCursor("orders", null), out var next); Assert.False(accepted); Assert.Same(tracker, next); @@ -211,7 +209,7 @@ public void EvictStreams_only_removes_stream_entries() var time = new ManualTimeProvider(now); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(time); - tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7)), out tracker); tracker.ProcessMessage(new OutboxSequenceToken(1, sender, now, now), out tracker); var evicted = tracker.EvictStreams(now); @@ -229,12 +227,12 @@ public void EvictOutboxes_only_removes_outbox_entries() var time = new ManualTimeProvider(now); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(time); - tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7)), out tracker); tracker.ProcessMessage(new OutboxSequenceToken(1, sender, now, now), out tracker); var evicted = tracker.EvictOutboxes(now); - Assert.Equal(new StreamCursor(streamId, new EventSequenceToken(7)), evicted.LatestStream(streamId)); + Assert.Equal(new StreamCursor("orders", new EventSequenceToken(7)), evicted.LatestStream(streamId)); Assert.Null(evicted.LatestOutbox(sender)); } @@ -246,12 +244,12 @@ public void Evict_stream_entry_respects_cutoff_for_targeted_remove() var time = new ManualTimeProvider(now); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(time); - tracker.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7)), out tracker); var notEvicted = tracker.Evict(streamId, now.AddTicks(-1)); var evicted = tracker.Evict(streamId, now); - Assert.Equal(new StreamCursor(streamId, new EventSequenceToken(7)), notEvicted.LatestStream(streamId)); + Assert.Equal(new StreamCursor("orders", new EventSequenceToken(7)), notEvicted.LatestStream(streamId)); Assert.Null(evicted.LatestStream(streamId)); } @@ -277,13 +275,12 @@ public void Equals_returns_false_when_tracked_entries_differ() { var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); var sender = GrainId.Create("test/sender", "one"); - var streamId = StreamId.Create("orders", "one"); var time = new ManualTimeProvider(now); var left = new MessageTracker(); var right = new MessageTracker(); left.RegisterTimeProvider(time); right.RegisterTimeProvider(time); - left.ProcessMessage(new StreamCursor(streamId, new EventSequenceToken(7)), out left); + left.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7)), out left); right.ProcessMessage(new OutboxSequenceToken(1, sender, now, now), out right); Assert.NotEqual(left, right); @@ -294,10 +291,10 @@ public void Equals_returns_false_when_stream_keys_differ_but_counts_match() { var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); var left = CreateTracker( - stream: new StreamCursor(StreamId.Create("orders", "one"), new EventSequenceToken(7)), + stream: new StreamCursor("orders", new EventSequenceToken(7)), received: now); var right = CreateTracker( - stream: new StreamCursor(StreamId.Create("invoices", "one"), new EventSequenceToken(7)), + stream: new StreamCursor("invoices", new EventSequenceToken(7)), received: now); Assert.NotEqual(left, right); @@ -307,9 +304,8 @@ public void Equals_returns_false_when_stream_keys_differ_but_counts_match() public void Equals_returns_false_when_stream_values_differ_but_keys_match() { var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var streamId = StreamId.Create("orders", "one"); - var left = CreateTracker(new StreamCursor(streamId, new EventSequenceToken(7)), now); - var right = CreateTracker(new StreamCursor(streamId, new EventSequenceToken(8)), now); + var left = CreateTracker(new StreamCursor("orders", new EventSequenceToken(7)), now); + var right = CreateTracker(new StreamCursor("orders", new EventSequenceToken(8)), now); Assert.NotEqual(left, right); } From 6145dd5477378b27346dc9f82e7b0a14d07dd4ac Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 16:12:28 +0000 Subject: [PATCH 34/82] feat(om): add stream token tracker overloads Allow MessageTracker callers to process stream sequence tokens directly with explicit stream namespace or provider-qualified namespace, and expose latest stream sequence token lookup for resume-token use cases. Clarify stream manager tests by using distinct provider-name and namespace constants for explicit subscriptions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 9 +- Egil.Orleans.Messaging/api-design.md | 41 +++++++-- .../Tracking/MessageTracker.cs | 75 +++++++++++++++++ .../MessagingTestClusterFixture.cs | 16 +--- .../Streams/StreamManagerTests.cs | 34 ++++++-- .../Tracking/MessageTrackerTests.cs | 83 +++++++++++++++++++ 6 files changed, 227 insertions(+), 31 deletions(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index 706a8246..87569d97 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -129,10 +129,10 @@ public sealed class OrderGrain : Grain, IOutboxGrain ## Receiver Dedup -`MessageTracker` accepts a message only when its stream cursor or outbox token advances the stored high-water mark: +`MessageTracker` accepts a message only when its stream token, stream cursor, or outbox token advances the stored high-water mark: ```csharp -if (!state.State.Tracker.ProcessMessage(token, out var tracker)) +if (!state.State.Tracker.ProcessMessage("prices", token, out var tracker)) { return; } @@ -140,6 +140,11 @@ if (!state.State.Tracker.ProcessMessage(token, out var tracker)) await state.WriteAsync(state.State with { Tracker = tracker }); ``` +Use `LatestStreamSequenceToken("prices")` when all you need is the previous +resume token. Keep using `LatestStream("prices")` when you need the full +cursor or must distinguish "no stream tracked" from "tracked stream with a +null token". + The tracker can also evict old sender or stream entries when your retention policy allows it. ## Streams diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index c40d54c3..722b1a1e 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -588,7 +588,7 @@ each upstream source. Two source kinds: [JsonConverter(typeof(MessageTrackerJsonConverter))] public sealed class MessageTracker { - [Id(0)] private ImmutableDictionary streams; + [Id(0)] private ImmutableDictionary streams; [Id(1)] private ImmutableDictionary outbox; // Non-persisted; no [Id]. Mutable. @@ -599,9 +599,22 @@ public sealed class MessageTracker public void RegisterTimeProvider(TimeProvider time) => this.time = time; public bool ProcessMessage(StreamCursor cursor, out MessageTracker next); + public bool ProcessMessage( + string streamNamespace, + StreamSequenceToken? token, + out MessageTracker next); + public bool ProcessMessage( + string streamProviderName, + string streamNamespace, + StreamSequenceToken? token, + out MessageTracker next); public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next); + public StreamCursor? LatestStream(string streamNamespace); + public StreamCursor? LatestStream(string streamProviderName, string streamNamespace); public StreamCursor? LatestStream(StreamId stream); + public StreamSequenceToken? LatestStreamSequenceToken(string streamNamespace); + public StreamSequenceToken? LatestStreamSequenceToken(string streamProviderName, string streamNamespace); public OutboxSequenceToken? LatestOutbox(GrainId sender); public MessageTracker Evict(DateTimeOffset olderThan); @@ -610,6 +623,11 @@ public sealed class MessageTracker public MessageTracker Evict(StreamId stream, DateTimeOffset olderThan); public MessageTracker Evict(GrainId sender, DateTimeOffset olderThan); + [GenerateSerializer] + private readonly record struct StreamSource( + [property: Id(0)] string StreamNamespace, + [property: Id(1)] string? ProviderName); + [GenerateSerializer] private readonly record struct StreamEntry( [property: Id(0)] StreamCursor LastPosition, @@ -627,12 +645,18 @@ public sealed class MessageTracker - Outbox identity: `OutboxSequenceToken.Sender` in the envelope. Payloads stay clean. -- Stream identity: `StreamId` from runtime. Sufficient under the - one-provider-per-namespace convention (code-review enforced, not - runtime enforced). This is Orleans's own constraint — the library - doesn't make it worse. +- Stream identity: stream namespace, plus provider name when available. + Stream keys are intentionally not part of `MessageTracker` state because + the tracker is scoped to one grain activation's durable state. + +### `ProcessMessage(StreamCursor)` and stream token semantics -### `ProcessMessage(StreamCursor)` semantics +Users can pass either a `StreamCursor` or the raw Orleans +`StreamSequenceToken` plus stream namespace. Provider-qualified overloads are +available when a grain consumes the same namespace from multiple providers. +The stream namespace stays required even for grains that currently subscribe +to one stream; making it optional would encode a fragile "single stream" +assumption into persisted high-water marks. | Prior entry | Decision | Effect | | -------------------------------- | -------- | ---------------------------------------------- | @@ -640,6 +664,11 @@ public sealed class MessageTracker | `cursor > stored.LastPosition` | Accept | Update position + received | | `cursor <= stored.LastPosition` | Reject | No change | +`LatestStreamSequenceToken(...)` is a convenience for resume-token lookup. +It returns `null` both when no stream is tracked and when the tracked cursor +has a null token; callers that need to distinguish those cases should use +`LatestStream(...)`. + ### `ProcessMessage(OutboxSequenceToken)` semantics | Prior entry | Comparison | Decision | Effect | diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs index f085628f..51a99ffe 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs @@ -134,6 +134,53 @@ public bool ProcessMessage(StreamCursor cursor, out MessageTracker next) return true; } + /// + /// Evaluates a stream message for acceptance. Returns true if the + /// advances past the stored high-water mark for + /// . + /// + /// The Orleans stream namespace within the grain. + /// The stream sequence token to evaluate. + /// + /// When accepted, a new with the updated + /// position. When rejected, equals this. + /// + /// true if accepted (new message); false if duplicate. + public bool ProcessMessage( + string streamNamespace, + StreamSequenceToken? token, + out MessageTracker next) + { + ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); + + return ProcessMessage(new StreamCursor(streamNamespace, token), out next); + } + + /// + /// Evaluates a provider-qualified stream message for acceptance. Returns + /// true if the advances past the stored + /// high-water mark for the provider and namespace. + /// + /// The Orleans stream provider name. + /// The Orleans stream namespace within the grain. + /// The stream sequence token to evaluate. + /// + /// When accepted, a new with the updated + /// position. When rejected, equals this. + /// + /// true if accepted (new message); false if duplicate. + public bool ProcessMessage( + string streamProviderName, + string streamNamespace, + StreamSequenceToken? token, + out MessageTracker next) + { + ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); + ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); + + return ProcessMessage(new StreamCursor(streamNamespace, token, streamProviderName), out next); + } + /// /// Evaluates an outbox message for acceptance. Returns true if the /// advances past the stored position or carries @@ -207,6 +254,20 @@ public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) return result; } + /// + /// Returns the last accepted for the + /// given , or null if no token is + /// tracked for that namespace. + /// + /// + /// A null result can mean either no stream has been tracked or the + /// latest tracked cursor has a null token. Use + /// when callers need to distinguish + /// those cases. + /// + public StreamSequenceToken? LatestStreamSequenceToken(string streamNamespace) => + LatestStream(streamNamespace)?.Token; + /// /// Returns the last accepted for the given /// and @@ -234,6 +295,20 @@ public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) : null; } + /// + /// Returns the last accepted for the + /// given provider and stream namespace, or null if no token is + /// tracked for that provider/namespace pair. + /// + /// + /// A null result can mean either no stream has been tracked or the + /// latest tracked cursor has a null token. Use + /// when callers need to + /// distinguish those cases. + /// + public StreamSequenceToken? LatestStreamSequenceToken(string streamProviderName, string streamNamespace) => + LatestStream(streamProviderName, streamNamespace)?.Token; + /// /// Returns the last accepted for the namespace /// in the given . diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs index df5fbb6a..38d69cf7 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs @@ -51,12 +51,6 @@ public TGrain GetUniqueGrain([CallerMemberName] string memberName = "") where TGrain : IGrain => (TGrain)GrainFactory.GetGrain(typeof(TGrain), Guid.NewGuid()); - public IAsyncStream GetStream(string streamNamespace, Guid key) - { - var provider = Cluster.Client.GetStreamProvider(streamNamespace); - return provider.GetStream(StreamId.Create(streamNamespace, key)); - } - public IAsyncStream GetStream(string providerName, string streamNamespace, Guid key) { var provider = Cluster.Client.GetStreamProvider(providerName); @@ -72,20 +66,14 @@ Task IGrainActivityWaiter.WaitForAssertionAsync( private static void AddStreamProviders(ISiloBuilder siloBuilder) { - siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.ValueTask); - siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Task); - siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Failure); - siloBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Resume); + siloBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Explicit); siloBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Implicit); siloBuilder.AddMemoryStreams(OutboxProcessorTestNamespaces.Events); } private static void AddStreamProviders(IClientBuilder clientBuilder) { - clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.ValueTask); - clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Task); - clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Failure); - clientBuilder.AddMemoryStreams(StreamManagerTestNamespaces.Resume); + clientBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Explicit); clientBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Implicit); clientBuilder.AddMemoryStreams(OutboxProcessorTestNamespaces.Events); } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs index aa83391e..d47d68e1 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs @@ -10,7 +10,10 @@ public async Task ConfigureExplicitSubscription_with_ValueTask_handler_delivers_ var grainKey = Guid.NewGuid(); var grain = fixture.GrainFactory.GetGrain(grainKey); await grain.EnsureActiveAsync(); - var stream = fixture.GetStream(StreamManagerTestNamespaces.ValueTask, grainKey); + var stream = fixture.GetStream( + StreamManagerTestProviderNames.Explicit, + StreamManagerTestNamespaces.ValueTask, + grainKey); await stream.OnNextAsync("value-task"); @@ -89,7 +92,10 @@ public async Task Explicit_subscription_does_not_duplicate_after_reactivation() grain = fixture.GrainFactory.GetGrain(grainKey); await grain.EnsureActiveAsync(); - var stream = fixture.GetStream(StreamManagerTestNamespaces.ValueTask, grainKey); + var stream = fixture.GetStream( + StreamManagerTestProviderNames.Explicit, + StreamManagerTestNamespaces.ValueTask, + grainKey); await stream.OnNextAsync("after-reactivation"); @@ -104,7 +110,10 @@ public async Task ConfigureExplicitSubscription_with_Task_handler_delivers_strea var grainKey = Guid.NewGuid(); var grain = fixture.GrainFactory.GetGrain(grainKey); await grain.EnsureActiveAsync(); - var stream = fixture.GetStream(StreamManagerTestNamespaces.Task, grainKey); + var stream = fixture.GetStream( + StreamManagerTestProviderNames.Explicit, + StreamManagerTestNamespaces.Task, + grainKey); await stream.OnNextAsync("task"); @@ -119,7 +128,10 @@ public async Task Default_error_handler_does_not_break_subscription_when_handler var grainKey = Guid.NewGuid(); var grain = fixture.GrainFactory.GetGrain(grainKey); await grain.EnsureActiveAsync(); - var stream = fixture.GetStream(StreamManagerTestNamespaces.Failure, grainKey); + var stream = fixture.GetStream( + StreamManagerTestProviderNames.Explicit, + StreamManagerTestNamespaces.Failure, + grainKey); await stream.OnNextAsync("fail"); await stream.OnNextAsync("after-failure"); @@ -135,7 +147,10 @@ public async Task EnsureExplicitSubscriptionsAsync_is_idempotent() var grainKey = Guid.NewGuid(); var grain = fixture.GrainFactory.GetGrain(grainKey); await grain.EnsureActiveAsync(); - var stream = fixture.GetStream(StreamManagerTestNamespaces.ValueTask, grainKey); + var stream = fixture.GetStream( + StreamManagerTestProviderNames.Explicit, + StreamManagerTestNamespaces.ValueTask, + grainKey); var exceptionType = await grain.EnsureExplicitSubscriptionsAgainAsync(); @@ -171,6 +186,7 @@ internal static class StreamManagerTestNamespaces internal static class StreamManagerTestProviderNames { + public const string Explicit = "stream-manager-explicit-provider"; public const string Implicit = "stream-manager-provider"; } @@ -218,10 +234,10 @@ public sealed class StreamManagerTestGrain( public override async Task OnActivateAsync(CancellationToken cancellationToken) { streamManager = this.RegisterStreamManager(state.State.Tracker) - .ConfigureExplicitSubscription(StreamManagerTestNamespaces.ValueTask, StreamManagerTestNamespaces.ValueTask, HandleValueTaskAsync) - .ConfigureExplicitSubscription(StreamManagerTestNamespaces.Task, StreamManagerTestNamespaces.Task, HandleTaskAsync) - .ConfigureExplicitSubscription(StreamManagerTestNamespaces.Failure, StreamManagerTestNamespaces.Failure, HandleFailureAsync) - .ConfigureExplicitSubscription(StreamManagerTestNamespaces.Resume, StreamManagerTestNamespaces.Resume, HandleResumeAsync); + .ConfigureExplicitSubscription(StreamManagerTestProviderNames.Explicit, StreamManagerTestNamespaces.ValueTask, HandleValueTaskAsync) + .ConfigureExplicitSubscription(StreamManagerTestProviderNames.Explicit, StreamManagerTestNamespaces.Task, HandleTaskAsync) + .ConfigureExplicitSubscription(StreamManagerTestProviderNames.Explicit, StreamManagerTestNamespaces.Failure, HandleFailureAsync) + .ConfigureExplicitSubscription(StreamManagerTestProviderNames.Explicit, StreamManagerTestNamespaces.Resume, HandleResumeAsync); await streamManager.EnsureExplicitSubscriptionsAsync(cancellationToken); await base.OnActivateAsync(cancellationToken); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs index 4179dbb8..ee9f75c1 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs @@ -22,6 +22,48 @@ public void ProcessMessage_accepts_first_stream_cursor_and_tracks_latest_positio Assert.NotSame(tracker, next); } + [Fact] + public void ProcessMessage_accepts_stream_sequence_token_and_namespace() + { + var time = new ManualTimeProvider(new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero)); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + + var accepted = tracker.ProcessMessage("orders", new EventSequenceToken(7), out var next); + + Assert.True(accepted); + Assert.Equal(new EventSequenceToken(7), next.LatestStreamSequenceToken("orders")); + } + + [Fact] + public void ProcessMessage_accepts_stream_sequence_token_provider_and_namespace() + { + var time = new ManualTimeProvider(new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero)); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + + tracker.ProcessMessage("provider-a", "orders", new EventSequenceToken(7), out tracker); + tracker.ProcessMessage("provider-b", "orders", new EventSequenceToken(9), out tracker); + + Assert.Equal(new EventSequenceToken(7), tracker.LatestStreamSequenceToken("provider-a", "orders")); + Assert.Equal(new EventSequenceToken(9), tracker.LatestStreamSequenceToken("provider-b", "orders")); + Assert.Null(tracker.LatestStreamSequenceToken("orders")); + } + + [Fact] + public void ProcessMessage_with_stream_sequence_token_rejects_duplicate_for_namespace() + { + var time = new ManualTimeProvider(new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero)); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(time); + tracker.ProcessMessage("orders", new EventSequenceToken(7), out tracker); + + var accepted = tracker.ProcessMessage("orders", new EventSequenceToken(7), out var next); + + Assert.False(accepted); + Assert.Same(tracker, next); + } + [Fact] public void ProcessMessage_accepts_first_outbox_token_and_tracks_latest_position() { @@ -103,6 +145,19 @@ public void LatestStream_with_provider_returns_matching_provider_cursor() Assert.Equal(new StreamCursor("orders", new EventSequenceToken(7), "provider-a"), latest); } + [Fact] + public void LatestStreamSequenceToken_returns_token_for_single_providerless_namespace() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(now)); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7)), out tracker); + + var latest = tracker.LatestStreamSequenceToken("orders"); + + Assert.Equal(new EventSequenceToken(7), latest); + } + [Fact] public void LatestStream_with_provider_does_not_fall_back_to_different_provider() { @@ -116,6 +171,34 @@ public void LatestStream_with_provider_does_not_fall_back_to_different_provider( Assert.Null(latest); } + [Fact] + public void LatestStreamSequenceToken_returns_null_when_namespace_has_multiple_providers() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(now)); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(7), "provider-a"), out tracker); + tracker.ProcessMessage(new StreamCursor("orders", new EventSequenceToken(9), "provider-b"), out tracker); + + var latest = tracker.LatestStreamSequenceToken("orders"); + + Assert.Null(latest); + } + + [Fact] + public void LatestStreamSequenceToken_returns_null_for_tracked_null_token() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(now)); + tracker.ProcessMessage("orders", token: null, out tracker); + + var latest = tracker.LatestStreamSequenceToken("orders"); + + Assert.Null(latest); + Assert.NotNull(tracker.LatestStream("orders")); + } + [Fact] public void ProcessMessage_accepts_non_null_stream_token_after_null_initial_position() { From 9f16760f534d95fede48f11c930db765d5e9c909 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 16:12:49 +0000 Subject: [PATCH 35/82] refactor(om): split state manager support types [skip notes] Move DefaultStateManager and WriteFailureKind into their own files without changing state manager behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../State/DefaultStateManager.cs | 20 +++++++++++ .../State/StateManager.cs | 35 ------------------- .../State/WriteFailureKind.cs | 17 +++++++++ 3 files changed, 37 insertions(+), 35 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/WriteFailureKind.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs new file mode 100644 index 00000000..d84d6329 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs @@ -0,0 +1,20 @@ +namespace Egil.Orleans.Messaging.State; + +/// +/// Default implementation for general storage providers. +/// +public sealed class DefaultStateManager : StateManagerBase + where T : class, IEquatable +{ + public DefaultStateManager(IPersistentState storage) + : base(storage) + { + } + + /// + protected override WriteFailureKind ClassifyWriteFailure(Exception exception) + { + ArgumentNullException.ThrowIfNull(exception); + return WriteFailureKind.UnknownOutcome; + } +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs index 753c50e9..87b33ffe 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs @@ -2,22 +2,6 @@ namespace Egil.Orleans.Messaging.State; -/// -/// Classifies a storage write failure for recovery decisions. -/// -public enum WriteFailureKind -{ - /// - /// Unknown outcome. The write may have persisted. Run read-back recovery. - /// - UnknownOutcome, - - /// - /// The write definitely did not persist. Skip read-back and rethrow. - /// - DidNotPersist -} - /// /// Base implementation of . Wraps an /// and provides committed-state fencing, @@ -240,22 +224,3 @@ private static bool IsEquivalent(T persisted, T attempted) return persisted.Equals(attempted); } } - -/// -/// Default implementation for general storage providers. -/// -public sealed class DefaultStateManager : StateManagerBase - where T : class, IEquatable -{ - public DefaultStateManager(IPersistentState storage) - : base(storage) - { - } - - /// - protected override WriteFailureKind ClassifyWriteFailure(Exception exception) - { - ArgumentNullException.ThrowIfNull(exception); - return WriteFailureKind.UnknownOutcome; - } -} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/WriteFailureKind.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/WriteFailureKind.cs new file mode 100644 index 00000000..ccf0e35a --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/WriteFailureKind.cs @@ -0,0 +1,17 @@ +namespace Egil.Orleans.Messaging.State; + +/// +/// Classifies a storage write failure for recovery decisions. +/// +public enum WriteFailureKind +{ + /// + /// Unknown outcome. The write may have persisted. Run read-back recovery. + /// + UnknownOutcome, + + /// + /// The write definitely did not persist. Skip read-back and rethrow. + /// + DidNotPersist +} From 8c50a4b02e7ac68c70f02a95c25eb6cd12645182 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 16:20:03 +0000 Subject: [PATCH 36/82] feat(om): add keyed outbox postman services Add IPostman, named postman attributes, IServiceCollection registration helpers, and an OutboxProcessor.AddPostman overload that resolves keyed postman services from the grain activation service provider. Cover keyed registration, scoped multi-contract resolution, successful keyed dispatch, and keyed dispatch failure reconciliation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 22 ++ Egil.Orleans.Messaging/api-design.md | 47 +++- .../outbox-postman-api-plan.md | 41 ++++ .../Outboxes/IPostman.cs | 16 ++ .../Outboxes/OutboxPostmanAttribute.cs | 14 ++ ...utboxPostmanServiceCollectionExtensions.cs | 112 ++++++++++ .../Outboxes/OutboxProcessor.cs | 16 ++ .../MessagingTestClusterFixture.cs | 6 + ...PostmanServiceCollectionExtensionsTests.cs | 96 ++++++++ .../Outboxes/OutboxProcessorTests.cs | 207 ++++++++++++++++++ 10 files changed, 573 insertions(+), 4 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IPostman.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanAttribute.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanServiceCollectionExtensions.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanServiceCollectionExtensionsTests.cs diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index 87569d97..34bcc16a 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -127,6 +127,28 @@ public sealed class OrderGrain : Grain, IOutboxGrain `IOutboxGrain` forwards reminder ticks to the attached processor. The grain remains responsible for its own message contracts, posting target, and dead-letter policy. +For reusable delivery code, implement and register keyed postman services: + +```csharp +[OutboxPostman("orders")] +public sealed class OrderEventPostman : IPostman +{ + public async ValueTask PostAsync(OrderSubmitted message, CancellationToken ct) + { + await publisher.PublishAsync(message, ct); + } +} + +services.AddOutboxPostman(); +``` + +Then resolve the postman by name from the grain activation service provider: + +```csharp +outboxProcessor = this.RegisterOutboxProcessor(options) + .AddPostman("orders"); +``` + ## Receiver Dedup `MessageTracker` accepts a message only when its stream token, stream cursor, or outbox token advances the stored high-water mark: diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 722b1a1e..4dab76a2 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -1144,10 +1144,47 @@ in the `Clever.PricingEngine` codebase. ### Postman dispatch -**Callback-based.** The grain registers one or more postmen via -`AddPostman(...)`, each handling a subtype of `TOutbox`. Matching -is first-registered-wins against the item's runtime type — order from -most specific to least specific (like a `switch`). +The grain registers one or more postmen via `AddPostman(...)`, each +handling a subtype of `TOutbox`. Matching is first-registered-wins against +the item's runtime type — order from most specific to least specific (like a +`switch`). + +Postmen can be inline callbacks for local/simple cases, or keyed +`IPostman` services for reusable delivery code: + +```csharp +public interface IPostman + where TMessage : notnull +{ + ValueTask PostAsync(TMessage message, CancellationToken cancellationToken); +} + +[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = false)] +public sealed class OutboxPostmanAttribute(string name) : Attribute +{ + public string Name { get; } = name; +} +``` + +```csharp +namespace Microsoft.Extensions.DependencyInjection; + +public static class OutboxPostmanServiceCollectionExtensions +{ + extension(IServiceCollection services) + { + public IServiceCollection AddOutboxPostman() + where TPostman : class; + + public IServiceCollection AddOutboxPostman(string postmanName) + where TPostman : class; + + public IServiceCollection AddOutboxPostman(string postmanName) + where TMessage : notnull + where TPostman : class, IPostman; + } +} +``` - Per-item exceptions are caught and surfaced through `ReconcileFailedAsync` with attempt count (in-memory, resets on reactivation) — the grain @@ -1324,6 +1361,8 @@ public sealed partial class OutboxProcessor : IOutboxComponent public OutboxProcessor AddPostman( Func postman) where TSub : TOutbox; + public OutboxProcessor AddPostman( + string postmanName) where TSub : TOutbox; /// Posts pending items. Safe to call from grain's task scheduler. /// Arms timer/reminder if items remain, unregisters if empty. diff --git a/Egil.Orleans.Messaging/outbox-postman-api-plan.md b/Egil.Orleans.Messaging/outbox-postman-api-plan.md index 49f982d7..a50afe17 100644 --- a/Egil.Orleans.Messaging/outbox-postman-api-plan.md +++ b/Egil.Orleans.Messaging/outbox-postman-api-plan.md @@ -200,3 +200,44 @@ design. - Throwing keyed postman goes through existing retry/reconcile path. - Cancellation from processor propagates to postman and leaves item pending. - Existing lambda-based `AddPostman(...)` behavior remains unchanged. + +## Implementation Steps + +1. Add the core public contract: + - `IPostman` + - `OutboxPostmanAttribute` + +2. Add service-registration APIs: + - `IServiceCollection.AddOutboxPostman()` + - `IServiceCollection.AddOutboxPostman(string postmanName)` + - `IServiceCollection.AddOutboxPostman(string postmanName)` + - Validate missing attributes, missing postman contracts, empty names, and open generic postman types. + +3. Add keyed processor registration: + - `OutboxProcessor.AddPostman(string postmanName)` + - Resolve from the grain activation service provider using keyed DI. + - Reuse the existing `AddPostmanCore` dispatch pipeline so retry, timeout, acknowledgement, reconciliation, and telemetry stay unchanged. + +4. Add focused registration tests: + - Attribute-based registration discovers all implemented contracts. + - Multiple contracts resolve to the same scoped concrete postman instance. + - Explicit-name registration works without an attribute. + - Missing attribute and missing postman contracts throw. + +5. Add Orleans processor behavior tests: + - Keyed postman success acknowledges and removes pending items. + - Keyed postman exceptions flow through existing failure reconciliation. + - Existing lambda postman tests remain green. + +6. Add cancellation/timeout tests: + - Processor cancellation propagates to keyed postman. + - Timeout leaves the pending item unacknowledged. + +7. Update user documentation: + - README examples for service postmen. + - API design section for the postman service contract. + +## Implementation Status + +- Steps 1-5 are implemented in the first pass. +- Steps 6-7 remain. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IPostman.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IPostman.cs new file mode 100644 index 00000000..36a6052e --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IPostman.cs @@ -0,0 +1,16 @@ +namespace Egil.Orleans.Messaging.Outboxes; + +/// +/// Delivers one outbox message under the processor-owned cancellation budget. +/// +/// The message type handled by this postman. +public interface IPostman + where TMessage : notnull +{ + /// + /// Posts the message to its target. + /// + /// The message to post. + /// Cancellation token owned by the outbox processor. + ValueTask PostAsync(TMessage message, CancellationToken cancellationToken); +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanAttribute.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanAttribute.cs new file mode 100644 index 00000000..19331783 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanAttribute.cs @@ -0,0 +1,14 @@ +namespace Egil.Orleans.Messaging.Outboxes; + +/// +/// Names a postman implementation for keyed outbox postman registration. +/// +/// The keyed postman name. +[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = false)] +public sealed class OutboxPostmanAttribute(string name) : Attribute +{ + /// + /// The keyed postman name. + /// + public string Name { get; } = name; +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanServiceCollectionExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanServiceCollectionExtensions.cs new file mode 100644 index 00000000..b62cf3f0 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanServiceCollectionExtensions.cs @@ -0,0 +1,112 @@ +using Egil.Orleans.Messaging.Outboxes; +using Microsoft.Extensions.DependencyInjection.Extensions; + +namespace Microsoft.Extensions.DependencyInjection; + +/// +/// Registration helpers for keyed outbox postman services. +/// +public static class OutboxPostmanServiceCollectionExtensions +{ + extension(IServiceCollection services) + { + /// + /// Registers a postman implementation using its + /// name. + /// + public IServiceCollection AddOutboxPostman() + where TPostman : class + { + ArgumentNullException.ThrowIfNull(services); + + var name = GetAttributeName(typeof(TPostman)); + return services.AddOutboxPostman(name); + } + + /// + /// Registers all closed contracts + /// implemented by with the given keyed + /// postman name. + /// + public IServiceCollection AddOutboxPostman(string postmanName) + where TPostman : class + { + ArgumentNullException.ThrowIfNull(services); + ValidatePostmanName(postmanName); + + var postmanType = typeof(TPostman); + if (postmanType.ContainsGenericParameters) + { + throw new ArgumentException("Open generic postman types are not supported.", nameof(TPostman)); + } + + var contracts = GetPostmanContracts(postmanType); + if (contracts.Length == 0) + { + throw new ArgumentException( + $"Type '{postmanType.FullName}' must implement at least one closed {typeof(IPostman<>).FullName} contract.", + nameof(TPostman)); + } + + services.TryAddScoped(); + foreach (var contract in contracts) + { + services.AddKeyedScoped( + contract, + postmanName, + static (provider, _) => provider.GetRequiredService()); + } + + return services; + } + + /// + /// Registers one keyed postman contract. + /// + public IServiceCollection AddOutboxPostman(string postmanName) + where TMessage : notnull + where TPostman : class, IPostman + { + ArgumentNullException.ThrowIfNull(services); + ValidatePostmanName(postmanName); + + services.TryAddScoped(); + services.AddKeyedScoped>( + postmanName, + static (provider, _) => provider.GetRequiredService()); + + return services; + } + } + + private static string GetAttributeName(Type postmanType) + { + var attribute = postmanType.GetCustomAttributes(typeof(OutboxPostmanAttribute), inherit: false) + .Cast() + .SingleOrDefault(); + + if (attribute is null) + { + throw new ArgumentException( + $"Type '{postmanType.FullName}' must declare {nameof(OutboxPostmanAttribute)} or be registered with an explicit postman name.", + nameof(postmanType)); + } + + ValidatePostmanName(attribute.Name); + return attribute.Name; + } + + private static Type[] GetPostmanContracts(Type postmanType) => + postmanType.GetInterfaces() + .Where(static candidate => + candidate.IsGenericType + && candidate.GetGenericTypeDefinition() == typeof(IPostman<>) + && !candidate.ContainsGenericParameters) + .Distinct() + .ToArray(); + + private static void ValidatePostmanName(string postmanName) + { + ArgumentException.ThrowIfNullOrWhiteSpace(postmanName); + } +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index 59b123e9..5c9e8f6d 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -1,5 +1,6 @@ using System.Collections.Immutable; using System.Diagnostics; +using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; namespace Egil.Orleans.Messaging.Outboxes; @@ -95,6 +96,21 @@ public OutboxProcessor AddPostman( return AddPostmanCore((item, cancellationToken) => postman(item, grainFactory, cancellationToken)); } + /// + /// Registers a keyed service that handles + /// items of type . + /// + public OutboxProcessor AddPostman(string postmanName) + where TSub : TOutbox + { + ArgumentException.ThrowIfNullOrWhiteSpace(postmanName); + + var postman = owner.GrainContext.ActivationServices + .GetRequiredKeyedService>(postmanName); + + return AddPostmanCore(postman.PostAsync); + } + /// /// Posts all pending items by dispatching each to its matching postman. /// Arms timer/reminder if items remain; unregisters retry work if empty. diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs index 38d69cf7..a1d65202 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs @@ -28,6 +28,12 @@ public async ValueTask InitializeAsync() siloBuilder.UseInMemoryReminderService(); AddStreamProviders(siloBuilder); + siloBuilder.ConfigureServices(services => + { + services + .AddOutboxPostman(OutboxProcessorTestPostmanNames.Success) + .AddOutboxPostman(OutboxProcessorTestPostmanNames.Failure); + }); siloBuilder.AddGrainActivityCollector(Collector) .CollectStorageActivityFrom("Default"); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanServiceCollectionExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanServiceCollectionExtensionsTests.cs new file mode 100644 index 00000000..7c071cb5 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanServiceCollectionExtensionsTests.cs @@ -0,0 +1,96 @@ +namespace Egil.Orleans.Messaging.Tests.Outboxes; + +public sealed class OutboxPostmanServiceCollectionExtensionsTests +{ + [Fact] + public void AddOutboxPostman_with_attribute_registers_all_postman_contracts_as_keyed_scoped() + { + var services = new ServiceCollection(); + + services.AddOutboxPostman(); + + Assert.Contains(services, service => + service.ServiceType == typeof(IPostman) + && Equals(service.ServiceKey, "multi") + && service.Lifetime == ServiceLifetime.Scoped); + Assert.Contains(services, service => + service.ServiceType == typeof(IPostman) + && Equals(service.ServiceKey, "multi") + && service.Lifetime == ServiceLifetime.Scoped); + } + + [Fact] + public void AddOutboxPostman_registers_one_scoped_concrete_instance_for_multiple_contracts() + { + var services = new ServiceCollection(); + services.AddOutboxPostman(); + + using var provider = services.BuildServiceProvider(); + using var scope = provider.CreateScope(); + + var testPostman = scope.ServiceProvider.GetRequiredKeyedService>("multi"); + var otherPostman = scope.ServiceProvider.GetRequiredKeyedService>("multi"); + + Assert.Same(testPostman, otherPostman); + } + + [Fact] + public void AddOutboxPostman_with_explicit_name_does_not_require_attribute() + { + var services = new ServiceCollection(); + + services.AddOutboxPostman("explicit"); + + using var provider = services.BuildServiceProvider(); + using var scope = provider.CreateScope(); + var postman = scope.ServiceProvider.GetRequiredKeyedService>("explicit"); + + Assert.IsType(postman); + } + + [Fact] + public void AddOutboxPostman_throws_when_attribute_name_is_missing() + { + var services = new ServiceCollection(); + + var ex = Assert.Throws(() => + services.AddOutboxPostman()); + + Assert.Equal("postmanType", ex.ParamName); + } + + [Fact] + public void AddOutboxPostman_throws_when_type_has_no_postman_contracts() + { + var services = new ServiceCollection(); + + var ex = Assert.Throws(() => + services.AddOutboxPostman("missing")); + + Assert.Equal("TPostman", ex.ParamName); + } + + private sealed record TestMessage(string Value); + + private sealed record OtherMessage(string Value); + + [OutboxPostman("multi")] + private sealed class MultiMessagePostman : + IPostman, + IPostman + { + public ValueTask PostAsync(TestMessage message, CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + public ValueTask PostAsync(OtherMessage message, CancellationToken cancellationToken) => + ValueTask.CompletedTask; + } + + private sealed class TestMessagePostman : IPostman + { + public ValueTask PostAsync(TestMessage message, CancellationToken cancellationToken) => + ValueTask.CompletedTask; + } + + private sealed class NoPostmanContracts; +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index 592b60b6..d2556224 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -73,6 +73,45 @@ await fixture.WaitForAssertionAsync( }, ct: TestContext.Current.CancellationToken); } + + [Fact] + public async Task PostInBackgroundAsync_resolves_keyed_postman_and_acknowledges() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + await source.PublishInBackgroundAsync("keyed"); + + await fixture.WaitForAssertionAsync( + source, + async () => + { + var state = await source.GetStateAsync(); + Assert.Equal(1, state.AcknowledgedCount); + Assert.Equal(0, state.FailedCount); + Assert.Equal(0, state.Outbox?.Count ?? 0); + }, + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task PostInBackgroundAsync_reports_keyed_postman_failure_and_reconciles_pending_item() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + await source.PublishInBackgroundAsync("keyed-failure"); + + await fixture.WaitForAssertionAsync( + source, + async () => + { + var state = await source.GetStateAsync(); + Assert.Equal(nameof(InvalidOperationException), state.LastFailureType); + Assert.Equal(1, state.FailedCount); + Assert.Equal(0, state.AcknowledgedCount); + Assert.Equal(0, state.Outbox?.Count ?? 0); + }, + ct: TestContext.Current.CancellationToken); + } } internal static class OutboxProcessorTestNamespaces @@ -80,6 +119,12 @@ internal static class OutboxProcessorTestNamespaces public const string Events = "outbox-processor-events"; } +internal static class OutboxProcessorTestPostmanNames +{ + public const string Success = "outbox-processor-success"; + public const string Failure = "outbox-processor-failure"; +} + public interface IOutboxProcessorSourceGrain : IGrainWithGuidKey { Task PublishInBackgroundAsync(string value); @@ -101,6 +146,20 @@ public interface IOutboxProcessorFailingPostmanGrain : IGrainWithGuidKey Task GetStateAsync(); } +public interface IOutboxProcessorKeyedPostmanGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorKeyedFailingPostmanGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value); + + Task GetStateAsync(); +} + public interface IOutboxProcessorSinkGrain : IGrainWithGuidKey { Task EnsureActiveAsync(); @@ -340,6 +399,154 @@ private Outbox EnsureOutbox() } } +public sealed class OutboxProcessorKeyedPostmanGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorKeyedPostmanGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>(OutboxProcessorTestPostmanNames.Success); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishInBackgroundAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } + + private async ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var failure in failures) + { + outbox = outbox.Remove(failure.Item.Token); + state.State.LastFailureType = failure.Error.GetType().Name; + } + + state.State.Outbox = outbox; + state.State.FailedCount += failures.Length; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() + { + return state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); + } +} + +public sealed class OutboxProcessorKeyedFailingPostmanGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorKeyedFailingPostmanGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>(OutboxProcessorTestPostmanNames.Failure); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishInBackgroundAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private async ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var failure in failures) + { + outbox = outbox.Remove(failure.Item.Token); + state.State.LastFailureType = failure.Error.GetType().Name; + } + + state.State.Outbox = outbox; + state.State.FailedCount += failures.Length; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() + { + return state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); + } +} + +public sealed class KeyedOutboxProcessorSuccessPostman : + IPostman> +{ + public ValueTask PostAsync( + OutboxMessageEnvelope message, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; +} + +public sealed class KeyedOutboxProcessorFailingPostman : + IPostman> +{ + public ValueTask PostAsync( + OutboxMessageEnvelope message, + CancellationToken cancellationToken) + { + throw new InvalidOperationException($"Cannot post {message.Message.Value}."); + } +} + public sealed class OutboxProcessorSinkGrain( [PersistentState("state", "Default")] IPersistentState state) : Grain, IOutboxProcessorSinkGrain From 57d02df9e585e6dfc035e688a66d1ef4e5503555 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 16:23:19 +0000 Subject: [PATCH 37/82] test(om): cover keyed postman cancellation Add keyed outbox postman tests for caller cancellation and processor timeout, verifying both paths leave the outbox item pending and do not acknowledge or reconcile it as a per-item failure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../outbox-postman-api-plan.md | 3 +- .../MessagingTestClusterFixture.cs | 3 +- .../Outboxes/OutboxProcessorTests.cs | 171 ++++++++++++++++++ 3 files changed, 174 insertions(+), 3 deletions(-) diff --git a/Egil.Orleans.Messaging/outbox-postman-api-plan.md b/Egil.Orleans.Messaging/outbox-postman-api-plan.md index a50afe17..d7b5b767 100644 --- a/Egil.Orleans.Messaging/outbox-postman-api-plan.md +++ b/Egil.Orleans.Messaging/outbox-postman-api-plan.md @@ -239,5 +239,4 @@ design. ## Implementation Status -- Steps 1-5 are implemented in the first pass. -- Steps 6-7 remain. +- Steps 1-7 are implemented. diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs index a1d65202..1df7fd58 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs @@ -32,7 +32,8 @@ public async ValueTask InitializeAsync() { services .AddOutboxPostman(OutboxProcessorTestPostmanNames.Success) - .AddOutboxPostman(OutboxProcessorTestPostmanNames.Failure); + .AddOutboxPostman(OutboxProcessorTestPostmanNames.Failure) + .AddOutboxPostman(OutboxProcessorTestPostmanNames.Delay); }); siloBuilder.AddGrainActivityCollector(Collector) diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index d2556224..6f8216b1 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -112,6 +112,34 @@ await fixture.WaitForAssertionAsync( }, ct: TestContext.Current.CancellationToken); } + + [Fact] + public async Task PostAsync_propagates_cancellation_to_keyed_postman_and_leaves_item_pending() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var exceptionType = await source.PublishWithCancellationAsync("cancel"); + var state = await source.GetStateAsync(); + + Assert.Equal(nameof(TaskCanceledException), exceptionType); + Assert.Equal(0, state.AcknowledgedCount); + Assert.Equal(0, state.FailedCount); + Assert.Equal(1, state.Outbox?.Count); + } + + [Fact] + public async Task PostAsync_timeout_from_keyed_postman_leaves_item_pending() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var exceptionType = await source.PublishWithTimeoutAsync("timeout"); + var state = await source.GetStateAsync(); + + Assert.Equal(nameof(TimeoutException), exceptionType); + Assert.Equal(0, state.AcknowledgedCount); + Assert.Equal(0, state.FailedCount); + Assert.Equal(1, state.Outbox?.Count); + } } internal static class OutboxProcessorTestNamespaces @@ -123,6 +151,7 @@ internal static class OutboxProcessorTestPostmanNames { public const string Success = "outbox-processor-success"; public const string Failure = "outbox-processor-failure"; + public const string Delay = "outbox-processor-delay"; } public interface IOutboxProcessorSourceGrain : IGrainWithGuidKey @@ -160,6 +189,20 @@ public interface IOutboxProcessorKeyedFailingPostmanGrain : IGrainWithGuidKey Task GetStateAsync(); } +public interface IOutboxProcessorKeyedCancellationPostmanGrain : IGrainWithGuidKey +{ + Task PublishWithCancellationAsync(string value); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorKeyedTimeoutPostmanGrain : IGrainWithGuidKey +{ + Task PublishWithTimeoutAsync(string value); + + Task GetStateAsync(); +} + public interface IOutboxProcessorSinkGrain : IGrainWithGuidKey { Task EnsureActiveAsync(); @@ -527,6 +570,123 @@ private Outbox EnsureOutbox() } } +public sealed class OutboxProcessorKeyedCancellationPostmanGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorKeyedCancellationPostmanGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + ProcessingTimeout = TimeSpan.FromSeconds(5), + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>(OutboxProcessorTestPostmanNames.Delay); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishWithCancellationAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + + using var cancellation = new CancellationTokenSource(TimeSpan.FromMilliseconds(20)); + try + { + await processor!.PostAsync(cancellation.Token); + return null; + } + catch (Exception ex) + { + return ex.GetType().Name; + } + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private Outbox EnsureOutbox() + { + return state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); + } +} + +public sealed class OutboxProcessorKeyedTimeoutPostmanGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorKeyedTimeoutPostmanGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + ProcessingTimeout = TimeSpan.FromMilliseconds(20), + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>(OutboxProcessorTestPostmanNames.Delay); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishWithTimeoutAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + + try + { + await processor!.PostAsync(); + return null; + } + catch (Exception ex) + { + return ex.GetType().Name; + } + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private Outbox EnsureOutbox() + { + return state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); + } +} + public sealed class KeyedOutboxProcessorSuccessPostman : IPostman> { @@ -547,6 +707,17 @@ public ValueTask PostAsync( } } +public sealed class KeyedOutboxProcessorDelayingPostman : + IPostman> +{ + public async ValueTask PostAsync( + OutboxMessageEnvelope message, + CancellationToken cancellationToken) + { + await Task.Delay(Timeout.InfiniteTimeSpan, cancellationToken); + } +} + public sealed class OutboxProcessorSinkGrain( [PersistentState("state", "Default")] IPersistentState state) : Grain, IOutboxProcessorSinkGrain From 255c98054eb6c1c8f681eda6e6919f4dcd9b1a07 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 16:35:11 +0000 Subject: [PATCH 38/82] feat(om): add outbox postman helper APIs Add built-in stream and grain postman helpers so common Orleans fan-out targets can use the outbox processor without hand-written dispatch callbacks. Cover helper behavior through Orleans tests and fill validation coverage for keyed postman registration edge cases. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 17 + Egil.Orleans.Messaging/api-design.md | 20 + .../outbox-postman-api-plan.md | 27 +- ...utboxPostmanServiceCollectionExtensions.cs | 62 +-- .../Outboxes/OutboxProcessor.cs | 53 +++ .../MessagingTestClusterFixture.cs | 4 +- .../Outboxes/OutboxPostmanHelperTests.cs | 395 ++++++++++++++++++ ...PostmanServiceCollectionExtensionsTests.cs | 71 +++- .../Outboxes/OutboxProcessorTests.cs | 9 +- 9 files changed, 622 insertions(+), 36 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanHelperTests.cs diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index 34bcc16a..afd2265c 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -149,6 +149,23 @@ outboxProcessor = this.RegisterOutboxProcessor(options) .AddPostman("orders"); ``` +For common Orleans targets, use the built-in helpers instead of writing the +callback by hand: + +```csharp +outboxProcessor = this.RegisterOutboxProcessor(options) + .AddStreamPostman( + "order-streams", + message => StreamId.Create("submitted-orders", message.OrderId)); +``` + +```csharp +outboxProcessor = this.RegisterOutboxProcessor(options) + .AddGrainPostman( + (message, grainFactory) => grainFactory.GetGrain(message.OrderId), + (grain, message) => grain.ApplyAsync(message)); +``` + ## Receiver Dedup `MessageTracker` accepts a message only when its stream token, stream cursor, or outbox token advances the stored high-water mark: diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 4dab76a2..ac0bd015 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -1166,6 +1166,12 @@ public sealed class OutboxPostmanAttribute(string name) : Attribute } ``` +Use inline callbacks when delivery logic is grain-local and small. Use keyed +`IPostman` services when the delivery code has dependencies, should +be tested outside Orleans, or is shared across grains. Registered postman +services are scoped to the grain activation service provider and the processor +does not create child scopes. + ```csharp namespace Microsoft.Extensions.DependencyInjection; @@ -1363,6 +1369,20 @@ public sealed partial class OutboxProcessor : IOutboxComponent where TSub : TOutbox; public OutboxProcessor AddPostman( string postmanName) where TSub : TOutbox; + public OutboxProcessor AddStreamPostman( + string streamProviderName, + Func streamId) + where TSub : TOutbox; + public OutboxProcessor AddStreamPostman( + string streamProviderName, + Func streamId, + Func project) + where TSub : TOutbox; + public OutboxProcessor AddGrainPostman( + Func resolveGrain, + Func call) + where TSub : TOutbox + where TGrain : IGrain; /// Posts pending items. Safe to call from grain's task scheduler. /// Arms timer/reminder if items remain, unregisters if empty. diff --git a/Egil.Orleans.Messaging/outbox-postman-api-plan.md b/Egil.Orleans.Messaging/outbox-postman-api-plan.md index d7b5b767..62a2c988 100644 --- a/Egil.Orleans.Messaging/outbox-postman-api-plan.md +++ b/Egil.Orleans.Messaging/outbox-postman-api-plan.md @@ -156,7 +156,7 @@ Processor behavior: ## Built-In Postman Helpers -Add later as convenience APIs, implemented on top of `AddPostmanCore`. +Convenience APIs implemented on top of `AddPostmanCore`. Stream postman: @@ -183,8 +183,9 @@ public OutboxProcessor AddGrainPostman( where TGrain : IGrain; ``` -These helpers are optional and should not block the initial `IPostman` -design. +These helpers are part of the completed postman API surface. They keep +common Orleans stream and grain fan-out cases on the same processor-owned +dispatch, retry, timeout, acknowledgement, reconciliation, and telemetry path. ## Tests @@ -193,12 +194,20 @@ design. - Multiple `IPostman` contracts on one concrete postman resolve to one scoped concrete instance. - Missing `[OutboxPostman]` without explicit name throws. +- Empty explicit names, empty attribute names, and open generic postman types + throw. - Concrete type with no `IPostman` contracts throws. - `OutboxProcessor.AddPostman("name")` resolves keyed postman from grain activation services. - Successful keyed postman call acknowledges item. - Throwing keyed postman goes through existing retry/reconcile path. - Cancellation from processor propagates to postman and leaves item pending. +- Stream postman helpers publish to the selected Orleans stream and + acknowledge on success. +- Projected stream postman helpers publish projected events and acknowledge on + success. +- Grain postman helpers resolve a grain through `IGrainFactory`, invoke it, and + acknowledge on success. - Existing lambda-based `AddPostman(...)` behavior remains unchanged. ## Implementation Steps @@ -237,6 +246,16 @@ design. - README examples for service postmen. - API design section for the postman service contract. +8. Add built-in helper APIs: + - `OutboxProcessor.AddStreamPostman(...)` + - `OutboxProcessor.AddStreamPostman(...)` + - `OutboxProcessor.AddGrainPostman(...)` + +9. Add helper behavior tests: + - Stream helper publishes the outbox item and acknowledges. + - Projected stream helper publishes the projected event and acknowledges. + - Grain helper resolves the target grain, invokes it, and acknowledges. + ## Implementation Status -- Steps 1-7 are implemented. +- Steps 1-9 are implemented. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanServiceCollectionExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanServiceCollectionExtensions.cs index b62cf3f0..9baac2ce 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanServiceCollectionExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanServiceCollectionExtensions.cs @@ -31,33 +31,7 @@ public IServiceCollection AddOutboxPostman() public IServiceCollection AddOutboxPostman(string postmanName) where TPostman : class { - ArgumentNullException.ThrowIfNull(services); - ValidatePostmanName(postmanName); - - var postmanType = typeof(TPostman); - if (postmanType.ContainsGenericParameters) - { - throw new ArgumentException("Open generic postman types are not supported.", nameof(TPostman)); - } - - var contracts = GetPostmanContracts(postmanType); - if (contracts.Length == 0) - { - throw new ArgumentException( - $"Type '{postmanType.FullName}' must implement at least one closed {typeof(IPostman<>).FullName} contract.", - nameof(TPostman)); - } - - services.TryAddScoped(); - foreach (var contract in contracts) - { - services.AddKeyedScoped( - contract, - postmanName, - static (provider, _) => provider.GetRequiredService()); - } - - return services; + return AddOutboxPostman(services, typeof(TPostman), postmanName); } /// @@ -79,6 +53,40 @@ public IServiceCollection AddOutboxPostman(string postmanNam } } + internal static IServiceCollection AddOutboxPostman( + IServiceCollection services, + Type postmanType, + string postmanName) + { + ArgumentNullException.ThrowIfNull(services); + ArgumentNullException.ThrowIfNull(postmanType); + ValidatePostmanName(postmanName); + + if (postmanType.ContainsGenericParameters) + { + throw new ArgumentException("Open generic postman types are not supported.", nameof(postmanType)); + } + + var contracts = GetPostmanContracts(postmanType); + if (contracts.Length == 0) + { + throw new ArgumentException( + $"Type '{postmanType.FullName}' must implement at least one closed {typeof(IPostman<>).FullName} contract.", + nameof(postmanType)); + } + + services.TryAddScoped(postmanType); + foreach (var contract in contracts) + { + services.AddKeyedScoped( + contract, + postmanName, + (provider, _) => provider.GetRequiredService(postmanType)); + } + + return services; + } + private static string GetAttributeName(Type postmanType) { var attribute = postmanType.GetCustomAttributes(typeof(OutboxPostmanAttribute), inherit: false) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index 5c9e8f6d..db412ce3 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -2,6 +2,7 @@ using System.Diagnostics; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; +using Orleans.Streams; namespace Egil.Orleans.Messaging.Outboxes; @@ -111,6 +112,58 @@ public OutboxProcessor AddPostman(string postmanName) return AddPostmanCore(postman.PostAsync); } + /// + /// Registers a postman that publishes each item to an Orleans stream. + /// + public OutboxProcessor AddStreamPostman( + string streamProviderName, + Func streamId) + where TSub : TOutbox => + AddStreamPostman( + streamProviderName, + streamId, + static message => message); + + /// + /// Registers a postman that projects each item and publishes the projected + /// event to an Orleans stream. + /// + public OutboxProcessor AddStreamPostman( + string streamProviderName, + Func streamId, + Func project) + where TSub : TOutbox + { + ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); + ArgumentNullException.ThrowIfNull(streamId); + ArgumentNullException.ThrowIfNull(project); + + var streamProvider = owner.GrainContext.ActivationServices + .GetRequiredKeyedService(streamProviderName); + + return AddPostmanCore((message, _) => + { + var stream = streamProvider.GetStream(streamId(message)); + return new ValueTask(stream.OnNextAsync(project(message))); + }); + } + + /// + /// Registers a postman that resolves a grain for each item and invokes it. + /// + public OutboxProcessor AddGrainPostman( + Func resolveGrain, + Func call) + where TSub : TOutbox + where TGrain : IGrain + { + ArgumentNullException.ThrowIfNull(resolveGrain); + ArgumentNullException.ThrowIfNull(call); + + return AddPostmanCore((message, _) => + new ValueTask(call(resolveGrain(message, grainFactory), message))); + } + /// /// Posts all pending items by dispatching each to its matching postman. /// Arms timer/reminder if items remain; unregisters retry work if empty. diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs index 1df7fd58..1e0634cd 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs @@ -75,13 +75,13 @@ private static void AddStreamProviders(ISiloBuilder siloBuilder) { siloBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Explicit); siloBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Implicit); - siloBuilder.AddMemoryStreams(OutboxProcessorTestNamespaces.Events); + siloBuilder.AddMemoryStreams(OutboxProcessorTestProviderNames.Events); } private static void AddStreamProviders(IClientBuilder clientBuilder) { clientBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Explicit); clientBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Implicit); - clientBuilder.AddMemoryStreams(OutboxProcessorTestNamespaces.Events); + clientBuilder.AddMemoryStreams(OutboxProcessorTestProviderNames.Events); } } \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanHelperTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanHelperTests.cs new file mode 100644 index 00000000..d67445f4 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanHelperTests.cs @@ -0,0 +1,395 @@ +using System.Collections.Immutable; +using Egil.Orleans.Testing; + +namespace Egil.Orleans.Messaging.Tests.Outboxes; + +public sealed class OutboxPostmanHelperTests(MessagingTestClusterFixture fixture) : IClassFixture +{ + [Fact] + public async Task Stream_postman_delivers_outbox_item_and_acknowledges() + { + var grainKey = Guid.NewGuid(); + var sink = fixture.GrainFactory.GetGrain(grainKey); + var source = fixture.GrainFactory.GetGrain(grainKey); + await sink.EnsureActiveAsync(); + + await source.PublishInBackgroundAsync("stream-helper"); + + await fixture.WaitForAssertionAsync( + sink, + async () => + { + var sinkState = await sink.GetStateAsync(); + Assert.Contains("stream-helper", sinkState.ReceivedValues); + }, + ct: TestContext.Current.CancellationToken); + + await fixture.WaitForAssertionAsync( + source, + async () => + { + var sourceState = await source.GetStateAsync(); + Assert.Equal(1, sourceState.AcknowledgedCount); + Assert.Equal(0, sourceState.Outbox?.Count ?? 0); + }, + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task Projected_stream_postman_delivers_projected_event_and_acknowledges() + { + var grainKey = Guid.NewGuid(); + var sink = fixture.GrainFactory.GetGrain(grainKey); + var source = fixture.GrainFactory.GetGrain(grainKey); + await sink.EnsureActiveAsync(); + + await source.PublishInBackgroundAsync("projected-stream-helper"); + + await fixture.WaitForAssertionAsync( + sink, + async () => + { + var sinkState = await sink.GetStateAsync(); + Assert.Contains("projected-stream-helper", sinkState.ReceivedValues); + }, + ct: TestContext.Current.CancellationToken); + + await fixture.WaitForAssertionAsync( + source, + async () => + { + var sourceState = await source.GetStateAsync(); + Assert.Equal(1, sourceState.AcknowledgedCount); + Assert.Equal(0, sourceState.Outbox?.Count ?? 0); + }, + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task Grain_postman_delivers_to_resolved_grain_and_acknowledges() + { + var grainKey = Guid.NewGuid(); + var target = fixture.GrainFactory.GetGrain(grainKey); + var source = fixture.GrainFactory.GetGrain(grainKey); + + await source.PublishInBackgroundAsync("grain-helper"); + + await fixture.WaitForAssertionAsync( + target, + async () => + { + var targetState = await target.GetStateAsync(); + Assert.Contains("grain-helper", targetState.ReceivedValues); + }, + ct: TestContext.Current.CancellationToken); + + await fixture.WaitForAssertionAsync( + source, + async () => + { + var sourceState = await source.GetStateAsync(); + Assert.Equal(1, sourceState.AcknowledgedCount); + Assert.Equal(0, sourceState.Outbox?.Count ?? 0); + }, + ct: TestContext.Current.CancellationToken); + } +} + +public interface IOutboxProcessorStreamPostmanGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorProjectedStreamPostmanGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorProjectedSinkGrain : IGrainWithGuidKey +{ + Task EnsureActiveAsync(); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorGrainPostmanSourceGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorGrainPostmanTargetGrain : IGrainWithGuidKey +{ + Task ReceiveAsync(string value); + + Task GetStateAsync(); +} + +public sealed class OutboxProcessorStreamPostmanGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorStreamPostmanGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(CreateOptions()) + .AddStreamPostman>( + OutboxProcessorTestProviderNames.Events, + envelope => StreamId.Create(OutboxProcessorTestNamespaces.Events, this.GetPrimaryKey())); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishInBackgroundAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private OutboxProcessorOptions> CreateOptions() => new() + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }; + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } + + private async ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var failure in failures) + { + outbox = outbox.Remove(failure.Item.Token); + state.State.LastFailureType = failure.Error.GetType().Name; + } + + state.State.Outbox = outbox; + state.State.FailedCount += failures.Length; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() => + state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); +} + +public sealed class OutboxProcessorProjectedStreamPostmanGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorProjectedStreamPostmanGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(CreateOptions()) + .AddStreamPostman, OutboxProcessorTestEvent>( + OutboxProcessorTestProviderNames.Events, + envelope => StreamId.Create(OutboxProcessorTestNamespaces.Events, this.GetPrimaryKey()), + envelope => envelope.Message); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishInBackgroundAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private OutboxProcessorOptions> CreateOptions() => new() + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }; + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } + + private async ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var failure in failures) + { + outbox = outbox.Remove(failure.Item.Token); + state.State.LastFailureType = failure.Error.GetType().Name; + } + + state.State.Outbox = outbox; + state.State.FailedCount += failures.Length; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() => + state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); +} + +public sealed class OutboxProcessorProjectedSinkGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorProjectedSinkGrain +{ + private StreamManager? streamManager; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + streamManager = this.RegisterStreamManager(state.State.Tracker) + .ConfigureExplicitSubscription( + OutboxProcessorTestProviderNames.Events, + OutboxProcessorTestNamespaces.Events, + HandleEventAsync); + + await streamManager.EnsureExplicitSubscriptionsAsync(cancellationToken); + await base.OnActivateAsync(cancellationToken); + } + + public Task EnsureActiveAsync() => Task.CompletedTask; + + public Task GetStateAsync() => Task.FromResult(state.State); + + private async ValueTask HandleEventAsync( + OutboxProcessorTestEvent message, + StreamCursor cursor) + { + if (!state.State.Tracker.ProcessMessage(cursor.StreamNamespace, cursor.Token, out var next)) + { + return; + } + + state.State.Tracker = next; + state.State.ReceivedValues = state.State.ReceivedValues.Add(message.Value); + await state.WriteStateAsync(); + } +} + +public sealed class OutboxProcessorGrainPostmanSourceGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorGrainPostmanSourceGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(CreateOptions()) + .AddGrainPostman, IOutboxProcessorGrainPostmanTargetGrain>( + (_, grainFactory) => grainFactory.GetGrain(this.GetPrimaryKey()), + (target, envelope) => target.ReceiveAsync(envelope.Message.Value)); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishInBackgroundAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private OutboxProcessorOptions> CreateOptions() => new() + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }; + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } + + private async ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var failure in failures) + { + outbox = outbox.Remove(failure.Item.Token); + state.State.LastFailureType = failure.Error.GetType().Name; + } + + state.State.Outbox = outbox; + state.State.FailedCount += failures.Length; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() => + state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); +} + +public sealed class OutboxProcessorGrainPostmanTargetGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorGrainPostmanTargetGrain +{ + public async Task ReceiveAsync(string value) + { + state.State.ReceivedValues = state.State.ReceivedValues.Add(value); + await state.WriteStateAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanServiceCollectionExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanServiceCollectionExtensionsTests.cs index 7c071cb5..3cf9f8b9 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanServiceCollectionExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanServiceCollectionExtensionsTests.cs @@ -48,6 +48,23 @@ public void AddOutboxPostman_with_explicit_name_does_not_require_attribute() Assert.IsType(postman); } + [Fact] + public void AddOutboxPostman_with_explicit_name_registers_all_postman_contracts_as_keyed_scoped() + { + var services = new ServiceCollection(); + + services.AddOutboxPostman("explicit"); + + Assert.Contains(services, service => + service.ServiceType == typeof(IPostman) + && Equals(service.ServiceKey, "explicit") + && service.Lifetime == ServiceLifetime.Scoped); + Assert.Contains(services, service => + service.ServiceType == typeof(IPostman) + && Equals(service.ServiceKey, "explicit") + && service.Lifetime == ServiceLifetime.Scoped); + } + [Fact] public void AddOutboxPostman_throws_when_attribute_name_is_missing() { @@ -59,6 +76,30 @@ public void AddOutboxPostman_throws_when_attribute_name_is_missing() Assert.Equal("postmanType", ex.ParamName); } + [Theory] + [InlineData("")] + [InlineData(" ")] + public void AddOutboxPostman_throws_when_explicit_name_is_missing(string postmanName) + { + var services = new ServiceCollection(); + + var ex = Assert.Throws(() => + services.AddOutboxPostman(postmanName)); + + Assert.Equal("postmanName", ex.ParamName); + } + + [Fact] + public void AddOutboxPostman_throws_when_attribute_name_is_empty() + { + var services = new ServiceCollection(); + + var ex = Assert.Throws(() => + services.AddOutboxPostman()); + + Assert.Equal("postmanName", ex.ParamName); + } + [Fact] public void AddOutboxPostman_throws_when_type_has_no_postman_contracts() { @@ -67,7 +108,21 @@ public void AddOutboxPostman_throws_when_type_has_no_postman_contracts() var ex = Assert.Throws(() => services.AddOutboxPostman("missing")); - Assert.Equal("TPostman", ex.ParamName); + Assert.Equal("postmanType", ex.ParamName); + } + + [Fact] + public void AddOutboxPostman_throws_when_postman_type_is_open_generic() + { + var services = new ServiceCollection(); + + var ex = Assert.Throws(() => + OutboxPostmanServiceCollectionExtensions.AddOutboxPostman( + services, + typeof(OpenGenericPostman<>), + "open")); + + Assert.Equal("postmanType", ex.ParamName); } private sealed record TestMessage(string Value); @@ -92,5 +147,19 @@ public ValueTask PostAsync(TestMessage message, CancellationToken cancellationTo ValueTask.CompletedTask; } + [OutboxPostman("")] + private sealed class EmptyNamePostman : IPostman + { + public ValueTask PostAsync(TestMessage message, CancellationToken cancellationToken) => + ValueTask.CompletedTask; + } + + private sealed class OpenGenericPostman : IPostman + where T : notnull + { + public ValueTask PostAsync(T message, CancellationToken cancellationToken) => + ValueTask.CompletedTask; + } + private sealed class NoPostmanContracts; } \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index 6f8216b1..93563e94 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -147,6 +147,11 @@ internal static class OutboxProcessorTestNamespaces public const string Events = "outbox-processor-events"; } +internal static class OutboxProcessorTestProviderNames +{ + public const string Events = "outbox-processor-stream-provider"; +} + internal static class OutboxProcessorTestPostmanNames { public const string Success = "outbox-processor-success"; @@ -275,7 +280,7 @@ private Task PublishEnvelopeAsync( OutboxMessageEnvelope envelope, CancellationToken cancellationToken) { - var stream = this.GetStreamProvider(OutboxProcessorTestNamespaces.Events) + var stream = this.GetStreamProvider(OutboxProcessorTestProviderNames.Events) .GetStream>( StreamId.Create(OutboxProcessorTestNamespaces.Events, this.GetPrimaryKey())); @@ -728,7 +733,7 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) { streamManager = this.RegisterStreamManager(state.State.Tracker) .ConfigureExplicitSubscription>( - OutboxProcessorTestNamespaces.Events, + OutboxProcessorTestProviderNames.Events, OutboxProcessorTestNamespaces.Events, HandleEnvelopeAsync); From 7a5009550d411f13985d8b1d80501447fa3bd17b Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 23:03:53 +0000 Subject: [PATCH 39/82] fix(om): tighten stream subscription configuration Fail fast when Orleans invokes an implicit stream subscription without a configured handler, and add explicit StreamId subscription overloads so explicit streams are not limited to the grain-keyed convention. Clarify outbox ownership, first-match postman dispatch, thread-pool postman safety, and future provider-specific package boundaries in public docs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 23 +++++ Egil.Orleans.Messaging/api-design.md | 65 +++++++++++-- .../Egil.Orleans.Messaging/Outboxes/Outbox.cs | 9 +- .../Outboxes/OutboxProcessor.cs | 5 + .../Outboxes/OutboxProcessorOptions.cs | 6 ++ .../Streams/StreamManager.cs | 93 ++++++++++++++++--- .../Streams/StreamManagerResumeTests.cs | 48 ++++++++++ 7 files changed, 221 insertions(+), 28 deletions(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index afd2265c..953668ba 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -126,6 +126,17 @@ public sealed class OrderGrain : Grain, IOutboxGrain ``` `IOutboxGrain` forwards reminder ticks to the attached processor. The grain remains responsible for its own message contracts, posting target, and dead-letter policy. +Postman matching is first-match-wins: register specific message types before +base interfaces or catch-all handlers. + +Failed dispatches are reported through `ReconcileFailedAsync`. That callback +is where the owning grain applies retry, dead-letter, max-depth, or trimming +policy, because the grain owns the durable outbox state. + +Postmen normally run on the Orleans activation scheduler. Use +`OutboxPostmanExecutionMode.ThreadPool` only for blocking or legacy delivery +code, and do not read or mutate activation-local grain state from those +callbacks. For reusable delivery code, implement and register keyed postman services: @@ -208,6 +219,18 @@ streamManager = this.RegisterStreamManager(state.State.Tracker) await streamManager.EnsureExplicitSubscriptionsAsync(cancellationToken); ``` +The string namespace overload derives a stream id from the receiving grain +identity. Use the `StreamId` overload for an explicit stream keyed by another +application id: + +```csharp +streamManager = this.RegisterStreamManager(state.State.Tracker) + .ConfigureExplicitSubscription( + "StreamProvider", + StreamId.Create("prices", customerId), + HandlePriceChangedAsync); +``` + Tracked resume tokens are a per-subscription choice. The default is to pass the previous token when a tracker snapshot is supplied. Opt out when a subscription should attach without a resume token: diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index ac0bd015..20be2ee4 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -45,6 +45,16 @@ One NuGet package for now. Internal boundaries (state management vs messaging vs streaming) are kept clean so a future split is mechanical. Split only when dependency weight becomes a concrete problem. +Provider-specific integrations are the preferred split point when this +becomes more than one package. Keep provider-neutral state/outbox/tracking +and core stream manager APIs in `Egil.Orleans.Messaging`; move optional +provider dependencies to companion packages such as +`Egil.Orleans.Messaging.Streams.EventHubs`, +`Egil.Orleans.Messaging.State.AzureTable`, and +`Egil.Orleans.Messaging.State.AzureBlob`. This follows Orleans' own package +shape, where optional stream and storage providers live behind provider +packages instead of the core runtime package. + ### Capability namespaces and folders Source and test files are grouped by library tool. Tests mirror production @@ -447,15 +457,15 @@ public sealed class Outbox : IReadOnlyList>, IEquata initialisation and deliberate ops-level sequence-space resets. Document and warn. -### Outbox depth telemetry and configurable maximum +### Outbox depth telemetry and owner policy The outbox can grow unbounded if postman targets are down. Mitigation: - **Telemetry:** gauge for outbox depth per grain type, emitted on every write. Operators see growth before it becomes a crisis. -- **Configurable maximum:** users can set a max outbox size when - configuring the outbox processor. When exceeded, oldest messages are - automatically dropped (FIFO). This is opt-in; default is no cap. +- **Owner policy:** the owning grain controls the outbox. In + `ReconcileFailedAsync`, it can leave failed items pending, remove them, + dead-letter them, or trim old entries according to domain policy. - **Documentation:** storage providers have entity size limits (e.g. Azure Table = 1MB). Document the risk of unbounded growth. @@ -793,6 +803,20 @@ public sealed class StreamManager Action? onError = default, bool useTrackedResumeToken = true); + public StreamManager ConfigureExplicitSubscription( + string streamProviderName, + StreamId streamId, + Func onNextAsync, + Action? onError = default, + bool useTrackedResumeToken = true); + + public StreamManager ConfigureExplicitSubscription( + string streamProviderName, + StreamId streamId, + Func onNextAsync, + Action? onError = default, + bool useTrackedResumeToken = true); + public Task ResumeExplicitSubscriptionsAsync( CancellationToken cancellationToken = default); @@ -815,7 +839,10 @@ parameter. Orleans provides the concrete provider and stream id through `IStreamSubscriptionHandleFactory` when the implicit subscription fires. `ConfigureExplicitSubscription` requires a provider name because the library -must ask Orleans for the stream and durable subscription handles itself. +must ask Orleans for the stream and durable subscription handles itself. The +string namespace overload follows the library's grain-keyed convention and +derives the stream id from the receiving grain identity. Use the `StreamId` +overload when subscribing to an arbitrary Orleans explicit stream. `RegisterStreamManager()` can be called without a `MessageTracker` when the grain does not persist stream high-water marks. In that mode the manager @@ -883,6 +910,16 @@ public override async Task OnActivateAsync(CancellationToken ct) The `TEvent` generic argument is usually inferred from the handler method group. Users only specify it for inline lambdas or ambiguous method groups. +Use the `StreamId` overload when the target explicit stream is not keyed by +the receiving grain: + +```csharp +streamManager = this.RegisterStreamManager(state.Tracker) + .ConfigureExplicitSubscription( + "StreamProvider", + StreamId.Create("tariff-events", customerId), + HandleTariffChangedAsync); +``` ### Implementation changes from the previous design @@ -914,6 +951,9 @@ group. Users only specify it for inline lambdas or ambiguous method groups. the activation-time `MessageTracker` snapshot when a cursor exists for the same provider and namespace and that subscription has `useTrackedResumeToken: true`. +- If no handler is configured for the implicit stream namespace, the manager + throws during `OnSubscribed(...)`. A mismatched attribute/configuration is a + delivery-path misconfiguration and should not be a log-only condition. - Orleans controls activation and target stream identity. If the grain is not active, a matching stream event can activate it. - Implicit subscriptions do not show up as explicit handles from @@ -922,8 +962,10 @@ group. Users only specify it for inline lambdas or ambiguous method groups. ### Explicit subscription semantics -- `ConfigureExplicitSubscription(...)` records the provider, namespace, event - type, handler, and error callback. +- `ConfigureExplicitSubscription(...)` records the provider, stream identity, + event type, handler, and error callback. The namespace overload derives the + stream id from the receiving grain identity; the `StreamId` overload uses + the caller-provided stream identity directly. - `ResumeExplicitSubscriptionsAsync(...)` resumes all existing durable handles for each configured explicit stream from the activation-time `MessageTracker` cursor when one exists and that subscription has @@ -1436,11 +1478,14 @@ public async Task ReceiveReminder(string name, TickStatus status) by the processor, not a method users call. The name describes the data supplied to the processor and avoids implying ad-hoc outbox operations. - **First-registered-wins postman matching.** Simple dispatch model. - Order most-specific first. Unmatched items → `NoPostmanRegisteredException` - via `ReconcileFailedAsync`. + The metaphor is a switch statement: the first matching case handles the + item. Order most-specific first. Unmatched items → + `NoPostmanRegisteredException` via `ReconcileFailedAsync`. - **`PostAsync` swallows per-item errors.** Grain observes failures via `ReconcileFailedAsync` with attempt count. Processor never drops items - silently unless the grain explicitly removes them. + silently unless the grain explicitly removes them. Dead-letter and + max-depth policies belong in this reconciliation callback because the grain + owns the durable outbox state. - **Acknowledgement is explicit.** Successfully posted items are not removed by the processor directly. The grain removes and persists them in `AcknowledgePostedAsync`, preserving the outbox invariant that all durable diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs index acfc3621..9abf8792 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs @@ -46,10 +46,11 @@ namespace Egil.Orleans.Messaging.Outboxes; /// /// /// Unbounded growth risk: If postman targets are down, the outbox grows -/// without limit. Mitigations: telemetry (gauge for depth per grain type), -/// configurable max via (oldest -/// messages auto-dropped FIFO), and documentation of storage-provider entity -/// size limits (e.g., Azure Table = 1 MB). +/// without limit unless the owning grain applies a policy. The processor +/// reports depth telemetry and passes failures to +/// , where +/// the grain can leave items pending, dead-letter them, or drop old entries +/// before storage-provider entity limits are reached. /// /// /// Serialization: Decorated with [GenerateSerializer] for Orleans. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index db412ce3..bad7be38 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -66,6 +66,11 @@ internal OutboxProcessor( /// /// Registers a postman that handles items of type . /// + /// + /// Postman matching is first-match-wins, like a switch statement. Register + /// more specific message types before base interfaces or catch-all types. + /// Each outbox item is dispatched to at most one postman. + /// public OutboxProcessor AddPostman( Func postman) where TSub : TOutbox { diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs index df2a6eab..079db814 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs @@ -15,6 +15,12 @@ public enum OutboxPostmanExecutionMode /// /// Executes postman callbacks on the .NET thread pool. /// + /// + /// Use only for blocking or legacy delivery code that must not run on the + /// Orleans activation scheduler. Thread-pool postmen must not read or + /// mutate activation-local grain state. Prefer static/state-free callbacks, + /// injected thread-safe services, or calls. + /// ThreadPool } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs index b7219089..43ae04b6 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs @@ -119,8 +119,14 @@ public StreamManager ConfigureImplicitSubscription( } /// - /// Configures a durable Orleans explicit stream subscription. + /// Configures a durable Orleans explicit stream subscription using the + /// library's grain-keyed stream id convention. /// + /// + /// The stream id is derived from the receiving grain identity and + /// . Use the + /// overload when the explicit stream is keyed by another application id. + /// public StreamManager ConfigureExplicitSubscription( string streamProviderName, string streamNamespace, @@ -143,7 +149,7 @@ public StreamManager ConfigureExplicitSubscription( explicitSubscriptions.Add(new ExplicitSubscription( streamProviderName, - streamNamespace, + getStreamId(streamNamespace), onNextAsync, onError, useTrackedResumeToken)); @@ -169,6 +175,62 @@ public StreamManager ConfigureExplicitSubscription( useTrackedResumeToken); } + /// + /// Configures a durable Orleans explicit stream subscription for the + /// specified stream identity. + /// + public StreamManager ConfigureExplicitSubscription( + string streamProviderName, + StreamId streamId, + Func onNextAsync, + Action? onError = default, + bool useTrackedResumeToken = true) + { + ArgumentException.ThrowIfNullOrWhiteSpace(streamProviderName); + ArgumentNullException.ThrowIfNull(onNextAsync); + EnsureCanConfigure(); + + if (streamId.GetNamespace() is null) + { + throw new ArgumentException("StreamId must have a namespace.", nameof(streamId)); + } + + if (explicitSubscriptions.Any(subscription => + subscription.StreamProviderName == streamProviderName + && subscription.StreamId.Equals(streamId))) + { + throw new InvalidOperationException( + $"An explicit stream subscription for provider '{streamProviderName}' and stream id '{streamId}' has already been configured."); + } + + explicitSubscriptions.Add(new ExplicitSubscription( + streamProviderName, + streamId, + onNextAsync, + onError, + useTrackedResumeToken)); + + return this; + } + + /// + public StreamManager ConfigureExplicitSubscription( + string streamProviderName, + StreamId streamId, + Func onNextAsync, + Action? onError = default, + bool useTrackedResumeToken = true) + { + ArgumentNullException.ThrowIfNull(onNextAsync); + + return ConfigureExplicitSubscription( + streamProviderName, + streamId, + (item, cursor) => new ValueTask(onNextAsync(item, cursor)), + onError, + useTrackedResumeToken); + } + /// /// Resumes existing durable explicit subscription handles without creating /// new subscriptions. @@ -211,10 +273,9 @@ async Task IStreamManagerComponent.OnSubscribedAsync(IStreamSubscriptionHandleFa if (streamNamespace is null || !implicitSubscriptions.TryGetValue(streamNamespace, out var subscription)) { - logger.LogWarning( - "No implicit stream subscription handler is configured for namespace {StreamNamespace}.", - streamNamespace); - return; + throw new InvalidOperationException( + $"No implicit stream subscription handler is configured for provider '{handleFactory.ProviderName}' and stream id '{handleFactory.StreamId}'. " + + $"Call ConfigureImplicitSubscription(...) for namespace '{streamNamespace ?? ""}' during activation."); } await subscription.ResumeAsync(this, handleFactory); @@ -253,18 +314,19 @@ private async Task> ResumeImplicitAsync private async Task ResumeExplicitAsync( string streamProviderName, - string streamNamespace, + StreamId streamId, Func onNextAsync, Action? onError, bool useTrackedResumeToken, bool createIfMissing, CancellationToken cancellationToken) { + var streamNamespace = streamId.GetNamespace()!; try { cancellationToken.ThrowIfCancellationRequested(); - var stream = GetExplicitStream(streamProviderName, streamNamespace); + var stream = GetExplicitStream(streamProviderName, streamId); var handles = await stream.GetAllSubscriptionHandles(); if (handles.Count == 0 && createIfMissing) { @@ -316,10 +378,9 @@ private async Task ResumeExplicitAsync( private IAsyncStream GetExplicitStream( string streamProviderName, - string streamNamespace) + StreamId streamId) { var streamProvider = getStreamProvider(streamProviderName); - var streamId = getStreamId(streamNamespace); return streamProvider.GetStream(streamId); } @@ -461,6 +522,8 @@ private interface IExplicitSubscription { string StreamProviderName { get; } + StreamId StreamId { get; } + string StreamNamespace { get; } Task ResumeExistingAsync(StreamManager manager, CancellationToken cancellationToken); @@ -481,7 +544,7 @@ public Task ResumeAsync(StreamManager manager, IStreamSubscriptionHandleFactory private sealed class ExplicitSubscription( string streamProviderName, - string streamNamespace, + StreamId streamId, Func onNextAsync, Action? onError, bool useTrackedResumeToken) @@ -489,12 +552,14 @@ private sealed class ExplicitSubscription( { public string StreamProviderName => streamProviderName; - public string StreamNamespace => streamNamespace; + public StreamId StreamId => streamId; + + public string StreamNamespace => streamId.GetNamespace()!; public Task ResumeExistingAsync(StreamManager manager, CancellationToken cancellationToken) => manager.ResumeExplicitAsync( streamProviderName, - streamNamespace, + streamId, onNextAsync, onError, useTrackedResumeToken, @@ -504,7 +569,7 @@ public Task ResumeExistingAsync(StreamManager manager, CancellationToken cancell public Task EnsureAsync(StreamManager manager, CancellationToken cancellationToken) => manager.ResumeExplicitAsync( streamProviderName, - streamNamespace, + streamId, onNextAsync, onError, useTrackedResumeToken, diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs index 6685340c..8b5c2bb4 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerResumeTests.cs @@ -55,6 +55,24 @@ await manager Assert.Equal(1, stream.SubscribeCount); } + [Fact] + public async Task EnsureExplicitSubscriptionsAsync_uses_configured_stream_id() + { + var streamId = StreamId.Create("orders", "external"); + var stream = new FakeStream("provider-a", streamId); + var manager = CreateManager(null, stream); + + await manager + .ConfigureExplicitSubscription( + "provider-a", + streamId, + static (_, _) => ValueTask.CompletedTask) + .EnsureExplicitSubscriptionsAsync(TestContext.Current.CancellationToken); + + Assert.Equal(streamId, stream.ResolvedStreamId); + Assert.Equal(1, stream.SubscribeCount); + } + [Fact] public async Task ResumeExplicitSubscriptionsAsync_uses_tracker_resume_token_when_resuming_existing_handle() { @@ -136,6 +154,23 @@ public async Task Implicit_subscription_resume_uses_null_resume_token_when_subsc Assert.Equal(1, handleFactory.StringHandle.ResumeCount); } + [Fact] + public async Task Implicit_subscription_without_configured_handler_throws() + { + var streamId = StreamId.Create("orders", "one"); + var handleFactory = new FakeStreamSubscriptionHandleFactory( + "provider-a", + streamId, + new FakeSubscriptionHandle("provider-a", streamId)); + var manager = CreateManager(null, new FakeStream("provider-a", streamId)); + + var exception = await Assert.ThrowsAsync( + () => ((IStreamManagerComponent)manager).OnSubscribedAsync(handleFactory)); + + Assert.Contains("No implicit stream subscription handler is configured", exception.Message); + Assert.Contains("provider-a", exception.Message); + } + private static StreamManager CreateManager( MessageTracker? tracker, FakeStream stream) @@ -175,6 +210,7 @@ public IAsyncStream GetStream(StreamId streamId) { if (stream is IAsyncStream typedStream) { + stream.MarkResolved(streamId); return typedStream; } @@ -192,6 +228,8 @@ private sealed class FakeStream( public StreamSequenceToken? SubscribeToken { get; private set; } + public StreamId? ResolvedStreamId { get; private set; } + public bool IsRewindable => true; public string ProviderName => providerName; @@ -251,6 +289,16 @@ public bool Equals(IAsyncStream? other) => other is not null && string.Equals(ProviderName, other.ProviderName, StringComparison.Ordinal) && StreamId.Equals(other.StreamId); + + public void MarkResolved(StreamId streamId) + { + if (!StreamId.Equals(streamId)) + { + throw new InvalidOperationException($"Unexpected stream id '{streamId}'."); + } + + ResolvedStreamId = streamId; + } } private sealed class FakeSubscriptionHandle( From c4da881599660e7b59d88456e1797adc90fdd245 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 25 May 2026 23:20:20 +0000 Subject: [PATCH 40/82] feat(om)!: split provider-specific messaging packages Move Event Hubs stream enrichment into Egil.Orleans.Messaging.Streams.EventHubs so the core package no longer carries the Event Hubs dependency. Add Egil.Orleans.Messaging.State.AzureStorage with Azure Table/Blob storage-aware state manager registration and failure classification for optimistic concurrency failures. Rename WriteFailureKind to StorageFailureKind so write and clear recovery can share the same provider classification model. Update CI to pack all messaging packages and document the companion package model. BREAKING CHANGE: Event Hubs enrichment types moved out of Egil.Orleans.Messaging into Egil.Orleans.Messaging.Streams.EventHubs, and WriteFailureKind was renamed to StorageFailureKind. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../workflows/egil-orleans-messaging-ci.yml | 6 +- .../Directory.Packages.props | 1 + .../Egil.Orleans.Messaging.slnx | 4 + Egil.Orleans.Messaging/README.md | 32 +++- Egil.Orleans.Messaging/api-design.md | 70 +++++--- .../AzureStorage/AzureStorageStateManager.cs | 70 ++++++++ ...orageStateManagerRegistrationExtensions.cs | 28 ++++ ...torageStateManagerSiloBuilderExtensions.cs | 24 +++ ...rleans.Messaging.State.AzureStorage.csproj | 54 ++++++ ...Orleans.Messaging.Streams.EventHubs.csproj | 54 ++++++ .../EventHubs/EnrichedEventHubAdapter.cs | 0 .../EnrichedEventHubAdapterExtensions.cs | 0 .../EnrichedEventHubSequenceToken.cs | 31 +++- .../Egil.Orleans.Messaging.csproj | 1 - .../State/DefaultStateManager.cs | 4 +- .../State/StateManager.cs | 23 ++- .../State/StorageFailureKind.cs | 17 ++ .../State/WriteFailureKind.cs | 17 -- .../Streams/IStreamSequenceTokenMetadata.cs | 23 +++ .../Streams/StreamCursor.cs | 56 +++---- .../Streams/StreamCursorJsonConverter.cs | 52 +----- .../Tracking/MessageTrackerJsonConverter.cs | 44 ----- ...StateManagerRegistrationExtensionsTests.cs | 30 ++++ .../AzureStorageStateManagerTests.cs | 155 ++++++++++++++++++ ....Messaging.State.AzureStorage.Tests.csproj | 40 +++++ ...s.Messaging.Streams.EventHubs.Tests.csproj | 40 +++++ .../EnrichedEventHubAdapterExtensionsTests.cs | 0 .../EventHubs/EnrichedEventHubAdapterTests.cs | 0 .../GlobalUsings.cs | 1 - .../Streams/StreamCursorJsonConverterTests.cs | 108 ------------ .../Streams/StreamCursorTests.cs | 51 ++++-- .../MessageTrackerJsonConverterTests.cs | 102 +----------- 32 files changed, 732 insertions(+), 406 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerRegistrationExtensions.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerSiloBuilderExtensions.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/Egil.Orleans.Messaging.Streams.EventHubs.csproj rename Egil.Orleans.Messaging/src/{Egil.Orleans.Messaging/Streams => Egil.Orleans.Messaging.Streams.EventHubs}/EventHubs/EnrichedEventHubAdapter.cs (100%) rename Egil.Orleans.Messaging/src/{Egil.Orleans.Messaging/Streams => Egil.Orleans.Messaging.Streams.EventHubs}/EventHubs/EnrichedEventHubAdapterExtensions.cs (100%) rename Egil.Orleans.Messaging/src/{Egil.Orleans.Messaging/Streams => Egil.Orleans.Messaging.Streams.EventHubs}/EventHubs/EnrichedEventHubSequenceToken.cs (85%) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StorageFailureKind.cs delete mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/WriteFailureKind.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamSequenceTokenMetadata.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerRegistrationExtensionsTests.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerTests.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/Egil.Orleans.Messaging.State.AzureStorage.Tests.csproj create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/Egil.Orleans.Messaging.Streams.EventHubs.Tests.csproj rename Egil.Orleans.Messaging/test/{Egil.Orleans.Messaging.Tests/Streams => Egil.Orleans.Messaging.Streams.EventHubs.Tests}/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs (100%) rename Egil.Orleans.Messaging/test/{Egil.Orleans.Messaging.Tests/Streams => Egil.Orleans.Messaging.Streams.EventHubs.Tests}/EventHubs/EnrichedEventHubAdapterTests.cs (100%) diff --git a/.github/workflows/egil-orleans-messaging-ci.yml b/.github/workflows/egil-orleans-messaging-ci.yml index f8ab40a4..f5f8cacf 100644 --- a/.github/workflows/egil-orleans-messaging-ci.yml +++ b/.github/workflows/egil-orleans-messaging-ci.yml @@ -49,8 +49,10 @@ jobs: - name: Build and pack run: | - dotnet build ./Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj -c Release -f net10.0 -p:ContinuousIntegrationBuild=true - dotnet pack ./Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj -c Release --no-build -p:ContinuousIntegrationBuild=true --output ${{ env.NuGetDirectory }} + dotnet build ./Egil.Orleans.Messaging/Egil.Orleans.Messaging.slnx -c Release -p:ContinuousIntegrationBuild=true + for project in ./Egil.Orleans.Messaging/src/*/*.csproj; do + dotnet pack "$project" -c Release --no-build -p:ContinuousIntegrationBuild=true --output ${{ env.NuGetDirectory }} + done - uses: actions/upload-artifact@v4 with: diff --git a/Egil.Orleans.Messaging/Directory.Packages.props b/Egil.Orleans.Messaging/Directory.Packages.props index ea96da40..bc28a862 100644 --- a/Egil.Orleans.Messaging/Directory.Packages.props +++ b/Egil.Orleans.Messaging/Directory.Packages.props @@ -14,6 +14,7 @@ + diff --git a/Egil.Orleans.Messaging/Egil.Orleans.Messaging.slnx b/Egil.Orleans.Messaging/Egil.Orleans.Messaging.slnx index 779ae7ba..489b35c4 100644 --- a/Egil.Orleans.Messaging/Egil.Orleans.Messaging.slnx +++ b/Egil.Orleans.Messaging/Egil.Orleans.Messaging.slnx @@ -1,8 +1,12 @@ + + + + diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index 953668ba..25040a8e 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -16,13 +16,19 @@ Composable messaging infrastructure for Microsoft Orleans grains. dotnet add package Egil.Orleans.Messaging ``` +Provider-specific integrations are shipped as companion packages: + +```shell +dotnet add package Egil.Orleans.Messaging.Streams.EventHubs +dotnet add package Egil.Orleans.Messaging.State.AzureStorage +``` + Use the capability namespaces for the tools you need: ```csharp using Egil.Orleans.Messaging.Outboxes; using Egil.Orleans.Messaging.State; using Egil.Orleans.Messaging.Streams; -using Egil.Orleans.Messaging.Streams.EventHubs; using Egil.Orleans.Messaging.Tracking; ``` @@ -43,6 +49,19 @@ Register the default state manager factory on the silo: siloBuilder.AddDefaultStateManager("state"); ``` +For Orleans Azure Table or Blob grain storage, install +`Egil.Orleans.Messaging.State.AzureStorage` and register the Azure-aware +factory instead: + +```csharp +siloBuilder.AddAzureStorageStateManager("state"); +``` + +The Azure-aware manager treats Azure Storage optimistic-concurrency failures +such as HTTP 412 as definite non-persistence, so writes and clears can fail +fast without an unnecessary recovery read. Ambiguous failures, such as +transient 5xx responses, still use read-back recovery. + Then wrap the Orleans persistent state facet during activation: ```csharp @@ -251,6 +270,17 @@ this.RegisterStreamManager() async (message, cursor) => await UpdateProjectionAsync(message)); ``` +Install `Egil.Orleans.Messaging.Streams.EventHubs` when using Orleans Event +Hubs streams and the enriched adapter/token support: + +```csharp +using Egil.Orleans.Messaging.Streams.EventHubs; +using Orleans.Hosting; +``` + +The core package can consume provider-specific token metadata through +`IStreamSequenceTokenMetadata` without taking a direct Event Hubs dependency. + ## Scope This package is messaging infrastructure, not an event-sourcing or CQRS framework. It wraps Orleans state, outbox dispatch, receiver deduplication, and stream subscription management while leaving domain modeling, read models, transport targets, and operational policy to the application. diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 20be2ee4..f5e6fc00 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -41,32 +41,38 @@ A set of composable building blocks for Orleans grains that need: ### Packaging -One NuGet package for now. Internal boundaries (state management vs -messaging vs streaming) are kept clean so a future split is mechanical. -Split only when dependency weight becomes a concrete problem. - -Provider-specific integrations are the preferred split point when this -becomes more than one package. Keep provider-neutral state/outbox/tracking -and core stream manager APIs in `Egil.Orleans.Messaging`; move optional -provider dependencies to companion packages such as -`Egil.Orleans.Messaging.Streams.EventHubs`, -`Egil.Orleans.Messaging.State.AzureTable`, and -`Egil.Orleans.Messaging.State.AzureBlob`. This follows Orleans' own package -shape, where optional stream and storage providers live behind provider -packages instead of the core runtime package. +Provider-neutral state/outbox/tracking and core stream manager APIs live in +`Egil.Orleans.Messaging`. Provider-specific integrations live in companion +packages so consumers do not inherit optional Azure/Event Hubs dependencies +unless they use those providers. + +Current packages: + +| Package | Contents | +| ------- | -------- | +| `Egil.Orleans.Messaging` | State manager base/default implementation, outbox, postman, tracking, stream manager, provider-neutral stream cursor/token metadata. | +| `Egil.Orleans.Messaging.Streams.EventHubs` | Orleans Event Hubs stream adapter and enriched Event Hubs sequence token support. | +| `Egil.Orleans.Messaging.State.AzureStorage` | Azure Table/Blob storage-aware state manager factory and failure classification. | + +Future provider integrations should follow the same shape, for example +dedicated packages for provider-specific state manager behavior rather than +adding those dependencies to the core package. This follows Orleans' own +package shape, where optional stream and storage providers live behind +provider packages instead of the core runtime package. ### Capability namespaces and folders Source and test files are grouped by library tool. Tests mirror production folders so behavior is easy to find from the type under test. -| Capability | Production namespace | Test folder | -| ---------- | -------------------- | ----------- | -| State management | `Egil.Orleans.Messaging.State` | `State/` | -| Outbox | `Egil.Orleans.Messaging.Outboxes` | `Outboxes/` | -| Receiver tracking | `Egil.Orleans.Messaging.Tracking` | `Tracking/` | -| Streams | `Egil.Orleans.Messaging.Streams` | `Streams/` | -| Event Hub stream enrichment | `Egil.Orleans.Messaging.Streams.EventHubs` | `Streams/EventHubs/` | +| Capability | Package | Production namespace | Test folder | +| ---------- | ------- | -------------------- | ----------- | +| State management | `Egil.Orleans.Messaging` | `Egil.Orleans.Messaging.State` | `State/` | +| Outbox | `Egil.Orleans.Messaging` | `Egil.Orleans.Messaging.Outboxes` | `Outboxes/` | +| Receiver tracking | `Egil.Orleans.Messaging` | `Egil.Orleans.Messaging.Tracking` | `Tracking/` | +| Streams | `Egil.Orleans.Messaging` | `Egil.Orleans.Messaging.Streams` | `Streams/` | +| Event Hub stream enrichment | `Egil.Orleans.Messaging.Streams.EventHubs` | `Egil.Orleans.Messaging.Streams.EventHubs` | `EventHubs/` | +| Azure Storage state manager | `Egil.Orleans.Messaging.State.AzureStorage` | `Egil.Orleans.Messaging.State.AzureStorage` | `AzureStorage/` | Extension entry points live in the namespace of the type they extend so they are discoverable from normal Orleans, hosting, and DI imports: @@ -264,11 +270,23 @@ Each storage provider we care to optimise for may ship its own `IStateManager` that: - Inspects provider-specific exception types to classify failures as - *DefinitelyDidNotPersist* (skip re-read, revert + rethrow) vs - *UnknownOutcome* (re-read). + `StorageFailureKind.DidNotPersist` (skip re-read, revert + rethrow) vs + `StorageFailureKind.UnknownOutcome` (re-read). - Optionally exploits provider features (conditional writes, blob versions) to avoid the re-read. +`StorageFailureKind` is intentionally named for storage operations rather +than only writes. The same classification is useful for clear operations: +an Azure Storage HTTP 412/ETag conflict definitely did not clear the record, +while transient 5xx errors remain ambiguous and require read-back recovery. +Read failures do not use the classification because no local committed state +has been tentatively changed. + +The Azure Storage companion package treats Orleans/Azure optimistic +concurrency failures (`InconsistentStateException` and Azure Storage HTTP +412) as `DidNotPersist` for writes and clears. Other provider failures stay +`UnknownOutcome`, preserving the default read-back recovery path. + ### Why a wrapper, not extension methods The wrapper was debated — recovery logic is stateless, could be an @@ -1521,8 +1539,12 @@ public async Task ReceiveReminder(string name, TickStatus status) ### Namespace -Flat: all public types in `Egil.Orleans.Messaging`. One `using` -statement. ~15-20 public types — not crowded enough to split. +Public types are grouped by capability namespace: +`Egil.Orleans.Messaging.State`, `Egil.Orleans.Messaging.Outboxes`, +`Egil.Orleans.Messaging.Tracking`, and `Egil.Orleans.Messaging.Streams`. +Provider-specific companion packages add provider namespaces such as +`Egil.Orleans.Messaging.Streams.EventHubs` and +`Egil.Orleans.Messaging.State.AzureStorage`. ### Orleans `[Alias]` on serializable types diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs new file mode 100644 index 00000000..4e47c723 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs @@ -0,0 +1,70 @@ +using Azure; +using Orleans.Storage; + +namespace Egil.Orleans.Messaging.State.AzureStorage; + +/// +/// Azure Table/Blob storage-aware . +/// +/// The grain state type. +public sealed class AzureStorageStateManager : StateManagerBase + where T : class, IEquatable +{ + /// + /// Creates a state manager around an Orleans Azure Storage-backed state facet. + /// + public AzureStorageStateManager(IPersistentState storage) + : base(storage) + { + } + + /// + protected override StorageFailureKind ClassifyWriteFailure(Exception exception) + { + ArgumentNullException.ThrowIfNull(exception); + return AzureStorageFailureClassifier.Classify(exception); + } + + /// + protected override StorageFailureKind ClassifyClearFailure(Exception exception) + { + ArgumentNullException.ThrowIfNull(exception); + return AzureStorageFailureClassifier.Classify(exception); + } +} + +/// +/// Factory that creates instances. +/// +public sealed class AzureStorageStateManagerFactory : IStateManagerFactory + where T : class, IEquatable +{ + /// + public IStateManager Create(IPersistentState storage) + { + ArgumentNullException.ThrowIfNull(storage); + return new AzureStorageStateManager(storage); + } +} + +internal static class AzureStorageFailureClassifier +{ + public static StorageFailureKind Classify(Exception exception) + { + for (var current = exception; current is not null; current = current.InnerException) + { + if (current is InconsistentStateException) + { + return StorageFailureKind.DidNotPersist; + } + + if (current is RequestFailedException requestFailed + && requestFailed.Status is 412) + { + return StorageFailureKind.DidNotPersist; + } + } + + return StorageFailureKind.UnknownOutcome; + } +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerRegistrationExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerRegistrationExtensions.cs new file mode 100644 index 00000000..a5d55d37 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerRegistrationExtensions.cs @@ -0,0 +1,28 @@ +using Egil.Orleans.Messaging.State; +using Egil.Orleans.Messaging.State.AzureStorage; + +namespace Microsoft.Extensions.DependencyInjection; + +/// +/// Registration helpers for Azure Storage-aware state managers. +/// +public static class AzureStorageStateManagerRegistrationExtensions +{ + extension(IServiceCollection services) + { + /// + /// Registers the Azure Storage-aware keyed + /// for the given storage name. + /// + public IServiceCollection AddAzureStorageStateManager(string storageName) + { + ArgumentNullException.ThrowIfNull(services); + ArgumentException.ThrowIfNullOrWhiteSpace(storageName); + + return services.AddKeyedSingleton( + typeof(IStateManagerFactory<>), + storageName, + typeof(AzureStorageStateManagerFactory<>)); + } + } +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerSiloBuilderExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerSiloBuilderExtensions.cs new file mode 100644 index 00000000..69603d63 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerSiloBuilderExtensions.cs @@ -0,0 +1,24 @@ +using Microsoft.Extensions.DependencyInjection; + +namespace Orleans.Hosting; + +/// +/// Silo builder registration helpers for Azure Storage-aware state managers. +/// +public static class AzureStorageStateManagerSiloBuilderExtensions +{ + extension(ISiloBuilder builder) + { + /// + /// Registers the Azure Storage-aware state manager factory on the silo builder. + /// + public ISiloBuilder AddAzureStorageStateManager(string storageName) + { + ArgumentNullException.ThrowIfNull(builder); + ArgumentException.ThrowIfNullOrWhiteSpace(storageName); + + builder.ConfigureServices(services => services.AddAzureStorageStateManager(storageName)); + return builder; + } + } +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj new file mode 100644 index 00000000..1db5144f --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj @@ -0,0 +1,54 @@ + + + + net10.0 + enable + enable + true + embedded + + + + Egil.Orleans.Messaging.State.AzureStorage + Orleans Messaging Azure Storage State Manager + Egil Hansen + Egil Hansen + + Azure Table and Blob storage-aware state manager support for Egil.Orleans.Messaging. + + README.md + orleans,messaging,state,azure-storage,table-storage,blob-storage,egil + Egil Hansen + https://github.com/egil/framework + https://github.com/egil/framework + git + LICENSE + true + true + true + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + + + + True + \ + + + True + \ + + + + diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/Egil.Orleans.Messaging.Streams.EventHubs.csproj b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/Egil.Orleans.Messaging.Streams.EventHubs.csproj new file mode 100644 index 00000000..9088fb40 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/Egil.Orleans.Messaging.Streams.EventHubs.csproj @@ -0,0 +1,54 @@ + + + + net10.0 + enable + enable + true + embedded + + + + Egil.Orleans.Messaging.Streams.EventHubs + Orleans Messaging Event Hubs Extensions + Egil Hansen + Egil Hansen + + Event Hubs-specific stream token enrichment for Egil.Orleans.Messaging. + + README.md + orleans,messaging,streams,eventhubs,event-hubs,egil + Egil Hansen + https://github.com/egil/framework + https://github.com/egil/framework + git + LICENSE + true + true + true + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + + + + True + \ + + + True + \ + + + + diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapter.cs similarity index 100% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapter.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapter.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapterExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapterExtensions.cs similarity index 100% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubAdapterExtensions.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapterExtensions.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs similarity index 85% rename from Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubSequenceToken.cs rename to Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs index b64b6e05..db319ac7 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/EventHubs/EnrichedEventHubSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs @@ -42,15 +42,15 @@ namespace Egil.Orleans.Messaging.Streams.EventHubs; /// /// /// Serialization: Inherits Orleans serialization from -/// . The library's STJ converter for -/// recognizes this subtype via a $kind -/// discriminator and round-trips , -/// , and fields. +/// . Persist stream positions through +/// or another application-owned checkpoint +/// model; the provider-neutral core package does not serialize Event +/// Hubs-specific token subtypes directly. /// /// [GenerateSerializer] [Alias("egil.orleans.messaging.EnrichedEventHubSequenceToken")] -public class EnrichedEventHubSequenceToken : EventHubSequenceTokenV2 +public class EnrichedEventHubSequenceToken : EventHubSequenceTokenV2, IStreamSequenceTokenMetadata { /// /// The wall-clock time the event was enqueued at the Event Hub broker. @@ -125,4 +125,25 @@ public EnrichedEventHubSequenceToken() { ProviderName = string.Empty; } + + /// + public bool TryGetEnqueuedTime(out DateTimeOffset enqueuedTime) + { + enqueuedTime = EnqueuedTime; + return true; + } + + /// + public bool TryGetProviderName([NotNullWhen(true)] out string? providerName) + { + providerName = ProviderName; + return !string.IsNullOrWhiteSpace(providerName); + } + + /// + public bool TryGetTraceParent([NotNullWhen(true)] out string? traceParent) + { + traceParent = TraceParent; + return traceParent is not null; + } } \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj index cdd51962..945c637b 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Egil.Orleans.Messaging.csproj @@ -38,7 +38,6 @@ - diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs index d84d6329..1007edb9 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs @@ -12,9 +12,9 @@ public DefaultStateManager(IPersistentState storage) } /// - protected override WriteFailureKind ClassifyWriteFailure(Exception exception) + protected override StorageFailureKind ClassifyWriteFailure(Exception exception) { ArgumentNullException.ThrowIfNull(exception); - return WriteFailureKind.UnknownOutcome; + return StorageFailureKind.UnknownOutcome; } } \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs index 87b33ffe..8e5caa49 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs @@ -117,7 +117,7 @@ public async Task WriteAsync(T newState) catch (Exception ex) { var failureKind = ClassifyWriteFailure(ex); - if (failureKind is WriteFailureKind.DidNotPersist) + if (failureKind is StorageFailureKind.DidNotPersist) { // Provider-specific classification says the write never reached durable storage. // Revert our local fence immediately and rethrow the original write error. @@ -178,8 +178,16 @@ public async Task ClearAsync() state = storage.State; return; } - catch + catch (Exception ex) { + var failureKind = ClassifyClearFailure(ex); + if (failureKind is StorageFailureKind.DidNotPersist) + { + state = previousState; + storage.State = previousState; + throw; + } + try { await storage.ReadStateAsync(); @@ -211,7 +219,16 @@ public async Task ClearAsync() /// /// Classifies a write failure so the base can decide recovery behavior. /// - protected abstract WriteFailureKind ClassifyWriteFailure(Exception exception); + protected abstract StorageFailureKind ClassifyWriteFailure(Exception exception); + + /// + /// Classifies a clear failure so the base can decide recovery behavior. + /// + protected virtual StorageFailureKind ClassifyClearFailure(Exception exception) + { + ArgumentNullException.ThrowIfNull(exception); + return StorageFailureKind.UnknownOutcome; + } private static bool IsEquivalent(T persisted, T attempted) { diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StorageFailureKind.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StorageFailureKind.cs new file mode 100644 index 00000000..b36956af --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StorageFailureKind.cs @@ -0,0 +1,17 @@ +namespace Egil.Orleans.Messaging.State; + +/// +/// Classifies a storage operation failure for recovery decisions. +/// +public enum StorageFailureKind +{ + /// + /// Unknown outcome. The operation may have persisted. Run read-back recovery. + /// + UnknownOutcome, + + /// + /// The operation definitely did not persist. Skip read-back and rethrow. + /// + DidNotPersist +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/WriteFailureKind.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/WriteFailureKind.cs deleted file mode 100644 index ccf0e35a..00000000 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/WriteFailureKind.cs +++ /dev/null @@ -1,17 +0,0 @@ -namespace Egil.Orleans.Messaging.State; - -/// -/// Classifies a storage write failure for recovery decisions. -/// -public enum WriteFailureKind -{ - /// - /// Unknown outcome. The write may have persisted. Run read-back recovery. - /// - UnknownOutcome, - - /// - /// The write definitely did not persist. Skip read-back and rethrow. - /// - DidNotPersist -} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamSequenceTokenMetadata.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamSequenceTokenMetadata.cs new file mode 100644 index 00000000..e41e9f96 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/IStreamSequenceTokenMetadata.cs @@ -0,0 +1,23 @@ +namespace Egil.Orleans.Messaging.Streams; + +/// +/// Optional metadata contract for stream sequence token implementations that +/// carry provider-specific diagnostics. +/// +public interface IStreamSequenceTokenMetadata +{ + /// + /// Attempts to extract the broker-side enqueue time from the token. + /// + bool TryGetEnqueuedTime(out DateTimeOffset enqueuedTime); + + /// + /// Attempts to extract the stream provider name from the token. + /// + bool TryGetProviderName([NotNullWhen(true)] out string? providerName); + + /// + /// Attempts to extract the W3C traceparent value from the token. + /// + bool TryGetTraceParent([NotNullWhen(true)] out string? traceParent); +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs index ade12aaf..e392a7d0 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs @@ -1,5 +1,4 @@ using System.Text.Json.Serialization; -using Egil.Orleans.Messaging.Streams.EventHubs; using Egil.Orleans.Messaging.Tracking; using Orleans.Streams; @@ -17,20 +16,15 @@ namespace Egil.Orleans.Messaging.Streams; /// subtypes via a $kind discriminator: /// /// EventSequenceToken — Orleans SimpleMessageStream -/// EventHubSequenceToken — Orleans Event Hub provider v1 -/// EventHubSequenceTokenV2 — Orleans Event Hub provider v2 -/// — library-shipped v2 -/// subclass carrying /// /// Unknown subtypes throw at serialization time — silently dropping the cursor /// would corrupt dedup state. /// /// -/// Enriched time: Use to extract the -/// broker-side enqueue time when the token is an . -/// Returns false for all other token types. This enables end-to-end lag -/// histograms in the telemetry without coupling the -/// cursor type to a specific streaming provider. +/// Provider-specific metadata: Tokens can implement +/// to expose broker-side enqueue +/// time, provider name, or trace context without coupling this core package to +/// a specific streaming provider. /// /// /// Serialization: Decorated with [GenerateSerializer] for Orleans @@ -61,25 +55,21 @@ public sealed record StreamCursor( /// . /// /// - /// Returns true only when the token is an - /// . Callers can use this to - /// compute end-to-end lag (now - enqueuedTime) without coupling - /// to a specific streaming provider. + /// Returns true when the token implements + /// and exposes an enqueue time. /// /// /// The time the event was enqueued at the broker, or default if /// the token type does not carry this information. /// /// - /// true if is an - /// and the time was extracted; - /// false otherwise. + /// true if the time was extracted; false otherwise. /// public bool TryGetEnqueuedTime(out DateTimeOffset enqueuedTime) { - if (Token is EnrichedEventHubSequenceToken token) + if (Token is IStreamSequenceTokenMetadata metadata + && metadata.TryGetEnqueuedTime(out enqueuedTime)) { - enqueuedTime = token.EnqueuedTime; return true; } @@ -92,24 +82,22 @@ public bool TryGetEnqueuedTime(out DateTimeOffset enqueuedTime) /// . /// /// - /// Returns true only when the token is an - /// . Enables provider-aware - /// dedup and diagnostics without coupling to a specific streaming provider. + /// Returns true when the token implements + /// and exposes a provider name, + /// or when is populated. /// /// /// The name of the stream provider that delivered this event, or /// null if the token type does not carry this information. /// /// - /// true if is an - /// or - /// is populated; false otherwise. + /// true if a provider name was extracted; false otherwise. /// public bool TryGetProviderName([NotNullWhen(true)] out string? providerName) { - if (Token is EnrichedEventHubSequenceToken token) + if (Token is IStreamSequenceTokenMetadata metadata + && metadata.TryGetProviderName(out providerName)) { - providerName = token.ProviderName; return true; } @@ -128,9 +116,8 @@ public bool TryGetProviderName([NotNullWhen(true)] out string? providerName) /// . /// /// - /// Returns true only when the token is an - /// whose - /// is not null. + /// Returns true when the token implements + /// and exposes a traceparent. /// uses this to create /// s — correlating consumer /// spans to producer spans without creating multi-hour parent-child traces. @@ -142,16 +129,13 @@ public bool TryGetProviderName([NotNullWhen(true)] out string? providerName) /// publish time. /// /// - /// true if is an - /// with a non-null - /// ; false - /// otherwise. + /// true if a traceparent was extracted; false otherwise. /// public bool TryGetTraceParent([NotNullWhen(true)] out string? traceParent) { - if (Token is EnrichedEventHubSequenceToken token && token.TraceParent is not null) + if (Token is IStreamSequenceTokenMetadata metadata + && metadata.TryGetTraceParent(out traceParent)) { - traceParent = token.TraceParent; return true; } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs index 7c24dbba..c804211d 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs @@ -1,8 +1,6 @@ using System.Text.Json; using System.Text.Json.Serialization; -using Egil.Orleans.Messaging.Streams.EventHubs; using Orleans.Providers.Streams.Common; -using Orleans.Streaming.EventHubs; using Orleans.Streams; namespace Egil.Orleans.Messaging.Streams; @@ -16,11 +14,8 @@ namespace Egil.Orleans.Messaging.Streams; /// /// wraps a stream namespace and an optional /// StreamSequenceToken. The token is polymorphic and this converter -/// supports Orleans built-in sequence token types only: -/// , -/// EventHubSequenceTokenV2, -/// EventHubSequenceToken, EventSequenceTokenV2, and -/// EventSequenceToken. +/// supports provider-neutral Orleans sequence token types only: +/// EventSequenceTokenV2 and EventSequenceToken. /// /// /// Registered on via [JsonConverter]. @@ -33,7 +28,7 @@ internal sealed class StreamCursorJsonConverter : JsonConverter private const string FeatureRequestUrl = "https://github.com/egil/framework/issues"; private const string SupportedKinds = - "EventSequenceToken, EventSequenceTokenV2, EventHubSequenceToken, EventHubSequenceTokenV2, EnrichedEventHubSequenceToken"; + "EventSequenceToken, EventSequenceTokenV2"; /// public override StreamCursor? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) @@ -70,21 +65,6 @@ public override void Write(Utf8JsonWriter writer, StreamCursor value, JsonSerial { StreamSequenceTokenKinds.EventSequenceToken => new EventSequenceToken(model.SequenceNumber, model.EventIndex), StreamSequenceTokenKinds.EventSequenceTokenV2 => new EventSequenceTokenV2(model.SequenceNumber, model.EventIndex), - StreamSequenceTokenKinds.EventHubSequenceToken => new EventHubSequenceToken( - model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), - model.SequenceNumber, - model.EventIndex), - StreamSequenceTokenKinds.EventHubSequenceTokenV2 => new EventHubSequenceTokenV2( - model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), - model.SequenceNumber, - model.EventIndex), - StreamSequenceTokenKinds.EnrichedEventHubSequenceToken => new EnrichedEventHubSequenceToken( - model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), - model.SequenceNumber, - model.EventIndex, - model.EnqueuedTime ?? throw new JsonException("Missing EnqueuedTime."), - model.ProviderName ?? throw new JsonException("Missing ProviderName."), - model.TraceParent), _ => throw CreateUnsupportedTokenException(model.Kind ?? "") }; } @@ -98,24 +78,6 @@ public override void Write(Utf8JsonWriter writer, StreamCursor value, JsonSerial return value switch { - EnrichedEventHubSequenceToken token => new StreamSequenceTokenJsonModel( - Kind: StreamSequenceTokenKinds.EnrichedEventHubSequenceToken, - SequenceNumber: token.SequenceNumber, - EventIndex: token.EventIndex, - EventHubOffset: token.EventHubOffset, - EnqueuedTime: token.EnqueuedTime.UtcDateTime, - ProviderName: token.ProviderName, - TraceParent: token.TraceParent), - EventHubSequenceTokenV2 token => new StreamSequenceTokenJsonModel( - Kind: StreamSequenceTokenKinds.EventHubSequenceTokenV2, - SequenceNumber: token.SequenceNumber, - EventIndex: token.EventIndex, - EventHubOffset: token.EventHubOffset), - EventHubSequenceToken token => new StreamSequenceTokenJsonModel( - Kind: StreamSequenceTokenKinds.EventHubSequenceToken, - SequenceNumber: token.SequenceNumber, - EventIndex: token.EventIndex, - EventHubOffset: token.EventHubOffset), EventSequenceTokenV2 token => new StreamSequenceTokenJsonModel( Kind: StreamSequenceTokenKinds.EventSequenceTokenV2, SequenceNumber: token.SequenceNumber, @@ -144,17 +106,11 @@ private sealed record StreamSequenceTokenJsonModel( string? Kind = null, long SequenceNumber = 0, int EventIndex = 0, - string? EventHubOffset = null, - DateTime? EnqueuedTime = null, - string? ProviderName = null, - string? TraceParent = null); + string? EventHubOffset = null); private static class StreamSequenceTokenKinds { public const string EventSequenceToken = "event-sequence"; public const string EventSequenceTokenV2 = "event-sequence-v2"; - public const string EventHubSequenceToken = "event-hub"; - public const string EventHubSequenceTokenV2 = "event-hub-v2"; - public const string EnrichedEventHubSequenceToken = "enriched-event-hub"; } } \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs index 9ae56029..b621c96f 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs @@ -3,9 +3,7 @@ using System.Text.Json; using System.Text.Json.Serialization; using Egil.Orleans.Messaging.Streams; -using Egil.Orleans.Messaging.Streams.EventHubs; using Orleans.Providers.Streams.Common; -using Orleans.Streaming.EventHubs; using Orleans.Streams; namespace Egil.Orleans.Messaging.Tracking; @@ -126,21 +124,6 @@ private static StreamCursor FromJsonModel(StreamCursorJsonModel model) => { StreamSequenceTokenKinds.EventSequenceToken => new EventSequenceToken(model.SequenceNumber, model.EventIndex), StreamSequenceTokenKinds.EventSequenceTokenV2 => new EventSequenceTokenV2(model.SequenceNumber, model.EventIndex), - StreamSequenceTokenKinds.EventHubSequenceToken => new EventHubSequenceToken( - model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), - model.SequenceNumber, - model.EventIndex), - StreamSequenceTokenKinds.EventHubSequenceTokenV2 => new EventHubSequenceTokenV2( - model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), - model.SequenceNumber, - model.EventIndex), - StreamSequenceTokenKinds.EnrichedEventHubSequenceToken => new EnrichedEventHubSequenceToken( - model.EventHubOffset ?? throw new JsonException("Missing EventHubOffset."), - model.SequenceNumber, - model.EventIndex, - model.EnqueuedTime ?? throw new JsonException("Missing EnqueuedTime."), - model.ProviderName ?? throw new JsonException("Missing ProviderName."), - model.TraceParent), _ => throw new JsonException($"Unsupported stream sequence token kind '{model.Kind}'.") }; } @@ -160,30 +143,6 @@ private static StreamCursorJsonModel ToJsonModel(StreamCursor value) => return value switch { - EnrichedEventHubSequenceToken token => new StreamSequenceTokenJsonModel( - StreamSequenceTokenKinds.EnrichedEventHubSequenceToken, - token.SequenceNumber, - token.EventIndex, - token.EventHubOffset, - token.EnqueuedTime, - token.ProviderName, - token.TraceParent), - EventHubSequenceTokenV2 token => new StreamSequenceTokenJsonModel( - StreamSequenceTokenKinds.EventHubSequenceTokenV2, - token.SequenceNumber, - token.EventIndex, - token.EventHubOffset, - null, - null, - null), - EventHubSequenceToken token => new StreamSequenceTokenJsonModel( - StreamSequenceTokenKinds.EventHubSequenceToken, - token.SequenceNumber, - token.EventIndex, - token.EventHubOffset, - null, - null, - null), EventSequenceTokenV2 token => new StreamSequenceTokenJsonModel( StreamSequenceTokenKinds.EventSequenceTokenV2, token.SequenceNumber, @@ -244,8 +203,5 @@ private static class StreamSequenceTokenKinds { public const string EventSequenceToken = "event-sequence"; public const string EventSequenceTokenV2 = "event-sequence-v2"; - public const string EventHubSequenceToken = "event-hub"; - public const string EventHubSequenceTokenV2 = "event-hub-v2"; - public const string EnrichedEventHubSequenceToken = "enriched-event-hub"; } } \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerRegistrationExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerRegistrationExtensionsTests.cs new file mode 100644 index 00000000..8478a0f6 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerRegistrationExtensionsTests.cs @@ -0,0 +1,30 @@ +namespace Egil.Orleans.Messaging.State.AzureStorage.Tests.AzureStorage; + +public sealed class AzureStorageStateManagerRegistrationExtensionsTests +{ + [Fact] + public void AddAzureStorageStateManager_registers_keyed_factory() + { + var services = new ServiceCollection(); + + services.AddAzureStorageStateManager("state"); + + using var provider = services.BuildServiceProvider(); + var factory = provider.GetRequiredKeyedService>("state"); + + Assert.IsType>(factory); + } + + [Fact] + public void AddAzureStorageStateManager_throws_for_empty_storage_name() + { + var services = new ServiceCollection(); + + var exception = Assert.Throws(() => + services.AddAzureStorageStateManager("")); + + Assert.Equal("storageName", exception.ParamName); + } + + private sealed record TestState(string Value); +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerTests.cs new file mode 100644 index 00000000..40b6b204 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerTests.cs @@ -0,0 +1,155 @@ +using Azure; + +namespace Egil.Orleans.Messaging.State.AzureStorage.Tests.AzureStorage; + +public sealed class AzureStorageStateManagerTests +{ + [Fact] + public async Task WriteAsync_with_precondition_failed_skips_read_back_and_rethrows() + { + var exception = new RequestFailedException(412, "Precondition failed."); + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = exception + }; + var manager = new AzureStorageStateManager(storage); + + var actual = await Assert.ThrowsAsync( + () => manager.WriteAsync(new TestState("next"))); + + Assert.Same(exception, actual); + Assert.Equal(0, storage.ReadCount); + Assert.Equal(new TestState("initial"), manager.State); + Assert.Equal(new TestState("initial"), storage.State); + } + + [Fact] + public async Task WriteAsync_with_nested_precondition_failed_skips_read_back_and_rethrows_outer_exception() + { + var exception = new InvalidOperationException( + "wrapped", + new RequestFailedException(412, "Precondition failed.")); + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = exception + }; + var manager = new AzureStorageStateManager(storage); + + var actual = await Assert.ThrowsAsync( + () => manager.WriteAsync(new TestState("next"))); + + Assert.Same(exception, actual); + Assert.Equal(0, storage.ReadCount); + Assert.Equal(new TestState("initial"), manager.State); + } + + [Fact] + public async Task WriteAsync_with_transient_storage_failure_uses_read_back_recovery() + { + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = new RequestFailedException(503, "Server busy."), + OnRead = state => state.State = new TestState("next") + }; + var manager = new AzureStorageStateManager(storage); + + await manager.WriteAsync(new TestState("next")); + + Assert.Equal(1, storage.ReadCount); + Assert.Equal(new TestState("next"), manager.State); + } + + [Fact] + public async Task ClearAsync_with_precondition_failed_skips_read_back_and_rethrows() + { + var exception = new RequestFailedException(412, "Precondition failed."); + var storage = new FakePersistentState(new TestState("initial")) + { + ClearException = exception + }; + var manager = new AzureStorageStateManager(storage); + + var actual = await Assert.ThrowsAsync( + () => manager.ClearAsync()); + + Assert.Same(exception, actual); + Assert.Equal(0, storage.ReadCount); + Assert.Equal(new TestState("initial"), manager.State); + Assert.Equal(new TestState("initial"), storage.State); + } + + [Fact] + public async Task ClearAsync_with_transient_storage_failure_uses_read_back_recovery() + { + var storage = new FakePersistentState(new TestState("initial")) + { + ClearException = new RequestFailedException(503, "Server busy."), + OnRead = state => + { + state.State = null!; + state.RecordExists = false; + } + }; + var manager = new AzureStorageStateManager(storage); + + await manager.ClearAsync(); + + Assert.Equal(1, storage.ReadCount); + Assert.False(storage.RecordExists); + Assert.Null(manager.State); + } + + private sealed record TestState(string Value); + + private sealed class FakePersistentState(TestState state) : IPersistentState + { + public Exception? WriteException { get; set; } + + public Exception? ClearException { get; set; } + + public Action? OnRead { get; set; } + + public int ReadCount { get; private set; } + + public string Etag { get; set; } = "etag-1"; + + public bool RecordExists { get; set; } = true; + + public TestState State { get; set; } = state; + + public Task ReadStateAsync() + { + ReadCount++; + OnRead?.Invoke(this); + return Task.CompletedTask; + } + + public Task WriteStateAsync() + { + if (WriteException is not null) + { + throw WriteException; + } + + return Task.CompletedTask; + } + + public Task ClearStateAsync() + { + if (ClearException is not null) + { + throw ClearException; + } + + State = null!; + RecordExists = false; + return Task.CompletedTask; + } + + public Task ReadStateAsync(CancellationToken cancellationToken) => ReadStateAsync(); + + public Task WriteStateAsync(CancellationToken cancellationToken) => WriteStateAsync(); + + public Task ClearStateAsync(CancellationToken cancellationToken) => ClearStateAsync(); + } +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/Egil.Orleans.Messaging.State.AzureStorage.Tests.csproj b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/Egil.Orleans.Messaging.State.AzureStorage.Tests.csproj new file mode 100644 index 00000000..b7a9e8d6 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/Egil.Orleans.Messaging.State.AzureStorage.Tests.csproj @@ -0,0 +1,40 @@ + + + + enable + enable + true + Exe + Egil.Orleans.Messaging.State.AzureStorage.Tests + net10.0 + true + [Egil.Orleans.Messaging.State.AzureStorage]* + [*.Tests]* + **/*Tests.cs + + + + + + + + + + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + + + + + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/Egil.Orleans.Messaging.Streams.EventHubs.Tests.csproj b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/Egil.Orleans.Messaging.Streams.EventHubs.Tests.csproj new file mode 100644 index 00000000..476fb2df --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/Egil.Orleans.Messaging.Streams.EventHubs.Tests.csproj @@ -0,0 +1,40 @@ + + + + enable + enable + true + Exe + Egil.Orleans.Messaging.Streams.EventHubs.Tests + net10.0 + true + [Egil.Orleans.Messaging.Streams.EventHubs]* + [*.Tests]* + **/*Tests.cs + + + + + + + + + + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + + + + + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs similarity index 100% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubAdapterExtensionsTests.cs diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubAdapterTests.cs similarity index 100% rename from Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/EventHubs/EnrichedEventHubAdapterTests.cs rename to Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubAdapterTests.cs diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs index 32bd2312..e4247886 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/GlobalUsings.cs @@ -1,7 +1,6 @@ global using Egil.Orleans.Messaging.Outboxes; global using Egil.Orleans.Messaging.State; global using Egil.Orleans.Messaging.Streams; -global using Egil.Orleans.Messaging.Streams.EventHubs; global using Egil.Orleans.Messaging.Tracking; global using Microsoft.Extensions.DependencyInjection; global using Orleans; diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs index 082bbcc5..09a8c392 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs @@ -1,7 +1,6 @@ using System.Text.Json; using System.Text.Json.Serialization; using Orleans.Providers.Streams.Common; -using Orleans.Streaming.EventHubs; using Orleans.Streams; namespace Egil.Orleans.Messaging.Tests.Streams; @@ -55,50 +54,6 @@ public void JsonSerializer_round_trips_stream_cursor_with_event_sequence_token_v Assert.Equal(cursor, roundTripped); } - [Fact] - public void JsonSerializer_round_trips_stream_cursor_with_event_hub_token() - { - var cursor = new StreamCursor("orders", new EventHubSequenceToken("41", 9, 2)); - - var json = JsonSerializer.Serialize(cursor); - var roundTripped = JsonSerializer.Deserialize(json); - - Assert.NotNull(roundTripped); - Assert.Equal(cursor, roundTripped); - } - - [Fact] - public void JsonSerializer_round_trips_stream_cursor_with_event_hub_token_v2() - { - var cursor = new StreamCursor("orders", new EventHubSequenceTokenV2("42", 10, 3)); - - var json = JsonSerializer.Serialize(cursor); - var roundTripped = JsonSerializer.Deserialize(json); - - Assert.NotNull(roundTripped); - Assert.Equal(cursor, roundTripped); - } - - [Fact] - public void JsonSerializer_round_trips_stream_cursor_with_enriched_event_hub_token() - { - var cursor = new StreamCursor( - "orders", - new EnrichedEventHubSequenceToken( - "43", - 11, - 4, - new DateTimeOffset(2026, 5, 24, 18, 30, 0, TimeSpan.Zero), - "provider-a", - "00-abc-xyz-01")); - - var json = JsonSerializer.Serialize(cursor); - var roundTripped = JsonSerializer.Deserialize(json); - - Assert.NotNull(roundTripped); - Assert.Equal(cursor, roundTripped); - } - [Fact] public void JsonSerializer_throws_for_stream_cursor_payload_with_unknown_token_kind() { @@ -130,69 +85,6 @@ public void JsonSerializer_throws_for_custom_token_type() Assert.Contains("supports only built-in Orleans token types", exception.Message); } - [Theory] - [InlineData("event-hub")] - [InlineData("event-hub-v2")] - [InlineData("enriched-event-hub")] - public void JsonSerializer_throws_for_event_hub_payload_without_offset(string kind) - { - var json = $$""" - { - "StreamNamespace": "orders", - "Token": { - "Kind": "{{kind}}", - "SequenceNumber": 1, - "EventIndex": 0, - "EnqueuedTime": "2026-05-23T12:30:00Z", - "ProviderName": "provider-a" - } - } - """; - - var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); - Assert.Equal("Missing EventHubOffset.", exception.Message); - } - - [Fact] - public void JsonSerializer_throws_for_enriched_event_hub_payload_without_enqueued_time() - { - var json = """ - { - "StreamNamespace": "orders", - "Token": { - "Kind": "enriched-event-hub", - "SequenceNumber": 1, - "EventIndex": 0, - "EventHubOffset": "42", - "ProviderName": "provider-a" - } - } - """; - - var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); - Assert.Equal("Missing EnqueuedTime.", exception.Message); - } - - [Fact] - public void JsonSerializer_throws_for_enriched_event_hub_payload_without_stream_provider_name() - { - var json = """ - { - "StreamNamespace": "orders", - "Token": { - "Kind": "enriched-event-hub", - "SequenceNumber": 1, - "EventIndex": 0, - "EventHubOffset": "42", - "EnqueuedTime": "2026-05-23T12:30:00Z" - } - } - """; - - var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); - Assert.Equal("Missing ProviderName.", exception.Message); - } - private sealed class UnsupportedCustomSequenceToken(long sequenceNumber, int eventIndex) : StreamSequenceToken { public override long SequenceNumber { get; protected set; } = sequenceNumber; diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs index 9d4ae8af..344ace6e 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorTests.cs @@ -5,12 +5,12 @@ namespace Egil.Orleans.Messaging.Tests.Streams; public sealed class StreamCursorTests { [Fact] - public void TryGetEnqueuedTime_returns_true_and_value_for_enriched_event_hub_token() + public void TryGetEnqueuedTime_returns_true_and_value_for_metadata_token() { var enqueuedTime = new DateTimeOffset(2026, 5, 24, 12, 30, 0, TimeSpan.Zero); var cursor = new StreamCursor( "orders", - new EnrichedEventHubSequenceToken("42", 10, 0, enqueuedTime, "provider-a")); + new MetadataSequenceToken(10, 0, enqueuedTime, "provider-a", null)); var found = cursor.TryGetEnqueuedTime(out var actual); @@ -32,16 +32,16 @@ public void TryGetEnqueuedTime_returns_false_for_non_enriched_token() } [Fact] - public void TryGetProviderName_returns_true_and_value_for_enriched_event_hub_token() + public void TryGetProviderName_returns_true_and_value_for_metadata_token() { var cursor = new StreamCursor( "orders", - new EnrichedEventHubSequenceToken( - "42", + new MetadataSequenceToken( 10, 0, new DateTimeOffset(2026, 5, 24, 12, 30, 0, TimeSpan.Zero), - "provider-a")); + "provider-a", + null)); var found = cursor.TryGetProviderName(out var providerName); @@ -63,12 +63,11 @@ public void TryGetProviderName_returns_false_for_non_enriched_token() } [Fact] - public void TryGetTraceParent_returns_true_and_value_when_present_on_enriched_event_hub_token() + public void TryGetTraceParent_returns_true_and_value_when_present_on_metadata_token() { var cursor = new StreamCursor( "orders", - new EnrichedEventHubSequenceToken( - "42", + new MetadataSequenceToken( 10, 0, new DateTimeOffset(2026, 5, 24, 12, 30, 0, TimeSpan.Zero), @@ -82,20 +81,46 @@ public void TryGetTraceParent_returns_true_and_value_when_present_on_enriched_ev } [Fact] - public void TryGetTraceParent_returns_false_when_missing_on_enriched_event_hub_token() + public void TryGetTraceParent_returns_false_when_missing_on_metadata_token() { var cursor = new StreamCursor( "orders", - new EnrichedEventHubSequenceToken( - "42", + new MetadataSequenceToken( 10, 0, new DateTimeOffset(2026, 5, 24, 12, 30, 0, TimeSpan.Zero), - "provider-a")); + "provider-a", + null)); var found = cursor.TryGetTraceParent(out var traceParent); Assert.False(found); Assert.Null(traceParent); } + + private sealed class MetadataSequenceToken( + long sequenceNumber, + int eventIndex, + DateTimeOffset enqueuedTime, + string providerName, + string? traceParent) : EventSequenceToken(sequenceNumber, eventIndex), IStreamSequenceTokenMetadata + { + public bool TryGetEnqueuedTime(out DateTimeOffset value) + { + value = enqueuedTime; + return true; + } + + public bool TryGetProviderName([NotNullWhen(true)] out string? value) + { + value = providerName; + return true; + } + + public bool TryGetTraceParent([NotNullWhen(true)] out string? value) + { + value = traceParent; + return value is not null; + } + } } \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs index a9fafab8..1376c722 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs @@ -1,6 +1,5 @@ using System.Text.Json; using Orleans.Providers.Streams.Common; -using Orleans.Streaming.EventHubs; using Orleans.Streams; using TimeProviderExtensions; @@ -11,16 +10,7 @@ public sealed class MessageTrackerJsonConverterTests public static TheoryData SupportedStreamTokens => new() { new EventSequenceToken(7, 1), - new EventSequenceTokenV2(8, 2), - new EventHubSequenceToken("42", 10, 3), - new EventHubSequenceTokenV2("43", 11, 4), - new EnrichedEventHubSequenceToken( - offset: "44", - sequenceNumber: 12, - eventIndex: 5, - enqueuedTime: new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero), - providerName: "provider-a", - traceParent: "00-abc-xyz-01") + new EventSequenceTokenV2(8, 2) }; [Theory] @@ -143,96 +133,6 @@ public void JsonSerializer_throws_for_unknown_stream_token_kind() Assert.Throws(() => JsonSerializer.Deserialize(json)); } - [Theory] - [InlineData("event-hub")] - [InlineData("event-hub-v2")] - [InlineData("enriched-event-hub")] - public void JsonSerializer_throws_for_event_hub_stream_token_without_offset(string kind) - { - var json = $$""" - { - "Streams": [ - { - "StreamNamespace": "orders", - "LastPosition": { - "StreamNamespace": "orders", - "Token": { - "Kind": "{{kind}}", - "SequenceNumber": 1, - "EventIndex": 0, - "EnqueuedTime": "2026-05-23T12:30:00+00:00", - "ProviderName": "provider-a" - } - }, - "Received": "2026-05-23T12:30:00+00:00" - } - ], - "Outboxes": [] - } - """; - - var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); - Assert.Equal("Missing EventHubOffset.", exception.Message); - } - - [Fact] - public void JsonSerializer_throws_for_enriched_stream_token_without_enqueued_time() - { - var json = """ - { - "Streams": [ - { - "StreamNamespace": "orders", - "LastPosition": { - "StreamNamespace": "orders", - "Token": { - "Kind": "enriched-event-hub", - "SequenceNumber": 1, - "EventIndex": 0, - "EventHubOffset": "42", - "ProviderName": "provider-a" - } - }, - "Received": "2026-05-23T12:30:00+00:00" - } - ], - "Outboxes": [] - } - """; - - var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); - Assert.Equal("Missing EnqueuedTime.", exception.Message); - } - - [Fact] - public void JsonSerializer_throws_for_enriched_stream_token_without_stream_provider_name() - { - var json = """ - { - "Streams": [ - { - "StreamNamespace": "orders", - "LastPosition": { - "StreamNamespace": "orders", - "Token": { - "Kind": "enriched-event-hub", - "SequenceNumber": 1, - "EventIndex": 0, - "EventHubOffset": "42", - "EnqueuedTime": "2026-05-23T12:30:00+00:00" - } - }, - "Received": "2026-05-23T12:30:00+00:00" - } - ], - "Outboxes": [] - } - """; - - var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); - Assert.Equal("Missing ProviderName.", exception.Message); - } - private sealed class UnsupportedSequenceToken(long sequenceNumber, int eventIndex) : StreamSequenceToken { public override long SequenceNumber { get; protected set; } = sequenceNumber; From 65fa696194a9f15de7b4d7242e16cac7e3724a80 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Tue, 26 May 2026 13:26:09 +0000 Subject: [PATCH 41/82] feat(om): classify azure storage failures Classify deterministic Azure Storage failures as non-persistence outcomes and keep ambiguous transient results on the read-back recovery path. This lets StateManager skip unnecessary recovery reads for rejected mutations while preserving correctness for throttling, timeouts, and service failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Directory.Packages.props | 5 + .../AzureStorage/AzureStorageStateManager.cs | 97 +++++++++-- ...rleans.Messaging.State.AzureStorage.csproj | 3 + .../AzureStorageStateManagerTests.cs | 151 ++++++++++++++++++ 4 files changed, 247 insertions(+), 9 deletions(-) diff --git a/Egil.Orleans.Messaging/Directory.Packages.props b/Egil.Orleans.Messaging/Directory.Packages.props index bc28a862..4f20e325 100644 --- a/Egil.Orleans.Messaging/Directory.Packages.props +++ b/Egil.Orleans.Messaging/Directory.Packages.props @@ -12,6 +12,11 @@ + + + + + diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs index 4e47c723..3bb3c9ee 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs @@ -1,4 +1,7 @@ using Azure; +using Azure.Data.Tables.Models; +using Azure.Storage.Blobs.Models; +using System.Net; using Orleans.Storage; namespace Egil.Orleans.Messaging.State.AzureStorage; @@ -51,20 +54,96 @@ internal static class AzureStorageFailureClassifier { public static StorageFailureKind Classify(Exception exception) { - for (var current = exception; current is not null; current = current.InnerException) + return TryClassify(exception) ?? StorageFailureKind.UnknownOutcome; + } + + private static StorageFailureKind? TryClassify(Exception exception) + { + if (exception is InconsistentStateException) { - if (current is InconsistentStateException) - { - return StorageFailureKind.DidNotPersist; - } + return StorageFailureKind.DidNotPersist; + } + + if (exception is RequestFailedException requestFailed) + { + return ClassifyRequestFailed(requestFailed); + } - if (current is RequestFailedException requestFailed - && requestFailed.Status is 412) + if (exception is AggregateException aggregateException) + { + var hasDidNotPersist = false; + foreach (var inner in aggregateException.Flatten().InnerExceptions) { - return StorageFailureKind.DidNotPersist; + var innerKind = TryClassify(inner); + if (innerKind is StorageFailureKind.UnknownOutcome) + { + return StorageFailureKind.UnknownOutcome; + } + + hasDidNotPersist |= innerKind is StorageFailureKind.DidNotPersist; } + + return hasDidNotPersist ? StorageFailureKind.DidNotPersist : null; } - return StorageFailureKind.UnknownOutcome; + return exception.InnerException is null ? null : TryClassify(exception.InnerException); + } + + private static StorageFailureKind ClassifyRequestFailed(RequestFailedException exception) + { + if (IsAmbiguousOrTransient(exception)) + { + return StorageFailureKind.UnknownOutcome; + } + + return IsRejectedBeforePersistence(exception) + ? StorageFailureKind.DidNotPersist + : StorageFailureKind.UnknownOutcome; + } + + private static bool IsAmbiguousOrTransient(RequestFailedException exception) + { + return exception.Status is 0 + or (int)HttpStatusCode.RequestTimeout + or 429 + or >= 500 + || IsErrorCode( + exception, + BlobErrorCode.OperationTimedOut.ToString(), + BlobErrorCode.ServerBusy.ToString(), + BlobErrorCode.InternalError.ToString(), + TableErrorCode.OperationTimedOut.ToString(), + "ServerBusy", + "InternalError", + "AccountIOPSLimitExceeded"); + } + + private static bool IsRejectedBeforePersistence(RequestFailedException exception) + { + return exception.Status is >= 400 and < 500 + || IsErrorCode( + exception, + BlobErrorCode.ConditionNotMet.ToString(), + BlobErrorCode.AppendPositionConditionNotMet.ToString(), + BlobErrorCode.MaxBlobSizeConditionNotMet.ToString(), + BlobErrorCode.SequenceNumberConditionNotMet.ToString(), + BlobErrorCode.SourceConditionNotMet.ToString(), + BlobErrorCode.TargetConditionNotMet.ToString(), + BlobErrorCode.BlobAlreadyExists.ToString(), + BlobErrorCode.BlobNotFound.ToString(), + BlobErrorCode.ContainerNotFound.ToString(), + BlobErrorCode.ResourceAlreadyExists.ToString(), + BlobErrorCode.ResourceNotFound.ToString(), + TableErrorCode.UpdateConditionNotSatisfied.ToString(), + TableErrorCode.EntityAlreadyExists.ToString(), + TableErrorCode.EntityNotFound.ToString(), + TableErrorCode.ResourceNotFound.ToString(), + TableErrorCode.TableNotFound.ToString()); + } + + private static bool IsErrorCode(RequestFailedException exception, params string[] errorCodes) + { + return !string.IsNullOrWhiteSpace(exception.ErrorCode) + && errorCodes.Contains(exception.ErrorCode, StringComparer.Ordinal); } } \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj index 1db5144f..f3623fcd 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj @@ -33,6 +33,9 @@ all runtime; build; native; contentfiles; analyzers; buildtransitive + + + diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerTests.cs index 40b6b204..5efc204d 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerTests.cs @@ -1,4 +1,6 @@ using Azure; +using Azure.Data.Tables.Models; +using Azure.Storage.Blobs.Models; namespace Egil.Orleans.Messaging.State.AzureStorage.Tests.AzureStorage; @@ -43,6 +45,50 @@ public async Task WriteAsync_with_nested_precondition_failed_skips_read_back_and Assert.Equal(new TestState("initial"), manager.State); } + [Fact] + public async Task WriteAsync_with_conflict_skips_read_back_and_rethrows() + { + var exception = new RequestFailedException( + 409, + "The specified blob already exists.", + BlobErrorCode.BlobAlreadyExists.ToString(), + innerException: null); + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = exception + }; + var manager = new AzureStorageStateManager(storage); + + var actual = await Assert.ThrowsAsync( + () => manager.WriteAsync(new TestState("next"))); + + Assert.Same(exception, actual); + Assert.Equal(0, storage.ReadCount); + Assert.Equal(new TestState("initial"), manager.State); + } + + [Fact] + public async Task WriteAsync_with_auth_failure_skips_read_back_and_rethrows() + { + var exception = new RequestFailedException( + 403, + "Authentication failed.", + BlobErrorCode.AuthenticationFailed.ToString(), + innerException: null); + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = exception + }; + var manager = new AzureStorageStateManager(storage); + + var actual = await Assert.ThrowsAsync( + () => manager.WriteAsync(new TestState("next"))); + + Assert.Same(exception, actual); + Assert.Equal(0, storage.ReadCount); + Assert.Equal(new TestState("initial"), manager.State); + } + [Fact] public async Task WriteAsync_with_transient_storage_failure_uses_read_back_recovery() { @@ -59,6 +105,111 @@ public async Task WriteAsync_with_transient_storage_failure_uses_read_back_recov Assert.Equal(new TestState("next"), manager.State); } + [Fact] + public async Task WriteAsync_with_operation_timed_out_uses_read_back_recovery() + { + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = new RequestFailedException( + 500, + "The operation could not be completed within the permitted time.", + TableErrorCode.OperationTimedOut.ToString(), + innerException: null), + OnRead = state => state.State = new TestState("next") + }; + var manager = new AzureStorageStateManager(storage); + + await manager.WriteAsync(new TestState("next")); + + Assert.Equal(1, storage.ReadCount); + Assert.Equal(new TestState("next"), manager.State); + } + + [Fact] + public async Task WriteAsync_with_no_response_uses_read_back_recovery() + { + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = new RequestFailedException( + 0, + "No response was received.", + errorCode: null, + innerException: null), + OnRead = state => state.State = new TestState("next") + }; + var manager = new AzureStorageStateManager(storage); + + await manager.WriteAsync(new TestState("next")); + + Assert.Equal(1, storage.ReadCount); + Assert.Equal(new TestState("next"), manager.State); + } + + [Fact] + public async Task WriteAsync_with_timeout_exception_uses_read_back_recovery() + { + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = new TimeoutException("storage timeout"), + OnRead = state => state.State = new TestState("next") + }; + var manager = new AzureStorageStateManager(storage); + + await manager.WriteAsync(new TestState("next")); + + Assert.Equal(1, storage.ReadCount); + Assert.Equal(new TestState("next"), manager.State); + } + + [Fact] + public async Task WriteAsync_with_aggregate_precondition_failed_skips_read_back_and_rethrows() + { + var exception = new AggregateException( + new RequestFailedException( + 412, + "The update condition specified in the request was not satisfied.", + TableErrorCode.UpdateConditionNotSatisfied.ToString(), + innerException: null)); + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = exception + }; + var manager = new AzureStorageStateManager(storage); + + var actual = await Assert.ThrowsAsync( + () => manager.WriteAsync(new TestState("next"))); + + Assert.Same(exception, actual); + Assert.Equal(0, storage.ReadCount); + Assert.Equal(new TestState("initial"), manager.State); + } + + [Fact] + public async Task WriteAsync_with_mixed_aggregate_failure_uses_read_back_recovery() + { + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = new AggregateException( + new RequestFailedException( + 412, + "Precondition failed.", + BlobErrorCode.ConditionNotMet.ToString(), + innerException: null), + new RequestFailedException( + 503, + "Server busy.", + BlobErrorCode.ServerBusy.ToString(), + innerException: null)), + OnRead = state => state.State = new TestState("next") + }; + var manager = new AzureStorageStateManager(storage); + + await manager.WriteAsync(new TestState("next")); + + Assert.Equal(1, storage.ReadCount); + Assert.Equal(new TestState("next"), manager.State); + } + [Fact] public async Task ClearAsync_with_precondition_failed_skips_read_back_and_rethrows() { From ae63b09e098a6a88ac702d5f5992a88da2c3d6ea Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Tue, 26 May 2026 13:26:43 +0000 Subject: [PATCH 42/82] feat(om): post outbox items concurrently Dispatch pending outbox items concurrently on the Orleans activation scheduler and remove the thread-pool execution mode. Background posting now allows delivery to interleave by default while durable acknowledgement and failure reconciliation remain non-interleaving unless explicitly opted in. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../MessagingTelemetry.cs | 11 +- .../Outboxes/IPostman.cs | 7 + .../Outboxes/OutboxProcessor.cs | 166 +++- .../Outboxes/OutboxProcessorOptions.cs | 60 +- .../Outboxes/OutboxProcessorCoverageTests.cs | 712 ++++++++++++++++++ .../Outboxes/OutboxProcessorTests.cs | 143 ++++ 6 files changed, 1022 insertions(+), 77 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs index f6005b0b..8fc21150 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/MessagingTelemetry.cs @@ -47,22 +47,20 @@ public static void RecordStreamHandlerDuration(string streamNamespace, string st public static void RecordOutboxPostDuration( string grainType, - OutboxPostmanExecutionMode executionMode, double milliseconds) { OutboxPostDuration.Record( milliseconds, - OutboxTags(grainType, executionMode)); + OutboxTags(grainType)); } public static void RecordOutboxPostItem( string grainType, string postmanType, - OutboxPostmanExecutionMode executionMode, bool success, double milliseconds) { - var tags = OutboxTags(grainType, executionMode); + var tags = OutboxTags(grainType); tags.Add("postman.type", postmanType); tags.Add("success", success); @@ -148,12 +146,11 @@ private static TagList StreamTags(string streamNamespace, string status) return tags; } - private static TagList OutboxTags(string grainType, OutboxPostmanExecutionMode executionMode) + private static TagList OutboxTags(string grainType) { return new TagList { - { "grain.type", grainType }, - { "postman.execution", executionMode is OutboxPostmanExecutionMode.ThreadPool ? "thread_pool" : "grain_scheduler" } + { "grain.type", grainType } }; } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IPostman.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IPostman.cs index 36a6052e..2d548bc2 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IPostman.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IPostman.cs @@ -3,6 +3,13 @@ namespace Egil.Orleans.Messaging.Outboxes; /// /// Delivers one outbox message under the processor-owned cancellation budget. /// +/// +/// Implementations are resolved from the grain activation service provider and +/// should be ordinary delivery services: use injected dependencies, the +/// message payload, and the cancellation token. They should not depend on the +/// owning grain's activation-local state. Use inline postman callbacks only +/// when delivery genuinely needs grain-local context. +/// /// The message type handled by this postman. public interface IPostman where TMessage : notnull diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index bad7be38..5be76b99 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -28,7 +28,9 @@ public sealed partial class OutboxProcessor : IOutboxComponent private readonly string grainType; private readonly string reminderName; private IGrainTimer? timer; + private IGrainTimer? reconciliationTimer; private IGrainReminder? reminder; + private ReconciliationBatch? pendingReconciliation; private bool drainActive; private bool drainRequested; @@ -187,7 +189,8 @@ public async ValueTask PostAsync(CancellationToken cancellationToken = default) do { drainRequested = false; - await DrainOnceAsync(cancellationToken); + var reconciliation = await DispatchOnceAsync(cancellationToken); + await ReconcileAsync(reconciliation, cancellationToken); } while (drainRequested); } @@ -220,7 +223,7 @@ public async ValueTask PostInBackgroundAsync(CancellationToken cancellationToken public ValueTask ReceiveReminderAsync(string reminderName, TickStatus status) { return string.Equals(reminderName, this.reminderName, StringComparison.Ordinal) - ? PostAsync() + ? PostInBackgroundAsync() : ValueTask.CompletedTask; } @@ -244,7 +247,56 @@ private OutboxProcessor AddPostmanCore( return this; } - private async Task DrainOnceAsync(CancellationToken cancellationToken) + private async Task DispatchInBackgroundAsync(CancellationToken cancellationToken) + { + if (drainActive) + { + drainRequested = true; + return; + } + + drainActive = true; + try + { + drainRequested = false; + var reconciliation = await DispatchOnceAsync(cancellationToken); + if (reconciliation.HasWork) + { + pendingReconciliation = reconciliation; + EnsureReconciliationTimer(TimeSpan.Zero); + return; + } + + drainActive = false; + } + catch + { + drainActive = false; + throw; + } + } + + private async Task ReconcileInBackgroundAsync(CancellationToken cancellationToken) + { + var reconciliation = pendingReconciliation; + pendingReconciliation = null; + + try + { + await ReconcileAsync(reconciliation.GetValueOrDefault(), cancellationToken); + } + finally + { + drainActive = false; + } + + if (drainRequested) + { + EnsureTimer(TimeSpan.Zero); + } + } + + private async Task DispatchOnceAsync(CancellationToken cancellationToken) { cancellationToken.ThrowIfCancellationRequested(); var pending = options.PendingItems(); @@ -253,7 +305,7 @@ private async Task DrainOnceAsync(CancellationToken cancellationToken) if (pending.IsDefaultOrEmpty) { await DisableRetryAsync(); - return; + return default; } using var timeout = new CancellationTokenSource(options.ProcessingTimeout); @@ -264,25 +316,27 @@ private async Task DrainOnceAsync(CancellationToken cancellationToken) try { - foreach (var item in pending) + var dispatchTasks = new Task[pending.Length]; + for (var i = 0; i < pending.Length; i++) { linked.Token.ThrowIfCancellationRequested(); - await DispatchItemAsync(item, posted, failed, linked.Token); + dispatchTasks[i] = DispatchItemAsync(pending[i], linked.Token); } - if (posted.Count > 0) + var dispatchResults = await Task.WhenAll(dispatchTasks); + foreach (var result in dispatchResults) { - await options.AcknowledgePostedAsync(posted.ToImmutable(), linked.Token); - foreach (var item in posted) + if (result.Error is null) { - attempts.Remove(item); + posted.Add(result.Item); + } + else + { + failed.Add((result.Item, result.Error, IncrementAttempt(result.Item))); } } - if (failed.Count > 0 && options.ReconcileFailedAsync is { } reconcileFailed) - { - await reconcileFailed(failed.ToImmutable(), linked.Token); - } + return new ReconciliationBatch(posted.ToImmutable(), failed.ToImmutable()); } catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested && timeout.IsCancellationRequested) { @@ -292,17 +346,38 @@ private async Task DrainOnceAsync(CancellationToken cancellationToken) { MessagingTelemetry.RecordOutboxPostDuration( grainType, - options.PostmanExecution, Stopwatch.GetElapsedTime(started).TotalMilliseconds); } + } + + private async Task ReconcileAsync( + ReconciliationBatch reconciliation, + CancellationToken cancellationToken) + { + if (!reconciliation.HasWork) + { + return; + } + + if (!reconciliation.Posted.IsDefaultOrEmpty) + { + await options.AcknowledgePostedAsync(reconciliation.Posted, cancellationToken); + foreach (var item in reconciliation.Posted) + { + attempts.Remove(item); + } + } + + if (!reconciliation.Failed.IsDefaultOrEmpty && options.ReconcileFailedAsync is { } reconcileFailed) + { + await reconcileFailed(reconciliation.Failed, cancellationToken); + } await ReconcileRetryStateAsync(); } - private async Task DispatchItemAsync( + private async Task DispatchItemAsync( TOutbox item, - ImmutableArray.Builder posted, - ImmutableArray<(TOutbox Item, Exception Error, int Attempt)>.Builder failed, CancellationToken cancellationToken) { var started = Stopwatch.GetTimestamp(); @@ -316,21 +391,19 @@ private async Task DispatchItemAsync( "No outbox postman registered for item type {OutboxItemType} on grain {GrainType}.", itemType.FullName, grainType); - failed.Add((item, error, IncrementAttempt(item))); MessagingTelemetry.RecordOutboxPostError(grainType, itemType.Name, error); - return; + return new DispatchResult(item, error); } try { await InvokePostmanAsync(postman, item, cancellationToken); - posted.Add(item); MessagingTelemetry.RecordOutboxPostItem( grainType, postman.ItemType.Name, - options.PostmanExecution, success: true, Stopwatch.GetElapsedTime(started).TotalMilliseconds); + return new DispatchResult(item); } catch (Exception ex) when (ex is not OperationCanceledException) { @@ -340,31 +413,21 @@ private async Task DispatchItemAsync( "Outbox postman failed for item type {OutboxItemType} on grain {GrainType}.", itemType.FullName, grainType); - failed.Add((item, ex, IncrementAttempt(item))); MessagingTelemetry.RecordOutboxPostError(grainType, itemType.Name, ex); MessagingTelemetry.RecordOutboxPostItem( grainType, postman.ItemType.Name, - options.PostmanExecution, success: false, Stopwatch.GetElapsedTime(started).TotalMilliseconds); + return new DispatchResult(item, ex); } } private ValueTask InvokePostmanAsync( PostmanRegistration postman, TOutbox item, - CancellationToken cancellationToken) - { - if (options.PostmanExecution is OutboxPostmanExecutionMode.GrainScheduler) - { - return postman.Invoke(item, cancellationToken); - } - - return new ValueTask(Task.Run( - async () => await postman.Invoke(item, cancellationToken), - cancellationToken)); - } + CancellationToken cancellationToken) => + postman.Invoke(item, cancellationToken); private PostmanRegistration? FindPostman(TOutbox item) { @@ -413,7 +476,7 @@ private void EnsureTimer(TimeSpan dueTime) if (timer is null) { timer = owner.RegisterGrainTimer( - static (processor, cancellationToken) => processor.PostAsync(cancellationToken).AsTask(), + static (processor, cancellationToken) => processor.DispatchInBackgroundAsync(cancellationToken), this, options); return; @@ -422,6 +485,26 @@ private void EnsureTimer(TimeSpan dueTime) timer.Change(dueTime, this.options.RetryDelay); } + private void EnsureReconciliationTimer(TimeSpan dueTime) + { + var options = new GrainTimerCreationOptions(dueTime, Timeout.InfiniteTimeSpan) + { + Interleave = this.options.InterleaveReconciliationCallbacks, + KeepAlive = this.options.KeepAlive + }; + + if (reconciliationTimer is null) + { + reconciliationTimer = owner.RegisterGrainTimer( + static (processor, cancellationToken) => processor.ReconcileInBackgroundAsync(cancellationToken), + this, + options); + return; + } + + reconciliationTimer.Change(dueTime, Timeout.InfiniteTimeSpan); + } + private async Task EnsureReminderAsync() { var period = Max(options.RetryDelay, TimeSpan.FromMinutes(1)); @@ -432,6 +515,8 @@ private async Task DisableRetryAsync() { timer?.Dispose(); timer = null; + reconciliationTimer?.Dispose(); + reconciliationTimer = null; var activeReminder = reminder ?? await owner.GetReminder(reminderName); if (activeReminder is not null) @@ -443,6 +528,13 @@ private async Task DisableRetryAsync() private static TimeSpan Max(TimeSpan left, TimeSpan right) => left >= right ? left : right; + private readonly record struct ReconciliationBatch( + ImmutableArray Posted, + ImmutableArray<(TOutbox Item, Exception Error, int Attempt)> Failed) + { + public bool HasWork => !Posted.IsDefaultOrEmpty || !Failed.IsDefaultOrEmpty; + } + private sealed record PostmanRegistration( Func ItemFilter, Func Postman, @@ -451,6 +543,8 @@ private sealed record PostmanRegistration( public ValueTask Invoke(TOutbox item, CancellationToken cancellationToken) => Postman(item, cancellationToken); } + + private readonly record struct DispatchResult(TOutbox Item, Exception? Error = null); } /// diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs index 079db814..e830e186 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs @@ -2,32 +2,10 @@ namespace Egil.Orleans.Messaging.Outboxes; -/// -/// Controls where outbox postman callbacks execute. -/// -public enum OutboxPostmanExecutionMode -{ - /// - /// Executes postman callbacks on the Orleans activation scheduler. - /// - GrainScheduler, - - /// - /// Executes postman callbacks on the .NET thread pool. - /// - /// - /// Use only for blocking or legacy delivery code that must not run on the - /// Orleans activation scheduler. Thread-pool postmen must not read or - /// mutate activation-local grain state. Prefer static/state-free callbacks, - /// injected thread-safe services, or calls. - /// - ThreadPool -} - /// /// Configuration for . Defines how the -/// processor reads pending items, reconciles successes/failures, and controls -/// timer/reminder scheduling. +/// processor reads pending items, reconciles successes/failures, and schedules +/// retry work. /// /// /// The base type of outbox items. Must match the type parameter of the @@ -66,24 +44,38 @@ public required Func, CancellationToken, ValueTask> public TimeSpan ProcessingTimeout { get; init; } = TimeSpan.FromSeconds(20); /// - /// Timer/reminder retry period. Reminder period is clamped to at least one - /// minute because Orleans reminders do not support sub-minute precision. + /// Delay before retrying remaining pending items. Cross-activation retry + /// is clamped to at least one minute because Orleans reminders do not + /// support sub-minute precision. /// public TimeSpan RetryDelay { get; init; } = TimeSpan.FromMinutes(2); /// - /// Whether timer callbacks may interleave with other grain turns. + /// Whether background posting may allow other grain calls to run while + /// postmen are awaiting asynchronous work. /// - public bool Interleave { get; init; } + /// + /// Defaults to so slow delivery does not block + /// unrelated calls to the grain. This controls the delivery phase only; + /// acknowledgement and failure callbacks use + /// . + /// + public bool Interleave { get; init; } = true; /// - /// Whether an active timer keeps the grain activation alive. + /// Whether acknowledgement and failure reconciliation callbacks may + /// interleave with other grain calls when posting runs in the background. /// - public bool KeepAlive { get; init; } + /// + /// Defaults to because these callbacks usually + /// update durable outbox state. Orleans reentrancy rules still apply: + /// reentrant grains may interleave these callbacks regardless. + /// + public bool InterleaveReconciliationCallbacks { get; init; } /// - /// Controls where postman callbacks execute. + /// Whether background retry work should keep the grain activation alive + /// while pending outbox items remain. /// - public OutboxPostmanExecutionMode PostmanExecution { get; init; } = - OutboxPostmanExecutionMode.GrainScheduler; -} \ No newline at end of file + public bool KeepAlive { get; init; } +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs new file mode 100644 index 00000000..7d32e30b --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs @@ -0,0 +1,712 @@ +using System.Collections.Immutable; +using Egil.Orleans.Testing; + +namespace Egil.Orleans.Messaging.Tests.Outboxes; + +public sealed class OutboxProcessorCoverageTests(MessagingTestClusterFixture fixture) : IClassFixture +{ + [Fact] + public async Task PostAsync_coalesces_reentrant_post_request_into_second_drain() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var state = await grain.PublishWithReentrantPostAsync("first", "second"); + + Assert.Equal(2, state.AcknowledgedCount); + Assert.Equal(0, state.Outbox?.Count); + } + + [Fact] + public async Task PostInBackgroundAsync_with_failed_item_left_pending_schedules_retry() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + await grain.PublishInBackgroundAsync("retry"); + + await fixture.WaitForAssertionAsync( + grain, + async () => + { + var state = await grain.GetStateAsync(); + Assert.True(state.FailedCount >= 1); + Assert.Equal(1, state.Outbox?.Count); + }, + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task ReceiveReminderAsync_ignores_unrelated_reminder_name() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var state = await grain.PublishAndReceiveUnrelatedReminderAsync("ignored"); + + Assert.Equal(0, state.AcknowledgedCount); + Assert.Equal(1, state.Outbox?.Count); + } + + [Fact] + public async Task ReceiveReminderAsync_schedules_matching_reminder_name() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + await grain.PublishAndReceiveOutboxReminderAsync("matched"); + + await fixture.WaitForAssertionAsync( + grain, + async () => + { + var state = await grain.GetStateAsync(); + Assert.Equal(1, state.AcknowledgedCount); + Assert.Equal(0, state.Outbox?.Count); + }, + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task PostInBackgroundAsync_with_empty_pending_snapshot_disables_retry() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var state = await grain.PostEmptyPendingInBackgroundAsync(); + + Assert.Equal(0, state.AcknowledgedCount); + Assert.Equal(0, state.FailedCount); + Assert.Null(state.Outbox); + } + + [Fact] + public async Task PostAsync_with_default_pending_snapshot_disables_retry() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var state = await grain.PostDefaultPendingAsync(); + + Assert.Equal(0, state.AcknowledgedCount); + Assert.Equal(0, state.FailedCount); + Assert.Null(state.Outbox); + } + + [Fact] + public async Task PostAsync_with_default_pending_snapshot_after_acknowledgement_disables_retry() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var state = await grain.PostPendingThenDefaultAfterAcknowledgeAsync(); + + Assert.Equal(1, state.AcknowledgedCount); + Assert.Null(state.Outbox); + } + + [Fact] + public async Task PostInBackgroundAsync_honors_canceled_token() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var exceptionType = await grain.PostInBackgroundWithCanceledTokenAsync(); + + Assert.Equal(nameof(OperationCanceledException), exceptionType); + } + + [Fact] + public async Task AddStreamPostman_two_argument_overload_validates_stream_id_delegate() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var paramName = await grain.ValidateNullStreamIdAsync(); + + Assert.Equal("streamId", paramName); + } + + [Fact] + public async Task RegisterOutboxProcessor_rejects_non_positive_processing_timeout() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var paramName = await grain.ValidateProcessingTimeoutAsync(); + + Assert.Equal("options", paramName); + } + + [Fact] + public async Task RegisterOutboxProcessor_rejects_non_positive_retry_delay() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var paramName = await grain.ValidateRetryDelayAsync(); + + Assert.Equal("options", paramName); + } + + [Fact] + public async Task Background_dispatch_allows_other_calls_while_postman_is_awaiting() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + var gate = OutboxProcessorSchedulingGate.For(grainKey); + + try + { + await grain.PublishInBackgroundAsync("dispatch", interleaveReconciliation: false); + await gate.DispatchStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + + await grain.RecordWriteAsync("during-dispatch"); + + gate.AllowDispatch.SetResult(); + await gate.ReconciliationStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + gate.AllowReconciliation.SetResult(); + + await fixture.WaitForAssertionAsync( + grain, + async () => + { + var state = await grain.GetSchedulingStateAsync(); + Assert.Contains("during-dispatch", state.Writes); + Assert.Equal(1, state.AcknowledgedCount); + }, + ct: TestContext.Current.CancellationToken); + } + finally + { + OutboxProcessorSchedulingGate.Remove(grainKey); + } + } + + [Fact] + public async Task Background_reconciliation_does_not_interleave_with_other_calls_by_default() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + var gate = OutboxProcessorSchedulingGate.For(grainKey); + + try + { + await grain.PublishInBackgroundAsync("non-interleaving-reconciliation", interleaveReconciliation: false); + gate.AllowDispatch.SetResult(); + await gate.ReconciliationStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + + var writeTask = grain.RecordWriteAsync("during-reconciliation"); + var completed = await Task.WhenAny(writeTask, Task.Delay(TimeSpan.FromMilliseconds(150), TestContext.Current.CancellationToken)); + Assert.NotSame(writeTask, completed); + + gate.AllowReconciliation.SetResult(); + await writeTask.WaitAsync(TestContext.Current.CancellationToken); + + var state = await grain.GetSchedulingStateAsync(); + Assert.Equal(["acknowledged", "during-reconciliation"], state.Writes); + } + finally + { + OutboxProcessorSchedulingGate.Remove(grainKey); + } + } + + [Fact] + public async Task Background_reconciliation_can_be_configured_to_interleave_with_other_calls() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + var gate = OutboxProcessorSchedulingGate.For(grainKey); + + try + { + await grain.PublishInBackgroundAsync("interleaving-reconciliation", interleaveReconciliation: true); + gate.AllowDispatch.SetResult(); + await gate.ReconciliationStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + + await grain.RecordWriteAsync("during-reconciliation"); + gate.AllowReconciliation.SetResult(); + + await fixture.WaitForAssertionAsync( + grain, + async () => + { + var state = await grain.GetSchedulingStateAsync(); + Assert.Equal(["during-reconciliation", "acknowledged"], state.Writes); + Assert.Equal(1, state.AcknowledgedCount); + }, + ct: TestContext.Current.CancellationToken); + } + finally + { + OutboxProcessorSchedulingGate.Remove(grainKey); + } + } +} + +public interface IOutboxProcessorReentrantPostGrain : IGrainWithGuidKey +{ + Task PublishWithReentrantPostAsync(string first, string second); +} + +public interface IOutboxProcessorRetryPendingGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorReminderCoverageGrain : IGrainWithGuidKey +{ + Task PublishAndReceiveUnrelatedReminderAsync(string value); + + Task PublishAndReceiveOutboxReminderAsync(string value); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorValidationCoverageGrain : IGrainWithGuidKey +{ + Task PostEmptyPendingInBackgroundAsync(); + + Task PostDefaultPendingAsync(); + + Task PostPendingThenDefaultAfterAcknowledgeAsync(); + + Task PostInBackgroundWithCanceledTokenAsync(); + + Task ValidateNullStreamIdAsync(); + + Task ValidateProcessingTimeoutAsync(); + + Task ValidateRetryDelayAsync(); +} + +public interface IOutboxProcessorReconciliationSchedulingGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value, bool interleaveReconciliation); + + Task RecordWriteAsync(string value); + + Task GetSchedulingStateAsync(); +} + +[GenerateSerializer] +public sealed record OutboxProcessorSchedulingState( + [property: Id(0)] int AcknowledgedCount, + [property: Id(1)] ImmutableArray Writes); + +internal sealed class OutboxProcessorSchedulingGate +{ + private static readonly Dictionary Gates = []; + + public TaskCompletionSource DispatchStarted { get; } = + new(TaskCreationOptions.RunContinuationsAsynchronously); + + public TaskCompletionSource AllowDispatch { get; } = + new(TaskCreationOptions.RunContinuationsAsynchronously); + + public TaskCompletionSource ReconciliationStarted { get; } = + new(TaskCreationOptions.RunContinuationsAsynchronously); + + public TaskCompletionSource AllowReconciliation { get; } = + new(TaskCreationOptions.RunContinuationsAsynchronously); + + public static OutboxProcessorSchedulingGate For(Guid grainKey) + { + lock (Gates) + { + return Gates.TryGetValue(grainKey, out var gate) + ? gate + : Gates[grainKey] = new OutboxProcessorSchedulingGate(); + } + } + + public static void Remove(Guid grainKey) + { + lock (Gates) + { + Gates.Remove(grainKey); + } + } +} + +public sealed class OutboxProcessorReentrantPostGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorReentrantPostGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + private string? followUpValue; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + processor = this.RegisterOutboxProcessor(CreateOptions()) + .AddPostman>(PostAndRequestAnotherDrainAsync); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishWithReentrantPostAsync(string first, string second) + { + followUpValue = second; + state.State.Outbox = EnsureOutbox().Add(new OutboxProcessorTestEvent(first)); + await state.WriteStateAsync(); + + await processor!.PostAsync(); + return state.State; + } + + private async Task PostAndRequestAnotherDrainAsync( + OutboxMessageEnvelope envelope, + CancellationToken cancellationToken) + { + if (followUpValue is null) + { + return; + } + + var nextValue = followUpValue; + followUpValue = null; + state.State.Outbox = EnsureOutbox().Add(new OutboxProcessorTestEvent(nextValue)); + await processor!.PostAsync(cancellationToken); + } + + private OutboxProcessorOptions> CreateOptions() => new() + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }; + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } + + private ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private Outbox EnsureOutbox() => + state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); +} + +public sealed class OutboxProcessorRetryPendingGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorRetryPendingGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>( + static _ => new ValueTask(Task.FromException( + new InvalidOperationException("Keep pending for retry.")))); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishInBackgroundAsync(string value) + { + state.State.Outbox = EnsureOutbox().Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private async ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + state.State.FailedCount += failures.Length; + state.State.LastFailureType = failures[0].Error.GetType().Name; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() => + state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); +} + +public sealed class OutboxProcessorReminderCoverageGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorReminderCoverageGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMinutes(2) + }) + .AddPostman>(static _ => ValueTask.CompletedTask); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishAndReceiveUnrelatedReminderAsync(string value) + { + state.State.Outbox = EnsureOutbox().Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.ReceiveReminderAsync("not-the-outbox-reminder", default); + return state.State; + } + + public async Task PublishAndReceiveOutboxReminderAsync(string value) + { + state.State.Outbox = EnsureOutbox().Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.ReceiveReminderAsync(processor.ReminderName, default); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } + + private ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private Outbox EnsureOutbox() => + state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); +} + +public sealed class OutboxProcessorValidationCoverageGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorValidationCoverageGrain, IOutboxGrain +{ + public async Task PostEmptyPendingInBackgroundAsync() + { + var processor = this.RegisterOutboxProcessor(CreateOptions(() => [])); + await processor.PostInBackgroundAsync(); + return state.State; + } + + public async Task PostDefaultPendingAsync() + { + var processor = this.RegisterOutboxProcessor(CreateOptions(static () => default)); + await processor.PostAsync(); + return state.State; + } + + public async Task PostPendingThenDefaultAfterAcknowledgeAsync() + { + var pending = Outbox + .Create(GrainContext.GrainId) + .Add(new OutboxProcessorTestEvent("default-after-ack")) + .ToImmutableArray(); + var returnDefault = false; + + var processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => returnDefault ? default : pending, + AcknowledgePostedAsync = (items, _) => + { + state.State.AcknowledgedCount += items.Length; + returnDefault = true; + return ValueTask.CompletedTask; + }, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>(static _ => ValueTask.CompletedTask); + + await processor.PostAsync(); + return state.State; + } + + public async Task PostInBackgroundWithCanceledTokenAsync() + { + var processor = this.RegisterOutboxProcessor(CreateOptions(() => [])); + + using var cancellation = new CancellationTokenSource(); + await cancellation.CancelAsync(); + + try + { + await processor.PostInBackgroundAsync(cancellation.Token); + return null; + } + catch (Exception ex) + { + return ex.GetType().Name; + } + } + + public Task ValidateNullStreamIdAsync() + { + try + { + _ = this.RegisterOutboxProcessor(CreateOptions(() => [])) + .AddStreamPostman>( + OutboxProcessorTestProviderNames.Events, + null!); + return Task.FromResult(null); + } + catch (ArgumentNullException ex) + { + return Task.FromResult(ex.ParamName); + } + } + + public Task ValidateProcessingTimeoutAsync() + { + try + { + _ = this.RegisterOutboxProcessor(CreateOptions( + () => [], + processingTimeout: TimeSpan.Zero)); + return Task.FromResult(null); + } + catch (ArgumentOutOfRangeException ex) + { + return Task.FromResult(ex.ParamName); + } + } + + public Task ValidateRetryDelayAsync() + { + try + { + _ = this.RegisterOutboxProcessor(CreateOptions( + () => [], + retryDelay: TimeSpan.Zero)); + return Task.FromResult(null); + } + catch (ArgumentOutOfRangeException ex) + { + return Task.FromResult(ex.ParamName); + } + } + + private OutboxProcessorOptions> CreateOptions( + Func>> pendingItems, + TimeSpan? processingTimeout = null, + TimeSpan? retryDelay = null) => new() + { + PendingItems = pendingItems, + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + ProcessingTimeout = processingTimeout ?? TimeSpan.FromSeconds(20), + RetryDelay = retryDelay ?? TimeSpan.FromMilliseconds(100) + }; + + private ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + state.State.AcknowledgedCount += items.Length; + return ValueTask.CompletedTask; + } + + private ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + state.State.FailedCount += failures.Length; + return ValueTask.CompletedTask; + } +} + +public sealed class OutboxProcessorReconciliationSchedulingGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorReconciliationSchedulingGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + private ImmutableArray writes = []; + + public async Task PublishInBackgroundAsync(string value, bool interleaveReconciliation) + { + processor ??= this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100), + InterleaveReconciliationCallbacks = interleaveReconciliation + }) + .AddPostman>(PostWithGateAsync); + + state.State.Outbox = EnsureOutbox().Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor.PostInBackgroundAsync(); + } + + public Task RecordWriteAsync(string value) + { + writes = writes.Add(value); + return Task.CompletedTask; + } + + public Task GetSchedulingStateAsync() => + Task.FromResult(new OutboxProcessorSchedulingState(state.State.AcknowledgedCount, writes)); + + private async Task PostWithGateAsync( + OutboxMessageEnvelope envelope, + CancellationToken cancellationToken) + { + var gate = OutboxProcessorSchedulingGate.For(this.GetPrimaryKey()); + gate.DispatchStarted.SetResult(); + await gate.AllowDispatch.Task.WaitAsync(cancellationToken); + } + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var gate = OutboxProcessorSchedulingGate.For(this.GetPrimaryKey()); + gate.ReconciliationStarted.SetResult(); + await gate.AllowReconciliation.Task.WaitAsync(cancellationToken); + + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + writes = writes.Add("acknowledged"); + await state.WriteStateAsync(cancellationToken); + } + + private ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private Outbox EnsureOutbox() => + state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index 93563e94..69ca0ac2 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -140,6 +140,39 @@ public async Task PostAsync_timeout_from_keyed_postman_leaves_item_pending() Assert.Equal(0, state.FailedCount); Assert.Equal(1, state.Outbox?.Count); } + + [Fact] + public async Task PostAsync_dispatches_pending_items_concurrently() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + await source.PublishTwoAsync("first", "second"); + var state = await source.GetStateAsync(); + + Assert.Equal(2, state.AcknowledgedCount); + Assert.Equal(0, state.FailedCount); + Assert.Equal(0, state.Outbox?.Count); + Assert.Equal(2, state.MaxConcurrentPostmen); + } + + [Fact] + public async Task PostInBackgroundAsync_allows_other_calls_while_postman_is_awaiting() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + await source.PublishInBackgroundAsync("background"); + + await fixture.WaitForAssertionAsync( + source, + async () => + { + var state = await source.GetStateAsync(); + Assert.Equal(1, state.MaxConcurrentPostmen); + Assert.Equal(0, state.AcknowledgedCount); + Assert.Equal(1, state.Outbox?.Count); + }, + ct: TestContext.Current.CancellationToken); + } } internal static class OutboxProcessorTestNamespaces @@ -208,6 +241,15 @@ public interface IOutboxProcessorKeyedTimeoutPostmanGrain : IGrainWithGuidKey Task GetStateAsync(); } +public interface IOutboxProcessorConcurrentPostmanGrain : IGrainWithGuidKey +{ + Task PublishInBackgroundAsync(string value); + + Task PublishTwoAsync(string first, string second); + + Task GetStateAsync(); +} + public interface IOutboxProcessorSinkGrain : IGrainWithGuidKey { Task EnsureActiveAsync(); @@ -229,6 +271,9 @@ public sealed class OutboxProcessorSourceState [Id(3)] public string? LastFailureType { get; set; } + + [Id(4)] + public int MaxConcurrentPostmen { get; set; } } [GenerateSerializer] @@ -692,6 +737,104 @@ private Outbox EnsureOutbox() } } +public sealed class OutboxProcessorConcurrentPostmanGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorConcurrentPostmanGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + private int activePostmen; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>(DelayAndTrackConcurrencyAsync); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishTwoAsync(string first, string second) + { + var outbox = EnsureOutbox() + .Add(new OutboxProcessorTestEvent(first)) + .Add(new OutboxProcessorTestEvent(second)); + + state.State.Outbox = outbox; + await state.WriteStateAsync(); + await processor!.PostAsync(); + } + + public async Task PublishInBackgroundAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private async Task DelayAndTrackConcurrencyAsync( + OutboxMessageEnvelope envelope, + CancellationToken cancellationToken) + { + activePostmen++; + state.State.MaxConcurrentPostmen = Math.Max(state.State.MaxConcurrentPostmen, activePostmen); + + try + { + await Task.Delay(TimeSpan.FromMilliseconds(500), cancellationToken); + } + finally + { + activePostmen--; + } + } + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } + + private async ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var failure in failures) + { + outbox = outbox.Remove(failure.Item.Token); + state.State.LastFailureType = failure.Error.GetType().Name; + } + + state.State.Outbox = outbox; + state.State.FailedCount += failures.Length; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() + { + return state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); + } +} + public sealed class KeyedOutboxProcessorSuccessPostman : IPostman> { From a045f117183309eb78a2a1abe412c06d9f5b7d0b Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Tue, 26 May 2026 13:27:11 +0000 Subject: [PATCH 43/82] feat(om): preserve stream token json types Add an explicit JsonConverter registry for concrete StreamSequenceToken types and route StreamCursor and MessageTracker token payloads through that registry. Register Event Hubs token converters from the enriched adapter setup so enriched checkpoints round-trip without downcasting provider metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../EnrichedEventHubAdapterExtensions.cs | 9 +- .../EnrichedEventHubSequenceToken.cs | 19 +- ...entHubStreamSequenceTokenJsonConverters.cs | 224 +++++++++++++ .../Streams/StreamCursor.cs | 13 +- .../Streams/StreamCursorJsonConverter.cs | 156 +++++---- ...sonConverterServiceCollectionExtensions.cs | 56 ++++ .../StreamSequenceTokenJsonConverters.cs | 300 ++++++++++++++++++ .../Tracking/MessageTrackerJsonConverter.cs | 84 +---- ...EventHubSequenceTokenJsonConverterTests.cs | 66 ++++ .../Streams/StreamCursorJsonConverterTests.cs | 16 +- .../MessageTrackerJsonConverterTests.cs | 8 +- 11 files changed, 780 insertions(+), 171 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EventHubStreamSequenceTokenJsonConverters.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverterServiceCollectionExtensions.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverters.cs create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubSequenceTokenJsonConverterTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapterExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapterExtensions.cs index 566c0154..d6fa4632 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapterExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapterExtensions.cs @@ -1,6 +1,7 @@ using Microsoft.Extensions.DependencyInjection; using Egil.Orleans.Messaging.Streams; using Egil.Orleans.Messaging.Streams.EventHubs; +using Egil.Orleans.Messaging.Tracking; using Orleans.Serialization; namespace Orleans.Hosting; @@ -29,7 +30,9 @@ namespace Orleans.Hosting; /// overrides GetStreamPosition and GetSequenceToken to return /// instances carrying the /// broker-side enqueue time and the stream provider name. No user-written -/// adapter code is needed. +/// adapter code is needed. It also registers Event Hubs sequence-token JSON +/// converters so and +/// can round-trip enriched checkpoints through System.Text.Json. /// /// /// Downstream access: Once registered, the enrichment is transparently @@ -62,10 +65,12 @@ public void UseEnrichedDataAdapter() { ArgumentNullException.ThrowIfNull(configurator); + EventHubStreamSequenceTokenJsonConverters.Register(); + configurator.UseDataAdapter((services, providerName) => new EnrichedEventHubAdapter( providerName, services.GetRequiredService())); } } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs index db319ac7..25622611 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs @@ -42,16 +42,23 @@ namespace Egil.Orleans.Messaging.Streams.EventHubs; /// /// /// Serialization: Inherits Orleans serialization from -/// . Persist stream positions through -/// or another application-owned checkpoint -/// model; the provider-neutral core package does not serialize Event -/// Hubs-specific token subtypes directly. +/// . The Event Hubs package also +/// registers from +/// , so +/// and can round-trip +/// this token through System.Text.Json without the core package referencing +/// Event Hubs. /// /// [GenerateSerializer] -[Alias("egil.orleans.messaging.EnrichedEventHubSequenceToken")] +[Alias(EnrichedEventHubSequenceToken.TypeAlias)] public class EnrichedEventHubSequenceToken : EventHubSequenceTokenV2, IStreamSequenceTokenMetadata { + /// + /// Stable Orleans/STJ token discriminator for this token type. + /// + public const string TypeAlias = "egil.orleans.messaging.EnrichedEventHubSequenceToken"; + /// /// The wall-clock time the event was enqueued at the Event Hub broker. /// Used by to compute @@ -146,4 +153,4 @@ public bool TryGetTraceParent([NotNullWhen(true)] out string? traceParent) traceParent = TraceParent; return traceParent is not null; } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EventHubStreamSequenceTokenJsonConverters.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EventHubStreamSequenceTokenJsonConverters.cs new file mode 100644 index 00000000..5e92b07e --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EventHubStreamSequenceTokenJsonConverters.cs @@ -0,0 +1,224 @@ +using System.Text.Json; +using System.Text.Json.Serialization; +using Orleans.Streaming.EventHubs; + +namespace Egil.Orleans.Messaging.Streams.EventHubs; + +internal static class EventHubStreamSequenceTokenJsonConverters +{ + public static void Register() + { + StreamSequenceTokenJsonConverters.Register( + "orleans.event-hubs.sequence-token", + new EventHubSequenceTokenJsonConverter()); + StreamSequenceTokenJsonConverters.Register( + "orleans.event-hubs.sequence-token-v2", + new EventHubSequenceTokenV2JsonConverter()); + StreamSequenceTokenJsonConverters.Register( + EnrichedEventHubSequenceToken.TypeAlias, + new EnrichedEventHubSequenceTokenJsonConverter()); + } +} + +/// +/// JSON converter for Orleans Event Hubs sequence tokens. +/// +public sealed class EventHubSequenceTokenJsonConverter : JsonConverter +{ + /// + public override EventHubSequenceToken Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + EventHubSequenceTokenJsonConverterHelpers.ExpectStartObject(ref reader, nameof(EventHubSequenceToken)); + var offset = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredStringProperty(ref reader, "EventHubOffset"); + var sequenceNumber = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt64Property(ref reader, "SequenceNumber"); + var eventIndex = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt32Property(ref reader, "EventIndex"); + EventHubSequenceTokenJsonConverterHelpers.ReadEndObject(ref reader); + + return new EventHubSequenceToken(offset, sequenceNumber, eventIndex); + } + + /// + public override void Write(Utf8JsonWriter writer, EventHubSequenceToken value, JsonSerializerOptions options) + { + writer.WriteStartObject(); + EventHubSequenceTokenJsonConverterHelpers.WriteBaseProperties( + writer, + value.EventHubOffset, + value.SequenceNumber, + value.EventIndex); + writer.WriteEndObject(); + } +} + +/// +/// JSON converter for Orleans Event Hubs V2 sequence tokens. +/// +public sealed class EventHubSequenceTokenV2JsonConverter : JsonConverter +{ + /// + public override EventHubSequenceTokenV2 Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + EventHubSequenceTokenJsonConverterHelpers.ExpectStartObject(ref reader, nameof(EventHubSequenceTokenV2)); + var offset = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredStringProperty(ref reader, "EventHubOffset"); + var sequenceNumber = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt64Property(ref reader, "SequenceNumber"); + var eventIndex = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt32Property(ref reader, "EventIndex"); + EventHubSequenceTokenJsonConverterHelpers.ReadEndObject(ref reader); + + return new EventHubSequenceTokenV2(offset, sequenceNumber, eventIndex); + } + + /// + public override void Write(Utf8JsonWriter writer, EventHubSequenceTokenV2 value, JsonSerializerOptions options) + { + writer.WriteStartObject(); + EventHubSequenceTokenJsonConverterHelpers.WriteBaseProperties( + writer, + value.EventHubOffset, + value.SequenceNumber, + value.EventIndex); + writer.WriteEndObject(); + } +} + +/// +/// JSON converter for . +/// +public sealed class EnrichedEventHubSequenceTokenJsonConverter : JsonConverter +{ + /// + public override EnrichedEventHubSequenceToken Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + EventHubSequenceTokenJsonConverterHelpers.ExpectStartObject(ref reader, nameof(EnrichedEventHubSequenceToken)); + var offset = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredStringProperty(ref reader, "EventHubOffset"); + var sequenceNumber = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt64Property(ref reader, "SequenceNumber"); + var eventIndex = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt32Property(ref reader, "EventIndex"); + var enqueuedTime = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredDateTimeOffsetProperty(ref reader, "EnqueuedTime"); + var providerName = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredStringProperty(ref reader, "ProviderName"); + var traceParent = EventHubSequenceTokenJsonConverterHelpers.ReadNullableStringProperty(ref reader, "TraceParent"); + EventHubSequenceTokenJsonConverterHelpers.ReadEndObject(ref reader); + + return new EnrichedEventHubSequenceToken( + offset, + sequenceNumber, + eventIndex, + enqueuedTime, + providerName, + traceParent); + } + + /// + public override void Write(Utf8JsonWriter writer, EnrichedEventHubSequenceToken value, JsonSerializerOptions options) + { + writer.WriteStartObject(); + EventHubSequenceTokenJsonConverterHelpers.WriteBaseProperties( + writer, + value.EventHubOffset, + value.SequenceNumber, + value.EventIndex); + writer.WriteString("EnqueuedTime", value.EnqueuedTime); + writer.WriteString("ProviderName", value.ProviderName); + writer.WriteString("TraceParent", value.TraceParent); + writer.WriteEndObject(); + } +} + +internal static class EventHubSequenceTokenJsonConverterHelpers +{ + public static void ExpectStartObject(ref Utf8JsonReader reader, string typeName) + { + if (reader.TokenType is not JsonTokenType.StartObject) + { + throw new JsonException($"Expected {typeName} object, got '{reader.TokenType}'."); + } + } + + public static void WriteBaseProperties( + Utf8JsonWriter writer, + string eventHubOffset, + long sequenceNumber, + int eventIndex) + { + writer.WriteString("EventHubOffset", eventHubOffset); + writer.WriteNumber("SequenceNumber", sequenceNumber); + writer.WriteNumber("EventIndex", eventIndex); + } + + public static string ReadRequiredStringProperty(ref Utf8JsonReader reader, string propertyName) + { + ReadRequiredPropertyName(ref reader, propertyName); + if (!reader.Read() || reader.TokenType is not JsonTokenType.String) + { + throw new JsonException($"Property '{propertyName}' must be a string."); + } + + return reader.GetString() + ?? throw new JsonException($"Property '{propertyName}' must not be null."); + } + + public static string? ReadNullableStringProperty(ref Utf8JsonReader reader, string propertyName) + { + ReadRequiredPropertyName(ref reader, propertyName); + if (!reader.Read()) + { + throw new JsonException($"Unexpected end of JSON while reading '{propertyName}'."); + } + + return reader.TokenType is JsonTokenType.Null + ? null + : reader.GetString(); + } + + public static long ReadRequiredInt64Property(ref Utf8JsonReader reader, string propertyName) + { + ReadRequiredPropertyName(ref reader, propertyName); + if (!reader.Read() || reader.TokenType is not JsonTokenType.Number) + { + throw new JsonException($"Property '{propertyName}' must be a number."); + } + + return reader.GetInt64(); + } + + public static int ReadRequiredInt32Property(ref Utf8JsonReader reader, string propertyName) + { + ReadRequiredPropertyName(ref reader, propertyName); + if (!reader.Read() || reader.TokenType is not JsonTokenType.Number) + { + throw new JsonException($"Property '{propertyName}' must be a number."); + } + + return reader.GetInt32(); + } + + public static DateTimeOffset ReadRequiredDateTimeOffsetProperty(ref Utf8JsonReader reader, string propertyName) + { + ReadRequiredPropertyName(ref reader, propertyName); + if (!reader.Read() || reader.TokenType is not JsonTokenType.String) + { + throw new JsonException($"Property '{propertyName}' must be a date-time string."); + } + + return reader.GetDateTimeOffset(); + } + + public static void ReadEndObject(ref Utf8JsonReader reader) + { + if (!reader.Read() || reader.TokenType is not JsonTokenType.EndObject) + { + throw new JsonException("Expected end of JSON object."); + } + } + + private static void ReadRequiredPropertyName(ref Utf8JsonReader reader, string propertyName) + { + if (!reader.Read() || reader.TokenType is not JsonTokenType.PropertyName) + { + throw new JsonException($"Expected property '{propertyName}'."); + } + + if (!reader.ValueTextEquals(propertyName)) + { + throw new JsonException($"Expected property '{propertyName}', got '{reader.GetString()}'."); + } + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs index e392a7d0..199e9619 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursor.cs @@ -12,13 +12,16 @@ namespace Egil.Orleans.Messaging.Streams; /// /// /// Projection constraint: is an abstract -/// type. The library ships an STJ converter that handles a closed set of known -/// subtypes via a $kind discriminator: +/// type. The library ships an STJ converter that handles explicitly +/// registered token converters via a Kind discriminator: /// /// EventSequenceToken — Orleans SimpleMessageStream +/// EventSequenceTokenV2 — Orleans SimpleMessageStream v2 /// -/// Unknown subtypes throw at serialization time — silently dropping the cursor -/// would corrupt dedup state. +/// Provider packages can register additional concrete token converters through +/// . Unknown subtypes throw at +/// serialization time — silently dropping the cursor would corrupt dedup +/// state. /// /// /// Provider-specific metadata: Tokens can implement @@ -142,4 +145,4 @@ public bool TryGetTraceParent([NotNullWhen(true)] out string? traceParent) traceParent = null; return false; } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs index c804211d..c8f5f1b4 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs @@ -1,7 +1,5 @@ using System.Text.Json; using System.Text.Json.Serialization; -using Orleans.Providers.Streams.Common; -using Orleans.Streams; namespace Egil.Orleans.Messaging.Streams; @@ -14,8 +12,7 @@ namespace Egil.Orleans.Messaging.Streams; /// /// wraps a stream namespace and an optional /// StreamSequenceToken. The token is polymorphic and this converter -/// supports provider-neutral Orleans sequence token types only: -/// EventSequenceTokenV2 and EventSequenceToken. +/// delegates concrete token payloads to . /// /// /// Registered on via [JsonConverter]. @@ -25,92 +22,117 @@ namespace Egil.Orleans.Messaging.Streams; /// internal sealed class StreamCursorJsonConverter : JsonConverter { - private const string FeatureRequestUrl = "https://github.com/egil/framework/issues"; - - private const string SupportedKinds = - "EventSequenceToken, EventSequenceTokenV2"; - /// public override StreamCursor? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { - var model = JsonSerializer.Deserialize(ref reader, options); - if (model is null) + if (reader.TokenType is JsonTokenType.None && !reader.Read()) + { + throw new JsonException("Unexpected end of StreamCursor JSON."); + } + + if (reader.TokenType is JsonTokenType.Null) { return null; } - return new StreamCursor(model.StreamNamespace, FromJsonModel(model.Token), model.ProviderName); + if (reader.TokenType is not JsonTokenType.StartObject) + { + throw new JsonException($"Expected StreamCursor object, got '{reader.TokenType}'."); + } + + var streamNamespace = ReadRequiredStringProperty(ref reader, nameof(StreamCursor.StreamNamespace)); + + ReadRequiredPropertyName(ref reader, nameof(StreamCursor.Token)); + if (!reader.Read()) + { + throw new JsonException("Unexpected end of StreamCursor token JSON."); + } + + var token = reader.TokenType is JsonTokenType.Null + ? null + : StreamSequenceTokenJsonConverters.Read(ref reader, options); + + string? providerName = null; + if (!reader.Read()) + { + throw new JsonException("Unexpected end of StreamCursor JSON."); + } + + if (reader.TokenType is JsonTokenType.PropertyName) + { + if (!reader.ValueTextEquals(nameof(StreamCursor.ProviderName))) + { + throw new JsonException($"Expected StreamCursor property '{nameof(StreamCursor.ProviderName)}', got '{reader.GetString()}'."); + } + + if (!reader.Read()) + { + throw new JsonException("Unexpected end of StreamCursor provider name JSON."); + } + + providerName = reader.TokenType is JsonTokenType.Null + ? null + : reader.GetString(); + + if (!reader.Read()) + { + throw new JsonException("Unexpected end of StreamCursor JSON."); + } + } + + if (reader.TokenType is not JsonTokenType.EndObject) + { + throw new JsonException("Expected end of StreamCursor object."); + } + + return new StreamCursor(streamNamespace, token, providerName); } /// public override void Write(Utf8JsonWriter writer, StreamCursor value, JsonSerializerOptions options) { - JsonSerializer.Serialize( - writer, - new StreamCursorJsonModel( - value.StreamNamespace, - ToJsonModel(value.Token), - value.ProviderName), - options); - } - - private static StreamSequenceToken? FromJsonModel(StreamSequenceTokenJsonModel? model) - { - if (model is null) + writer.WriteStartObject(); + writer.WriteString(nameof(StreamCursor.StreamNamespace), value.StreamNamespace); + writer.WritePropertyName(nameof(StreamCursor.Token)); + if (value.Token is null) { - return null; + writer.WriteNullValue(); + } + else + { + StreamSequenceTokenJsonConverters.Write(writer, value.Token, options); } - return model.Kind switch + if (value.ProviderName is not null) { - StreamSequenceTokenKinds.EventSequenceToken => new EventSequenceToken(model.SequenceNumber, model.EventIndex), - StreamSequenceTokenKinds.EventSequenceTokenV2 => new EventSequenceTokenV2(model.SequenceNumber, model.EventIndex), - _ => throw CreateUnsupportedTokenException(model.Kind ?? "") - }; + writer.WriteString(nameof(StreamCursor.ProviderName), value.ProviderName); + } + + writer.WriteEndObject(); } - private static StreamSequenceTokenJsonModel? ToJsonModel(StreamSequenceToken? value) + private static string ReadRequiredStringProperty(ref Utf8JsonReader reader, string propertyName) { - if (value is null) + ReadRequiredPropertyName(ref reader, propertyName); + if (!reader.Read() || reader.TokenType is not JsonTokenType.String) { - return null; + throw new JsonException($"StreamCursor property '{propertyName}' must be a string."); } - return value switch - { - EventSequenceTokenV2 token => new StreamSequenceTokenJsonModel( - Kind: StreamSequenceTokenKinds.EventSequenceTokenV2, - SequenceNumber: token.SequenceNumber, - EventIndex: token.EventIndex), - EventSequenceToken token => new StreamSequenceTokenJsonModel( - Kind: StreamSequenceTokenKinds.EventSequenceToken, - SequenceNumber: token.SequenceNumber, - EventIndex: token.EventIndex), - _ => throw CreateUnsupportedTokenException(value.GetType().FullName ?? value.GetType().Name) - }; + return reader.GetString() + ?? throw new JsonException($"StreamCursor property '{propertyName}' must not be null."); } - private static NotSupportedException CreateUnsupportedTokenException(string tokenIdentifier) => - new( - $"Unsupported stream sequence token '{tokenIdentifier}'. " + - $"This converter supports only built-in Orleans token types: {SupportedKinds}. " + - "Custom token support is intentionally disabled to keep persisted StreamCursor JSON format stable. " + - $"If you need this feature, please request it at {FeatureRequestUrl}."); - - private sealed record StreamCursorJsonModel( - string StreamNamespace, - StreamSequenceTokenJsonModel? Token, - string? ProviderName = null); - - private sealed record StreamSequenceTokenJsonModel( - string? Kind = null, - long SequenceNumber = 0, - int EventIndex = 0, - string? EventHubOffset = null); - - private static class StreamSequenceTokenKinds + private static void ReadRequiredPropertyName(ref Utf8JsonReader reader, string propertyName) { - public const string EventSequenceToken = "event-sequence"; - public const string EventSequenceTokenV2 = "event-sequence-v2"; + if (!reader.Read() || reader.TokenType is not JsonTokenType.PropertyName) + { + throw new JsonException($"Expected StreamCursor property '{propertyName}'."); + } + + if (!reader.ValueTextEquals(propertyName)) + { + throw new JsonException($"Expected StreamCursor property '{propertyName}', got '{reader.GetString()}'."); + } } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverterServiceCollectionExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverterServiceCollectionExtensions.cs new file mode 100644 index 00000000..692513b4 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverterServiceCollectionExtensions.cs @@ -0,0 +1,56 @@ +using System.Text.Json.Serialization; +using Egil.Orleans.Messaging.Streams; +using Orleans.Streams; + +namespace Microsoft.Extensions.DependencyInjection; + +/// +/// Registration helpers for provider-specific stream sequence token JSON +/// converters. +/// +public static class StreamSequenceTokenJsonConverterServiceCollectionExtensions +{ + extension(IServiceCollection services) + { + /// + /// Registers a JSON converter for a concrete stream sequence token type. + /// + /// + /// The converter is registered in the process-wide registry used by the + /// built-in System.Text.Json converters, since grain-state converters + /// are not resolved from the application service provider. + /// + public IServiceCollection AddStreamSequenceTokenJsonConverter( + string typeDescriptor) + where TToken : StreamSequenceToken + where TConverter : JsonConverter, new() + { + ArgumentNullException.ThrowIfNull(services); + + StreamSequenceTokenJsonConverters.Register(typeDescriptor); + + return services; + } + + /// + /// Registers a JSON converter instance for a concrete stream sequence + /// token type. + /// + /// + /// The converter is registered in the process-wide registry used by the + /// built-in System.Text.Json converters, since grain-state converters + /// are not resolved from the application service provider. + /// + public IServiceCollection AddStreamSequenceTokenJsonConverter( + string typeDescriptor, + JsonConverter converter) + where TToken : StreamSequenceToken + { + ArgumentNullException.ThrowIfNull(services); + + StreamSequenceTokenJsonConverters.Register(typeDescriptor, converter); + + return services; + } + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverters.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverters.cs new file mode 100644 index 00000000..86834f14 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverters.cs @@ -0,0 +1,300 @@ +using System.Collections.Immutable; +using System.Text.Json; +using System.Text.Json.Serialization; +using Orleans.Providers.Streams.Common; +using Orleans.Streams; + +namespace Egil.Orleans.Messaging.Streams; + +/// +/// Process-wide registry of explicitly registered JSON converters for concrete +/// types. +/// +/// +/// and use +/// this registry to write a small discriminator envelope and then delegate the +/// token payload directly to the registered . +/// Converters should be registered during silo startup before persisted stream +/// positions are serialized or deserialized. +/// +public static class StreamSequenceTokenJsonConverters +{ + private static readonly object Sync = new(); + + private static ImmutableArray registrations = + [ + Registration.Create("event-sequence", new EventSequenceTokenJsonConverter()), + Registration.Create("event-sequence-v2", new EventSequenceTokenV2JsonConverter()) + ]; + + /// + /// Registers a converter for a concrete stream sequence token type. + /// + /// The concrete token type handled by the converter. + /// Stable discriminator written to the token JSON envelope. + /// The converter that reads and writes the token payload. + /// + /// Thrown when another converter already owns the same type descriptor. + /// + public static void Register( + string typeDescriptor, + JsonConverter converter) + where TToken : StreamSequenceToken + { + ArgumentException.ThrowIfNullOrWhiteSpace(typeDescriptor); + ArgumentNullException.ThrowIfNull(converter); + + Register(Registration.Create(typeDescriptor, converter)); + } + + /// + /// Registers a converter with a public parameterless constructor for a + /// concrete stream sequence token type. + /// + /// The concrete token type handled by the converter. + /// The converter type. + /// Stable discriminator written to the token JSON envelope. + public static void Register(string typeDescriptor) + where TToken : StreamSequenceToken + where TConverter : JsonConverter, new() => + Register(typeDescriptor, new TConverter()); + + internal static void Write(Utf8JsonWriter writer, StreamSequenceToken token, JsonSerializerOptions options) + { + ArgumentNullException.ThrowIfNull(writer); + ArgumentNullException.ThrowIfNull(token); + + var registration = registrations.FirstOrDefault(x => x.CanWrite(token)) + ?? throw CreateUnsupportedTokenException(token.GetType().FullName ?? token.GetType().Name); + + writer.WriteStartObject(); + writer.WriteString("Kind", registration.TypeDescriptor); + writer.WritePropertyName("Payload"); + registration.Write(writer, token, options); + writer.WriteEndObject(); + } + + internal static StreamSequenceToken Read(ref Utf8JsonReader reader, JsonSerializerOptions options) + { + if (reader.TokenType is JsonTokenType.None && !reader.Read()) + { + throw new JsonException("Unexpected end of stream sequence token JSON."); + } + + if (reader.TokenType is not JsonTokenType.StartObject) + { + throw new JsonException($"Expected stream sequence token object, got '{reader.TokenType}'."); + } + + var typeDescriptor = ReadRequiredStringProperty(ref reader, "Kind"); + var registration = registrations.FirstOrDefault(x => x.TypeDescriptor == typeDescriptor); + if (registration is null) + { + throw new JsonException(CreateUnsupportedTokenMessage(typeDescriptor)); + } + + ReadRequiredPropertyName(ref reader, "Payload"); + if (!reader.Read()) + { + throw new JsonException("Unexpected end of stream sequence token payload."); + } + + var token = registration.Read(ref reader, options); + + if (!reader.Read() || reader.TokenType is not JsonTokenType.EndObject) + { + throw new JsonException("Expected end of stream sequence token object."); + } + + return token; + } + + internal static NotSupportedException CreateUnsupportedTokenException(string tokenIdentifier) => + new(CreateUnsupportedTokenMessage(tokenIdentifier)); + + private static void Register(Registration registration) + { + lock (Sync) + { + var existing = registrations.FirstOrDefault(x => x.TypeDescriptor == registration.TypeDescriptor); + if (existing is null) + { + registrations = registrations.Add(registration); + return; + } + + if (existing.TokenType == registration.TokenType + && existing.ConverterType == registration.ConverterType) + { + return; + } + + throw new InvalidOperationException( + $"A stream sequence token JSON converter for '{registration.TypeDescriptor}' is already registered by " + + $"'{existing.ConverterType.FullName}'."); + } + } + + private static string ReadRequiredStringProperty(ref Utf8JsonReader reader, string propertyName) + { + ReadRequiredPropertyName(ref reader, propertyName); + if (!reader.Read() || reader.TokenType is not JsonTokenType.String) + { + throw new JsonException($"Stream sequence token property '{propertyName}' must be a string."); + } + + var value = reader.GetString(); + if (string.IsNullOrWhiteSpace(value)) + { + throw new JsonException($"Stream sequence token property '{propertyName}' must not be empty."); + } + + return value; + } + + private static void ReadRequiredPropertyName(ref Utf8JsonReader reader, string propertyName) + { + if (!reader.Read() || reader.TokenType is not JsonTokenType.PropertyName) + { + throw new JsonException($"Expected stream sequence token property '{propertyName}'."); + } + + if (!reader.ValueTextEquals(propertyName)) + { + throw new JsonException($"Expected stream sequence token property '{propertyName}', got '{reader.GetString()}'."); + } + } + + private static string CreateUnsupportedTokenMessage(string tokenIdentifier) => + $"Unsupported stream sequence token '{tokenIdentifier}'. " + + "Register a JsonConverter for provider-specific StreamSequenceToken types before " + + "serializing or deserializing MessageTracker or StreamCursor values."; + + private abstract class Registration + { + protected Registration(string typeDescriptor, Type tokenType, Type converterType) + { + TypeDescriptor = typeDescriptor; + TokenType = tokenType; + ConverterType = converterType; + } + + public string TypeDescriptor { get; } + + public Type TokenType { get; } + + public Type ConverterType { get; } + + public static Registration Create( + string typeDescriptor, + JsonConverter converter) + where TToken : StreamSequenceToken => + new Registration(typeDescriptor, converter); + + public bool CanWrite(StreamSequenceToken token) => + token.GetType() == TokenType; + + public abstract void Write(Utf8JsonWriter writer, StreamSequenceToken token, JsonSerializerOptions options); + + public abstract StreamSequenceToken Read(ref Utf8JsonReader reader, JsonSerializerOptions options); + } + + private sealed class Registration : Registration + where TToken : StreamSequenceToken + { + private readonly JsonConverter converter; + + public Registration(string typeDescriptor, JsonConverter converter) + : base(typeDescriptor, typeof(TToken), converter.GetType()) + { + this.converter = converter; + } + + public override void Write(Utf8JsonWriter writer, StreamSequenceToken token, JsonSerializerOptions options) => + converter.Write(writer, (TToken)token, options); + + public override StreamSequenceToken Read(ref Utf8JsonReader reader, JsonSerializerOptions options) => + converter.Read(ref reader, typeof(TToken), options) + ?? throw new JsonException($"Converter '{converter.GetType().FullName}' returned null."); + } + + private sealed class EventSequenceTokenJsonConverter : JsonConverter + { + public override EventSequenceToken Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + if (reader.TokenType is not JsonTokenType.StartObject) + { + throw new JsonException($"Expected {nameof(EventSequenceToken)} object, got '{reader.TokenType}'."); + } + + var sequenceNumber = ReadRequiredInt64Property(ref reader, "SequenceNumber"); + var eventIndex = ReadRequiredInt32Property(ref reader, "EventIndex"); + ReadEndObject(ref reader); + + return new EventSequenceToken(sequenceNumber, eventIndex); + } + + public override void Write(Utf8JsonWriter writer, EventSequenceToken value, JsonSerializerOptions options) + { + writer.WriteStartObject(); + writer.WriteNumber("SequenceNumber", value.SequenceNumber); + writer.WriteNumber("EventIndex", value.EventIndex); + writer.WriteEndObject(); + } + } + + private sealed class EventSequenceTokenV2JsonConverter : JsonConverter + { + public override EventSequenceTokenV2 Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + if (reader.TokenType is not JsonTokenType.StartObject) + { + throw new JsonException($"Expected {nameof(EventSequenceTokenV2)} object, got '{reader.TokenType}'."); + } + + var sequenceNumber = ReadRequiredInt64Property(ref reader, "SequenceNumber"); + var eventIndex = ReadRequiredInt32Property(ref reader, "EventIndex"); + ReadEndObject(ref reader); + + return new EventSequenceTokenV2(sequenceNumber, eventIndex); + } + + public override void Write(Utf8JsonWriter writer, EventSequenceTokenV2 value, JsonSerializerOptions options) + { + writer.WriteStartObject(); + writer.WriteNumber("SequenceNumber", value.SequenceNumber); + writer.WriteNumber("EventIndex", value.EventIndex); + writer.WriteEndObject(); + } + } + + private static long ReadRequiredInt64Property(ref Utf8JsonReader reader, string propertyName) + { + ReadRequiredPropertyName(ref reader, propertyName); + if (!reader.Read() || reader.TokenType is not JsonTokenType.Number) + { + throw new JsonException($"Property '{propertyName}' must be a number."); + } + + return reader.GetInt64(); + } + + private static int ReadRequiredInt32Property(ref Utf8JsonReader reader, string propertyName) + { + ReadRequiredPropertyName(ref reader, propertyName); + if (!reader.Read() || reader.TokenType is not JsonTokenType.Number) + { + throw new JsonException($"Property '{propertyName}' must be a number."); + } + + return reader.GetInt32(); + } + + private static void ReadEndObject(ref Utf8JsonReader reader) + { + if (!reader.Read() || reader.TokenType is not JsonTokenType.EndObject) + { + throw new JsonException("Expected end of JSON object."); + } + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs index b621c96f..039cf1d9 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs @@ -3,8 +3,6 @@ using System.Text.Json; using System.Text.Json.Serialization; using Egil.Orleans.Messaging.Streams; -using Orleans.Providers.Streams.Common; -using Orleans.Streams; namespace Egil.Orleans.Messaging.Tracking; @@ -49,7 +47,7 @@ internal sealed class MessageTrackerJsonConverter : JsonConverter(); foreach (var item in model.Streams ?? Array.Empty()) { - var lastPosition = FromJsonModel(item.LastPosition); + var lastPosition = item.LastPosition; streams.Add( MessageTracker.StreamSource.From(lastPosition), new MessageTracker.StreamEntry(lastPosition, item.Received)); @@ -80,7 +78,7 @@ public override void Write(Utf8JsonWriter writer, MessageTracker value, JsonSeri foreach (var item in streams) { streamModels[streamIndex++] = new StreamEntryJsonModel( - ToJsonModel(item.Value.LastPosition), + item.Value.LastPosition, item.Value.Received); } @@ -110,60 +108,6 @@ public override void Write(Utf8JsonWriter writer, MessageTracker value, JsonSeri private static GrainId ParseGrainId(GrainIdJsonModel value) => GrainId.Create(value.Type, value.Key); - private static StreamCursor FromJsonModel(StreamCursorJsonModel model) => - new(model.StreamNamespace, FromJsonModel(model.Token), model.ProviderName); - - private static StreamSequenceToken? FromJsonModel(StreamSequenceTokenJsonModel? model) - { - if (model is null) - { - return null; - } - - return model.Kind switch - { - StreamSequenceTokenKinds.EventSequenceToken => new EventSequenceToken(model.SequenceNumber, model.EventIndex), - StreamSequenceTokenKinds.EventSequenceTokenV2 => new EventSequenceTokenV2(model.SequenceNumber, model.EventIndex), - _ => throw new JsonException($"Unsupported stream sequence token kind '{model.Kind}'.") - }; - } - - private static StreamCursorJsonModel ToJsonModel(StreamCursor value) => - new( - value.StreamNamespace, - ToJsonModel(value.Token), - value.ProviderName); - - private static StreamSequenceTokenJsonModel? ToJsonModel(StreamSequenceToken? value) - { - if (value is null) - { - return null; - } - - return value switch - { - EventSequenceTokenV2 token => new StreamSequenceTokenJsonModel( - StreamSequenceTokenKinds.EventSequenceTokenV2, - token.SequenceNumber, - token.EventIndex, - null, - null, - null, - null), - EventSequenceToken token => new StreamSequenceTokenJsonModel( - StreamSequenceTokenKinds.EventSequenceToken, - token.SequenceNumber, - token.EventIndex, - null, - null, - null, - null), - _ => throw new NotSupportedException( - $"Unsupported stream sequence token type '{value.GetType().FullName}'.") - }; - } - private static GrainIdJsonModel ToJsonModel(GrainId grainId) => new(grainId.Type.ToString()!, grainId.Key.ToString()!); @@ -172,7 +116,7 @@ private sealed record MessageTrackerJsonModel( OutboxEntryJsonModel[]? Outboxes); private sealed record StreamEntryJsonModel( - StreamCursorJsonModel LastPosition, + StreamCursor LastPosition, DateTimeOffset Received); private sealed record OutboxEntryJsonModel( @@ -181,27 +125,7 @@ private sealed record OutboxEntryJsonModel( long LastSequenceNumber, DateTimeOffset Received); - private sealed record StreamCursorJsonModel( - string StreamNamespace, - StreamSequenceTokenJsonModel? Token, - string? ProviderName); - - private sealed record StreamSequenceTokenJsonModel( - string Kind, - long SequenceNumber, - int EventIndex, - string? EventHubOffset, - DateTimeOffset? EnqueuedTime, - string? ProviderName, - string? TraceParent); - private sealed record GrainIdJsonModel( string Type, string Key); - - private static class StreamSequenceTokenKinds - { - public const string EventSequenceToken = "event-sequence"; - public const string EventSequenceTokenV2 = "event-sequence-v2"; - } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubSequenceTokenJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubSequenceTokenJsonConverterTests.cs new file mode 100644 index 00000000..f061db9b --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubSequenceTokenJsonConverterTests.cs @@ -0,0 +1,66 @@ +using System.Text.Json; +using Egil.Orleans.Messaging.Tracking; + +namespace Egil.Orleans.Messaging.Streams.EventHubs.Tests.EventHubs; + +public sealed class EnrichedEventHubSequenceTokenJsonConverterTests +{ + [Fact] + public void StreamCursor_round_trips_enriched_event_hub_token() + { + StreamSequenceTokenJsonConverters.Register( + EnrichedEventHubSequenceToken.TypeAlias, + new EnrichedEventHubSequenceTokenJsonConverter()); + var token = CreateToken(); + var cursor = new StreamCursor("orders", token); + + var json = JsonSerializer.Serialize(cursor); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.Contains(EnrichedEventHubSequenceToken.TypeAlias, json); + Assert.Contains("\"Payload\"", json); + Assert.NotNull(roundTripped); + AssertEnrichedToken(token, Assert.IsType(roundTripped.Token)); + } + + [Fact] + public void MessageTracker_round_trips_enriched_event_hub_token() + { + StreamSequenceTokenJsonConverters.Register( + EnrichedEventHubSequenceToken.TypeAlias, + new EnrichedEventHubSequenceTokenJsonConverter()); + var token = CreateToken(); + var tracker = new MessageTracker(); + tracker.ProcessMessage(new StreamCursor("orders", token), out tracker); + + var json = JsonSerializer.Serialize(tracker); + var roundTripped = JsonSerializer.Deserialize(json); + + Assert.Contains(EnrichedEventHubSequenceToken.TypeAlias, json); + Assert.Contains("\"Payload\"", json); + Assert.NotNull(roundTripped); + var latest = roundTripped.LatestStreamSequenceToken("event-hubs", "orders"); + AssertEnrichedToken(token, Assert.IsType(latest)); + } + + private static EnrichedEventHubSequenceToken CreateToken() => + new( + "12345", + 42, + 2, + new DateTimeOffset(2026, 5, 26, 10, 15, 0, TimeSpan.Zero), + "event-hubs", + "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01"); + + private static void AssertEnrichedToken( + EnrichedEventHubSequenceToken expected, + EnrichedEventHubSequenceToken actual) + { + Assert.Equal(expected.EventHubOffset, actual.EventHubOffset); + Assert.Equal(expected.SequenceNumber, actual.SequenceNumber); + Assert.Equal(expected.EventIndex, actual.EventIndex); + Assert.Equal(expected.EnqueuedTime, actual.EnqueuedTime); + Assert.Equal(expected.ProviderName, actual.ProviderName); + Assert.Equal(expected.TraceParent, actual.TraceParent); + } +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs index 09a8c392..d5d9d934 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs @@ -62,15 +62,16 @@ public void JsonSerializer_throws_for_stream_cursor_payload_with_unknown_token_k "StreamNamespace": "orders", "Token": { "Kind": "unknown-kind", - "SequenceNumber": 1, - "EventIndex": 0 + "Payload": { + "SequenceNumber": 1, + "EventIndex": 0 + } } } """; - var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); - Assert.Contains("https://github.com/egil/framework/issues", exception.Message); - Assert.Contains("supports only built-in Orleans token types", exception.Message); + var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); + Assert.Contains("Register a JsonConverter", exception.Message); } [Fact] @@ -81,8 +82,7 @@ public void JsonSerializer_throws_for_custom_token_type() new UnsupportedCustomSequenceToken(21, 3)); var exception = Assert.Throws(() => JsonSerializer.Serialize(cursor)); - Assert.Contains("https://github.com/egil/framework/issues", exception.Message); - Assert.Contains("supports only built-in Orleans token types", exception.Message); + Assert.Contains("Register a JsonConverter", exception.Message); } private sealed class UnsupportedCustomSequenceToken(long sequenceNumber, int eventIndex) : StreamSequenceToken @@ -110,4 +110,4 @@ public override int CompareTo(StreamSequenceToken? other) return EventIndex.CompareTo(other.EventIndex); } } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs index 1376c722..e4b5582e 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs @@ -119,8 +119,10 @@ public void JsonSerializer_throws_for_unknown_stream_token_kind() "StreamNamespace": "orders", "Token": { "Kind": "unknown-kind", - "SequenceNumber": 1, - "EventIndex": 0 + "Payload": { + "SequenceNumber": 1, + "EventIndex": 0 + } } }, "Received": "2026-05-23T12:30:00+00:00" @@ -158,4 +160,4 @@ public override int CompareTo(StreamSequenceToken? other) return EventIndex.CompareTo(other.EventIndex); } } -} \ No newline at end of file +} From 2eb3a850a6a2f720d4ce6ec3eafed3a174e25097 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Tue, 26 May 2026 13:28:04 +0000 Subject: [PATCH 44/82] docs(om): update messaging api design Document the Azure Storage classification behavior, outbox background posting model, and explicit stream token JSON converter registry. Keep the public README focused on observable behavior while leaving Orleans scheduling rationale in the API design notes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 44 +++- Egil.Orleans.Messaging/api-design.md | 290 +++++++++++++++++---------- 2 files changed, 222 insertions(+), 112 deletions(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index 25040a8e..6755c002 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -57,10 +57,14 @@ factory instead: siloBuilder.AddAzureStorageStateManager("state"); ``` -The Azure-aware manager treats Azure Storage optimistic-concurrency failures -such as HTTP 412 as definite non-persistence, so writes and clears can fail -fast without an unnecessary recovery read. Ambiguous failures, such as -transient 5xx responses, still use read-back recovery. +The Azure-aware manager uses Azure SDK `RequestFailedException.Status` and +`ErrorCode` values to decide recovery. Optimistic-concurrency and rejected +request failures such as HTTP 412, 409, 404, authentication/authorization +failures, and payload/validation failures are treated as definite +non-persistence, so writes and clears fail fast without an unnecessary +recovery read. Ambiguous or transient outcomes, including HTTP 503 +`ServerBusy`, HTTP 500 `OperationTimedOut`, HTTP 429 throttling, no-response +failures, and timeout exceptions, still use read-back recovery. Then wrap the Orleans persistent state facet during activation: @@ -152,10 +156,19 @@ Failed dispatches are reported through `ReconcileFailedAsync`. That callback is where the owning grain applies retry, dead-letter, max-depth, or trimming policy, because the grain owns the durable outbox state. -Postmen normally run on the Orleans activation scheduler. Use -`OutboxPostmanExecutionMode.ThreadPool` only for blocking or legacy delivery -code, and do not read or mutate activation-local grain state from those -callbacks. +Background outbox postage allows unrelated grain calls to continue while +postmen await I/O by default. `IPostman` services should be state-free with +respect to the owning grain. Inline lambda postmen may read activation-local +state, but should not write it; durable changes belong in +`AcknowledgePostedAsync` or `ReconcileFailedAsync`. +Postmen run on Orleans' activation scheduler, not on the .NET thread pool. +Acknowledgement and failure callbacks are non-interleaving by default, so they +do not interleave with normal grain calls unless +`InterleaveReconciliationCallbacks` is enabled. Reentrant grains can still +interleave according to Orleans' normal scheduling rules. +Pending items in a post run are dispatched concurrently. Successful items are +still acknowledged as one ordered batch after all dispatches complete, and +failed items are reconciled as one batch. For reusable delivery code, implement and register keyed postman services: @@ -278,8 +291,23 @@ using Egil.Orleans.Messaging.Streams.EventHubs; using Orleans.Hosting; ``` +Registering the enriched adapter also registers Event Hubs sequence-token JSON +converters, so `MessageTracker` and `StreamCursor` can persist and restore +`EnrichedEventHubSequenceToken` without downcasting it to the Orleans base +event token: + +```csharp +siloBuilder.AddEventHubStreams("event-hubs", configurator => +{ + configurator.UseEnrichedDataAdapter(); +}); +``` + The core package can consume provider-specific token metadata through `IStreamSequenceTokenMetadata` without taking a direct Event Hubs dependency. +Custom stream providers that expose custom `StreamSequenceToken` types should +register a `JsonConverter` with `StreamSequenceTokenJsonConverters` +during startup. ## Scope diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index f5e6fc00..58425497 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -282,10 +282,31 @@ while transient 5xx errors remain ambiguous and require read-back recovery. Read failures do not use the classification because no local committed state has been tentatively changed. -The Azure Storage companion package treats Orleans/Azure optimistic -concurrency failures (`InconsistentStateException` and Azure Storage HTTP -412) as `DidNotPersist` for writes and clears. Other provider failures stay -`UnknownOutcome`, preserving the default read-back recovery path. +The Azure Storage companion package follows Orleans' Azure provider semantics: +the Orleans provider wraps Azure Table/Blob optimistic-update failures +(precondition failed, conflict, and not found during conditional write/clear) +as `InconsistentStateException`, so those are classified as +`DidNotPersist`. + +It also understands Azure SDK `RequestFailedException` directly: + +- `DidNotPersist`: rejected/client-side service responses where Azure has + definitively not applied the mutation, including HTTP 400/401/403/404/409, + HTTP 412, payload-too-large responses, and Azure Table/Blob error codes such + as `UpdateConditionNotSatisfied`, `ConditionNotMet`, `BlobAlreadyExists`, + `BlobNotFound`, `ContainerNotFound`, and table entity not-found/already-exists + errors. +- `UnknownOutcome`: ambiguous or transient outcomes where a write/clear may + have landed before the caller observed failure, including no HTTP response, + HTTP 408, HTTP 429, all 5xx responses, `ServerBusy`, `OperationTimedOut`, + `InternalError`, account IOPS throttling, `TimeoutException`, and + cancellation-like failures. + +For aggregate Azure SDK failures, any ambiguous inner failure makes the whole +operation `UnknownOutcome`; otherwise, all deterministic storage failures can +be treated as `DidNotPersist`. This preserves correctness over optimization: +read-back is skipped only when the provider response proves the mutation did +not persist. ### Why a wrapper, not extension methods @@ -1027,19 +1048,25 @@ ambiguity is visible in code. ### `StreamCursor` projection constraint -`StreamCursor` carries an opaque `StreamSequenceToken`. Library ships an -STJ converter for a closed set of known subtypes discriminated on -`$kind`: +`StreamCursor` carries an opaque `StreamSequenceToken`. The core package ships +an STJ converter which writes a small discriminator envelope and delegates the +token payload directly to explicitly registered `JsonConverter` +instances. The discriminator is the token JSON `Kind` property. | Subtype | Source | | ------------------------------- | ------------------------------- | | `EventSequenceToken` | Orleans SimpleMessageStream | -| `EventHubSequenceToken` | Orleans EH provider v1 | -| `EventHubSequenceTokenV2` | Orleans EH provider v2 | -| `EnrichedEventHubSequenceToken` | Library-shipped v2 subclass | +| `EventSequenceTokenV2` | Orleans SimpleMessageStream v2 | +| `EventHubSequenceToken` | Event Hubs companion package | +| `EventHubSequenceTokenV2` | Event Hubs companion package | +| `EnrichedEventHubSequenceToken` | Event Hubs companion package | Unknown subtype throws at serialization time — silently dropping the -cursor would corrupt dedup. +cursor would corrupt dedup. Unknown discriminator throws at deserialization +time. Provider packages register converters during their normal setup; custom +stream providers can register their own converter with +`AddStreamSequenceTokenJsonConverter(...)` or +`StreamSequenceTokenJsonConverters.Register(...)`. ### Event Hub adapter for enriched tokens @@ -1064,6 +1091,12 @@ siloBuilder.AddEventHubStreams("orders", b => }); ``` +`UseEnrichedDataAdapter()` also registers Event Hubs sequence-token JSON +converters. That lets `MessageTracker` and `StreamCursor` round-trip +`EnrichedEventHubSequenceToken` without the core package referencing Event +Hubs and without losing `EventHubOffset`, `EnqueuedTime`, `ProviderName`, or +`TraceParent`. + Users who need custom adapter behavior can subclass `EnrichedEventHubAdapter` and register their subclass via Orleans' `UseDataAdapter` directly. The library intentionally provides no @@ -1186,21 +1219,21 @@ in the `Clever.PricingEngine` codebase. - **Grain-scoped, not silo-scoped.** Each grain with an outbox gets its own `OutboxProcessor`. No external scan, no registry, no second store. -- **GrainTimer** for in-process fast retry while activated. +- **In-process retry** for fast retry while activated. - **Durable Reminder** for cross-activation recovery. Reactivates the - grain if it deactivates with pending items. Timer arms on activation. + grain if it deactivates with pending items. Retry scheduling is updated on + activation. - **Single active drain.** At most one send attempt may run per activation. - `PostAsync`, `PostInBackgroundAsync`, timer callbacks, and reminder - callbacks all coalesce through the same drain gate. If a post run is - already active, additional requests mark another run as desired rather - than starting concurrently. -- **Callback interleaving is explicit.** `Interleave` mirrors Orleans' - `GrainTimerCreationOptions.Interleave` and controls whether processor - callbacks run interleaved with other grain turns. It does not by itself - permit overlapping post runs. -- **Keep-alive is explicit.** `KeepAlive` mirrors Orleans' - `GrainTimerCreationOptions.KeepAlive` and controls whether an active - processor timer keeps the grain activation alive. + `PostAsync`, `PostInBackgroundAsync`, retry callbacks, and reminder + callbacks all coalesce through the same drain gate. If a post run is already + active, additional requests mark another run as desired rather than starting + concurrently. +- **Background postage interleaves by default.** `Interleave` defaults to + `true`, allowing unrelated grain calls to continue while postmen await I/O. + It does not by itself permit overlapping post runs. +- **Keep-alive is explicit.** `KeepAlive` controls whether background retry + work should keep the grain activation alive while pending outbox items + remain. ### Postman dispatch @@ -1226,11 +1259,15 @@ public sealed class OutboxPostmanAttribute(string name) : Attribute } ``` -Use inline callbacks when delivery logic is grain-local and small. Use keyed -`IPostman` services when the delivery code has dependencies, should -be tested outside Orleans, or is shared across grains. Registered postman -services are scoped to the grain activation service provider and the processor -does not create child scopes. +Use inline callbacks when delivery logic is grain-local and small. Inline +callbacks are the only postman shape which can naturally close over +activation-local grain state, so they need the most care when background +postage interleaves with other grain turns. Use keyed `IPostman` +services when the delivery code has dependencies, should be tested outside +Orleans, or is shared across grains. Registered postman services are scoped to +the grain activation service provider and the processor does not create child +scopes. They should be state-free with respect to the owning grain: use the +message payload, injected services, and `IGrainFactory`, not grain fields. ```csharp namespace Microsoft.Extensions.DependencyInjection; @@ -1256,62 +1293,102 @@ public static class OutboxPostmanServiceCollectionExtensions with attempt count (in-memory, resets on reactivation) — the grain decides: leave item in state to retry, or remove to dead-letter after N attempts. +- Pending items dispatch concurrently within a single drain. The processor + starts all matching postman calls, waits for all of them, then invokes + `AcknowledgePostedAsync` with successful items in original pending order + and `ReconcileFailedAsync` with failed items. This avoids head-of-line + blocking between independent postmen while keeping durable reconciliation + batched and ordered. - Each item dispatches to exactly **one** postman (first-registered-wins). Items whose runtime type matches no postman → reported as failed with `NoPostmanRegisteredException`. - `PostAsync` only throws `TimeoutException` (per-run timeout), `OperationCanceledException` (caller token), or callback exceptions. -Postman callbacks can run in one of two execution modes: +Postman callbacks run on Orleans' activation scheduler. Keyed +`IPostman` services should not depend on activation-local grain +state. Inline postmen may close over and read activation-local state, but they +should not mutate it; durable changes belong in `AcknowledgePostedAsync` and +`ReconcileFailedAsync`. -```csharp -public enum OutboxPostmanExecutionMode -{ - /// - /// Executes postman callbacks on the Orleans activation scheduler. - /// - /// - /// This is the default and recommended mode. - /// - /// The Orleans activation scheduler is the per-activation scheduler that - /// provides Orleans' turn-based execution model. Work scheduled on it - /// runs as grain work: it is single-threaded per activation and obeys - /// Orleans request scheduling, reentrancy, and timer interleaving rules. - /// - /// Use this mode for normal async postmen. Well-behaved async libraries - /// should be awaited directly and do not need to be moved to the .NET - /// thread pool. - /// - GrainScheduler, - - /// - /// Executes postman callbacks on the .NET thread pool, outside the - /// Orleans activation scheduler. - /// - /// - /// This is an escape hatch for blocking or legacy postmen which would - /// otherwise block the grain scheduler. It should not be used merely - /// because a postman performs asynchronous I/O. - /// - /// Postmen in this mode must not read or mutate activation-local grain - /// state. They may call other grains through IGrainFactory or use - /// external services which are safe to use from thread-pool code. - /// PendingItems, AcknowledgePostedAsync, and ReconcileFailedAsync still - /// execute on the Orleans activation scheduler. - /// - ThreadPool -} -``` +Background postage uses Orleans activation scheduling and `Interleave` defaults +to `true`, so other grain calls can run while postmen await I/O. Orleans still +executes only one turn at a time on the activation. Reconciliation uses +`InterleaveReconciliationCallbacks` and defaults to non-interleaving, so +`AcknowledgePostedAsync` and `ReconcileFailedAsync` do not interleave with +ordinary grain calls unless the user opts in or the grain is reentrant. + +The scheduling goal is to keep external postage fast without letting durable +outbox reconciliation interleave with ordinary grain writes: -`GrainScheduler` is the safe default. Postmen may close over activation -state because they execute under Orleans activation scheduling. +```mermaid +sequenceDiagram + participant Grain + participant Dispatch as "Interleaving dispatch turn" + participant Postmen + participant Reconcile as "Non-interleaving reconciliation turn" + + Grain->>Dispatch: "PostInBackgroundAsync schedules dispatch" + Dispatch->>Grain: "PendingItems() snapshot" + Dispatch->>Postmen: "Dispatch all pending items concurrently" + Note over Dispatch,Grain: "Other grain calls may run while postmen await" + Postmen-->>Dispatch: "Success/failure results" + Dispatch->>Reconcile: "Enqueue reconciliation" + Reconcile->>Grain: "AcknowledgePostedAsync / ReconcileFailedAsync" + Note over Reconcile,Grain: "Must not interleave with normal writes" + Reconcile->>Grain: "PendingItems() and retry/reminder update" +``` -`ThreadPool` is for blocking or legacy postmen. In this mode postmen must -not read or mutate activation-local grain state. They should be static or -otherwise state-free and use services such as `IGrainFactory` to deliver the -message. Only the fast grain callbacks (`PendingItems`, -`AcknowledgePostedAsync`, `ReconcileFailedAsync`) run on the Orleans -activation scheduler and respect `Interleave`. +Orleans has two relevant scheduling layers: + +- **Task scheduling** through `IGrainContext.Scheduler` / + `IWorkItemScheduler` queues work on the activation scheduler, but it does + not create a grain request with interleaving metadata. It is not enough to + make an async reconciliation callback non-interleaving until its returned + task completes. +- **Request scheduling** applies to grain calls, reminder calls, and timer + callbacks. This is the layer that understands `Interleave`, `[ReadOnly]`, + `[AlwaysInterleave]`, reentrancy, and ordinary non-interleaving grain turns. + +This follows directly from Orleans 10.1.0 source: `GrainTimer` schedules timer +ticks by creating a local Orleans message and setting +`msg.IsAlwaysInterleave = _interleave`, while `WorkItemGroup` / +`IWorkItemScheduler` only enqueue and execute `Task` instances on the +activation scheduler. + +The two technically valid ways to get the diagram above are: + +1. Enqueue reconciliation through a second due-now `GrainTimer` with + `Interleave = false`. This works but is conceptually heavy: the timer is + used as a request-scheduling primitive, not because reconciliation is + time-based. +2. Move pending/acknowledge/reconcile callbacks onto an outbox grain interface + and have the processor call the owning grain through its self-reference. + Those methods are then ordinary Orleans grain calls. `PendingItems` should + not be `[ReadOnly]` if it must wait behind writes; `[ReadOnly]` only + interleaves with other read-only calls, not arbitrary writes. + +Decision: use the second timer internally for the callback/reconciliation +phase while preserving the callback-based public API. + +- The dispatch timer uses `Interleave = true`. +- The reconciliation timer uses `Interleave = false`. +- The reconciliation timer is not a time-based feature; it is a supported + Orleans request-scheduling primitive which lets the processor enqueue a + local non-interleaving activation turn without using Orleans internals. +- This guarantee is bounded by Orleans' normal scheduling rules. If the grain + class is `[Reentrant]`, Orleans allows timer callbacks to interleave + regardless of `GrainTimerCreationOptions.Interleave = false`. The processor + should not try to skip the reconciliation timer for reentrant grains; instead + documentation should state that reentrant grains opt out of the + non-interleaving reconciliation guarantee. +- `InterleaveReconciliationCallbacks` defaults to `false`. Most users should + keep reconciliation non-interleaving because those callbacks usually update + durable outbox state. +- `PostAsync()` remains a direct awaitable drain. It does not use the dispatch + or reconciliation timer, because the caller explicitly chose to wait for + postage. On ordinary non-reentrant grains, that means the caller's grain + turn remains occupied until dispatch and reconciliation complete. ### Grain integration pattern @@ -1377,18 +1454,17 @@ public sealed class OutboxProcessorOptions where TOutbox : notnull /// Timer + reminder period. Orleans reminders fire at most once/minute. public TimeSpan RetryDelay { get; init; } = TimeSpan.FromMinutes(2); - /// Whether processor timer callbacks may interleave with other grain turns. - /// Mirrors GrainTimerCreationOptions.Interleave. - public bool Interleave { get; init; } = false; + /// Whether background posting may allow other grain calls to run while + /// postmen are awaiting asynchronous work. + public bool Interleave { get; init; } = true; - /// Whether an active processor timer keeps the activation alive. - /// Mirrors GrainTimerCreationOptions.KeepAlive. - public bool KeepAlive { get; init; } = false; + /// Whether acknowledgement/failure reconciliation callbacks may interleave + /// when posting runs in the background. + public bool InterleaveReconciliationCallbacks { get; init; } = false; - /// Controls whether postmen run on the Orleans activation scheduler or on - /// the .NET thread pool. - public OutboxPostmanExecutionMode PostmanExecution { get; init; } = - OutboxPostmanExecutionMode.GrainScheduler; + /// Whether background retry work should keep the grain activation alive + /// while pending outbox items remain. + public bool KeepAlive { get; init; } = false; } ``` @@ -1445,12 +1521,12 @@ public sealed partial class OutboxProcessor : IOutboxComponent where TGrain : IGrain; /// Posts pending items. Safe to call from grain's task scheduler. - /// Arms timer/reminder if items remain, unregisters if empty. + /// Schedules retry if items remain; unregisters retry work if empty. public ValueTask PostAsync(CancellationToken cancellationToken = default); - /// Schedules posting through the same timer-backed drain path used for - /// retry. Returns after the run has been scheduled, not after posting - /// completes. Multiple calls coalesce into a single active drain. + /// Schedules posting through the same background drain path used for retry. + /// Returns after the run has been scheduled, not after posting completes. + /// Multiple calls coalesce into a single active drain. public ValueTask PostInBackgroundAsync( CancellationToken cancellationToken = default); @@ -1471,8 +1547,8 @@ Two obligations (both compiler-enforced): 1. Implement `IOutboxGrain`. 2. Call `RegisterOutboxProcessor(...)` in `OnActivateAsync`. -No `ReceiveReminder` override needed (DIM handles it). No manual -timer/reminder lifecycle. No telemetry wiring. +No `ReceiveReminder` override needed (DIM handles it). No manual retry +lifecycle. No telemetry wiring. Escape hatch for grains with their own reminders: @@ -1488,7 +1564,7 @@ public async Task ReceiveReminder(string name, TickStatus status) - **Grain-scoped, not silo-scoped.** Grain already knows its own outbox state. No external scan needed. Reminder ensures cross-activation - recovery. Timer handles fast retry. Pattern Orleans itself uses. + recovery. Activation-local scheduling handles fast retry. - **Callback-based, not DI service.** Grain controls dispatch logic, can pass its own state to the postman. DI service adds indirection without clear benefit. @@ -1509,20 +1585,18 @@ public async Task ReceiveReminder(string name, TickStatus status) `AcknowledgePostedAsync`, preserving the outbox invariant that all durable state changes go through the owning grain's state manager. - **`PostInBackgroundAsync` uses the same path as retry.** It schedules a - timer-backed drain and returns quickly so a grain command can commit an - outbox item and return to its caller without waiting for external - delivery. + background drain and returns quickly so a grain command can commit an outbox + item and return to its caller without waiting for external delivery. Reminder + ticks schedule this background path too, so recovery postage follows the same + interleaving policy. - **No overlapping drains, regardless of `Interleave`.** `Interleave` controls Orleans callback scheduling only. Internal drain gating still ensures one send attempt at a time. -- **`ThreadPool` is opt-in.** Blocking or legacy delivery can run outside - the Orleans activation scheduler, but postmen in that mode must not - touch activation-local grain state. Use `IGrainFactory` or injected - services instead. -- **Do not offload normal async libraries.** Orleans guidance is to await - well-behaved async libraries directly. `ThreadPool` exists for code - that blocks synchronously or otherwise must escape the Orleans - scheduler. +- **No thread-pool execution mode.** With interleavable background postage, + normal async postmen should be awaited on the Orleans activation scheduler. + Blocking or legacy synchronous delivery should be moved out to an injected + service or another grain instead of making the outbox processor schedule it + directly. - **One postman per item.** First-registered-wins, not broadcast. Simpler error semantics, no partial-success ambiguity. - **DIM on `IOutboxGrain`.** Zero ceremony. Grain author never writes @@ -1605,6 +1679,14 @@ Documented as a known limitation. | `StreamCursor` | `[JsonConverter(typeof(StreamCursorJsonConverter))]` | | `VersionedState` | No custom converter — `[JsonInclude]` on `Version` property makes `internal set` visible to STJ | +`StreamCursorJsonConverter` and `MessageTrackerJsonConverter` share the +process-wide `StreamSequenceTokenJsonConverters` registry for polymorphic +`StreamSequenceToken` payloads. The registry ships with provider-neutral +Orleans token converters and provider packages register their own +`JsonConverter` instances during setup. This is intentionally explicit: +unknown token types fail loudly instead of being silently downcast to an +ancestor token shape. + **Why custom converters (not `[JsonInclude]` on private fields):** `Outbox` and `MessageTracker` are sealed classes with private From 2ee8d4107dea8a5ebab1ed39caae40989b5469d5 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Tue, 26 May 2026 18:37:28 +0000 Subject: [PATCH 45/82] refactor(om): simplify state manager factory registration Use a non-generic IStateManagerFactory with a generic Create method so keyed registrations are easier to reason about and provider-specific factories do not need open generic registrations. Also simplifies Azure Storage failure code matching and documents the ambiguous clear recovery path. --- .../AzureStorage/AzureStorageStateManager.cs | 74 ++++++++++--------- ...orageStateManagerRegistrationExtensions.cs | 8 +- .../State/DefaultStateManagerFactory.cs | 15 ++++ .../State/IStateManagerFactory.cs | 26 +------ .../State/StateManager.cs | 14 +++- .../State/StateManagerExtensions.cs | 12 +-- .../StateManagerRegistrationExtensions.cs | 56 ++++++-------- .../StateManagerSiloBuilderExtensions.cs | 21 +++--- ...StateManagerRegistrationExtensionsTests.cs | 6 +- ...StateManagerRegistrationExtensionsTests.cs | 61 +++++++-------- 10 files changed, 145 insertions(+), 148 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManagerFactory.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs index 3bb3c9ee..208bc1d8 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManager.cs @@ -1,6 +1,4 @@ using Azure; -using Azure.Data.Tables.Models; -using Azure.Storage.Blobs.Models; using System.Net; using Orleans.Storage; @@ -39,11 +37,11 @@ protected override StorageFailureKind ClassifyClearFailure(Exception exception) /// /// Factory that creates instances. /// -public sealed class AzureStorageStateManagerFactory : IStateManagerFactory - where T : class, IEquatable +public sealed class AzureStorageStateManagerFactory : IStateManagerFactory { /// - public IStateManager Create(IPersistentState storage) + public IStateManager Create(IPersistentState storage) + where T : class, IEquatable { ArgumentNullException.ThrowIfNull(storage); return new AzureStorageStateManager(storage); @@ -107,43 +105,47 @@ private static bool IsAmbiguousOrTransient(RequestFailedException exception) or (int)HttpStatusCode.RequestTimeout or 429 or >= 500 - || IsErrorCode( - exception, - BlobErrorCode.OperationTimedOut.ToString(), - BlobErrorCode.ServerBusy.ToString(), - BlobErrorCode.InternalError.ToString(), - TableErrorCode.OperationTimedOut.ToString(), - "ServerBusy", - "InternalError", - "AccountIOPSLimitExceeded"); + || IsAmbiguousErrorCode(exception.ErrorCode); } private static bool IsRejectedBeforePersistence(RequestFailedException exception) { return exception.Status is >= 400 and < 500 - || IsErrorCode( - exception, - BlobErrorCode.ConditionNotMet.ToString(), - BlobErrorCode.AppendPositionConditionNotMet.ToString(), - BlobErrorCode.MaxBlobSizeConditionNotMet.ToString(), - BlobErrorCode.SequenceNumberConditionNotMet.ToString(), - BlobErrorCode.SourceConditionNotMet.ToString(), - BlobErrorCode.TargetConditionNotMet.ToString(), - BlobErrorCode.BlobAlreadyExists.ToString(), - BlobErrorCode.BlobNotFound.ToString(), - BlobErrorCode.ContainerNotFound.ToString(), - BlobErrorCode.ResourceAlreadyExists.ToString(), - BlobErrorCode.ResourceNotFound.ToString(), - TableErrorCode.UpdateConditionNotSatisfied.ToString(), - TableErrorCode.EntityAlreadyExists.ToString(), - TableErrorCode.EntityNotFound.ToString(), - TableErrorCode.ResourceNotFound.ToString(), - TableErrorCode.TableNotFound.ToString()); + || IsRejectedErrorCode(exception.ErrorCode); } - private static bool IsErrorCode(RequestFailedException exception, params string[] errorCodes) + private static bool IsAmbiguousErrorCode(string? errorCode) { - return !string.IsNullOrWhiteSpace(exception.ErrorCode) - && errorCodes.Contains(exception.ErrorCode, StringComparer.Ordinal); + return errorCode switch + { + "OperationTimedOut" => true, + "ServerBusy" => true, + "InternalError" => true, + "AccountIOPSLimitExceeded" => true, + _ => false + }; } -} \ No newline at end of file + + private static bool IsRejectedErrorCode(string? errorCode) + { + return errorCode switch + { + "ConditionNotMet" => true, + "AppendPositionConditionNotMet" => true, + "MaxBlobSizeConditionNotMet" => true, + "SequenceNumberConditionNotMet" => true, + "SourceConditionNotMet" => true, + "TargetConditionNotMet" => true, + "BlobAlreadyExists" => true, + "BlobNotFound" => true, + "ContainerNotFound" => true, + "ResourceAlreadyExists" => true, + "ResourceNotFound" => true, + "UpdateConditionNotSatisfied" => true, + "EntityAlreadyExists" => true, + "EntityNotFound" => true, + "TableNotFound" => true, + _ => false + }; + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerRegistrationExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerRegistrationExtensions.cs index a5d55d37..6caeac1e 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerRegistrationExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/AzureStorage/AzureStorageStateManagerRegistrationExtensions.cs @@ -12,7 +12,7 @@ public static class AzureStorageStateManagerRegistrationExtensions { /// /// Registers the Azure Storage-aware keyed - /// for the given storage name. + /// for the given storage name. /// public IServiceCollection AddAzureStorageStateManager(string storageName) { @@ -20,9 +20,9 @@ public IServiceCollection AddAzureStorageStateManager(string storageName) ArgumentException.ThrowIfNullOrWhiteSpace(storageName); return services.AddKeyedSingleton( - typeof(IStateManagerFactory<>), + typeof(IStateManagerFactory), storageName, - typeof(AzureStorageStateManagerFactory<>)); + typeof(AzureStorageStateManagerFactory)); } } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManagerFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManagerFactory.cs new file mode 100644 index 00000000..07b75612 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManagerFactory.cs @@ -0,0 +1,15 @@ +namespace Egil.Orleans.Messaging.State; + +/// +/// Default factory that creates instances. +/// +public sealed class DefaultStateManagerFactory : IStateManagerFactory +{ + /// + public IStateManager Create(IPersistentState storage) + where T : class, IEquatable + { + ArgumentNullException.ThrowIfNull(storage); + return new DefaultStateManager(storage); + } +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs index cc88e4aa..ee39b84a 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManagerFactory.cs @@ -4,32 +4,12 @@ namespace Egil.Orleans.Messaging.State; /// Factory contract for constructing provider-specific /// instances from Orleans-managed persistent state. /// -/// -/// The grain state type. -/// -public interface IStateManagerFactory - where T : class, IEquatable +public interface IStateManagerFactory { /// /// Creates an for the given /// facet. /// - IStateManager Create(IPersistentState storage); + IStateManager Create(IPersistentState storage) + where T : class, IEquatable; } - -/// -/// Default factory that creates instances. -/// -/// -/// The grain state type. -/// -public sealed class DefaultStateManagerFactory : IStateManagerFactory - where T : class, IEquatable -{ - /// - public IStateManager Create(IPersistentState storage) - { - ArgumentNullException.ThrowIfNull(storage); - return new DefaultStateManager(storage); - } -} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs index 8e5caa49..0edb7d33 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs @@ -183,6 +183,9 @@ public async Task ClearAsync() var failureKind = ClassifyClearFailure(ex); if (failureKind is StorageFailureKind.DidNotPersist) { + // The provider proved the clear was rejected before it could + // persist, so the in-memory view must stay at the pre-clear + // state and the original error should be surfaced. state = previousState; storage.State = previousState; throw; @@ -190,6 +193,9 @@ public async Task ClearAsync() try { + // Ambiguous failures may have persisted even though the caller + // observed an exception. Read back before deciding whether the + // local state should represent "cleared" or the previous value. await storage.ReadStateAsync(); recoveryReadSucceeded = true; @@ -203,11 +209,17 @@ public async Task ClearAsync() } catch { - // no-op; rethrow original clear error below + // Preserve the original clear exception. A recovery-read + // failure tells us nothing useful about whether the clear + // landed, so surfacing the original write-path error is the + // least surprising failure for the caller. } if (!recoveryReadSucceeded) { + // Without read-back confirmation, keep the local activation + // conservative. The next successful read/write can reconcile + // against storage. state = previousState; storage.State = previousState; } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs index 408cc41c..9197614d 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerExtensions.cs @@ -14,7 +14,7 @@ public static class StateManagerExtensions { /// /// Creates an for the given grain using - /// a keyed registration. + /// a keyed registration. /// /// /// @@ -32,7 +32,7 @@ public static class StateManagerExtensions /// /// /// Registration: A keyed - /// must be registered for + /// must be registered for /// via the provided registration helpers. /// Missing registrations fail fast with a descriptive /// . @@ -79,15 +79,15 @@ internal static IStateManager RegisterStateManagerCore( ArgumentNullException.ThrowIfNull(storage); ArgumentNullException.ThrowIfNull(grainType); - var factory = activationServices.GetKeyedService>(storageName); + var factory = activationServices.GetKeyedService(storageName); if (factory is null) { throw new InvalidOperationException( - $"No keyed IStateManagerFactory<{typeof(TState).Name}> registration was found for storage name '{storageName}', " + + $"No keyed IStateManagerFactory registration was found for storage name '{storageName}', " + $"state type '{typeof(TState).FullName}', grain type '{grainType.FullName}'. " + "Register one via AddDefaultStateManager(...) or AddStateManagerFactory(...)."); } - return factory.Create(storage); + return factory.Create(storage); } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs index a8080957..962a26df 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerRegistrationExtensions.cs @@ -3,7 +3,7 @@ namespace Microsoft.Extensions.DependencyInjection; /// -/// Registration helpers for keyed +/// Registration helpers for keyed /// services on . /// public static class StateManagerRegistrationExtensions @@ -11,7 +11,7 @@ public static class StateManagerRegistrationExtensions extension(IServiceCollection services) { /// - /// Registers the default keyed for the + /// Registers the default keyed for the /// given . /// public IServiceCollection AddDefaultStateManager(string storageName) @@ -20,40 +20,39 @@ public IServiceCollection AddDefaultStateManager(string storageName) ValidateStorageName(storageName); return services.AddKeyedSingleton( - typeof(IStateManagerFactory<>), + typeof(IStateManagerFactory), storageName, - typeof(DefaultStateManagerFactory<>)); + typeof(DefaultStateManagerFactory)); } /// - /// Registers a custom keyed open-generic + /// Registers a custom keyed state manager factory /// implementation for the given . /// public IServiceCollection AddStateManagerFactory( string storageName, - Type openGenericFactoryType) + Type factoryType) { ArgumentNullException.ThrowIfNull(services); ValidateStorageName(storageName); - ValidateOpenGenericFactoryType(openGenericFactoryType); + ValidateFactoryType(factoryType); return services.AddKeyedSingleton( - typeof(IStateManagerFactory<>), + typeof(IStateManagerFactory), storageName, - openGenericFactoryType); + factoryType); } /// - /// Registers a custom keyed factory for a specific state type. + /// Registers a custom keyed state manager factory. /// - public IServiceCollection AddStateManagerFactory(string storageName) - where TState : class, IEquatable - where TFactory : class, IStateManagerFactory + public IServiceCollection AddStateManagerFactory(string storageName) + where TFactory : class, IStateManagerFactory { ArgumentNullException.ThrowIfNull(services); ValidateStorageName(storageName); - return services.AddKeyedSingleton, TFactory>(storageName); + return services.AddKeyedSingleton(storageName); } } @@ -62,33 +61,22 @@ internal static void ValidateStorageName(string storageName) ArgumentException.ThrowIfNullOrWhiteSpace(storageName); } - internal static void ValidateOpenGenericFactoryType(Type openGenericFactoryType) + internal static void ValidateFactoryType(Type factoryType) { - ArgumentNullException.ThrowIfNull(openGenericFactoryType); + ArgumentNullException.ThrowIfNull(factoryType); - if (!openGenericFactoryType.IsClass || openGenericFactoryType.IsAbstract) + if (!factoryType.IsClass || factoryType.IsAbstract) { throw new ArgumentException( - $"Type '{openGenericFactoryType.FullName}' must be a non-abstract class.", - nameof(openGenericFactoryType)); + $"Type '{factoryType.FullName}' must be a non-abstract class.", + nameof(factoryType)); } - if (!openGenericFactoryType.IsGenericTypeDefinition) + if (!typeof(IStateManagerFactory).IsAssignableFrom(factoryType)) { throw new ArgumentException( - $"Type '{openGenericFactoryType.FullName}' must be an open generic type definition.", - nameof(openGenericFactoryType)); - } - - var implementsFactory = openGenericFactoryType.GetInterfaces().Any( - iface => iface.IsGenericType - && iface.GetGenericTypeDefinition() == typeof(IStateManagerFactory<>)); - - if (!implementsFactory) - { - throw new ArgumentException( - $"Type '{openGenericFactoryType.FullName}' must implement IStateManagerFactory<>.", - nameof(openGenericFactoryType)); + $"Type '{factoryType.FullName}' must implement IStateManagerFactory.", + nameof(factoryType)); } } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs index 14714de1..f06def63 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManagerSiloBuilderExtensions.cs @@ -4,7 +4,7 @@ namespace Orleans.Hosting; /// -/// Registration helpers for keyed +/// Registration helpers for keyed /// services on . /// public static class StateManagerSiloBuilderExtensions @@ -24,34 +24,33 @@ public ISiloBuilder AddDefaultStateManager(string storageName) } /// - /// Registers a custom keyed open-generic state manager factory on the silo builder. + /// Registers a custom keyed state manager factory on the silo builder. /// public ISiloBuilder AddStateManagerFactory( string storageName, - Type openGenericFactoryType) + Type factoryType) { ArgumentNullException.ThrowIfNull(builder); StateManagerRegistrationExtensions.ValidateStorageName(storageName); - StateManagerRegistrationExtensions.ValidateOpenGenericFactoryType(openGenericFactoryType); + StateManagerRegistrationExtensions.ValidateFactoryType(factoryType); builder.ConfigureServices(services => - services.AddStateManagerFactory(storageName, openGenericFactoryType)); + services.AddStateManagerFactory(storageName, factoryType)); return builder; } /// - /// Registers a custom keyed state manager factory for a specific state type on the silo builder. + /// Registers a custom keyed state manager factory on the silo builder. /// - public ISiloBuilder AddStateManagerFactory(string storageName) - where TState : class, IEquatable - where TFactory : class, IStateManagerFactory + public ISiloBuilder AddStateManagerFactory(string storageName) + where TFactory : class, IStateManagerFactory { ArgumentNullException.ThrowIfNull(builder); StateManagerRegistrationExtensions.ValidateStorageName(storageName); builder.ConfigureServices(services => - services.AddStateManagerFactory(storageName)); + services.AddStateManagerFactory(storageName)); return builder; } } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerRegistrationExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerRegistrationExtensionsTests.cs index 8478a0f6..a0cb2a61 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerRegistrationExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/AzureStorage/AzureStorageStateManagerRegistrationExtensionsTests.cs @@ -10,9 +10,9 @@ public void AddAzureStorageStateManager_registers_keyed_factory() services.AddAzureStorageStateManager("state"); using var provider = services.BuildServiceProvider(); - var factory = provider.GetRequiredKeyedService>("state"); + var factory = provider.GetRequiredKeyedService("state"); - Assert.IsType>(factory); + Assert.IsType(factory); } [Fact] @@ -27,4 +27,4 @@ public void AddAzureStorageStateManager_throws_for_empty_storage_name() } private sealed record TestState(string Value); -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs index 19ef800a..01270470 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerRegistrationExtensionsTests.cs @@ -10,38 +10,38 @@ public void AddDefaultStateManager_registers_keyed_default_factory_as_singleton( services.AddDefaultStateManager("state"); var descriptor = services.Single(service => - service.ServiceType == typeof(IStateManagerFactory<>) + service.ServiceType == typeof(IStateManagerFactory) && Equals(service.ServiceKey, "state")); Assert.Equal(ServiceLifetime.Singleton, descriptor.Lifetime); var provider = services.BuildServiceProvider(); - var first = provider.GetKeyedService>("state"); - var second = provider.GetKeyedService>("state"); + var first = provider.GetKeyedService("state"); + var second = provider.GetKeyedService("state"); Assert.NotNull(first); Assert.Same(first, second); - Assert.IsType>(first); + Assert.IsType(first); } [Fact] - public void AddStateManagerFactory_registers_custom_open_generic_factory_as_singleton() + public void AddStateManagerFactory_registers_custom_factory_as_singleton() { var services = new ServiceCollection(); - services.AddStateManagerFactory("state", typeof(CustomStateManagerFactory<>)); + services.AddStateManagerFactory("state", typeof(CustomStateManagerFactory)); var descriptor = services.Single(service => - service.ServiceType == typeof(IStateManagerFactory<>) + service.ServiceType == typeof(IStateManagerFactory) && Equals(service.ServiceKey, "state")); Assert.Equal(ServiceLifetime.Singleton, descriptor.Lifetime); var provider = services.BuildServiceProvider(); - var first = provider.GetKeyedService>("state"); - var second = provider.GetKeyedService>("state"); + var first = provider.GetKeyedService("state"); + var second = provider.GetKeyedService("state"); Assert.NotNull(first); Assert.Same(first, second); - Assert.IsType>(first); + Assert.IsType(first); } [Fact] @@ -49,16 +49,16 @@ public void AddStateManagerFactory_registers_custom_typed_factory_as_singleton() { var services = new ServiceCollection(); - services.AddStateManagerFactory("state"); + services.AddStateManagerFactory("state"); var descriptor = services.Single(service => - service.ServiceType == typeof(IStateManagerFactory) + service.ServiceType == typeof(IStateManagerFactory) && Equals(service.ServiceKey, "state")); Assert.Equal(ServiceLifetime.Singleton, descriptor.Lifetime); var provider = services.BuildServiceProvider(); - var first = provider.GetKeyedService>("state"); - var second = provider.GetKeyedService>("state"); + var first = provider.GetKeyedService("state"); + var second = provider.GetKeyedService("state"); Assert.NotNull(first); Assert.Same(first, second); @@ -66,14 +66,14 @@ public void AddStateManagerFactory_registers_custom_typed_factory_as_singleton() } [Fact] - public void AddStateManagerFactory_throws_for_invalid_open_generic_type() + public void AddStateManagerFactory_throws_for_invalid_factory_type() { var services = new ServiceCollection(); var ex = Assert.Throws(() => services.AddStateManagerFactory("state", typeof(string))); - Assert.Equal("openGenericFactoryType", ex.ParamName); + Assert.Equal("factoryType", ex.ParamName); } [Fact] @@ -94,31 +94,32 @@ public void ISiloBuilder_extensions_delegate_to_service_collection_registration( builder .AddDefaultStateManager("default-state") - .AddStateManagerFactory("custom-state", typeof(CustomStateManagerFactory<>)) - .AddStateManagerFactory("typed-state"); + .AddStateManagerFactory("custom-state", typeof(CustomStateManagerFactory)) + .AddStateManagerFactory("typed-state"); var provider = builder.Services.BuildServiceProvider(); - Assert.IsType>( - provider.GetKeyedService>("default-state")); - Assert.IsType>( - provider.GetKeyedService>("custom-state")); + Assert.IsType( + provider.GetKeyedService("default-state")); + Assert.IsType( + provider.GetKeyedService("custom-state")); Assert.IsType( - provider.GetKeyedService>("typed-state")); + provider.GetKeyedService("typed-state")); } private sealed record TestState(string Value) : IEquatable; - private sealed class CustomStateManagerFactory : IStateManagerFactory - where TState : class, IEquatable + private sealed class CustomStateManagerFactory : IStateManagerFactory { - public IStateManager Create(IPersistentState storage) => + public IStateManager Create(IPersistentState storage) + where TState : class, IEquatable => new DefaultStateManager(storage); } - private sealed class TestStateManagerFactory : IStateManagerFactory + private sealed class TestStateManagerFactory : IStateManagerFactory { - public IStateManager Create(IPersistentState storage) => - new DefaultStateManager(storage); + public IStateManager Create(IPersistentState storage) + where TState : class, IEquatable => + new DefaultStateManager(storage); } private sealed class FakeSiloBuilder : ISiloBuilder @@ -128,4 +129,4 @@ private sealed class FakeSiloBuilder : ISiloBuilder public Microsoft.Extensions.Configuration.IConfiguration Configuration { get; } = new Microsoft.Extensions.Configuration.ConfigurationManager(); } -} \ No newline at end of file +} From 62ad15e545890d2cbe933665a159cc88366334be Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Tue, 26 May 2026 18:38:17 +0000 Subject: [PATCH 46/82] fix(om): harden outbox review behavior Remove posted outbox tokens by identity so successful items after a failed gap can still be acknowledged. Also makes outbox token/envelope types Orleans-friendly with required init properties and rewrites the STJ converters to delegate through typed converters while preserving the existing JSON shape. --- .../Outboxes/GrainIdJsonConverter.cs | 63 +++++++ .../Egil.Orleans.Messaging/Outboxes/Outbox.cs | 21 ++- .../Outboxes/OutboxJsonConverterFactory.cs | 171 ++++++++++++------ .../Outboxes/OutboxMessageEnvelope.cs | 45 ++++- ...tboxMessageEnvelopeJsonConverterFactory.cs | 92 ++++++++-- .../Outboxes/OutboxProcessorOptions.cs | 15 +- .../Outboxes/OutboxSequenceToken.cs | 82 ++++++--- .../OutboxSequenceTokenJsonConverter.cs | 98 +++++++--- .../Outboxes/OutboxTests.cs | 7 +- 9 files changed, 439 insertions(+), 155 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/GrainIdJsonConverter.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/GrainIdJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/GrainIdJsonConverter.cs new file mode 100644 index 00000000..6e0b8473 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/GrainIdJsonConverter.cs @@ -0,0 +1,63 @@ +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace Egil.Orleans.Messaging.Outboxes; + +internal sealed class GrainIdJsonConverter : JsonConverter +{ + public static GrainIdJsonConverter Instance { get; } = new(); + + public override GrainId Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + if (reader.TokenType is not JsonTokenType.StartObject) + { + throw new JsonException($"Expected grain id object, got '{reader.TokenType}'."); + } + + string? type = null; + string? key = null; + while (reader.Read()) + { + if (reader.TokenType is JsonTokenType.EndObject) + { + return GrainId.Create( + type ?? throw new JsonException("Missing Type."), + key ?? throw new JsonException("Missing Key.")); + } + + if (reader.TokenType is not JsonTokenType.PropertyName) + { + throw new JsonException($"Expected grain id property, got '{reader.TokenType}'."); + } + + var propertyName = reader.GetString(); + if (!reader.Read()) + { + throw new JsonException("Unexpected end of grain id JSON."); + } + + switch (propertyName) + { + case "Type": + type = reader.GetString(); + break; + case "Key": + key = reader.GetString(); + break; + default: + reader.Skip(); + break; + } + } + + throw new JsonException("Unexpected end of grain id JSON."); + } + + public override void Write(Utf8JsonWriter writer, GrainId value, JsonSerializerOptions options) + { + writer.WriteStartObject(); + writer.WriteString("Type", value.Type.ToString()); + writer.WriteString("Key", value.Key.ToString()); + writer.WriteEndObject(); + } +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs index 9abf8792..1a41f912 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs @@ -240,9 +240,9 @@ public Outbox Remove(OutboxSequenceToken token) /// Batch-removes messages identified by . /// /// - /// Removes the longest FIFO prefix whose tokens appear in - /// . Tokens not found at the head of the outbox - /// are silently ignored. + /// Removes all pending messages whose tokens appear in + /// and preserves the original order of messages + /// that remain pending. Tokens not found in the outbox are silently ignored. /// /// The tokens of the messages to remove. /// A new outbox without the specified messages. @@ -254,18 +254,21 @@ public Outbox RemoveRange(IEnumerable tokens) return this; } - var removeCount = 0; - while (removeCount < items.Length && tokenSet.Contains(items[removeCount].Token)) + var remainingBuilder = ImmutableArray.CreateBuilder>(items.Length); + foreach (var item in items) { - removeCount++; + if (!tokenSet.Contains(item.Token)) + { + remainingBuilder.Add(item); + } } - var remaining = items.RemoveRange(0, removeCount); - if (remaining.Length == items.Length) + if (remainingBuilder.Count == items.Length) { return this; } + var remaining = remainingBuilder.ToImmutable(); var next = new Outbox( sender, latestSequenceNumber, @@ -345,4 +348,4 @@ public override int GetHashCode() private long LastSequenceNumber => items.IsDefaultOrEmpty ? 0 : items[^1].Token.SequenceNumber; -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs index 096c6816..75bf6686 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs @@ -34,65 +34,93 @@ public override bool CanConvert(Type typeToConvert) { var messageType = typeToConvert.GetGenericArguments()[0]; var converterType = typeof(OutboxJsonConverter<>).MakeGenericType(messageType); + var envelopeType = typeof(OutboxMessageEnvelope<>).MakeGenericType(messageType); + var envelopeConverter = options.GetConverter(envelopeType); - return (JsonConverter?)Activator.CreateInstance(converterType); + return (JsonConverter?)Activator.CreateInstance(converterType, envelopeConverter); } - private sealed class OutboxJsonConverter : JsonConverter> + private sealed class OutboxJsonConverter( + JsonConverter> envelopeConverter) : JsonConverter> { public override Outbox? Read( ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { - var model = JsonSerializer.Deserialize>(ref reader, options); - if (model is null) + if (reader.TokenType is JsonTokenType.None && !reader.Read()) { - return null; + throw new JsonException("Unexpected end of outbox JSON."); } - if (model.Sender is null) + if (reader.TokenType is JsonTokenType.Null) { - throw new JsonException("Missing Sender."); + return null; } - if (model.Items.IsDefault) + if (reader.TokenType is not JsonTokenType.StartObject) { - throw new JsonException("Missing Items."); + throw new JsonException($"Expected outbox object, got '{reader.TokenType}'."); } - var items = ImmutableArray.CreateBuilder>(model.Items.Length); - foreach (var item in model.Items) + GrainId? sender = null; + long latestSequenceNumber = 0; + var hasLatestSequenceNumber = false; + DateTimeOffset? epoch = null; + ImmutableArray> items = default; + var hasItems = false; + + while (reader.Read()) { - if (item is null) + if (reader.TokenType is JsonTokenType.EndObject) { - throw new JsonException("Missing Item."); + return new Outbox( + sender ?? throw new JsonException("Missing Sender."), + hasLatestSequenceNumber + ? latestSequenceNumber + : throw new JsonException("Missing LatestSequenceNumber."), + hasItems ? items : throw new JsonException("Missing Items."), + epoch); } - if (item.Token is null) + if (reader.TokenType is not JsonTokenType.PropertyName) { - throw new JsonException("Missing Token."); + throw new JsonException($"Expected outbox property, got '{reader.TokenType}'."); } - if (item.Token.Sender is null) + var propertyName = reader.GetString(); + if (!reader.Read()) { - throw new JsonException("Missing Token.Sender."); + throw new JsonException("Unexpected end of outbox JSON."); } - items.Add(new OutboxMessageEnvelope( - new OutboxSequenceToken( - item.Token.SequenceNumber, - GrainId.Create(item.Token.Sender.Type, item.Token.Sender.Key), - item.Token.Timestamp, - item.Token.Epoch), - item.Message)); + switch (propertyName) + { + case nameof(Outbox.Sender): + sender = reader.TokenType is JsonTokenType.Null + ? null + : GrainIdJsonConverter.Instance.Read(ref reader, typeof(GrainId), options); + break; + case nameof(Outbox.LatestSequenceNumber): + latestSequenceNumber = reader.GetInt64(); + hasLatestSequenceNumber = true; + break; + case nameof(Outbox.Epoch): + epoch = reader.TokenType is JsonTokenType.Null + ? null + : reader.GetDateTimeOffset(); + break; + case "Items": + items = ReadItems(ref reader, options); + hasItems = true; + break; + default: + reader.Skip(); + break; + } } - return new Outbox( - GrainId.Create(model.Sender.Type, model.Sender.Key), - model.LatestSequenceNumber, - items.ToImmutable(), - model.Epoch); + throw new JsonException("Unexpected end of outbox JSON."); } public override void Write( @@ -100,40 +128,63 @@ public override void Write( Outbox value, JsonSerializerOptions options) { - var model = new OutboxJsonModel( - ToJsonModel(value.Sender), - value.LatestSequenceNumber, - value.Epoch, - [.. value.Select(static item => new OutboxMessageEnvelopeJsonModel( - new OutboxSequenceTokenJsonModel( - item.Token.SequenceNumber, - ToJsonModel(item.Token.Sender), - item.Token.Timestamp, - item.Token.Epoch), - item.Message))]); - - JsonSerializer.Serialize(writer, model, options); + writer.WriteStartObject(); + writer.WritePropertyName(nameof(Outbox.Sender)); + GrainIdJsonConverter.Instance.Write(writer, value.Sender, options); + writer.WriteNumber(nameof(Outbox.LatestSequenceNumber), value.LatestSequenceNumber); + writer.WritePropertyName(nameof(Outbox.Epoch)); + if (value.Epoch is { } epoch) + { + writer.WriteStringValue(epoch); + } + else + { + writer.WriteNullValue(); + } + + writer.WritePropertyName("Items"); + writer.WriteStartArray(); + foreach (var item in value) + { + envelopeConverter.Write(writer, item, options); + } + + writer.WriteEndArray(); + writer.WriteEndObject(); } - } - private sealed record OutboxJsonModel( - GrainIdJsonModel? Sender, - long LatestSequenceNumber, - DateTimeOffset? Epoch, - ImmutableArray?> Items); + private ImmutableArray> ReadItems( + ref Utf8JsonReader reader, + JsonSerializerOptions options) + { + if (reader.TokenType is JsonTokenType.Null) + { + throw new JsonException("Missing Items."); + } + + if (reader.TokenType is not JsonTokenType.StartArray) + { + throw new JsonException($"Expected outbox items array, got '{reader.TokenType}'."); + } - private sealed record OutboxMessageEnvelopeJsonModel( - OutboxSequenceTokenJsonModel? Token, - T Message); + var items = ImmutableArray.CreateBuilder>(); + while (reader.Read()) + { + if (reader.TokenType is JsonTokenType.EndArray) + { + return items.ToImmutable(); + } - private sealed record OutboxSequenceTokenJsonModel( - long SequenceNumber, - GrainIdJsonModel? Sender, - DateTimeOffset Timestamp, - DateTimeOffset Epoch); + if (reader.TokenType is JsonTokenType.Null) + { + throw new JsonException("Missing Item."); + } - private sealed record GrainIdJsonModel(string Type, string Key); + var item = envelopeConverter.Read(ref reader, typeof(OutboxMessageEnvelope), options); + items.Add(item ?? throw new JsonException("Missing Item.")); + } - private static GrainIdJsonModel ToJsonModel(GrainId grainId) => - new(grainId.Type.ToString()!, grainId.Key.ToString()!); -} \ No newline at end of file + throw new JsonException("Unexpected end of outbox items JSON."); + } + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs index 53e56e9e..5f652429 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs @@ -34,14 +34,43 @@ namespace Egil.Orleans.Messaging.Outboxes; /// ([GenerateSerializer]) and System.Text.Json if the storage provider /// uses STJ. /// -/// -/// The sequence token identifying this message. Assigned by -/// — never user-constructed. -/// -/// The user-defined message payload. [GenerateSerializer] [Alias("egil.orleans.messaging.OutboxMessageEnvelope`1")] [JsonConverter(typeof(OutboxMessageEnvelopeJsonConverterFactory))] -public sealed record OutboxMessageEnvelope( - [property: Id(0)] OutboxSequenceToken Token, - [property: Id(1)] T Message); \ No newline at end of file +public sealed record OutboxMessageEnvelope +{ + /// + /// Creates an empty envelope for serializer use. + /// + [SetsRequiredMembers] + public OutboxMessageEnvelope() + { + Token = new OutboxSequenceToken(); + Message = default!; + } + + /// + /// Creates an envelope for the given token and message payload. + /// + [SetsRequiredMembers] + public OutboxMessageEnvelope(OutboxSequenceToken token, T message) + { + ArgumentNullException.ThrowIfNull(token); + + Token = token; + Message = message; + } + + /// + /// The sequence token identifying this message. Assigned by + /// — never user-constructed. + /// + [Id(0)] + public required OutboxSequenceToken Token { get; init; } + + /// + /// The user-defined message payload. + /// + [Id(1)] + public required T Message { get; init; } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs index c28f0946..3907ab76 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelopeJsonConverterFactory.cs @@ -28,29 +28,83 @@ public override bool CanConvert(Type typeToConvert) { var messageType = typeToConvert.GetGenericArguments()[0]; var converterType = typeof(OutboxMessageEnvelopeJsonConverter<>).MakeGenericType(messageType); + var messageConverter = options.GetConverter(messageType); - return (JsonConverter?)Activator.CreateInstance(converterType); + return (JsonConverter?)Activator.CreateInstance(converterType, messageConverter); } - private sealed class OutboxMessageEnvelopeJsonConverter : JsonConverter> + private sealed class OutboxMessageEnvelopeJsonConverter( + JsonConverter messageConverter) : JsonConverter> { public override OutboxMessageEnvelope? Read( ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { - var model = JsonSerializer.Deserialize>(ref reader, options); - if (model is null) + if (reader.TokenType is JsonTokenType.None && !reader.Read()) + { + throw new JsonException("Unexpected end of outbox message envelope JSON."); + } + + if (reader.TokenType is JsonTokenType.Null) { return null; } - if (model.Token is null) + if (reader.TokenType is not JsonTokenType.StartObject) + { + throw new JsonException($"Expected outbox message envelope object, got '{reader.TokenType}'."); + } + + OutboxSequenceToken? token = null; + T? message = default; + var hasMessage = false; + while (reader.Read()) { - throw new JsonException("Missing Token."); + if (reader.TokenType is JsonTokenType.EndObject) + { + return new OutboxMessageEnvelope( + token ?? throw new JsonException("Missing Token."), + hasMessage ? message! : throw new JsonException("Missing Message.")); + } + + if (reader.TokenType is not JsonTokenType.PropertyName) + { + throw new JsonException($"Expected outbox message envelope property, got '{reader.TokenType}'."); + } + + var propertyName = reader.GetString(); + if (!reader.Read()) + { + throw new JsonException("Unexpected end of outbox message envelope JSON."); + } + + switch (propertyName) + { + case nameof(OutboxMessageEnvelope.Token): + try + { + token = JsonSerializer.Deserialize(ref reader, options); + } + catch (JsonException exception) when (exception.Message == "Missing Sender.") + { + throw new JsonException("Missing Token.Sender.", exception); + } + + break; + case nameof(OutboxMessageEnvelope.Message): + message = reader.TokenType is JsonTokenType.Null + ? default + : messageConverter.Read(ref reader, typeof(T), options); + hasMessage = true; + break; + default: + reader.Skip(); + break; + } } - return new OutboxMessageEnvelope(model.Token, model.Message); + throw new JsonException("Unexpected end of outbox message envelope JSON."); } public override void Write( @@ -58,14 +112,20 @@ public override void Write( OutboxMessageEnvelope value, JsonSerializerOptions options) { - JsonSerializer.Serialize( - writer, - new OutboxMessageEnvelopeJsonModel(value.Token, value.Message), - options); + writer.WriteStartObject(); + writer.WritePropertyName(nameof(OutboxMessageEnvelope.Token)); + JsonSerializer.Serialize(writer, value.Token, options); + writer.WritePropertyName(nameof(OutboxMessageEnvelope.Message)); + if (value.Message is null) + { + writer.WriteNullValue(); + } + else + { + messageConverter.Write(writer, value.Message, options); + } + + writer.WriteEndObject(); } } - - private sealed record OutboxMessageEnvelopeJsonModel( - OutboxSequenceToken? Token, - T Message); -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs index e830e186..b593d140 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs @@ -25,18 +25,14 @@ public sealed class OutboxProcessorOptions /// The grain is expected to remove these items from durable outbox state /// and persist the update. /// - public required Func, CancellationToken, ValueTask> - AcknowledgePostedAsync - { get; init; } + public required Func, CancellationToken, ValueTask> AcknowledgePostedAsync { get; init; } /// /// Called with items that failed dispatch, along with the exception and /// the in-memory attempt count. The grain decides whether to leave them /// pending, remove them, or move them to dead-letter state. /// - public Func, - CancellationToken, ValueTask>? ReconcileFailedAsync - { get; init; } + public Func, CancellationToken, ValueTask>? ReconcileFailedAsync { get; init; } /// /// Maximum time per post run. Default: 20 seconds. @@ -57,8 +53,11 @@ public required Func, CancellationToken, ValueTask> /// /// Defaults to so slow delivery does not block /// unrelated calls to the grain. This controls the delivery phase only; - /// acknowledgement and failure callbacks use - /// . + /// and + /// use + /// . Snapshot reads from + /// can also happen after a background delivery + /// pass to decide whether retry work remains. /// public bool Interleave { get; init; } = true; diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs index 788865f6..af4240b2 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs @@ -33,30 +33,64 @@ namespace Egil.Orleans.Messaging.Outboxes; /// correctly through any Orleans storage provider using either serializer. /// /// -/// -/// Monotonically increasing sequence number within a single . -/// Assigned by — callers cannot fabricate or choose -/// sequence numbers. Starts at 1 for each new epoch. -/// -/// -/// The of the grain that owns the outbox. Receivers use -/// this to partition dedup tracking per sender. -/// -/// -/// Wall-clock time when the message was added to the outbox. Informational — -/// not used for ordering or dedup. Stamped by the sender's -/// . -/// -/// -/// Opaque marker that changes only on (full -/// sequence-space reset). Receivers compare epochs to detect resets. A token -/// with a newer epoch supersedes all prior sequence numbers from the same sender. -/// [GenerateSerializer] [Alias("egil.orleans.messaging.OutboxSequenceToken")] [JsonConverter(typeof(OutboxSequenceTokenJsonConverter))] -public sealed record OutboxSequenceToken( - [property: Id(0)] long SequenceNumber, - [property: Id(1)] GrainId Sender, - [property: Id(2)] DateTimeOffset Timestamp, - [property: Id(3)] DateTimeOffset Epoch); \ No newline at end of file +public sealed record OutboxSequenceToken +{ + /// + /// Creates an empty token for serializer use. + /// + [SetsRequiredMembers] + public OutboxSequenceToken() + { + } + + /// + /// Creates a token for the specified sender, sequence number, timestamp, + /// and epoch. + /// + [SetsRequiredMembers] + public OutboxSequenceToken( + long sequenceNumber, + GrainId sender, + DateTimeOffset timestamp, + DateTimeOffset epoch) + { + SequenceNumber = sequenceNumber; + Sender = sender; + Timestamp = timestamp; + Epoch = epoch; + } + + /// + /// Monotonically increasing sequence number within a single . + /// Assigned by — callers cannot fabricate or choose + /// sequence numbers. Starts at 1 for each new epoch. + /// + [Id(0)] + public required long SequenceNumber { get; init; } + + /// + /// The of the grain that owns the outbox. Receivers use + /// this to partition dedup tracking per sender. + /// + [Id(1)] + public required GrainId Sender { get; init; } + + /// + /// Wall-clock time when the message was added to the outbox. Informational — + /// not used for ordering or dedup. Stamped by the sender's + /// . + /// + [Id(2)] + public required DateTimeOffset Timestamp { get; init; } + + /// + /// Opaque marker that changes only on (full + /// sequence-space reset). Receivers compare epochs to detect resets. A token + /// with a newer epoch supersedes all prior sequence numbers from the same sender. + /// + [Id(3)] + public required DateTimeOffset Epoch { get; init; } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs index 7f4c9607..2c4948ea 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceTokenJsonConverter.cs @@ -21,42 +21,88 @@ internal sealed class OutboxSequenceTokenJsonConverter : JsonConverter public override OutboxSequenceToken? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { - var model = JsonSerializer.Deserialize(ref reader, options); - if (model is null) + if (reader.TokenType is JsonTokenType.None && !reader.Read()) + { + throw new JsonException("Unexpected end of outbox sequence token JSON."); + } + + if (reader.TokenType is JsonTokenType.Null) { return null; } - if (model.Sender is null) + if (reader.TokenType is not JsonTokenType.StartObject) + { + throw new JsonException($"Expected outbox sequence token object, got '{reader.TokenType}'."); + } + + long sequenceNumber = 0; + var hasSequenceNumber = false; + GrainId? sender = null; + DateTimeOffset timestamp = default; + var hasTimestamp = false; + DateTimeOffset epoch = default; + var hasEpoch = false; + + while (reader.Read()) { - throw new JsonException("Missing Sender."); + if (reader.TokenType is JsonTokenType.EndObject) + { + return new OutboxSequenceToken( + hasSequenceNumber ? sequenceNumber : throw new JsonException("Missing SequenceNumber."), + sender ?? throw new JsonException("Missing Sender."), + hasTimestamp ? timestamp : throw new JsonException("Missing Timestamp."), + hasEpoch ? epoch : throw new JsonException("Missing Epoch.")); + } + + if (reader.TokenType is not JsonTokenType.PropertyName) + { + throw new JsonException($"Expected outbox sequence token property, got '{reader.TokenType}'."); + } + + var propertyName = reader.GetString(); + if (!reader.Read()) + { + throw new JsonException("Unexpected end of outbox sequence token JSON."); + } + + switch (propertyName) + { + case nameof(OutboxSequenceToken.SequenceNumber): + sequenceNumber = reader.GetInt64(); + hasSequenceNumber = true; + break; + case nameof(OutboxSequenceToken.Sender): + sender = reader.TokenType is JsonTokenType.Null + ? null + : GrainIdJsonConverter.Instance.Read(ref reader, typeof(GrainId), options); + break; + case nameof(OutboxSequenceToken.Timestamp): + timestamp = reader.GetDateTimeOffset(); + hasTimestamp = true; + break; + case nameof(OutboxSequenceToken.Epoch): + epoch = reader.GetDateTimeOffset(); + hasEpoch = true; + break; + default: + reader.Skip(); + break; + } } - return new OutboxSequenceToken( - model.SequenceNumber, - GrainId.Create(model.Sender.Type, model.Sender.Key), - model.Timestamp, - model.Epoch); + throw new JsonException("Unexpected end of outbox sequence token JSON."); } /// public override void Write(Utf8JsonWriter writer, OutboxSequenceToken value, JsonSerializerOptions options) { - JsonSerializer.Serialize( - writer, - new OutboxSequenceTokenJsonModel( - value.SequenceNumber, - new GrainIdJsonModel(value.Sender.Type.ToString()!, value.Sender.Key.ToString()!), - value.Timestamp, - value.Epoch), - options); + writer.WriteStartObject(); + writer.WriteNumber(nameof(OutboxSequenceToken.SequenceNumber), value.SequenceNumber); + writer.WritePropertyName(nameof(OutboxSequenceToken.Sender)); + GrainIdJsonConverter.Instance.Write(writer, value.Sender, options); + writer.WriteString(nameof(OutboxSequenceToken.Timestamp), value.Timestamp); + writer.WriteString(nameof(OutboxSequenceToken.Epoch), value.Epoch); + writer.WriteEndObject(); } - - private sealed record OutboxSequenceTokenJsonModel( - long SequenceNumber, - GrainIdJsonModel? Sender, - DateTimeOffset Timestamp, - DateTimeOffset Epoch); - - private sealed record GrainIdJsonModel(string Type, string Key); -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs index c972d37c..7e6e3bbc 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs @@ -116,7 +116,7 @@ public void RemoveRange_deletes_matching_fifo_prefix_and_ignores_missing_tokens( } [Fact] - public void RemoveRange_stops_at_first_token_gap_to_preserve_fifo_order() + public void RemoveRange_removes_matching_tokens_after_gap_and_preserves_remaining_order() { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); @@ -127,9 +127,8 @@ public void RemoveRange_stops_at_first_token_gap_to_preserve_fifo_order() var next = outbox.RemoveRange([outbox[0].Token, outbox[2].Token]); - Assert.Equal(2, next.Count); + Assert.Single(next); Assert.Equal("second", next[0].Message); - Assert.Equal("third", next[1].Message); } [Fact] @@ -208,4 +207,4 @@ public void Equals_returns_false_when_sequence_metadata_differs() Assert.NotEqual(left, right); } -} \ No newline at end of file +} From e124ea9d3c3ddbc157525bc4c31b4afee7a46527 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Tue, 26 May 2026 18:38:43 +0000 Subject: [PATCH 47/82] fix(om): accept tokenless stream messages Do not persist a stream tracking entry when Orleans delivers a null StreamSequenceToken. Without a token there is no high-water mark to compare, so tokenless stream messages are accepted without changing the tracker instead of causing later tokenless messages to look like duplicates. --- .../Tracking/MessageTracker.cs | 9 ++++++++- .../Tracking/MessageTrackerTests.cs | 18 ++++++++++-------- 2 files changed, 18 insertions(+), 9 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs index 51a99ffe..6b40ece6 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs @@ -29,6 +29,7 @@ namespace Egil.Orleans.Messaging.Tracking; /// /// ProcessMessage semantics (streams): /// +/// cursor.Token is null → Accept, no tracking update. /// No prior entry → Accept, insert (LastPosition = cursor, Received = now). /// cursor > stored.LastPosition → Accept, update position + received. /// cursor <= stored.LastPosition → Reject (duplicate), no change. @@ -114,6 +115,12 @@ internal MessageTracker( public bool ProcessMessage(StreamCursor cursor, out MessageTracker next) { var now = time.GetUtcNow(); + if (cursor.Token is null) + { + MessagingTelemetry.RecordStreamReceiveLag(cursor, now); + next = this; + return true; + } var source = StreamSource.From(cursor); if (!streams.TryGetValue(source, out var entry)) @@ -556,4 +563,4 @@ internal readonly record struct OutboxEntry( [property: Id(0)] DateTimeOffset Epoch, [property: Id(1)] long LastSequenceNumber, [property: Id(2)] DateTimeOffset Received); -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs index ee9f75c1..df3d22d2 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs @@ -186,17 +186,18 @@ public void LatestStreamSequenceToken_returns_null_when_namespace_has_multiple_p } [Fact] - public void LatestStreamSequenceToken_returns_null_for_tracked_null_token() + public void ProcessMessage_accepts_null_stream_token_without_tracking_position() { var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); var tracker = new MessageTracker(); tracker.RegisterTimeProvider(new ManualTimeProvider(now)); - tracker.ProcessMessage("orders", token: null, out tracker); - var latest = tracker.LatestStreamSequenceToken("orders"); + var accepted = tracker.ProcessMessage("orders", token: null, out var next); - Assert.Null(latest); - Assert.NotNull(tracker.LatestStream("orders")); + Assert.True(accepted); + Assert.Same(tracker, next); + Assert.Null(next.LatestStreamSequenceToken("orders")); + Assert.Null(next.LatestStream("orders")); } [Fact] @@ -216,7 +217,7 @@ public void ProcessMessage_accepts_non_null_stream_token_after_null_initial_posi } [Fact] - public void ProcessMessage_rejects_null_stream_token_after_position_is_tracked() + public void ProcessMessage_accepts_null_stream_token_without_changing_tracked_position() { var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); var time = new ManualTimeProvider(now); @@ -227,8 +228,9 @@ public void ProcessMessage_rejects_null_stream_token_after_position_is_tracked() var accepted = tracker.ProcessMessage(new StreamCursor("orders", null), out var next); - Assert.False(accepted); + Assert.True(accepted); Assert.Same(tracker, next); + Assert.Equal(new StreamCursor("orders", new EventSequenceToken(1)), next.LatestStream(streamId)); } [Fact] @@ -436,4 +438,4 @@ private static MessageTracker CreateTracker(OutboxSequenceToken outbox, DateTime return tracker; } -} \ No newline at end of file +} From 8d120fb83fe34a7c060546783425afc58ee94f50 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sat, 30 May 2026 09:54:54 +0000 Subject: [PATCH 48/82] feat(om): coordinate outbox postman processing Refactor outbox processing into dedicated dispatch, postman registry, and reconciliation collaborators. Coordinate manual and timer-backed drains through a single active drain so callers wait for the current pass before processing the next pending snapshot. Preserve per-postman ordering while allowing different postmen to dispatch concurrently. Remove the prototype OutboxProcessor2 implementation from the compiled package. --- .../Outboxes/OutboxDispatchResult.cs | 6 + .../Outboxes/OutboxDispatcher.cs | 151 +++++++ .../Outboxes/OutboxMessageEnvelope.cs | 7 + .../Outboxes/OutboxPostmanRegistry.cs | 39 ++ .../Outboxes/OutboxProcessor.cs | 394 ++++++++---------- .../Outboxes/OutboxReconciler.cs | 63 +++ .../Outboxes/OutboxReconciliationBatch.cs | 11 + .../Outboxes/OutboxProcessorCoverageTests.cs | 135 +++++- .../Outboxes/OutboxProcessorTests.cs | 177 +++++++- 9 files changed, 763 insertions(+), 220 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatchResult.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatcher.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanRegistry.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciler.cs create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciliationBatch.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatchResult.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatchResult.cs new file mode 100644 index 00000000..0f2f0825 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatchResult.cs @@ -0,0 +1,6 @@ +namespace Egil.Orleans.Messaging.Outboxes; + +internal readonly record struct OutboxDispatchResult( + TOutbox Item, + Exception? Error = null) + where TOutbox : notnull; \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatcher.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatcher.cs new file mode 100644 index 00000000..3cba711e --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatcher.cs @@ -0,0 +1,151 @@ +using System.Collections.Immutable; +using System.Diagnostics; +using Microsoft.Extensions.Logging; + +namespace Egil.Orleans.Messaging.Outboxes; + +internal sealed class OutboxDispatcher( + OutboxPostmanRegistry postmen, + ILogger logger, + string grainType) + where TOutbox : notnull +{ + public async Task>> DispatchAsync( + ImmutableArray pending, + TimeSpan processingTimeout, + CancellationToken cancellationToken) + { + using var timeout = new CancellationTokenSource(processingTimeout); + using var linked = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, timeout.Token); + var started = Stopwatch.GetTimestamp(); + + try + { + var groups = CreateDispatchGroups(pending, linked.Token); + var dispatchTasks = new Task>>[groups.Count]; + for (var i = 0; i < groups.Count; i++) + { + dispatchTasks[i] = DispatchGroupAsync(groups[i], linked.Token); + } + + var groupResults = await Task.WhenAll(dispatchTasks); + var results = ImmutableArray.CreateBuilder>(pending.Length); + foreach (var groupResult in groupResults) + { + results.AddRange(groupResult); + } + + return results.ToImmutable(); + } + catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested && timeout.IsCancellationRequested) + { + throw new TimeoutException($"Outbox post run exceeded the configured timeout of {processingTimeout}."); + } + finally + { + MessagingTelemetry.RecordOutboxPostDuration( + grainType, + Stopwatch.GetElapsedTime(started).TotalMilliseconds); + } + } + + private List CreateDispatchGroups( + ImmutableArray pending, + CancellationToken cancellationToken) + { + var groups = new List(); + foreach (var item in pending) + { + cancellationToken.ThrowIfCancellationRequested(); + + var postman = postmen.Find(item); + var group = groups.FirstOrDefault(candidate => candidate.Postman == postman); + if (group is null) + { + group = new DispatchGroup(postman); + groups.Add(group); + } + + group.Items.Add(item); + } + + return groups; + } + + private async Task>> DispatchGroupAsync( + DispatchGroup group, + CancellationToken cancellationToken) + { + var results = ImmutableArray.CreateBuilder>(group.Items.Count); + foreach (var item in group.Items) + { + cancellationToken.ThrowIfCancellationRequested(); + + var result = group.Postman is null + ? DispatchMissingPostman(item) + : await DispatchItemAsync(group.Postman, item, cancellationToken); + + results.Add(result); + if (result.Error is not null) + { + break; + } + } + + return results.ToImmutable(); + } + + private OutboxDispatchResult DispatchMissingPostman(TOutbox item) + { + var itemType = item.GetType(); + var error = new NoPostmanRegisteredException(itemType); + logger.LogWarning( + error, + "No outbox postman registered for item type {OutboxItemType} on grain {GrainType}.", + itemType.FullName, + grainType); + MessagingTelemetry.RecordOutboxPostError(grainType, itemType.Name, error); + return new OutboxDispatchResult(item, error); + } + + private async Task> DispatchItemAsync( + OutboxPostmanRegistration postman, + TOutbox item, + CancellationToken cancellationToken) + { + var started = Stopwatch.GetTimestamp(); + try + { + await postman.Invoke(item, cancellationToken); + MessagingTelemetry.RecordOutboxPostItem( + grainType, + postman.ItemType.Name, + success: true, + Stopwatch.GetElapsedTime(started).TotalMilliseconds); + return new OutboxDispatchResult(item); + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + var itemType = item.GetType(); + logger.LogError( + ex, + "Outbox postman failed for item type {OutboxItemType} on grain {GrainType}.", + itemType.FullName, + grainType); + MessagingTelemetry.RecordOutboxPostError(grainType, itemType.Name, ex); + MessagingTelemetry.RecordOutboxPostItem( + grainType, + postman.ItemType.Name, + success: false, + Stopwatch.GetElapsedTime(started).TotalMilliseconds); + return new OutboxDispatchResult(item, ex); + } + } + + private sealed class DispatchGroup(OutboxPostmanRegistration? postman) + { + public OutboxPostmanRegistration? Postman { get; } = postman; + + public List Items { get; } = []; + } +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs index 5f652429..da24baeb 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs @@ -73,4 +73,11 @@ public OutboxMessageEnvelope(OutboxSequenceToken token, T message) /// [Id(1)] public required T Message { get; init; } + + [Id(2)] + public int SendAttempts { get; internal set; } + + [JsonIgnore] + [field: NonSerialized] + public Exception? LastSendError { get; internal set; } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanRegistry.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanRegistry.cs new file mode 100644 index 00000000..371433c3 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxPostmanRegistry.cs @@ -0,0 +1,39 @@ +namespace Egil.Orleans.Messaging.Outboxes; + +internal sealed class OutboxPostmanRegistry + where TOutbox : notnull +{ + private readonly List> postmen = []; + + public void Add(Func postman) + where TSub : TOutbox + { + postmen.Add(new OutboxPostmanRegistration( + item => item is TSub, + (item, cancellationToken) => postman((TSub)item, cancellationToken), + typeof(TSub))); + } + + public OutboxPostmanRegistration? Find(TOutbox item) + { + foreach (var postman in postmen) + { + if (postman.ItemFilter(item)) + { + return postman; + } + } + + return null; + } +} + +internal sealed record OutboxPostmanRegistration( + Func ItemFilter, + Func Postman, + Type ItemType) + where TOutbox : notnull +{ + public ValueTask Invoke(TOutbox item, CancellationToken cancellationToken) + => Postman(item, cancellationToken); +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index 5be76b99..649d8d05 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -1,5 +1,3 @@ -using System.Collections.Immutable; -using System.Diagnostics; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; using Orleans.Streams; @@ -14,24 +12,35 @@ namespace Egil.Orleans.Messaging.Outboxes; /// The base type of items in the outbox. Postmen can handle subtypes via /// . /// +/// +/// Matching remains first-match-wins across registered postmen. During a post +/// run, each postman receives its matching items sequentially in the order +/// returned by . A +/// failure stops that postman's sequence so later matching items remain +/// pending until the owning grain removes or reconciles the failed item. +/// Different postmen are dispatched concurrently. +/// public sealed partial class OutboxProcessor : IOutboxComponent where TOutbox : notnull { private const string ReminderPrefix = "egil.orleans.messaging.outbox."; + private static readonly AsyncLocal?> ActiveDrain = new(); private readonly IGrainBase owner; private readonly IGrainFactory grainFactory; private readonly OutboxProcessorOptions options; - private readonly ILogger logger; - private readonly List postmen = []; - private readonly Dictionary attempts = []; + private readonly object drainGate = new(); + private readonly OutboxPostmanRegistry postmen = new(); + private readonly OutboxDispatcher dispatcher; + private readonly OutboxReconciler reconciler; private readonly string grainType; private readonly string reminderName; - private IGrainTimer? timer; + private IGrainTimer? dispatchTimer; private IGrainTimer? reconciliationTimer; private IGrainReminder? reminder; - private ReconciliationBatch? pendingReconciliation; - private bool drainActive; + private OutboxReconciliationBatch? pendingReconciliation; + private TaskCompletionSource? activeDrain; + private bool backgroundDrainOwnsActiveDrain; private bool drainRequested; internal OutboxProcessor( @@ -58,9 +67,12 @@ internal OutboxProcessor( this.owner = owner; this.grainFactory = grainFactory; this.options = options; - this.logger = logger; grainType = owner.GetType().Name; reminderName = ReminderPrefix + typeof(TOutbox).FullName; + dispatcher = new OutboxDispatcher(postmen, logger, grainType); + reconciler = new OutboxReconciler( + options.AcknowledgePostedAsync, + options.ReconcileFailedAsync); } internal string ReminderName => reminderName; @@ -77,7 +89,8 @@ public OutboxProcessor AddPostman( Func postman) where TSub : TOutbox { ArgumentNullException.ThrowIfNull(postman); - return AddPostmanCore((item, _) => postman(item)); + postmen.Add((item, _) => postman(item)); + return this; } /// @@ -85,7 +98,8 @@ public OutboxProcessor AddPostman( Func postman) where TSub : TOutbox { ArgumentNullException.ThrowIfNull(postman); - return AddPostmanCore((item, _) => new ValueTask(postman(item))); + postmen.Add((item, _) => new ValueTask(postman(item))); + return this; } /// @@ -93,7 +107,8 @@ public OutboxProcessor AddPostman( Func postman) where TSub : TOutbox { ArgumentNullException.ThrowIfNull(postman); - return AddPostmanCore((item, cancellationToken) => new ValueTask(postman(item, cancellationToken))); + postmen.Add((item, cancellationToken) => new ValueTask(postman(item, cancellationToken))); + return this; } /// @@ -101,7 +116,8 @@ public OutboxProcessor AddPostman( Func postman) where TSub : TOutbox { ArgumentNullException.ThrowIfNull(postman); - return AddPostmanCore((item, cancellationToken) => postman(item, grainFactory, cancellationToken)); + postmen.Add((item, cancellationToken) => postman(item, grainFactory, cancellationToken)); + return this; } /// @@ -116,7 +132,8 @@ public OutboxProcessor AddPostman(string postmanName) var postman = owner.GrainContext.ActivationServices .GetRequiredKeyedService>(postmanName); - return AddPostmanCore(postman.PostAsync); + postmen.Add(postman.PostAsync); + return this; } /// @@ -148,11 +165,12 @@ public OutboxProcessor AddStreamPostman( var streamProvider = owner.GrainContext.ActivationServices .GetRequiredKeyedService(streamProviderName); - return AddPostmanCore((message, _) => + postmen.Add((message, _) => { var stream = streamProvider.GetStream(streamId(message)); return new ValueTask(stream.OnNextAsync(project(message))); }); + return this; } /// @@ -167,37 +185,34 @@ public OutboxProcessor AddGrainPostman( ArgumentNullException.ThrowIfNull(resolveGrain); ArgumentNullException.ThrowIfNull(call); - return AddPostmanCore((message, _) => - new ValueTask(call(resolveGrain(message, grainFactory), message))); + postmen.Add((message, _) => new ValueTask(call(resolveGrain(message, grainFactory), message))); + return this; } /// - /// Posts all pending items by dispatching each to its matching postman. - /// Arms timer/reminder if items remain; unregisters retry work if empty. + /// Posts one pending snapshot by dispatching each item to its matching + /// postman. Arms timer/reminder if items remain; unregisters retry work if + /// empty. /// public async ValueTask PostAsync(CancellationToken cancellationToken = default) { - if (drainActive) + if (ReferenceEquals(ActiveDrain.Value, this)) { drainRequested = true; return; } - drainActive = true; + await WaitForTurnAsync(cancellationToken); try { - do - { - drainRequested = false; - var reconciliation = await DispatchOnceAsync(cancellationToken); - await ReconcileAsync(reconciliation, cancellationToken); - } - while (drainRequested); + await DrainOnceAsync(cancellationToken); } finally { - drainActive = false; + CompleteDrain(); } + + await ScheduleRequestedDrainAsync(); } /// @@ -212,7 +227,7 @@ public async ValueTask PostInBackgroundAsync(CancellationToken cancellationToken return; } - EnsureTimer(TimeSpan.Zero); + EnsureDispatchTimer(TimeSpan.Zero); await EnsureReminderAsync(); } @@ -236,42 +251,36 @@ internal void AttachToGrain() owner.GrainContext.SetComponent(this); } - private OutboxProcessor AddPostmanCore( - Func postman) where TSub : TOutbox - { - postmen.Add(new PostmanRegistration( - item => item is TSub, - (item, cancellationToken) => postman((TSub)item, cancellationToken), - typeof(TSub))); - - return this; - } - private async Task DispatchInBackgroundAsync(CancellationToken cancellationToken) { - if (drainActive) - { - drainRequested = true; - return; - } + cancellationToken.ThrowIfCancellationRequested(); + await WaitForTurnAsync(cancellationToken); - drainActive = true; + var drainCompleted = false; try { - drainRequested = false; - var reconciliation = await DispatchOnceAsync(cancellationToken); + var reconciliation = await RunAsActiveDrainAsync( + () => ProcessPendingItemsAsync(cancellationToken)); + if (reconciliation.HasWork) { pendingReconciliation = reconciliation; + backgroundDrainOwnsActiveDrain = true; EnsureReconciliationTimer(TimeSpan.Zero); return; } - drainActive = false; + CompleteDrain(); + drainCompleted = true; + await ScheduleRequestedDrainAsync(); } catch { - drainActive = false; + if (!drainCompleted) + { + CompleteDrain(); + } + throw; } } @@ -283,171 +292,100 @@ private async Task ReconcileInBackgroundAsync(CancellationToken cancellationToke try { - await ReconcileAsync(reconciliation.GetValueOrDefault(), cancellationToken); + await RunAsActiveDrainAsync( + () => ReconcileAsync(reconciliation.GetValueOrDefault(), cancellationToken)); } finally { - drainActive = false; + if (backgroundDrainOwnsActiveDrain) + { + backgroundDrainOwnsActiveDrain = false; + CompleteDrain(); + } } - if (drainRequested) - { - EnsureTimer(TimeSpan.Zero); - } + await ScheduleRequestedDrainAsync(); } - private async Task DispatchOnceAsync(CancellationToken cancellationToken) + private async Task WaitForTurnAsync(CancellationToken cancellationToken) { - cancellationToken.ThrowIfCancellationRequested(); - var pending = options.PendingItems(); - MessagingTelemetry.RecordOutboxDepth(grainType, pending.IsDefault ? 0 : pending.Length); - - if (pending.IsDefaultOrEmpty) - { - await DisableRetryAsync(); - return default; - } - - using var timeout = new CancellationTokenSource(options.ProcessingTimeout); - using var linked = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, timeout.Token); - var started = Stopwatch.GetTimestamp(); - var posted = ImmutableArray.CreateBuilder(); - var failed = ImmutableArray.CreateBuilder<(TOutbox Item, Exception Error, int Attempt)>(); - - try + while (true) { - var dispatchTasks = new Task[pending.Length]; - for (var i = 0; i < pending.Length; i++) - { - linked.Token.ThrowIfCancellationRequested(); - dispatchTasks[i] = DispatchItemAsync(pending[i], linked.Token); - } - - var dispatchResults = await Task.WhenAll(dispatchTasks); - foreach (var result in dispatchResults) + var currentDrain = TryBeginDrainOrGetCurrent(); + if (currentDrain is null) { - if (result.Error is null) - { - posted.Add(result.Item); - } - else - { - failed.Add((result.Item, result.Error, IncrementAttempt(result.Item))); - } + return; } - return new ReconciliationBatch(posted.ToImmutable(), failed.ToImmutable()); - } - catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested && timeout.IsCancellationRequested) - { - throw new TimeoutException($"Outbox post run exceeded the configured timeout of {options.ProcessingTimeout}."); - } - finally - { - MessagingTelemetry.RecordOutboxPostDuration( - grainType, - Stopwatch.GetElapsedTime(started).TotalMilliseconds); + await currentDrain.WaitAsync(cancellationToken); } } - private async Task ReconcileAsync( - ReconciliationBatch reconciliation, - CancellationToken cancellationToken) + private async Task DrainOnceAsync(CancellationToken cancellationToken) { - if (!reconciliation.HasWork) + await RunAsActiveDrainAsync(async () => { - return; - } + drainRequested = false; + var reconciliation = await ProcessPendingItemsAsync(cancellationToken); + await ReconcileAsync(reconciliation, cancellationToken); + }); + } - if (!reconciliation.Posted.IsDefaultOrEmpty) + private async Task RunAsActiveDrainAsync(Func action) + { + var previous = ActiveDrain.Value; + ActiveDrain.Value = this; + try { - await options.AcknowledgePostedAsync(reconciliation.Posted, cancellationToken); - foreach (var item in reconciliation.Posted) - { - attempts.Remove(item); - } + await action(); } - - if (!reconciliation.Failed.IsDefaultOrEmpty && options.ReconcileFailedAsync is { } reconcileFailed) + finally { - await reconcileFailed(reconciliation.Failed, cancellationToken); + ActiveDrain.Value = previous; } - - await ReconcileRetryStateAsync(); } - private async Task DispatchItemAsync( - TOutbox item, - CancellationToken cancellationToken) + private async Task RunAsActiveDrainAsync(Func> action) { - var started = Stopwatch.GetTimestamp(); - var postman = FindPostman(item); - if (postman is null) - { - var itemType = item.GetType(); - var error = new NoPostmanRegisteredException(itemType); - logger.LogWarning( - error, - "No outbox postman registered for item type {OutboxItemType} on grain {GrainType}.", - itemType.FullName, - grainType); - MessagingTelemetry.RecordOutboxPostError(grainType, itemType.Name, error); - return new DispatchResult(item, error); - } - + var previous = ActiveDrain.Value; + ActiveDrain.Value = this; try { - await InvokePostmanAsync(postman, item, cancellationToken); - MessagingTelemetry.RecordOutboxPostItem( - grainType, - postman.ItemType.Name, - success: true, - Stopwatch.GetElapsedTime(started).TotalMilliseconds); - return new DispatchResult(item); + return await action(); } - catch (Exception ex) when (ex is not OperationCanceledException) + finally { - var itemType = item.GetType(); - logger.LogError( - ex, - "Outbox postman failed for item type {OutboxItemType} on grain {GrainType}.", - itemType.FullName, - grainType); - MessagingTelemetry.RecordOutboxPostError(grainType, itemType.Name, ex); - MessagingTelemetry.RecordOutboxPostItem( - grainType, - postman.ItemType.Name, - success: false, - Stopwatch.GetElapsedTime(started).TotalMilliseconds); - return new DispatchResult(item, ex); + ActiveDrain.Value = previous; } } - private ValueTask InvokePostmanAsync( - PostmanRegistration postman, - TOutbox item, - CancellationToken cancellationToken) => - postman.Invoke(item, cancellationToken); - - private PostmanRegistration? FindPostman(TOutbox item) + private async Task> ProcessPendingItemsAsync( + CancellationToken cancellationToken) { - foreach (var postman in postmen) + cancellationToken.ThrowIfCancellationRequested(); + var pending = options.PendingItems(); + MessagingTelemetry.RecordOutboxDepth(grainType, pending.IsDefault ? 0 : pending.Length); + + if (pending.IsDefaultOrEmpty) { - if (postman.ItemFilter(item)) - { - return postman; - } + await DisableRetryAsync(); + return default; } - return null; + var results = await dispatcher.DispatchAsync( + pending, + options.ProcessingTimeout, + cancellationToken); + + return reconciler.CreateBatch(results); } - private int IncrementAttempt(TOutbox item) + private async Task ReconcileAsync( + OutboxReconciliationBatch reconciliation, + CancellationToken cancellationToken) { - attempts.TryGetValue(item, out var current); - var next = current + 1; - attempts[item] = next; - return next; + await reconciler.ReconcileAsync(reconciliation, cancellationToken); + await ReconcileRetryStateAsync(); } private async Task ReconcileRetryStateAsync() @@ -461,36 +399,81 @@ private async Task ReconcileRetryStateAsync() return; } - EnsureTimer(options.RetryDelay); + EnsureDispatchTimer(options.RetryDelay); await EnsureReminderAsync(); } - private void EnsureTimer(TimeSpan dueTime) + private async Task ScheduleRequestedDrainAsync() { - var options = new GrainTimerCreationOptions(dueTime, this.options.RetryDelay) + if (!drainRequested) { - Interleave = this.options.Interleave, - KeepAlive = this.options.KeepAlive + return; + } + + drainRequested = false; + if (options.PendingItems().IsDefaultOrEmpty) + { + await DisableRetryAsync(); + return; + } + + EnsureDispatchTimer(TimeSpan.Zero); + await EnsureReminderAsync(); + } + + private Task? TryBeginDrainOrGetCurrent() + { + lock (drainGate) + { + if (activeDrain is not null) + { + return activeDrain.Task; + } + + activeDrain = new TaskCompletionSource( + TaskCreationOptions.RunContinuationsAsynchronously); + return null; + } + } + + private void CompleteDrain() + { + TaskCompletionSource? completedDrain; + lock (drainGate) + { + completedDrain = activeDrain; + activeDrain = null; + } + + completedDrain?.TrySetResult(); + } + + private void EnsureDispatchTimer(TimeSpan dueTime) + { + var timerOptions = new GrainTimerCreationOptions(dueTime, options.RetryDelay) + { + Interleave = options.Interleave, + KeepAlive = options.KeepAlive }; - if (timer is null) + if (dispatchTimer is null) { - timer = owner.RegisterGrainTimer( + dispatchTimer = owner.RegisterGrainTimer( static (processor, cancellationToken) => processor.DispatchInBackgroundAsync(cancellationToken), this, - options); + timerOptions); return; } - timer.Change(dueTime, this.options.RetryDelay); + dispatchTimer.Change(dueTime, options.RetryDelay); } private void EnsureReconciliationTimer(TimeSpan dueTime) { - var options = new GrainTimerCreationOptions(dueTime, Timeout.InfiniteTimeSpan) + var timerOptions = new GrainTimerCreationOptions(dueTime, Timeout.InfiniteTimeSpan) { - Interleave = this.options.InterleaveReconciliationCallbacks, - KeepAlive = this.options.KeepAlive + Interleave = options.InterleaveReconciliationCallbacks, + KeepAlive = options.KeepAlive }; if (reconciliationTimer is null) @@ -498,7 +481,7 @@ private void EnsureReconciliationTimer(TimeSpan dueTime) reconciliationTimer = owner.RegisterGrainTimer( static (processor, cancellationToken) => processor.ReconcileInBackgroundAsync(cancellationToken), this, - options); + timerOptions); return; } @@ -507,14 +490,17 @@ private void EnsureReconciliationTimer(TimeSpan dueTime) private async Task EnsureReminderAsync() { - var period = Max(options.RetryDelay, TimeSpan.FromMinutes(1)); + var period = options.RetryDelay >= TimeSpan.FromMinutes(1) + ? options.RetryDelay + : TimeSpan.FromMinutes(1); + reminder = await owner.RegisterOrUpdateReminder(reminderName, period, period); } private async Task DisableRetryAsync() { - timer?.Dispose(); - timer = null; + dispatchTimer?.Dispose(); + dispatchTimer = null; reconciliationTimer?.Dispose(); reconciliationTimer = null; @@ -525,26 +511,6 @@ private async Task DisableRetryAsync() reminder = null; } } - - private static TimeSpan Max(TimeSpan left, TimeSpan right) => left >= right ? left : right; - - private readonly record struct ReconciliationBatch( - ImmutableArray Posted, - ImmutableArray<(TOutbox Item, Exception Error, int Attempt)> Failed) - { - public bool HasWork => !Posted.IsDefaultOrEmpty || !Failed.IsDefaultOrEmpty; - } - - private sealed record PostmanRegistration( - Func ItemFilter, - Func Postman, - Type ItemType) - { - public ValueTask Invoke(TOutbox item, CancellationToken cancellationToken) => - Postman(item, cancellationToken); - } - - private readonly record struct DispatchResult(TOutbox Item, Exception? Error = null); } /// @@ -556,4 +522,4 @@ internal interface IOutboxComponent { /// Forwards a reminder callback to the outbox processor. ValueTask ReceiveReminderAsync(string reminderName, TickStatus status); -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciler.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciler.cs new file mode 100644 index 00000000..966c33b6 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciler.cs @@ -0,0 +1,63 @@ +using System.Collections.Immutable; + +namespace Egil.Orleans.Messaging.Outboxes; + +internal sealed class OutboxReconciler( + Func, CancellationToken, ValueTask> acknowledgePostedAsync, + Func, CancellationToken, ValueTask>? reconcileFailedAsync) + where TOutbox : notnull +{ + private readonly Dictionary attempts = []; + + public OutboxReconciliationBatch CreateBatch( + ImmutableArray> results) + { + var posted = ImmutableArray.CreateBuilder(); + var failed = ImmutableArray.CreateBuilder<(TOutbox Item, Exception Error, int Attempt)>(); + foreach (var result in results) + { + if (result.Error is null) + { + posted.Add(result.Item); + } + else + { + failed.Add((result.Item, result.Error, IncrementAttempt(result.Item))); + } + } + + return new OutboxReconciliationBatch(posted.ToImmutable(), failed.ToImmutable()); + } + + public async Task ReconcileAsync( + OutboxReconciliationBatch reconciliation, + CancellationToken cancellationToken) + { + if (!reconciliation.HasWork) + { + return; + } + + if (!reconciliation.Posted.IsDefaultOrEmpty) + { + await acknowledgePostedAsync(reconciliation.Posted, cancellationToken); + foreach (var item in reconciliation.Posted) + { + attempts.Remove(item); + } + } + + if (!reconciliation.Failed.IsDefaultOrEmpty && reconcileFailedAsync is not null) + { + await reconcileFailedAsync(reconciliation.Failed, cancellationToken); + } + } + + private int IncrementAttempt(TOutbox item) + { + attempts.TryGetValue(item, out var current); + var next = current + 1; + attempts[item] = next; + return next; + } +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciliationBatch.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciliationBatch.cs new file mode 100644 index 00000000..a62315ff --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciliationBatch.cs @@ -0,0 +1,11 @@ +using System.Collections.Immutable; + +namespace Egil.Orleans.Messaging.Outboxes; + +internal readonly record struct OutboxReconciliationBatch( + ImmutableArray Posted, + ImmutableArray<(TOutbox Item, Exception Error, int Attempt)> Failed) + where TOutbox : notnull +{ + public bool HasWork => !Posted.IsDefaultOrEmpty || !Failed.IsDefaultOrEmpty; +} \ No newline at end of file diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs index 7d32e30b..a1d5b67a 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs @@ -6,14 +6,55 @@ namespace Egil.Orleans.Messaging.Tests.Outboxes; public sealed class OutboxProcessorCoverageTests(MessagingTestClusterFixture fixture) : IClassFixture { [Fact] - public async Task PostAsync_coalesces_reentrant_post_request_into_second_drain() + public async Task PostAsync_schedules_reentrant_post_request_after_current_drain() { var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); var state = await grain.PublishWithReentrantPostAsync("first", "second"); - Assert.Equal(2, state.AcknowledgedCount); - Assert.Equal(0, state.Outbox?.Count); + Assert.Equal(1, state.AcknowledgedCount); + Assert.Equal(1, state.Outbox?.Count); + + await fixture.WaitForAssertionAsync( + grain, + async () => + { + var current = await grain.GetStateAsync(); + Assert.Equal(2, current.AcknowledgedCount); + Assert.Equal(0, current.Outbox?.Count); + }, + ct: TestContext.Current.CancellationToken); + } + + [Fact] + public async Task PostAsync_concurrent_call_waits_then_drains_current_outbox() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + var gate = OutboxProcessorSchedulingGate.For(grainKey); + + try + { + var firstPost = grain.PublishAndBlockPostAsync("first"); + await gate.DispatchStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + + var secondPost = grain.PublishAndPostAsync("second"); + var completed = await Task.WhenAny( + secondPost, + Task.Delay(TimeSpan.FromMilliseconds(150), TestContext.Current.CancellationToken)); + Assert.NotSame(secondPost, completed); + + gate.AllowDispatch.SetResult(); + await firstPost.WaitAsync(TestContext.Current.CancellationToken); + + var state = await secondPost.WaitAsync(TestContext.Current.CancellationToken); + Assert.Equal(2, state.AcknowledgedCount); + Assert.Equal(0, state.Outbox?.Count); + } + finally + { + OutboxProcessorSchedulingGate.Remove(grainKey); + } } [Fact] @@ -237,6 +278,15 @@ await fixture.WaitForAssertionAsync( public interface IOutboxProcessorReentrantPostGrain : IGrainWithGuidKey { Task PublishWithReentrantPostAsync(string first, string second); + + Task GetStateAsync(); +} + +public interface IOutboxProcessorConcurrentManualPostGrain : IGrainWithGuidKey +{ + Task PublishAndBlockPostAsync(string value); + + Task PublishAndPostAsync(string value); } public interface IOutboxProcessorRetryPendingGrain : IGrainWithGuidKey @@ -346,6 +396,8 @@ public async Task PublishWithReentrantPostAsync(stri return state.State; } + public Task GetStateAsync() => Task.FromResult(state.State); + private async Task PostAndRequestAnotherDrainAsync( OutboxMessageEnvelope envelope, CancellationToken cancellationToken) @@ -393,6 +445,81 @@ private Outbox EnsureOutbox() => state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); } +[global::Orleans.Concurrency.Reentrant] +public sealed class OutboxProcessorConcurrentManualPostGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorConcurrentManualPostGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>(PostWithGateAsync); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishAndBlockPostAsync(string value) + { + state.State.Outbox = EnsureOutbox().Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostAsync(); + return state.State; + } + + public async Task PublishAndPostAsync(string value) + { + state.State.Outbox = EnsureOutbox().Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostAsync(); + return state.State; + } + + private async Task PostWithGateAsync( + OutboxMessageEnvelope envelope, + CancellationToken cancellationToken) + { + if (envelope.Message.Value != "first") + { + return; + } + + var gate = OutboxProcessorSchedulingGate.For(this.GetPrimaryKey()); + gate.DispatchStarted.SetResult(); + await gate.AllowDispatch.Task.WaitAsync(cancellationToken); + } + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } + + private ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) => + ValueTask.CompletedTask; + + private Outbox EnsureOutbox() => + state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); +} + public sealed class OutboxProcessorRetryPendingGrain( [PersistentState("state", "Default")] IPersistentState state) : Grain, IOutboxProcessorRetryPendingGrain, IOutboxGrain @@ -709,4 +836,4 @@ private ValueTask ReconcileFailedAsync( private Outbox EnsureOutbox() => state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index 69ca0ac2..f2ffd362 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -142,7 +142,7 @@ public async Task PostAsync_timeout_from_keyed_postman_leaves_item_pending() } [Fact] - public async Task PostAsync_dispatches_pending_items_concurrently() + public async Task PostAsync_dispatches_items_for_same_postman_sequentially() { var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); @@ -152,9 +152,35 @@ public async Task PostAsync_dispatches_pending_items_concurrently() Assert.Equal(2, state.AcknowledgedCount); Assert.Equal(0, state.FailedCount); Assert.Equal(0, state.Outbox?.Count); + Assert.Equal(1, state.MaxConcurrentPostmen); + } + + [Fact] + public async Task PostAsync_dispatches_different_postmen_concurrently() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var state = await source.PublishTwoPostmenAsync("primary", "secondary"); + + Assert.Equal(["primary", "secondary"], state.PostedValues.Sort()); + Assert.Empty(state.FailedValues); + Assert.Empty(state.PendingValues); Assert.Equal(2, state.MaxConcurrentPostmen); } + [Fact] + public async Task PostAsync_blocks_later_items_for_same_postman_after_failure() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var state = await source.PublishBlockedPostmanBatchAsync("fail-first", "secondary", "blocked-later"); + + Assert.Equal(["secondary"], state.PostedValues); + Assert.Equal(["fail-first"], state.FailedValues); + Assert.Equal(["fail-first", "blocked-later"], state.PendingValues); + Assert.DoesNotContain("blocked-later", state.AttemptedValues); + } + [Fact] public async Task PostInBackgroundAsync_allows_other_calls_while_postman_is_awaiting() { @@ -250,6 +276,16 @@ public interface IOutboxProcessorConcurrentPostmanGrain : IGrainWithGuidKey Task GetStateAsync(); } +public interface IOutboxProcessorOrderedPostmanGrain : IGrainWithGuidKey +{ + Task PublishTwoPostmenAsync(string primary, string secondary); + + Task PublishBlockedPostmanBatchAsync( + string failingPrimary, + string secondary, + string blockedPrimary); +} + public interface IOutboxProcessorSinkGrain : IGrainWithGuidKey { Task EnsureActiveAsync(); @@ -286,9 +322,23 @@ public sealed class OutboxProcessorSinkState public ImmutableArray ReceivedValues { get; set; } = []; } +[GenerateSerializer] +public sealed record OutboxProcessorOrderedPostmanState( + [property: Id(0)] ImmutableArray PostedValues, + [property: Id(1)] ImmutableArray FailedValues, + [property: Id(2)] ImmutableArray PendingValues, + [property: Id(3)] ImmutableArray AttemptedValues, + [property: Id(4)] int MaxConcurrentPostmen); + [GenerateSerializer] public sealed record OutboxProcessorTestEvent([property: Id(0)] string Value); +public abstract record OutboxProcessorOrderedMessage(string Value); + +public sealed record OutboxProcessorPrimaryMessage(string Value) : OutboxProcessorOrderedMessage(Value); + +public sealed record OutboxProcessorSecondaryMessage(string Value) : OutboxProcessorOrderedMessage(Value); + public sealed class OutboxProcessorSourceGrain( [PersistentState("state", "Default")] IPersistentState state) : Grain, IOutboxProcessorSourceGrain, IOutboxGrain @@ -835,6 +885,129 @@ private Outbox EnsureOutbox() } } +public sealed class OutboxProcessorOrderedPostmanGrain + : Grain, IOutboxProcessorOrderedPostmanGrain, IOutboxGrain +{ + private OutboxProcessor? processor; + private ImmutableArray pending = []; + private ImmutableArray postedValues = []; + private ImmutableArray failedValues = []; + private ImmutableArray attemptedValues = []; + private int activePostmen; + private int maxConcurrentPostmen; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions + { + PendingItems = () => pending, + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = ReconcileFailedAsync, + RetryDelay = TimeSpan.FromMinutes(10) + }) + .AddPostman(PostPrimaryAsync) + .AddPostman(PostSecondaryAsync); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishTwoPostmenAsync( + string primary, + string secondary) + { + pending = + [ + new OutboxProcessorPrimaryMessage(primary), + new OutboxProcessorSecondaryMessage(secondary) + ]; + + await processor!.PostAsync(); + return CreateState(); + } + + public async Task PublishBlockedPostmanBatchAsync( + string failingPrimary, + string secondary, + string blockedPrimary) + { + pending = + [ + new OutboxProcessorPrimaryMessage(failingPrimary), + new OutboxProcessorSecondaryMessage(secondary), + new OutboxProcessorPrimaryMessage(blockedPrimary) + ]; + + await processor!.PostAsync(); + return CreateState(); + } + + private Task PostPrimaryAsync( + OutboxProcessorPrimaryMessage message, + CancellationToken cancellationToken) => + PostTrackedAsync(message, cancellationToken); + + private Task PostSecondaryAsync( + OutboxProcessorSecondaryMessage message, + CancellationToken cancellationToken) => + PostTrackedAsync(message, cancellationToken); + + private async Task PostTrackedAsync( + OutboxProcessorOrderedMessage message, + CancellationToken cancellationToken) + { + attemptedValues = attemptedValues.Add(message.Value); + activePostmen++; + maxConcurrentPostmen = Math.Max(maxConcurrentPostmen, activePostmen); + + try + { + if (message.Value.StartsWith("fail-", StringComparison.Ordinal)) + { + throw new InvalidOperationException($"Cannot post {message.Value}."); + } + + await Task.Delay(TimeSpan.FromMilliseconds(200), cancellationToken); + } + finally + { + activePostmen--; + } + } + + private ValueTask AcknowledgePostedAsync( + ImmutableArray items, + CancellationToken cancellationToken) + { + foreach (var item in items) + { + pending = pending.Remove(item); + postedValues = postedValues.Add(item.Value); + } + + return ValueTask.CompletedTask; + } + + private ValueTask ReconcileFailedAsync( + ImmutableArray<(OutboxProcessorOrderedMessage Item, Exception Error, int Attempt)> failures, + CancellationToken cancellationToken) + { + foreach (var failure in failures) + { + failedValues = failedValues.Add(failure.Item.Value); + } + + return ValueTask.CompletedTask; + } + + private OutboxProcessorOrderedPostmanState CreateState() => + new( + postedValues, + failedValues, + pending.Select(static item => item.Value).ToImmutableArray(), + attemptedValues, + maxConcurrentPostmen); +} + public sealed class KeyedOutboxProcessorSuccessPostman : IPostman> { @@ -901,4 +1074,4 @@ private async ValueTask HandleEnvelopeAsync( state.State.ReceivedValues = state.State.ReceivedValues.Add(envelope.Message.Value); await state.WriteStateAsync(); } -} \ No newline at end of file +} From e9194e553bf7e70228554bdf22235ce8c844b917 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 09:47:57 +0000 Subject: [PATCH 49/82] fix(om): arm outbox retry when a foreground post run fails A failed PostAsync run - processing timeout, cancellation, or an exception from an acknowledgement or reconciliation callback - previously skipped the retry scheduling that normally happens after reconciliation. On a first-ever foreground post no timer or reminder existed yet, so announced messages stayed in the durable outbox until the next post or activation. The processor now arms its retry timer and reminder before rethrowing the post-run failure, but only on the failure path, so successful posts still avoid the reminder registration write. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 6 ++ .../Outboxes/OutboxProcessor.cs | 40 +++++++- .../Outboxes/OutboxProcessorTests.cs | 99 +++++++++++++++++++ 3 files changed, 144 insertions(+), 1 deletion(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index 6755c002..8f122e59 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -156,6 +156,12 @@ Failed dispatches are reported through `ReconcileFailedAsync`. That callback is where the owning grain applies retry, dead-letter, max-depth, or trimming policy, because the grain owns the durable outbox state. +If a foreground `PostAsync` run fails before reconciliation completes — for +example when the run exceeds `ProcessingTimeout` or an acknowledgement +callback throws — the processor arms its retry timer and reminder before +rethrowing, so pending items are retried without requiring another explicit +post. Successful runs never pay the reminder registration cost. + Background outbox postage allows unrelated grain calls to continue while postmen await I/O by default. `IPostman` services should be state-free with respect to the owning grain. Inline lambda postmen may read activation-local diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index 649d8d05..e6577c1d 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -33,6 +33,7 @@ public sealed partial class OutboxProcessor : IOutboxComponent private readonly OutboxPostmanRegistry postmen = new(); private readonly OutboxDispatcher dispatcher; private readonly OutboxReconciler reconciler; + private readonly ILogger logger; private readonly string grainType; private readonly string reminderName; private IGrainTimer? dispatchTimer; @@ -67,6 +68,7 @@ internal OutboxProcessor( this.owner = owner; this.grainFactory = grainFactory; this.options = options; + this.logger = logger; grainType = owner.GetType().Name; reminderName = ReminderPrefix + typeof(TOutbox).FullName; dispatcher = new OutboxDispatcher(postmen, logger, grainType); @@ -192,7 +194,11 @@ public OutboxProcessor AddGrainPostman( /// /// Posts one pending snapshot by dispatching each item to its matching /// postman. Arms timer/reminder if items remain; unregisters retry work if - /// empty. + /// empty. If the run itself fails — for example with a + /// when + /// elapses, + /// or when a reconciliation callback throws — retry work is armed before + /// the exception is rethrown so pending items are not stranded. /// public async ValueTask PostAsync(CancellationToken cancellationToken = default) { @@ -207,6 +213,16 @@ public async ValueTask PostAsync(CancellationToken cancellationToken = default) { await DrainOnceAsync(cancellationToken); } + catch + { + // A failed foreground run (timeout, cancellation, or a reconciliation + // callback error) skips ReconcileRetryStateAsync, and unlike the + // background path no timer/reminder was armed beforehand. Arm retry + // here — only on failure — so announced items still get delivered + // without paying the reminder write on every successful post. + await TrySchedulePendingRetryAsync(); + throw; + } finally { CompleteDrain(); @@ -421,6 +437,28 @@ private async Task ScheduleRequestedDrainAsync() await EnsureReminderAsync(); } + private async Task TrySchedulePendingRetryAsync() + { + try + { + if (options.PendingItems().IsDefaultOrEmpty) + { + return; + } + + EnsureDispatchTimer(options.RetryDelay); + await EnsureReminderAsync(); + } + catch (Exception ex) + { + // Swallow so the original post-run failure stays the surfaced error. + logger.LogWarning( + ex, + "Failed to arm outbox retry after a failed post run on grain {GrainType}. Pending items remain until the next post or activation.", + grainType); + } + } + private Task? TryBeginDrainOrGetCurrent() { lock (drainGate) diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index f2ffd362..b8a3a157 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -141,6 +141,26 @@ public async Task PostAsync_timeout_from_keyed_postman_leaves_item_pending() Assert.Equal(1, state.Outbox?.Count); } + [Fact] + public async Task PostAsync_timeout_arms_retry_and_later_run_delivers_pending_item() + { + var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var exceptionType = await source.PublishWithTimeoutAsync("timeout-retry"); + + Assert.Equal(nameof(TimeoutException), exceptionType); + await fixture.WaitForAssertionAsync( + source, + async () => + { + var state = await source.GetStateAsync(); + Assert.Equal(1, state.AcknowledgedCount); + Assert.Equal(0, state.FailedCount); + Assert.Equal(0, state.Outbox?.Count ?? 0); + }, + ct: TestContext.Current.CancellationToken); + } + [Fact] public async Task PostAsync_dispatches_items_for_same_postman_sequentially() { @@ -267,6 +287,13 @@ public interface IOutboxProcessorKeyedTimeoutPostmanGrain : IGrainWithGuidKey Task GetStateAsync(); } +public interface IOutboxProcessorTimeoutRetryGrain : IGrainWithGuidKey +{ + Task PublishWithTimeoutAsync(string value); + + Task GetStateAsync(); +} + public interface IOutboxProcessorConcurrentPostmanGrain : IGrainWithGuidKey { Task PublishInBackgroundAsync(string value); @@ -787,6 +814,78 @@ private Outbox EnsureOutbox() } } +public sealed class OutboxProcessorTimeoutRetryGrain( + [PersistentState("state", "Default")] IPersistentState state) + : Grain, IOutboxProcessorTimeoutRetryGrain, IOutboxGrain +{ + private OutboxProcessor>? processor; + private int attempts; + + public override async Task OnActivateAsync(CancellationToken cancellationToken) + { + EnsureOutbox(); + + processor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], + AcknowledgePostedAsync = AcknowledgePostedAsync, + ReconcileFailedAsync = (_, _) => ValueTask.CompletedTask, + ProcessingTimeout = TimeSpan.FromMilliseconds(250), + RetryDelay = TimeSpan.FromMilliseconds(100) + }) + .AddPostman>((_, ct) => FirstAttemptBlocksAsync(ct)); + + await base.OnActivateAsync(cancellationToken); + } + + public async Task PublishWithTimeoutAsync(string value) + { + var outbox = EnsureOutbox(); + state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + + try + { + await processor!.PostAsync(); + return null; + } + catch (Exception ex) + { + return ex.GetType().Name; + } + } + + public Task GetStateAsync() => Task.FromResult(state.State); + + private async Task FirstAttemptBlocksAsync(CancellationToken cancellationToken) + { + if (++attempts == 1) + { + await Task.Delay(Timeout.InfiniteTimeSpan, cancellationToken); + } + } + + private async ValueTask AcknowledgePostedAsync( + ImmutableArray> items, + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } + + private Outbox EnsureOutbox() + { + return state.State.Outbox ??= Outbox.Create(GrainContext.GrainId); + } +} + public sealed class OutboxProcessorConcurrentPostmanGrain( [PersistentState("state", "Default")] IPersistentState state) : Grain, IOutboxProcessorConcurrentPostmanGrain, IOutboxGrain From 87e644f5531625c100913844e9aa1c0001c60b02 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 09:59:34 +0000 Subject: [PATCH 50/82] fix(om): detect diverged outbox tails during write recovery Outbox equality now compares the full first and last pending tokens, including their timestamps, instead of only their sequence numbers. Items are only appended at the tail, so two outbox histories that diverged by adding different messages - for example duplicate grain activations racing an ambiguous storage write - always differ in their highest pending token. Comparing the full token lets the state-manager recovery read-back distinguish such diverged states instead of mistaking them for a lost response, which could silently drop the local activation's message. Histories that diverged only by removals can still compare equal; recovery then at worst re-delivers an already posted item (at-least-once) but never loses a pending message. Equality remains O(1) and still ignores payloads. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Egil.Orleans.Messaging/Outboxes/Outbox.cs | 37 ++++++++++--------- .../Outboxes/OutboxTests.cs | 25 +++++++++++++ 2 files changed, 45 insertions(+), 17 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs index 1a41f912..1717eeb4 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs @@ -41,8 +41,8 @@ namespace Egil.Orleans.Messaging.Outboxes; /// /// /// Equality: Two instances are equal when -/// sender, sequence metadata, epoch, count, first sequence number, and last -/// sequence number are equal. Equality is O(1). +/// sender, sequence metadata, epoch, count, and the full first and last +/// pending tokens are equal. Equality is O(1) and ignores message payloads. /// /// /// Unbounded growth risk: If postman targets are down, the outbox grows @@ -308,15 +308,18 @@ public Outbox Clear() /// /// O(1) equality over the outbox identity and sequence fingerprint: - /// sender, latest sequence number, epoch, count, first sequence number, - /// and last sequence number. + /// sender, latest sequence number, epoch, count, and the full first and + /// last pending tokens (including their timestamps). /// /// - /// This intentionally does not compare message payloads. The outbox owns - /// sequence assignment, removal is FIFO-only, and sequence numbers are never - /// reused within an epoch. Under those invariants the metadata fingerprint - /// uniquely represents the pending outbox contents for equality/recovery - /// purposes while avoiding an O(n) payload walk during state comparisons. + /// This intentionally does not compare message payloads. Items are only + /// ever appended at the tail, so two histories that diverged by adding + /// different messages — for example duplicate grain activations racing an + /// ambiguous storage write — always differ in their highest pending + /// token, whose timestamp is stamped by the producing activation. + /// Histories that diverged only by removals can still compare equal; + /// the state-manager recovery path then at worst re-delivers an already + /// posted item (at-least-once) but never loses a pending message. /// public bool Equals(Outbox? other) { @@ -326,8 +329,8 @@ public bool Equals(Outbox? other) && latestSequenceNumber == other.latestSequenceNumber && epoch == other.epoch && items.Length == other.items.Length - && FirstSequenceNumber == other.FirstSequenceNumber - && LastSequenceNumber == other.LastSequenceNumber); + && Equals(FirstToken, other.FirstToken) + && Equals(LastToken, other.LastToken)); } /// @@ -340,12 +343,12 @@ public override int GetHashCode() latestSequenceNumber, epoch, items.Length, - FirstSequenceNumber, - LastSequenceNumber); + FirstToken, + LastToken); - private long FirstSequenceNumber => - items.IsDefaultOrEmpty ? 0 : items[0].Token.SequenceNumber; + private OutboxSequenceToken? FirstToken => + items.IsDefaultOrEmpty ? null : items[0].Token; - private long LastSequenceNumber => - items.IsDefaultOrEmpty ? 0 : items[^1].Token.SequenceNumber; + private OutboxSequenceToken? LastToken => + items.IsDefaultOrEmpty ? null : items[^1].Token; } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs index 7e6e3bbc..048f9007 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs @@ -207,4 +207,29 @@ public void Equals_returns_false_when_sequence_metadata_differs() Assert.NotEqual(left, right); } + [Fact] + public void Equals_returns_false_when_diverged_tail_items_have_different_timestamps() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var leftTime = new ManualTimeProvider(now); + var rightTime = new ManualTimeProvider(now); + var left = Outbox.Create(sender); + var right = Outbox.Create(sender); + left.RegisterTimeProvider(leftTime); + right.RegisterTimeProvider(rightTime); + + // Shared history establishes the same epoch and head token, then two + // duplicate activations each append their own message at the same + // sequence number but at different wall-clock instants. + left = left.Add("base"); + right = right.Add("base"); + leftTime.Advance(TimeSpan.FromMilliseconds(1)); + rightTime.Advance(TimeSpan.FromMilliseconds(2)); + left = left.Add("from-activation-a"); + right = right.Add("from-activation-b"); + + Assert.NotEqual(left, right); + } + } From 65317d3be6d59694acf96319c0abf9d2009d044e Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 09:59:35 +0000 Subject: [PATCH 51/82] perf(om): register outbox retry reminder lazily PostInBackgroundAsync and follow-up drain scheduling no longer register the durable reminder up front; they arm only the in-memory grain timer. The reminder - which costs a storage write - is now registered only when a post run fails or completes with items still pending, including when a background dispatch or reconciliation callback throws. Unregistering also caches the one-time lookup for a leftover reminder from a previous activation, so an empty outbox no longer pays a reminder-table read on every successful drain. Successful posts therefore incur no reminder I/O. The trade-off is that a silo crash between scheduling a background post and the first failed run loses the cross-activation retry trigger; pending items then wait for the next activation's post instead of a reminder tick. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 12 ++++--- .../Outboxes/OutboxProcessor.cs | 32 +++++++++++++++++-- 2 files changed, 36 insertions(+), 8 deletions(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index 8f122e59..c8a1c85a 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -156,11 +156,13 @@ Failed dispatches are reported through `ReconcileFailedAsync`. That callback is where the owning grain applies retry, dead-letter, max-depth, or trimming policy, because the grain owns the durable outbox state. -If a foreground `PostAsync` run fails before reconciliation completes — for -example when the run exceeds `ProcessingTimeout` or an acknowledgement -callback throws — the processor arms its retry timer and reminder before -rethrowing, so pending items are retried without requiring another explicit -post. Successful runs never pay the reminder registration cost. +If a post run fails before reconciliation completes — for example when the +run exceeds `ProcessingTimeout` or an acknowledgement callback throws — the +processor arms its retry timer and durable reminder before rethrowing, so +pending items are retried without requiring another explicit post. Successful +posts never pay reminder I/O: `PostInBackgroundAsync` schedules an in-memory +grain timer only, and the durable reminder is registered lazily when a run +fails or leaves items pending. Background outbox postage allows unrelated grain calls to continue while postmen await I/O by default. `IPostman` services should be state-free with diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index e6577c1d..a89baee6 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -39,6 +39,7 @@ public sealed partial class OutboxProcessor : IOutboxComponent private IGrainTimer? dispatchTimer; private IGrainTimer? reconciliationTimer; private IGrainReminder? reminder; + private bool checkedForExistingReminder; private OutboxReconciliationBatch? pendingReconciliation; private TaskCompletionSource? activeDrain; private bool backgroundDrainOwnsActiveDrain; @@ -234,6 +235,12 @@ public async ValueTask PostAsync(CancellationToken cancellationToken = default) /// /// Schedules a timer-backed post run and returns after scheduling. /// + /// + /// Only an in-memory grain timer is armed here. The durable reminder — + /// which costs a storage write — is registered lazily, when a post run + /// fails or leaves items pending, so successful posts incur no reminder + /// I/O. + /// public async ValueTask PostInBackgroundAsync(CancellationToken cancellationToken = default) { cancellationToken.ThrowIfCancellationRequested(); @@ -244,7 +251,6 @@ public async ValueTask PostInBackgroundAsync(CancellationToken cancellationToken } EnsureDispatchTimer(TimeSpan.Zero); - await EnsureReminderAsync(); } /// @@ -294,6 +300,10 @@ private async Task DispatchInBackgroundAsync(CancellationToken cancellationToken { if (!drainCompleted) { + // The reminder is armed lazily, so a run that throws before + // reconciliation must arm retry itself or pending items would + // only survive in this activation's timer. + await TrySchedulePendingRetryAsync(); CompleteDrain(); } @@ -311,6 +321,14 @@ private async Task ReconcileInBackgroundAsync(CancellationToken cancellationToke await RunAsActiveDrainAsync( () => ReconcileAsync(reconciliation.GetValueOrDefault(), cancellationToken)); } + catch + { + // A failed acknowledgement/reconciliation callback skips + // ReconcileRetryStateAsync; arm retry so pending items are not + // stranded if the activation goes away. + await TrySchedulePendingRetryAsync(); + throw; + } finally { if (backgroundDrainOwnsActiveDrain) @@ -434,7 +452,6 @@ private async Task ScheduleRequestedDrainAsync() } EnsureDispatchTimer(TimeSpan.Zero); - await EnsureReminderAsync(); } private async Task TrySchedulePendingRetryAsync() @@ -542,7 +559,16 @@ private async Task DisableRetryAsync() reconciliationTimer?.Dispose(); reconciliationTimer = null; - var activeReminder = reminder ?? await owner.GetReminder(reminderName); + // Look up a leftover reminder from a previous activation at most once; + // afterwards the local field is authoritative, so an empty outbox does + // not pay a reminder-table read on every successful drain. + var activeReminder = reminder; + if (activeReminder is null && !checkedForExistingReminder) + { + activeReminder = await owner.GetReminder(reminderName); + checkedForExistingReminder = true; + } + if (activeReminder is not null) { await owner.UnregisterReminder(activeReminder); From 13f71a447d1ddb4970c03312f1b59714c1daf46a Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 10:05:25 +0000 Subject: [PATCH 52/82] docs(om): document outbox equality recovery contract [skip notes] Expand the Outbox.Equals XML docs to fully explain the state-manager write-recovery contract the fingerprint equality must satisfy: why a false positive would drop pending messages, why the append-at-tail invariant plus full first/last tokens (incl. activation timestamps) keep the O(1) comparison loss-safe, that ETag conflicts are rethrown separately and equality only decides ambiguous-outcome writes, and the accepted at-least-once trade-off for removal-only divergence. Also document the same contract at the consumer side on StateManagerBase.IsEquivalent and note on FirstToken/LastToken why full tokens are compared instead of bare sequence numbers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Egil.Orleans.Messaging/Outboxes/Outbox.cs | 49 ++++++++++++++++--- .../State/StateManager.cs | 12 +++++ 2 files changed, 53 insertions(+), 8 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs index 1717eeb4..ca34f2a9 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs @@ -312,14 +312,42 @@ public Outbox Clear() /// last pending tokens (including their timestamps). /// /// - /// This intentionally does not compare message payloads. Items are only - /// ever appended at the tail, so two histories that diverged by adding - /// different messages — for example duplicate grain activations racing an - /// ambiguous storage write — always differ in their highest pending - /// token, whose timestamp is stamped by the producing activation. - /// Histories that diverged only by removals can still compare equal; - /// the state-manager recovery path then at worst re-delivers an already - /// posted item (at-least-once) but never loses a pending message. + /// + /// This equality is the contract the state-manager write recovery relies + /// on: after an ambiguous storage write (the call threw but the write may + /// have landed, e.g. a timeout or lost response), the state manager reads + /// the persisted state back and uses Equals to decide between + /// "lost response — the write landed, swallow the error" and "the write + /// did not land — rethrow so the caller retries". A false positive here + /// would make recovery adopt a foreign state and silently drop pending + /// messages, so the fingerprint must never compare equal for a history + /// that is missing items this instance added. + /// + /// + /// The fingerprint intentionally does not compare message payloads, to + /// avoid an O(n) scan on every recovery. It is still loss-safe because + /// items are only ever appended at the tail: two histories that diverged + /// by adding different messages — for example duplicate grain + /// activations of the same grain in two silos racing an ambiguous write — + /// always differ in count or in their highest pending token. The token + /// timestamp is stamped by the producing activation's clock, which also + /// distinguishes same-count races unless both activations produced the + /// exact same timestamp. + /// + /// + /// Note that direct optimistic-concurrency conflicts are not handled + /// here: when the storage provider reports an ETag conflict + /// (InconsistentStateException) the state manager rethrows it even + /// if the values happen to match, so this equality only decides the + /// ambiguous-outcome cases described above. + /// + /// + /// Accepted trade-off: histories that diverged only by removals + /// (or the exact-timestamp tie above) can still compare equal. Recovery + /// then at worst re-delivers an already posted item — duplicate delivery + /// is acceptable under the at-least-once contract, losing a pending + /// message is not. + /// /// public bool Equals(Outbox? other) { @@ -346,6 +374,11 @@ public override int GetHashCode() FirstToken, LastToken); + // Full tokens (sequence number + epoch + timestamp) rather than bare + // sequence numbers: the timestamp comes from the producing activation's + // clock, so it disambiguates duplicate activations that appended different + // items yet reached the same sequence number. See Equals for the recovery + // contract this protects. private OutboxSequenceToken? FirstToken => items.IsDefaultOrEmpty ? null : items[0].Token; diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs index 0edb7d33..26e4d918 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs @@ -242,6 +242,18 @@ protected virtual StorageFailureKind ClassifyClearFailure(Exception exception) return StorageFailureKind.UnknownOutcome; } + /// + /// Decides whether a read-back state proves an ambiguous write actually + /// landed (the "lost response" case in ). + /// + /// + /// For non- types this delegates to the state + /// type's own equality, which therefore carries recovery semantics: it + /// must never return true when the persisted state is missing data + /// the attempted write contained, or recovery would adopt a foreign state + /// and silently lose that data. Outbox<T>.Equals documents + /// how its O(1) fingerprint satisfies this contract. + /// private static bool IsEquivalent(T persisted, T attempted) { if (persisted is VersionedState persistedVersioned From e1182805cac6beb68c7bc8e5fe9acae696e7e3a8 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 10:13:14 +0000 Subject: [PATCH 53/82] fix(om): prune in-memory outbox attempt counters after reconciliation The outbox processor tracks per-item dispatch attempt counts in an in-memory dictionary keyed by item equality. Entries were only removed when an item posted successfully, so items that left the outbox any other way - dead-lettered or dropped in ReconcileFailedAsync, or removed directly by the grain - kept their entries for the lifetime of the activation, a slow memory leak invisible to the owning grain. The processor now prunes counters for items no longer pending after each reconciliation. A consequence is that an equal item enqueued after its predecessor was dead-lettered starts a fresh attempt sequence instead of inheriting stale counts. Also documents the attempt-count semantics on ReconcileFailedAsync: counts are per-activation and keyed by item equality, so retry and dead-letter policies that must survive activation restarts need to persist their own counters. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Outboxes/OutboxProcessor.cs | 5 ++ .../Outboxes/OutboxProcessorOptions.cs | 9 +++ .../Outboxes/OutboxReconciler.cs | 34 +++++++++ .../Outboxes/OutboxReconcilerTests.cs | 75 +++++++++++++++++++ 4 files changed, 123 insertions(+) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxReconcilerTests.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index a89baee6..c4233e1c 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -427,6 +427,11 @@ private async Task ReconcileRetryStateAsync() var pending = options.PendingItems(); MessagingTelemetry.RecordOutboxDepth(grainType, pending.IsDefault ? 0 : pending.Length); + // Items the grain removed without a successful post (dead-lettered or + // dropped in ReconcileFailedAsync) would otherwise leak their attempt + // counters for the activation lifetime. + reconciler.PruneAttempts(pending); + if (pending.IsDefaultOrEmpty) { await DisableRetryAsync(); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs index b593d140..713bf022 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs @@ -32,6 +32,15 @@ public sealed class OutboxProcessorOptions /// the in-memory attempt count. The grain decides whether to leave them /// pending, remove them, or move them to dead-letter state. /// + /// + /// Attempt counts are tracked in memory only, keyed by item equality: + /// they reset to one when the grain activation recycles, and item types + /// without stable value equality (for example mutable classes mutated + /// after enqueue) make counts restart silently. Counters are pruned when + /// the item is no longer pending after reconciliation. Policies that must + /// survive activation restarts (max attempts before dead-letter, etc.) + /// should persist their own counters on the items or grain state. + /// public Func, CancellationToken, ValueTask>? ReconcileFailedAsync { get; init; } /// diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciler.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciler.cs index 966c33b6..f6a245f0 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciler.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxReconciler.cs @@ -53,6 +53,40 @@ public async Task ReconcileAsync( } } + /// + /// Drops attempt counters for items that are no longer pending. + /// + /// + /// Attempt counts are keyed by item value equality and only removed by + /// when the item posts successfully. Items + /// that leave the outbox any other way — dead-lettered or dropped by + /// ReconcileFailedAsync, or removed directly by the grain — would + /// otherwise keep their entries for the lifetime of the activation, a + /// slow memory leak that the owning grain cannot observe or clean up. + /// The processor calls this with a fresh pending snapshot after each + /// reconciliation. + /// + public void PruneAttempts(ImmutableArray pending) + { + if (attempts.Count == 0) + { + return; + } + + if (pending.IsDefaultOrEmpty) + { + attempts.Clear(); + return; + } + + var stillPending = new HashSet(pending); + var stale = attempts.Keys.Where(item => !stillPending.Contains(item)).ToList(); + foreach (var item in stale) + { + attempts.Remove(item); + } + } + private int IncrementAttempt(TOutbox item) { attempts.TryGetValue(item, out var current); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxReconcilerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxReconcilerTests.cs new file mode 100644 index 00000000..2f4dc2fe --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxReconcilerTests.cs @@ -0,0 +1,75 @@ +namespace Egil.Orleans.Messaging.Tests.Outboxes; + +public sealed class OutboxReconcilerTests +{ + private static readonly Exception DispatchError = new InvalidOperationException("dispatch failed"); + + [Fact] + public void Attempt_count_increments_while_item_stays_pending() + { + var reconciler = CreateReconciler(); + + var first = reconciler.CreateBatch([new OutboxDispatchResult("a", DispatchError)]); + reconciler.PruneAttempts(["a"]); + var second = reconciler.CreateBatch([new OutboxDispatchResult("a", DispatchError)]); + + Assert.Equal(1, first.Failed[0].Attempt); + Assert.Equal(2, second.Failed[0].Attempt); + } + + [Fact] + public void Prune_resets_attempt_count_for_item_removed_without_successful_post() + { + var reconciler = CreateReconciler(); + reconciler.CreateBatch([new OutboxDispatchResult("a", DispatchError)]); + + // Grain dead-lettered "a" in ReconcileFailedAsync; it is no longer pending. + reconciler.PruneAttempts([]); + + // An equal item enqueued later must start a fresh attempt sequence. + var batch = reconciler.CreateBatch([new OutboxDispatchResult("a", DispatchError)]); + + Assert.Equal(1, batch.Failed[0].Attempt); + } + + [Fact] + public void Prune_keeps_counts_for_items_still_pending_and_drops_the_rest() + { + var reconciler = CreateReconciler(); + reconciler.CreateBatch( + [ + new OutboxDispatchResult("kept", DispatchError), + new OutboxDispatchResult("dropped", DispatchError), + ]); + + reconciler.PruneAttempts(["kept"]); + + var batch = reconciler.CreateBatch( + [ + new OutboxDispatchResult("kept", DispatchError), + new OutboxDispatchResult("dropped", DispatchError), + ]); + + Assert.Equal(2, batch.Failed.Single(f => f.Item == "kept").Attempt); + Assert.Equal(1, batch.Failed.Single(f => f.Item == "dropped").Attempt); + } + + [Fact] + public async Task Successful_post_removes_attempt_count() + { + var reconciler = CreateReconciler(); + reconciler.CreateBatch([new OutboxDispatchResult("a", DispatchError)]); + + var posted = reconciler.CreateBatch([new OutboxDispatchResult("a")]); + await reconciler.ReconcileAsync(posted, CancellationToken.None); + + var batch = reconciler.CreateBatch([new OutboxDispatchResult("a", DispatchError)]); + + Assert.Equal(1, batch.Failed[0].Attempt); + } + + private static OutboxReconciler CreateReconciler() + => new( + acknowledgePostedAsync: (_, _) => ValueTask.CompletedTask, + reconcileFailedAsync: (_, _) => ValueTask.CompletedTask); +} From 8ccc2fc5b5576cb3120fa063ffabd01f53e66db4 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 10:13:25 +0000 Subject: [PATCH 54/82] docs(om): document standalone outbox usage guarantees The library is a toolbox: grains may use Outbox and OutboxProcessor without the state manager. Document what those users can rely on and where the sharp edges are so problems stay detectable: - README: the outbox pipeline is at-least-once on its own - items only leave durable state via AcknowledgePostedAsync after a successful post, so failed or ambiguous state writes cause duplicate delivery, never loss. Also notes that attempt counts are per-activation and that Outbox.Equals is a fingerprint, not payload equality. - Outbox class docs: equality is exact within a single history lineage (every mutation changes the fingerprint, so dirty-checks are safe) and can only compare equal across divergent activation histories that differ by removals. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 15 ++++++++++++++- .../src/Egil.Orleans.Messaging/Outboxes/Outbox.cs | 11 ++++++++++- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index c8a1c85a..e88e70cb 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -154,7 +154,20 @@ base interfaces or catch-all handlers. Failed dispatches are reported through `ReconcileFailedAsync`. That callback is where the owning grain applies retry, dead-letter, max-depth, or trimming -policy, because the grain owns the durable outbox state. +policy, because the grain owns the durable outbox state. The attempt counts +passed to the callback are in-memory per activation (and pruned once an item +is no longer pending), so policies that must survive activation restarts need +to persist their own counters on the items or grain state. + +The outbox tools do not require the state manager. When persisting the outbox +with plain `IPersistentState` writes, the pipeline stays at-least-once on +its own: items only leave durable state when the grain removes them in +`AcknowledgePostedAsync` after a successful post, so a failed or ambiguous +state write leaves them pending and at worst causes duplicate delivery, never +loss. Be aware that `Outbox.Equals` is an O(1) fingerprint (sender, +sequence metadata, count, and first/last pending tokens), not deep payload +equality — safe for dirty-checks and write recovery, but not a substitute for +comparing message contents item by item. If a post run fails before reconciliation completes — for example when the run exceeds `ProcessingTimeout` or an acknowledgement callback throws — the diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs index ca34f2a9..9d1fc3d0 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs @@ -42,7 +42,16 @@ namespace Egil.Orleans.Messaging.Outboxes; /// /// Equality: Two instances are equal when /// sender, sequence metadata, epoch, count, and the full first and last -/// pending tokens are equal. Equality is O(1) and ignores message payloads. +/// pending tokens are equal. Equality is O(1) and ignores message payloads — +/// it is a fingerprint, not deep content equality. Within a single history +/// lineage every mutation changes the fingerprint ( changes +/// the last token, removals change the count or tokens), so dirty-check +/// "skip write if unchanged" logic is safe. Outboxes from divergent +/// histories (duplicate activations of the same grain) can compare equal when +/// they diverged only by removals; treating them as interchangeable +/// re-delivers already posted items but never loses pending ones. See +/// for the full contract before relying on +/// equality for anything beyond dirty-checks or write recovery. /// /// /// Unbounded growth risk: If postman targets are down, the outbox grows From bc0f05842db7cf8249d682a1fc285f85ea905718 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 10:48:59 +0000 Subject: [PATCH 55/82] refactor(om): remove unused envelope retry diagnostics OutboxMessageEnvelope carried SendAttempts and LastSendError members that nothing in the library read or wrote. They contradicted the envelope's documented design - retry diagnostics are tracked in-memory by the outbox processor, not persisted on the envelope - and SendAttempts was needlessly serialized with grain state. Removed before the first stable release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Outboxes/OutboxMessageEnvelope.cs | 7 ------- 1 file changed, 7 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs index da24baeb..5f652429 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs @@ -73,11 +73,4 @@ public OutboxMessageEnvelope(OutboxSequenceToken token, T message) /// [Id(1)] public required T Message { get; init; } - - [Id(2)] - public int SendAttempts { get; internal set; } - - [JsonIgnore] - [field: NonSerialized] - public Exception? LastSendError { get; internal set; } } From 6d5ac03835b8219a4c929701cb62d019216305ee Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 10:49:00 +0000 Subject: [PATCH 56/82] docs(om): document remaining public constructors and drop completed plan doc [skip notes] Adds XML docs to the StateManagerBase and DefaultStateManager constructors, the last undocumented public members in the shipped packages, and removes outbox-postman-api-plan.md now that the keyed postman API it planned is implemented and described in api-design.md and the README. The plan remains available in git history. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../outbox-postman-api-plan.md | 261 ------------------ .../State/DefaultStateManager.cs | 9 + .../State/StateManager.cs | 5 + 3 files changed, 14 insertions(+), 261 deletions(-) delete mode 100644 Egil.Orleans.Messaging/outbox-postman-api-plan.md diff --git a/Egil.Orleans.Messaging/outbox-postman-api-plan.md b/Egil.Orleans.Messaging/outbox-postman-api-plan.md deleted file mode 100644 index 62a2c988..00000000 --- a/Egil.Orleans.Messaging/outbox-postman-api-plan.md +++ /dev/null @@ -1,261 +0,0 @@ -# Outbox Postman API Plan - -## Goal - -Make `OutboxProcessor` less dependent on inline grain lambdas by -introducing reusable postman services. Grains should declare which postmen -handle which outbox message types, while delivery implementation lives in DI -services that can be tested outside Orleans. - -## Core Decisions - -- Add `IPostman` as the delivery abstraction. -- Keep current `AddPostman(...)` lambda overloads for local/simple cases. -- Add `OutboxProcessor.AddPostman(string postmanName)` to - resolve keyed `IPostman` from the grain activation service - provider. -- Use the grain-scoped service provider. `OutboxProcessor` must not create or - dispose child scopes. -- Register postmen through explicit outbox APIs named `AddOutboxPostman`, not - generic `AddPostman` service-registration APIs. -- Allow one concrete postman to implement multiple `IPostman` - interfaces and register all of them with one call. -- Prefer attribute-based postman names for discovery and one-line - registration. -- Keep `OutboxProcessor` responsible for orchestration, timeout/cancellation, - retry scheduling, acknowledgements, and telemetry. -- Keep `IPostman` responsible only for delivery work under the - processor-owned cancellation budget. - -## Proposed Public API - -```csharp -public interface IPostman - where TMessage : notnull -{ - ValueTask PostAsync( - TMessage message, - CancellationToken cancellationToken); -} - -[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = false)] -public sealed class OutboxPostmanAttribute(string name) : Attribute -{ - public string Name { get; } = name; -} -``` - -```csharp -public sealed partial class OutboxProcessor - where TOutbox : notnull -{ - public OutboxProcessor AddPostman( - string postmanName) - where TMessage : TOutbox; -} -``` - -```csharp -namespace Microsoft.Extensions.DependencyInjection; - -public static class OutboxPostmanServiceCollectionExtensions -{ - extension(IServiceCollection services) - { - public IServiceCollection AddOutboxPostman() - where TPostman : class; - - public IServiceCollection AddOutboxPostman( - string postmanName) - where TPostman : class; - - public IServiceCollection AddOutboxPostman( - string postmanName) - where TMessage : notnull - where TPostman : class, IPostman; - } -} -``` - -## Registration Behavior - -`AddOutboxPostman()`: - -- Reads `[OutboxPostman("name")]` from `TPostman`. -- Finds all closed `IPostman` interfaces implemented by `TPostman`. -- Registers `TPostman` as scoped. -- Registers every discovered `IPostman` as keyed scoped using the - attribute name. -- Throws if `TPostman` does not implement any closed `IPostman`. -- Throws if no attribute exists and no explicit name was provided. -- Rejects open generic postman types in v1. - -Important lifetime detail: - -```csharp -services.AddScoped(); - -services.AddKeyedScoped( - typeof(IPostman<>).MakeGenericType(messageType), - postmanName, - (sp, _) => sp.GetRequiredService()); -``` - -This lets one concrete scoped postman instance satisfy multiple message -contracts during one grain activation. - -## Usage Examples - -```csharp -[OutboxPostman("eventhub")] -public sealed class EventHubOrderPostman : - IPostman, - IPostman -{ - public ValueTask PostAsync(OrderPlaced message, CancellationToken ct) - { - // Send to Event Hubs. - } - - public ValueTask PostAsync(OrderCancelled message, CancellationToken ct) - { - // Send to Event Hubs. - } -} -``` - -```csharp -services.AddOutboxPostman(); -``` - -```csharp -outboxProcessor = this.RegisterOutboxProcessor(options) - .AddPostman("eventhub") - .AddPostman("eventhub"); -``` - -## Postman Execution Contract - -`OutboxProcessor` owns delivery budget. A postman may retry internally, use SDK -retry policies, or perform target-specific backoff, but it must: - -- Pass the cancellation token to async SDK/API calls. -- Stop promptly when cancellation is requested. -- Let `OperationCanceledException` escape. -- Avoid acknowledging or removing outbox items. -- Avoid mutating grain state. -- Be idempotent when the external target can receive duplicate attempts. - -Processor behavior: - -- Successful `PostAsync` call marks the item as posted. -- Exceptions are treated as failed delivery and reconciled through existing - retry behavior. -- Cancellation/timeout remains processor-owned and leaves the item pending. -- Processor telemetry measures total postman call duration. - -## Built-In Postman Helpers - -Convenience APIs implemented on top of `AddPostmanCore`. - -Stream postman: - -```csharp -public OutboxProcessor AddStreamPostman( - string streamProviderName, - Func streamId) - where TMessage : TOutbox; - -public OutboxProcessor AddStreamPostman( - string streamProviderName, - Func streamId, - Func project) - where TMessage : TOutbox; -``` - -Grain postman: - -```csharp -public OutboxProcessor AddGrainPostman( - Func resolveGrain, - Func call) - where TMessage : TOutbox - where TGrain : IGrain; -``` - -These helpers are part of the completed postman API surface. They keep -common Orleans stream and grain fan-out cases on the same processor-owned -dispatch, retry, timeout, acknowledgement, reconciliation, and telemetry path. - -## Tests - -- `AddOutboxPostman()` registers all implemented - `IPostman` contracts. -- Multiple `IPostman` contracts on one concrete postman resolve to - one scoped concrete instance. -- Missing `[OutboxPostman]` without explicit name throws. -- Empty explicit names, empty attribute names, and open generic postman types - throw. -- Concrete type with no `IPostman` contracts throws. -- `OutboxProcessor.AddPostman("name")` resolves keyed postman from - grain activation services. -- Successful keyed postman call acknowledges item. -- Throwing keyed postman goes through existing retry/reconcile path. -- Cancellation from processor propagates to postman and leaves item pending. -- Stream postman helpers publish to the selected Orleans stream and - acknowledge on success. -- Projected stream postman helpers publish projected events and acknowledge on - success. -- Grain postman helpers resolve a grain through `IGrainFactory`, invoke it, and - acknowledge on success. -- Existing lambda-based `AddPostman(...)` behavior remains unchanged. - -## Implementation Steps - -1. Add the core public contract: - - `IPostman` - - `OutboxPostmanAttribute` - -2. Add service-registration APIs: - - `IServiceCollection.AddOutboxPostman()` - - `IServiceCollection.AddOutboxPostman(string postmanName)` - - `IServiceCollection.AddOutboxPostman(string postmanName)` - - Validate missing attributes, missing postman contracts, empty names, and open generic postman types. - -3. Add keyed processor registration: - - `OutboxProcessor.AddPostman(string postmanName)` - - Resolve from the grain activation service provider using keyed DI. - - Reuse the existing `AddPostmanCore` dispatch pipeline so retry, timeout, acknowledgement, reconciliation, and telemetry stay unchanged. - -4. Add focused registration tests: - - Attribute-based registration discovers all implemented contracts. - - Multiple contracts resolve to the same scoped concrete postman instance. - - Explicit-name registration works without an attribute. - - Missing attribute and missing postman contracts throw. - -5. Add Orleans processor behavior tests: - - Keyed postman success acknowledges and removes pending items. - - Keyed postman exceptions flow through existing failure reconciliation. - - Existing lambda postman tests remain green. - -6. Add cancellation/timeout tests: - - Processor cancellation propagates to keyed postman. - - Timeout leaves the pending item unacknowledged. - -7. Update user documentation: - - README examples for service postmen. - - API design section for the postman service contract. - -8. Add built-in helper APIs: - - `OutboxProcessor.AddStreamPostman(...)` - - `OutboxProcessor.AddStreamPostman(...)` - - `OutboxProcessor.AddGrainPostman(...)` - -9. Add helper behavior tests: - - Stream helper publishes the outbox item and acknowledges. - - Projected stream helper publishes the projected event and acknowledges. - - Grain helper resolves the target grain, invokes it, and acknowledges. - -## Implementation Status - -- Steps 1-9 are implemented. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs index 1007edb9..e4a21fad 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/DefaultStateManager.cs @@ -3,9 +3,18 @@ namespace Egil.Orleans.Messaging.State; /// /// Default implementation for general storage providers. /// +/// +/// +/// public sealed class DefaultStateManager : StateManagerBase where T : class, IEquatable { + /// + /// Creates a manager over the grain's persistent state facet. Treats all + /// write failures as + /// because general providers give no reliable signal that a write was + /// rejected before persisting. + /// public DefaultStateManager(IPersistentState storage) : base(storage) { diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs index 26e4d918..475d7e79 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs @@ -74,6 +74,11 @@ public abstract class StateManagerBase : IStateManager private readonly IPersistentState storage; private T state; + /// + /// Initializes the manager over the grain's persistent state facet, + /// adopting the currently loaded IPersistentState<T>.State + /// as the committed snapshot. + /// protected StateManagerBase(IPersistentState storage) { ArgumentNullException.ThrowIfNull(storage); From 6e2f382764fc2a6817aa60579e4af31e2e17f01d Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 10:49:00 +0000 Subject: [PATCH 57/82] build(om): enforce XML doc coverage for shipped packages [skip notes] Adds a src-level Directory.Build.props that re-enables CS1591 (missing XML comment on publicly visible member), which the repo-wide props file suppresses. All public API in the three shipped packages is documented, and Release builds treat warnings as errors, so undocumented public members now fail CI for the src projects while tests keep the suppression. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/src/Directory.Build.props | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 Egil.Orleans.Messaging/src/Directory.Build.props diff --git a/Egil.Orleans.Messaging/src/Directory.Build.props b/Egil.Orleans.Messaging/src/Directory.Build.props new file mode 100644 index 00000000..9f91c2d5 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Directory.Build.props @@ -0,0 +1,16 @@ + + + + + + + + $(NoWarn.Replace('1591', '')) + + + From 2689d0b5823fdf3e889fa1b398dcca7ffc35b0d3 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 11:16:27 +0000 Subject: [PATCH 58/82] docs(om): fix unsafe positional acknowledgement example The README outbox example snapshotted only message payloads and acknowledged with Outbox.Take(items.Length), assuming posted items form a FIFO prefix. They do not: postmen dispatch their groups concurrently, and an item without a matching postman fails in place while later items succeed. Copying the example into such a setup removes a failed, undelivered item from the outbox and loses it. The example now uses OutboxMessageEnvelope as the processor item type and removes exactly the posted items by token. The AcknowledgePostedAsync contract documents that the batch is not necessarily a contiguous prefix and that acknowledgement must match items, never positions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Egil.Orleans.Messaging/README.md | 28 ++++++++++++------- .../Outboxes/OutboxProcessorExtensions.cs | 3 +- .../Outboxes/OutboxProcessorOptions.cs | 11 ++++++++ 3 files changed, 31 insertions(+), 11 deletions(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index e88e70cb..ef6c1410 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -116,38 +116,46 @@ public async Task SubmitAsync() } ``` -Use `OutboxProcessor` to dispatch pending items and acknowledge only the items that were posted successfully: +Use `OutboxProcessor` to dispatch pending items and acknowledge only the items that were posted successfully. Use `OutboxMessageEnvelope` as the processor item type so the acknowledgement callback can remove exactly the posted items by token: ```csharp public sealed class OrderGrain : Grain, IOutboxGrain { - private OutboxProcessor outboxProcessor = default!; + private OutboxProcessor> outboxProcessor = default!; public override Task OnActivateAsync(CancellationToken cancellationToken) { - outboxProcessor = this.RegisterOutboxProcessor(new OutboxProcessorOptions + outboxProcessor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> { - PendingItems = () => state.State.Outbox.Select(item => item.Message).ToImmutableArray(), + PendingItems = () => [.. state.State.Outbox], AcknowledgePostedAsync = async (items, ct) => { - var posted = state.State.Outbox - .Take(items.Length) - .Select(item => item.Token); - await state.WriteAsync(state.State with { - Outbox = state.State.Outbox.RemoveRange(posted) + Outbox = state.State.Outbox.RemoveRange(items.Select(item => item.Token)) }); }, ReconcileFailedAsync = (_, _) => ValueTask.CompletedTask, }) - .AddPostman(PublishSubmittedAsync); + .AddPostman>( + envelope => PublishSubmittedAsync(envelope.Message)); return Task.CompletedTask; } } ``` +`AcknowledgePostedAsync` receives exactly the items that posted successfully — +**not necessarily a contiguous prefix of the pending list**. Different message +types dispatch through different postmen concurrently, and an item without a +matching postman fails in place while later items can still succeed. Never +acknowledge by position (for example `outbox.Take(items.Length)`); that can +remove a failed item and lose it. When the processor item type is a bare +message type instead of the envelope, the callback must map each received item +back to its token — which is only reliable when payloads are unique — so +prefer the envelope form above unless per-message-type postman registration +(shown below) is required. + `IOutboxGrain` forwards reminder ticks to the attached processor. The grain remains responsible for its own message contracts, posting target, and dead-letter policy. Postman matching is first-match-wins: register specific message types before base interfaces or catch-all handlers. diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs index d51a9ce1..68c6ca3f 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs @@ -36,7 +36,8 @@ public static class OutboxProcessorExtensions /// .Select(e => e.Message).ToImmutableArray(), /// AcknowledgePostedAsync = async (items, ct) => /// { - /// // remove delivered items and persist + /// // remove exactly the delivered items (match items or their + /// // tokens, never positions) and persist /// }, /// ReconcileFailedAsync = async (failures, ct) => /// { diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs index 713bf022..a3f5aa03 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs @@ -25,6 +25,17 @@ public sealed class OutboxProcessorOptions /// The grain is expected to remove these items from durable outbox state /// and persist the update. /// + /// + /// The batch contains exactly the items that posted successfully and is + /// not necessarily a contiguous prefix of the + /// snapshot: different postmen dispatch their + /// groups concurrently, and an item without a matching postman fails in + /// place while later items can still succeed. Remove the received items + /// themselves (for example by their OutboxSequenceToken when + /// is OutboxMessageEnvelope<T>), + /// never by position or count — positional removal can drop a failed, + /// undelivered item and lose it. + /// public required Func, CancellationToken, ValueTask> AcknowledgePostedAsync { get; init; } /// From fb5327654f69d6c23aab8a6d0ee2f7bdfc69eff7 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 11 Jun 2026 11:16:38 +0000 Subject: [PATCH 59/82] fix(om): preserve sender timestamp in tracked outbox tokens MessageTracker.LatestOutbox reconstructed the last accepted OutboxSequenceToken using the receiver's Received wall-clock time as the token Timestamp. Token equality includes Timestamp, so whenever sender and receiver clocks differ the returned token did not equal the token that was actually accepted, breaking equality-based consumers and any logic reading the timestamp as sender time. Outbox entries now store the sender-stamped timestamp of the last accepted token alongside the receiver-side Received time (which remains the basis for eviction). LatestOutbox returns the original timestamp. JSON payloads written before the new property fall back to Received, matching the previous behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../Tracking/MessageTracker.cs | 25 ++++++++++++++----- .../Tracking/MessageTrackerJsonConverter.cs | 12 ++++++--- .../Tracking/MessageTrackerTests.cs | 17 +++++++++++++ 3 files changed, 45 insertions(+), 9 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs index 6b40ece6..ecb1e8b3 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTracker.cs @@ -206,21 +206,21 @@ public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) if (!outbox.TryGetValue(token.Sender, out var entry)) { MessagingTelemetry.RecordOutboxReceiveLag(token, now); - next = CreateTracker(streams, outbox.Add(token.Sender, new OutboxEntry(token.Epoch, token.SequenceNumber, now))); + next = CreateTracker(streams, outbox.Add(token.Sender, new OutboxEntry(token.Epoch, token.SequenceNumber, now, token.Timestamp))); return true; } if (token.Epoch > entry.Epoch) { MessagingTelemetry.RecordOutboxReceiveLag(token, now); - next = CreateTracker(streams, outbox.SetItem(token.Sender, new OutboxEntry(token.Epoch, token.SequenceNumber, now))); + next = CreateTracker(streams, outbox.SetItem(token.Sender, new OutboxEntry(token.Epoch, token.SequenceNumber, now, token.Timestamp))); return true; } if (token.Epoch == entry.Epoch && token.SequenceNumber > entry.LastSequenceNumber) { MessagingTelemetry.RecordOutboxReceiveLag(token, now); - next = CreateTracker(streams, outbox.SetItem(token.Sender, new OutboxEntry(entry.Epoch, token.SequenceNumber, now))); + next = CreateTracker(streams, outbox.SetItem(token.Sender, new OutboxEntry(entry.Epoch, token.SequenceNumber, now, token.Timestamp))); return true; } @@ -330,8 +330,12 @@ public bool ProcessMessage(OutboxSequenceToken token, out MessageTracker next) /// public OutboxSequenceToken? LatestOutbox(GrainId sender) { + // Reconstruct with the sender-stamped LastTimestamp, not Received: + // OutboxSequenceToken equality includes Timestamp, so using receiver + // time would make the returned token differ from the token that was + // actually accepted whenever sender and receiver clocks drift. return outbox.TryGetValue(sender, out var entry) - ? new OutboxSequenceToken(entry.LastSequenceNumber, sender, entry.Received, entry.Epoch) + ? new OutboxSequenceToken(entry.LastSequenceNumber, sender, entry.LastTimestamp, entry.Epoch) : null; } @@ -556,11 +560,20 @@ internal readonly record struct StreamEntry( /// /// Internal entry tracking an outbox source's last known epoch, - /// sequence number, and the wall-clock time it was received. + /// sequence number, the sender-stamped timestamp of the last accepted + /// token, and the wall-clock time it was received. /// + /// + /// LastTimestamp preserves the sender's original token timestamp + /// so can reconstruct the token faithfully. + /// Received uses the receiver's clock and exists for eviction + /// policies only — the two differ whenever sender and receiver clocks + /// drift. + /// [GenerateSerializer] internal readonly record struct OutboxEntry( [property: Id(0)] DateTimeOffset Epoch, [property: Id(1)] long LastSequenceNumber, - [property: Id(2)] DateTimeOffset Received); + [property: Id(2)] DateTimeOffset Received, + [property: Id(3)] DateTimeOffset LastTimestamp); } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs index 039cf1d9..829b0248 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs @@ -61,7 +61,11 @@ internal sealed class MessageTrackerJsonConverter : JsonConverter Date: Wed, 22 Jul 2026 20:13:22 +0000 Subject: [PATCH 60/82] fix(om): preserve outbox behavior after rehydration Fall back to TimeProvider.System when Orleans rehydrates an outbox without its non-serialized clock. Document custom-provider re-registration and cover the Orleans deep-copy path so the first mutation no longer fails. --- .../Egil.Orleans.Messaging/Outboxes/Outbox.cs | 33 +++++++++++++++---- .../Outboxes/OutboxTests.cs | 19 +++++++++++ 2 files changed, 45 insertions(+), 7 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs index 9d1fc3d0..4fdf2092 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs @@ -95,9 +95,13 @@ public sealed class Outbox : IReadOnlyList>, IEquata /// Non-persisted service reference. No [Id], no serialization. /// Falls back to when not explicitly set. /// - [NonSerialized] + [field: NonSerialized] [JsonIgnore] - private TimeProvider time = TimeProvider.System; + private TimeProvider Time + { + get => field ?? TimeProvider.System; + set => field = value; + } = TimeProvider.System; /// /// Internal constructor used by mutation methods to produce new instances. @@ -140,12 +144,26 @@ public static Outbox Create(GrainId sender) => /// Must be called after deserialization to inject a test-friendly clock. /// /// - /// This is a void mutator on a non-persisted field — it does not produce + /// + /// This is a void mutator on a non-persisted property — it does not produce /// a new instance. The provider is carried forward /// to successor instances created by , , /// , and . + /// + /// + /// The provider is intentionally not serialized. Every serialization and + /// deserialization round trip, including Orleans deep copies and state + /// rehydration, discards a previously registered custom provider. The + /// resulting outbox falls back to . + /// + /// + /// If timestamp generation depends on a custom provider for tests or + /// business logic, call this method on the resulting outbox after every + /// deserialization or rehydration and before the next . + /// Registering the provider only on the original instance is insufficient. + /// /// - public void RegisterTimeProvider(TimeProvider time) => this.time = time; + public void RegisterTimeProvider(TimeProvider time) => Time = time; /// /// The of the grain that owns this outbox. Baked @@ -203,6 +221,7 @@ public IEnumerator> GetEnumerator() /// A new outbox containing the appended message. public Outbox Add(T message) { + var time = Time; var now = time.GetUtcNow(); var epoch = this.epoch ?? now; var sequenceNumber = latestSequenceNumber + 1; @@ -241,7 +260,7 @@ public Outbox Remove(OutboxSequenceToken token) items.RemoveAt(0), epoch); - next.RegisterTimeProvider(time); + next.RegisterTimeProvider(Time); return next; } @@ -283,7 +302,7 @@ public Outbox RemoveRange(IEnumerable tokens) latestSequenceNumber, remaining, epoch); - next.RegisterTimeProvider(time); + next.RegisterTimeProvider(Time); return next; } @@ -311,7 +330,7 @@ public Outbox Clear() [], epoch); - next.RegisterTimeProvider(time); + next.RegisterTimeProvider(Time); return next; } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs index 048f9007..2ac30c12 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs @@ -1,9 +1,28 @@ +using Orleans.Serialization; using TimeProviderExtensions; namespace Egil.Orleans.Messaging.Tests.Outboxes; public sealed class OutboxTests { + [Fact] + public void Add_after_Orleans_deep_copy_appends_without_registered_time_provider() + { + var sender = GrainId.Create("test/sender", "one"); + var outbox = Outbox.Create(sender).Add("first"); + var services = new ServiceCollection(); + services.AddSerializer(builder => builder.AddAssembly(typeof(Outbox<>).Assembly)); + using var serviceProvider = services.BuildServiceProvider(); + var copied = serviceProvider.GetRequiredService().Copy(outbox); + + var next = copied.Add("second"); + + Assert.Equal(2, next.Count); + Assert.Equal(2, next.LatestSequenceNumber); + Assert.Equal("second", next[1].Message); + Assert.Equal(2, next[1].Token.SequenceNumber); + } + [Fact] public void Add_appends_message_with_next_sequence_sender_and_time() { From b822c2230d9c0e87b768a33db953c94296a13f45 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 11:51:18 +0000 Subject: [PATCH 61/82] fix(om): prevent outbox drain handoff deadlock Release the drain gate between background dispatch and non-interleaving reconciliation, and let a foreground drain finish any pending batch before taking a new snapshot. This prevents non-reentrant grain calls from blocking the reconciliation turn while preserving direct PostAsync completion semantics. --- .../Outboxes/OutboxProcessor.cs | 39 +++++++++++--- .../Outboxes/OutboxProcessorCoverageTests.cs | 51 +++++++++++++++++++ 2 files changed, 83 insertions(+), 7 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index c4233e1c..cee36f63 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -42,7 +42,6 @@ public sealed partial class OutboxProcessor : IOutboxComponent private bool checkedForExistingReminder; private OutboxReconciliationBatch? pendingReconciliation; private TaskCompletionSource? activeDrain; - private bool backgroundDrainOwnsActiveDrain; private bool drainRequested; internal OutboxProcessor( @@ -281,14 +280,31 @@ private async Task DispatchInBackgroundAsync(CancellationToken cancellationToken var drainCompleted = false; try { + if (pendingReconciliation is not null) + { + // A dispatch callback can win the drain gate before the queued + // reconciliation callback. Defer instead of posting the same + // durable snapshot again. + EnsureReconciliationTimer(TimeSpan.Zero); + CompleteDrain(); + drainCompleted = true; + return; + } + var reconciliation = await RunAsActiveDrainAsync( () => ProcessPendingItemsAsync(cancellationToken)); if (reconciliation.HasWork) { pendingReconciliation = reconciliation; - backgroundDrainOwnsActiveDrain = true; EnsureReconciliationTimer(TimeSpan.Zero); + + // Reconciliation is a separate Orleans turn. Keeping the gate + // across that boundary lets a non-reentrant foreground PostAsync + // occupy the activation while waiting for the queued callback, + // so neither operation can complete. + CompleteDrain(); + drainCompleted = true; return; } @@ -313,6 +329,7 @@ private async Task DispatchInBackgroundAsync(CancellationToken cancellationToken private async Task ReconcileInBackgroundAsync(CancellationToken cancellationToken) { + await WaitForTurnAsync(cancellationToken); var reconciliation = pendingReconciliation; pendingReconciliation = null; @@ -331,11 +348,7 @@ await RunAsActiveDrainAsync( } finally { - if (backgroundDrainOwnsActiveDrain) - { - backgroundDrainOwnsActiveDrain = false; - CompleteDrain(); - } + CompleteDrain(); } await ScheduleRequestedDrainAsync(); @@ -360,6 +373,18 @@ private async Task DrainOnceAsync(CancellationToken cancellationToken) await RunAsActiveDrainAsync(async () => { drainRequested = false; + + // A foreground caller can acquire the gate between background + // dispatch and its queued reconciliation turn. Finish that batch + // first so already-posted items are acknowledged before taking a + // new durable snapshot for dispatch. + var pending = pendingReconciliation; + pendingReconciliation = null; + if (pending is not null) + { + await ReconcileAsync(pending.Value, cancellationToken); + } + var reconciliation = await ProcessPendingItemsAsync(cancellationToken); await ReconcileAsync(reconciliation, cancellationToken); }); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs index a1d5b67a..d96c9ce3 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs @@ -213,6 +213,45 @@ await fixture.WaitForAssertionAsync( } } + [Fact] + public async Task PostAsync_coalesces_with_background_dispatch_before_non_interleaving_reconciliation() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + var gate = OutboxProcessorSchedulingGate.For(grainKey); + + try + { + await grain.PublishInBackgroundAsync("dispatch", interleaveReconciliation: false); + await gate.DispatchStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + using var cancellation = CancellationTokenSource.CreateLinkedTokenSource(TestContext.Current.CancellationToken); + cancellation.CancelAfter(TimeSpan.FromSeconds(2)); + + var postAgain = grain.PostAgainAsync(cancellation.Token); + await gate.ForegroundPostStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + gate.AllowDispatch.SetResult(); + gate.AllowReconciliation.SetResult(); + + await postAgain; + await gate.ReconciliationStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + + await fixture.WaitForAssertionAsync( + grain, + async () => + { + var state = await grain.GetSchedulingStateAsync(); + Assert.Equal(1, state.AcknowledgedCount); + }, + ct: TestContext.Current.CancellationToken); + } + finally + { + gate.AllowDispatch.TrySetResult(); + gate.AllowReconciliation.TrySetResult(); + OutboxProcessorSchedulingGate.Remove(grainKey); + } + } + [Fact] public async Task Background_reconciliation_does_not_interleave_with_other_calls_by_default() { @@ -326,6 +365,8 @@ public interface IOutboxProcessorReconciliationSchedulingGrain : IGrainWithGuidK { Task PublishInBackgroundAsync(string value, bool interleaveReconciliation); + Task PostAgainAsync(CancellationToken cancellationToken); + Task RecordWriteAsync(string value); Task GetSchedulingStateAsync(); @@ -346,6 +387,9 @@ internal sealed class OutboxProcessorSchedulingGate public TaskCompletionSource AllowDispatch { get; } = new(TaskCreationOptions.RunContinuationsAsynchronously); + public TaskCompletionSource ForegroundPostStarted { get; } = + new(TaskCreationOptions.RunContinuationsAsynchronously); + public TaskCompletionSource ReconciliationStarted { get; } = new(TaskCreationOptions.RunContinuationsAsynchronously); @@ -791,6 +835,13 @@ public async Task PublishInBackgroundAsync(string value, bool interleaveReconcil await processor.PostInBackgroundAsync(); } + public async Task PostAgainAsync(CancellationToken cancellationToken) + { + var gate = OutboxProcessorSchedulingGate.For(this.GetPrimaryKey()); + gate.ForegroundPostStarted.SetResult(); + await processor!.PostAsync(cancellationToken); + } + public Task RecordWriteAsync(string value) { writes = writes.Add(value); From 5a10776ab536b26defedef66bcb391ad75de0e58 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 14:57:30 +0000 Subject: [PATCH 62/82] fix(om): keep transient clocks out of persisted outboxes Accept an explicit UTC append timestamp while preserving the system-clock convenience overload. Persisted outboxes no longer retain a TimeProvider, so serialization and rehydration need no clock re-registration. --- Egil.Orleans.Messaging/README.md | 6 + .../Egil.Orleans.Messaging/Outboxes/Outbox.cs | 122 +++++++----------- .../Outboxes/OutboxJsonConverterFactory.cs | 4 +- .../Outboxes/OutboxMessageEnvelope.cs | 2 +- .../Outboxes/OutboxSequenceToken.cs | 2 +- .../OutboxJsonConverterFactoryTests.cs | 12 +- .../Outboxes/OutboxTests.cs | 101 ++++++--------- 7 files changed, 100 insertions(+), 149 deletions(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index ef6c1410..09c06f0e 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -116,6 +116,12 @@ public async Task SubmitAsync() } ``` +`Add(message)` uses system UTC. When a grain uses an injected clock, sample it +at the call site and pass the instant with +`Add(message, timeProvider.GetUtcNow())`. The persisted outbox never retains +the provider, so serialization and state rehydration need no clock +re-registration. + Use `OutboxProcessor` to dispatch pending items and acknowledge only the items that were posted successfully. Use `OutboxMessageEnvelope` as the processor item type so the acknowledgement callback can remove exactly the posted items by token: ```csharp diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs index 4fdf2092..078b3771 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/Outbox.cs @@ -14,12 +14,12 @@ namespace Egil.Orleans.Messaging.Outboxes; /// /// Immutable-collection semantics: Behaves like /// — read-only iteration, indexer access, -/// mutators (, , , +/// mutators (, , , /// ) return new instances. The original is never /// modified. Assign the return value back to the state property and write. /// /// -/// Sequence ownership: Only assigns sequence numbers. +/// Sequence ownership: Only assigns sequence numbers. /// Callers supply the payload; the outbox stamps /// with a monotonically increasing and the /// current . This is a hard invariant — there is no public @@ -31,8 +31,8 @@ namespace Egil.Orleans.Messaging.Outboxes; /// → Epoch = null, /// LatestSequenceNumber = 0. Use at construction time or for deliberate /// ops-level sequence-space resets. -/// First → stamps Epoch = now. Persisted with state. -/// Subsequent → same epoch, incrementing sequence number. +/// First → stamps Epoch = now. Persisted with state. +/// Subsequent → same epoch, incrementing sequence number. /// → removes all items but preserves /// and . This is the normal /// "postman drained successfully" path. @@ -44,7 +44,7 @@ namespace Egil.Orleans.Messaging.Outboxes; /// sender, sequence metadata, epoch, count, and the full first and last /// pending tokens are equal. Equality is O(1) and ignores message payloads — /// it is a fingerprint, not deep content equality. Within a single history -/// lineage every mutation changes the fingerprint ( changes +/// lineage every mutation changes the fingerprint ( changes /// the last token, removals change the count or tokens), so dirty-check /// "skip write if unchanged" logic is safe. Outboxes from divergent /// histories (duplicate activations of the same grain) can compare equal when @@ -68,13 +68,6 @@ namespace Egil.Orleans.Messaging.Outboxes; /// Newtonsoft.Json is not supported out of the box; users can write and /// register their own Newtonsoft converter if needed. /// -/// -/// TimeProvider: The time field is non-persisted -/// ([NonSerialized], [JsonIgnore], no [Id]). After -/// deserialization the grain must call to -/// inject a test-friendly clock. Falls back to -/// if skipped — correct for production, breaks fake-clock tests. -/// /// /// /// The user-defined message payload type. Must be serializable by Orleans @@ -91,22 +84,10 @@ public sealed class Outbox : IReadOnlyList>, IEquata [Id(2)] private readonly ImmutableArray> items; [Id(3)] private readonly DateTimeOffset? epoch; - /// - /// Non-persisted service reference. No [Id], no serialization. - /// Falls back to when not explicitly set. - /// - [field: NonSerialized] - [JsonIgnore] - private TimeProvider Time - { - get => field ?? TimeProvider.System; - set => field = value; - } = TimeProvider.System; - /// /// Internal constructor used by mutation methods to produce new instances. /// Not user-callable — use to create the - /// initial outbox, then to append messages. + /// initial outbox, then to append messages. /// internal Outbox( GrainId sender, @@ -123,48 +104,22 @@ internal Outbox( /// /// Creates a fresh, empty outbox for the given . /// is null and - /// is 0. The next stamps a new epoch. + /// is 0. The next stamps a new epoch. /// /// /// Use at grain-state construction time (default property initializer). /// Calling on an active outbox is a nuclear reset — the next - /// starts a fresh epoch. Receivers see the epoch change + /// starts a fresh epoch. Receivers see the epoch change /// and accept unconditionally. Prefer for the normal /// "postman drained" path. /// /// /// The of the grain that owns this outbox. Baked - /// into every produced by . + /// into every produced by . /// public static Outbox Create(GrainId sender) => new(sender, latestSequenceNumber: 0, items: [], epoch: null); - /// - /// Registers a for timestamp generation. - /// Must be called after deserialization to inject a test-friendly clock. - /// - /// - /// - /// This is a void mutator on a non-persisted property — it does not produce - /// a new instance. The provider is carried forward - /// to successor instances created by , , - /// , and . - /// - /// - /// The provider is intentionally not serialized. Every serialization and - /// deserialization round trip, including Orleans deep copies and state - /// rehydration, discards a previously registered custom provider. The - /// resulting outbox falls back to . - /// - /// - /// If timestamp generation depends on a custom provider for tests or - /// business logic, call this method on the resulting outbox after every - /// deserialization or rehydration and before the next . - /// Registering the provider only on the original instance is insufficient. - /// - /// - public void RegisterTimeProvider(TimeProvider time) => Time = time; - /// /// The of the grain that owns this outbox. Baked /// into every . @@ -179,7 +134,7 @@ public static Outbox Create(GrainId sender) => public long LatestSequenceNumber => latestSequenceNumber; /// - /// The epoch marker stamped on the first call. null + /// The epoch marker stamped on the first call. null /// only for a freshly constructed () outbox that /// has never had an item added. /// @@ -219,21 +174,44 @@ public IEnumerator> GetEnumerator() /// /// The user-defined payload to enqueue. /// A new outbox containing the appended message. - public Outbox Add(T message) + public Outbox Add(T message) => + Add(message, TimeProvider.System.GetUtcNow()); + + /// + /// Appends a message using an explicitly supplied current UTC instant, + /// assigning the next sequence number and stamping the epoch on first call. + /// + /// + /// + /// Use this overload when timestamp generation must use an injected clock. + /// Pass the clock's current instant for each append: + /// + /// state = state with + /// { + /// Outbox = state.Outbox.Add(myEvent, timeProvider.GetUtcNow()) + /// }; + /// + /// + /// + /// is the append instant, not the message's + /// domain timestamp. It becomes the token timestamp and, for the first + /// append, the outbox epoch. Non-UTC offsets are normalized to UTC. + /// + /// + /// The user-defined payload to enqueue. + /// The current instant used to timestamp this append. + /// A new outbox containing the appended message. + public Outbox Add(T message, DateTimeOffset utcNow) { - var time = Time; - var now = time.GetUtcNow(); - var epoch = this.epoch ?? now; + utcNow = utcNow.ToUniversalTime(); + var epoch = this.epoch ?? utcNow; var sequenceNumber = latestSequenceNumber + 1; - var token = new OutboxSequenceToken(sequenceNumber, sender, now, epoch); - var next = new Outbox( + var token = new OutboxSequenceToken(sequenceNumber, sender, utcNow, epoch); + return new Outbox( sender, sequenceNumber, items.Add(new OutboxMessageEnvelope(token, message)), epoch); - - next.RegisterTimeProvider(time); - return next; } /// @@ -254,14 +232,11 @@ public Outbox Remove(OutboxSequenceToken token) return this; } - var next = new Outbox( + return new Outbox( sender, latestSequenceNumber, items.RemoveAt(0), epoch); - - next.RegisterTimeProvider(Time); - return next; } /// @@ -297,13 +272,11 @@ public Outbox RemoveRange(IEnumerable tokens) } var remaining = remainingBuilder.ToImmutable(); - var next = new Outbox( + return new Outbox( sender, latestSequenceNumber, remaining, epoch); - next.RegisterTimeProvider(Time); - return next; } /// @@ -312,7 +285,7 @@ public Outbox RemoveRange(IEnumerable tokens) /// /// /// This is the normal path after the postman has successfully drained all - /// items. The high-water mark persists so subsequent calls + /// items. The high-water mark persists so subsequent calls /// continue the sequence without gaps. Receivers see monotonically increasing /// sequence numbers within the same epoch. /// @@ -324,14 +297,11 @@ public Outbox Clear() return this; } - var next = new Outbox( + return new Outbox( sender, latestSequenceNumber, [], epoch); - - next.RegisterTimeProvider(Time); - return next; } /// diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs index 75bf6686..6b8527f0 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxJsonConverterFactory.cs @@ -16,8 +16,8 @@ namespace Egil.Orleans.Messaging.Outboxes; /// /// /// The converter serializes only the structural data needed to reconstruct -/// the outbox (sender, epoch, sequence numbers, items). Internal -/// non-restorable service references such as the time provider are excluded. +/// the outbox (sender, epoch, sequence numbers, items) while keeping its +/// persisted representation encapsulated behind read-only properties. /// /// internal sealed class OutboxJsonConverterFactory : JsonConverterFactory diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs index 5f652429..b53e5eb1 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxMessageEnvelope.cs @@ -63,7 +63,7 @@ public OutboxMessageEnvelope(OutboxSequenceToken token, T message) /// /// The sequence token identifying this message. Assigned by - /// — never user-constructed. + /// — never user-constructed. /// [Id(0)] public required OutboxSequenceToken Token { get; init; } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs index af4240b2..6bddc5a9 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxSequenceToken.cs @@ -65,7 +65,7 @@ public OutboxSequenceToken( /// /// Monotonically increasing sequence number within a single . - /// Assigned by — callers cannot fabricate or choose + /// Assigned by — callers cannot fabricate or choose /// sequence numbers. Starts at 1 for each new epoch. /// [Id(0)] diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs index 1cedd5ea..141697c4 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxJsonConverterFactoryTests.cs @@ -1,5 +1,4 @@ using System.Text.Json; -using TimeProviderExtensions; namespace Egil.Orleans.Messaging.Tests.Outboxes; @@ -11,10 +10,8 @@ public void JsonSerializer_round_trips_outbox_with_string_messages(string first, { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var time = new ManualTimeProvider(now); var outbox = Outbox.Create(sender); - outbox.RegisterTimeProvider(time); - outbox = outbox.Add(first).Add(second); + outbox = outbox.Add(first, now).Add(second, now); var json = JsonSerializer.Serialize(outbox); var roundTripped = JsonSerializer.Deserialize>(json); @@ -30,13 +27,12 @@ public void JsonSerializer_round_trips_outbox_with_complex_messages() { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var time = new ManualTimeProvider(now); var outbox = Outbox.Create(sender); - outbox.RegisterTimeProvider(time); outbox = outbox.Add(new ComplexMessage( "order-17", 42, - new NestedMessage("north", true))); + new NestedMessage("north", true)), + now); var json = JsonSerializer.Serialize(outbox); var roundTripped = JsonSerializer.Deserialize>(json); @@ -145,4 +141,4 @@ public void JsonSerializer_throws_json_exception_for_missing_item_token_sender() private sealed record ComplexMessage(string OrderId, int Quantity, NestedMessage Route); private sealed record NestedMessage(string Name, bool IsExpress); -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs index 2ac30c12..42a64cbc 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs @@ -1,26 +1,27 @@ using Orleans.Serialization; -using TimeProviderExtensions; namespace Egil.Orleans.Messaging.Tests.Outboxes; public sealed class OutboxTests { [Fact] - public void Add_after_Orleans_deep_copy_appends_without_registered_time_provider() + public void Add_after_Orleans_deep_copy_uses_explicit_timestamp() { var sender = GrainId.Create("test/sender", "one"); - var outbox = Outbox.Create(sender).Add("first"); + var epoch = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var later = epoch.AddMinutes(5); + var outbox = Outbox.Create(sender).Add("first", epoch); var services = new ServiceCollection(); services.AddSerializer(builder => builder.AddAssembly(typeof(Outbox<>).Assembly)); using var serviceProvider = services.BuildServiceProvider(); var copied = serviceProvider.GetRequiredService().Copy(outbox); - var next = copied.Add("second"); + var next = copied.Add("second", later); Assert.Equal(2, next.Count); Assert.Equal(2, next.LatestSequenceNumber); Assert.Equal("second", next[1].Message); - Assert.Equal(2, next[1].Token.SequenceNumber); + Assert.Equal(new OutboxSequenceToken(2, sender, later, epoch), next[1].Token); } [Fact] @@ -28,11 +29,9 @@ public void Add_appends_message_with_next_sequence_sender_and_time() { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var time = new ManualTimeProvider(now); var outbox = Outbox.Create(sender); - outbox.RegisterTimeProvider(time); - var next = outbox.Add("created"); + var next = outbox.Add("created", now); Assert.Empty(outbox); Assert.Single(next); @@ -42,19 +41,30 @@ public void Add_appends_message_with_next_sequence_sender_and_time() Assert.Equal(new OutboxSequenceToken(1, sender, now, now), next[0].Token); } + [Fact] + public void Add_normalizes_supplied_timestamp_to_UTC() + { + var sender = GrainId.Create("test/sender", "one"); + var localNow = new DateTimeOffset(2026, 5, 23, 14, 30, 0, TimeSpan.FromHours(2)); + var utcNow = localNow.ToUniversalTime(); + + var next = Outbox.Create(sender).Add("created", localNow); + + Assert.Equal(utcNow, next.Epoch); + Assert.Equal(utcNow, next[0].Token.Timestamp); + Assert.Equal(TimeSpan.Zero, next[0].Token.Timestamp.Offset); + } + [Fact] public void Add_preserves_epoch_and_increments_sequence_for_later_messages() { var sender = GrainId.Create("test/sender", "one"); var epoch = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); var later = epoch.AddMinutes(5); - var time = new ManualTimeProvider(epoch); var outbox = Outbox.Create(sender); - outbox.RegisterTimeProvider(time); - var next = outbox.Add("first"); - time.Advance(TimeSpan.FromMinutes(5)); - next = next.Add("second"); + var next = outbox.Add("first", epoch); + next = next.Add("second", later); Assert.Equal(2, next.Count); Assert.Equal(2, next.LatestSequenceNumber); @@ -68,10 +78,8 @@ public void Remove_deletes_matching_sequence_and_preserves_sequence_metadata() { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var time = new ManualTimeProvider(now); var outbox = Outbox.Create(sender); - outbox.RegisterTimeProvider(time); - outbox = outbox.Add("first").Add("second"); + outbox = outbox.Add("first", now).Add("second", now); var next = outbox.Remove(outbox[0].Token); @@ -87,10 +95,8 @@ public void Remove_ignores_token_from_different_sender_with_same_sequence() var sender = GrainId.Create("test/sender", "one"); var otherSender = GrainId.Create("test/sender", "two"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var time = new ManualTimeProvider(now); var outbox = Outbox.Create(sender); - outbox.RegisterTimeProvider(time); - outbox = outbox.Add("first"); + outbox = outbox.Add("first", now); var otherToken = new OutboxSequenceToken(1, otherSender, now, now); var next = outbox.Remove(otherToken); @@ -104,10 +110,8 @@ public void Remove_ignores_matching_token_that_is_not_fifo_head() { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var time = new ManualTimeProvider(now); var outbox = Outbox.Create(sender); - outbox.RegisterTimeProvider(time); - outbox = outbox.Add("first").Add("second"); + outbox = outbox.Add("first", now).Add("second", now); var next = outbox.Remove(outbox[1].Token); @@ -120,10 +124,8 @@ public void RemoveRange_deletes_matching_fifo_prefix_and_ignores_missing_tokens( { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var time = new ManualTimeProvider(now); var outbox = Outbox.Create(sender); - outbox.RegisterTimeProvider(time); - outbox = outbox.Add("first").Add("second").Add("third"); + outbox = outbox.Add("first", now).Add("second", now).Add("third", now); var missing = new OutboxSequenceToken(99, sender, now, now); var next = outbox.RemoveRange([outbox[0].Token, outbox[1].Token, missing]); @@ -139,10 +141,8 @@ public void RemoveRange_removes_matching_tokens_after_gap_and_preserves_remainin { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var time = new ManualTimeProvider(now); var outbox = Outbox.Create(sender); - outbox.RegisterTimeProvider(time); - outbox = outbox.Add("first").Add("second").Add("third"); + outbox = outbox.Add("first", now).Add("second", now).Add("third", now); var next = outbox.RemoveRange([outbox[0].Token, outbox[2].Token]); @@ -156,14 +156,11 @@ public void Clear_removes_items_and_keeps_sequence_metadata_for_next_add() var sender = GrainId.Create("test/sender", "one"); var epoch = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); var later = epoch.AddMinutes(5); - var time = new ManualTimeProvider(epoch); var outbox = Outbox.Create(sender); - outbox.RegisterTimeProvider(time); - outbox = outbox.Add("first").Add("second"); + outbox = outbox.Add("first", epoch).Add("second", epoch); var cleared = outbox.Clear(); - time.Advance(TimeSpan.FromMinutes(5)); - var next = cleared.Add("third"); + var next = cleared.Add("third", later); Assert.Empty(cleared); Assert.Equal(2, cleared.LatestSequenceNumber); @@ -176,15 +173,11 @@ public void Equals_treats_matching_sequence_window_as_same_pending_items() { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var leftTime = new ManualTimeProvider(now); - var rightTime = new ManualTimeProvider(now); var left = Outbox.Create(sender); var right = Outbox.Create(sender); - left.RegisterTimeProvider(leftTime); - right.RegisterTimeProvider(rightTime); - left = left.Add("left").Add("middle-left").Add("last"); - right = right.Add("right").Add("middle-right").Add("last"); + left = left.Add("left", now).Add("middle-left", now).Add("last", now); + right = right.Add("right", now).Add("middle-right", now).Add("last", now); Assert.Equal(left, right); } @@ -194,15 +187,11 @@ public void Equals_returns_true_when_sender_sequence_epoch_and_envelopes_match() { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var leftTime = new ManualTimeProvider(now); - var rightTime = new ManualTimeProvider(now); var left = Outbox.Create(sender); var right = Outbox.Create(sender); - left.RegisterTimeProvider(leftTime); - right.RegisterTimeProvider(rightTime); - left = left.Add("first").Add("second"); - right = right.Add("first").Add("second"); + left = left.Add("first", now).Add("second", now); + right = right.Add("first", now).Add("second", now); Assert.Equal(left, right); Assert.Equal(left.GetHashCode(), right.GetHashCode()); @@ -213,15 +202,11 @@ public void Equals_returns_false_when_sequence_metadata_differs() { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var leftTime = new ManualTimeProvider(now); - var rightTime = new ManualTimeProvider(now); var left = Outbox.Create(sender); var right = Outbox.Create(sender); - left.RegisterTimeProvider(leftTime); - right.RegisterTimeProvider(rightTime); - left = left.Add("first").Add("second"); - right = right.Add("first").Add("second").Add("third"); + left = left.Add("first", now).Add("second", now); + right = right.Add("first", now).Add("second", now).Add("third", now); Assert.NotEqual(left, right); } @@ -231,22 +216,16 @@ public void Equals_returns_false_when_diverged_tail_items_have_different_timesta { var sender = GrainId.Create("test/sender", "one"); var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); - var leftTime = new ManualTimeProvider(now); - var rightTime = new ManualTimeProvider(now); var left = Outbox.Create(sender); var right = Outbox.Create(sender); - left.RegisterTimeProvider(leftTime); - right.RegisterTimeProvider(rightTime); // Shared history establishes the same epoch and head token, then two // duplicate activations each append their own message at the same // sequence number but at different wall-clock instants. - left = left.Add("base"); - right = right.Add("base"); - leftTime.Advance(TimeSpan.FromMilliseconds(1)); - rightTime.Advance(TimeSpan.FromMilliseconds(2)); - left = left.Add("from-activation-a"); - right = right.Add("from-activation-b"); + left = left.Add("base", now); + right = right.Add("base", now); + left = left.Add("from-activation-a", now.AddMilliseconds(1)); + right = right.Add("from-activation-b", now.AddMilliseconds(2)); Assert.NotEqual(left, right); } From 4d20df9b2ec38d7bed5a7e3d58235988179ab126 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 14:57:53 +0000 Subject: [PATCH 63/82] fix(om): preserve active outbox reconciliation Keep reconciliation ownership visible until callbacks complete so an interleaving empty post cannot cancel acknowledged work. Processing timeouts now accept a scoped TimeProvider, and tests use deterministic gates and clocks. --- Egil.Orleans.Messaging/api-design.md | 55 +++-- .../Outboxes/OutboxDispatcher.cs | 7 +- .../Outboxes/OutboxProcessor.cs | 31 ++- .../Outboxes/OutboxProcessorExtensions.cs | 2 +- .../Outboxes/OutboxProcessorOptions.cs | 10 + .../MessagingTestClusterFixture.cs | 7 +- .../Outboxes/OutboxProcessorCoverageTests.cs | 59 +++++- .../Outboxes/OutboxProcessorTests.cs | 197 +++++++++++++++--- 8 files changed, 296 insertions(+), 72 deletions(-) diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 58425497..09df53ae 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -410,7 +410,8 @@ instances — with three additions: - A baked-in `Sender` (grain id). - A monotonic `LatestSequenceNumber` that persists independently of the message array contents. -- An `Add(T payload)` mutator that owns sequence assignment. +- `Add` mutators that own sequence assignment and either sample system + UTC or accept the caller's current UTC instant explicitly. ### Shape @@ -435,11 +436,6 @@ public sealed class Outbox : IReadOnlyList>, IEquata [Id(2)] private readonly ImmutableArray> items; [Id(3)] private readonly DateTimeOffset? epoch; - // Non-persisted; no [Id]. Mutable, registered post-construction. - [NonSerialized] - [JsonIgnore] - private TimeProvider time = TimeProvider.System; - internal Outbox( GrainId sender, long latestSequenceNumber, @@ -455,8 +451,6 @@ public sealed class Outbox : IReadOnlyList>, IEquata public static Outbox Create(GrainId sender) => new(sender, latestSequenceNumber: 0, items: [], epoch: null); - public void RegisterTimeProvider(TimeProvider time) => this.time = time; - public GrainId Sender => sender; public long LatestSequenceNumber => latestSequenceNumber; public DateTimeOffset? Epoch => epoch; @@ -468,7 +462,9 @@ public sealed class Outbox : IReadOnlyList>, IEquata => ((IEnumerable>)items).GetEnumerator(); IEnumerator IEnumerable.GetEnumerator() => GetEnumerator(); - public Outbox Add(T message) { /* increments seq, stamps epoch on first call */ } + public Outbox Add(T message) { /* delegates with system UTC */ } + public Outbox Add(T message, DateTimeOffset utcNow) + { /* increments seq, stamps epoch on first call */ } public Outbox Remove(OutboxSequenceToken token) { /* removes FIFO head if token matches */ } public Outbox RemoveRange(IEnumerable tokens) { /* removes matching FIFO prefix */ } public Outbox Clear() { /* removes all items, preserves LatestSequenceNumber + Epoch */ } @@ -529,12 +525,14 @@ detect that invalid state. ### Why these choices - **Sealed class, not record.** No `with`, no synthesized copy - constructor. Mutation through four methods only. Reference type because - `RegisterTimeProvider` is a void mutator. + constructor. Mutation through four method families only, preserving + sequence and epoch invariants. - **`Add(T payload)` not `Add(envelope)`.** Outbox owns sequence assignment. Callers cannot fabricate sequence numbers. -- **`RegisterTimeProvider` not per-call argument.** Matches - `MessageTracker`. Successor instances carry the provider forward. +- **Time is per append, never retained.** `Add(T)` samples system UTC; + `Add(T, DateTimeOffset)` accepts the caller's current instant and + normalizes it to UTC. Persisted values therefore contain no transient + clock reference and require no post-deserialization registration. - **Lazy `Epoch`.** A grain that never sends doesn't burn a fresh epoch on storage. - **`Remove(token)` not `Remove(envelope)`.** Token is the identity, but @@ -1162,7 +1160,7 @@ Grains using this library should follow a functional-command model: fields. 2. **Commands run sequentially.** Methods that mutate state ("commands") - produce a new state value from `(currentState, commandPayload)` and + produce a new state value from `(currentState, commandPayload, utcNow)` and call `IStateManager.WriteAsync(newState)`. They run under Orleans' default non-reentrant turn-based concurrency — one at a time, no interleaving. @@ -1182,7 +1180,7 @@ Grains using this library should follow a functional-command model: ### Why this works -- **Deterministic commands.** Same state + same payload → same result. +- **Deterministic commands.** Same state + same payload + same UTC input → same result. Easy to test, easy to reason about, no ambient dependencies. - **Safe concurrency.** Reads never block writes. Writes are serialised by the runtime. No custom locks. @@ -1418,11 +1416,11 @@ extension(TGrain grain) where TGrain : IOutboxGrain, IGrainBase { var services = grain.GrainContext.ActivationServices; var processor = new OutboxProcessor( - grain, options, + grain, + services.GetRequiredService(), + options, services.GetRequiredService() - .CreateLogger($"OutboxProcessor<{typeof(TOutbox).Name}>"), - services.GetService() ?? TimeProvider.System, - services.GetRequiredService()); + .CreateLogger>()); processor.AttachToGrain(); return processor; } @@ -1451,6 +1449,10 @@ public sealed class OutboxProcessorOptions where TOutbox : notnull /// Max time per post run. Set below grain's response timeout. public TimeSpan ProcessingTimeout { get; init; } = TimeSpan.FromSeconds(20); + /// Clock used only to enforce ProcessingTimeout. Orleans owns the grain + /// timers and reminders used for RetryDelay. + public TimeProvider TimeProvider { get; init; } = TimeProvider.System; + /// Timer + reminder period. Orleans reminders fire at most once/minute. public TimeSpan RetryDelay { get; init; } = TimeSpan.FromMinutes(2); @@ -1468,6 +1470,12 @@ public sealed class OutboxProcessorOptions where TOutbox : notnull } ``` +An activation-scoped clock is supplied independently at both call sites: +assign it to `OutboxProcessorOptions.TimeProvider` for deterministic +processing-timeout behavior, and pass `timeProvider.GetUtcNow()` to +`Outbox.Add(message, utcNow)` when appending. The processor does not own the +persisted outbox or re-inject transient services into it. + Naming note: `PendingItems` intentionally names the role of the callback rather than an imperative method (`GetPending`). `OutboxAccessor` was considered, but it is less precise because the processor does not need @@ -1705,7 +1713,7 @@ generic types. The factory's `CreateConverter` method creates the closed ### Non-serialized fields -Service-reference fields like `TimeProvider time` get both +`MessageTracker`'s transient `TimeProvider time` field gets both `[NonSerialized]` and `[JsonIgnore]` — belt-and-suspenders: - **No `[Id]`** → Orleans `[GenerateSerializer]` skips them. @@ -1714,9 +1722,10 @@ Service-reference fields like `TimeProvider time` get both bypasses our custom converter and falls back to reflection. - **Custom converters** also skip them explicitly. -All four layers prevent accidental serialization of non-restorable -service references. `RegisterTimeProvider()` re-injects after -deserialization. +All four layers prevent accidental serialization of the non-restorable +reference. `MessageTracker.RegisterTimeProvider()` re-injects it after +deserialization. `Outbox` deliberately does not use this pattern: its +clock input is sampled by the caller for each append. ### Telemetry diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatcher.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatcher.cs index 3cba711e..6eb10b05 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatcher.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxDispatcher.cs @@ -7,7 +7,8 @@ namespace Egil.Orleans.Messaging.Outboxes; internal sealed class OutboxDispatcher( OutboxPostmanRegistry postmen, ILogger logger, - string grainType) + string grainType, + TimeProvider timeProvider) where TOutbox : notnull { public async Task>> DispatchAsync( @@ -15,7 +16,7 @@ public async Task>> DispatchAsync( TimeSpan processingTimeout, CancellationToken cancellationToken) { - using var timeout = new CancellationTokenSource(processingTimeout); + using var timeout = new CancellationTokenSource(processingTimeout, timeProvider); using var linked = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, timeout.Token); var started = Stopwatch.GetTimestamp(); @@ -148,4 +149,4 @@ private sealed class DispatchGroup(OutboxPostmanRegistration? postman) public List Items { get; } = []; } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index cee36f63..1fd40b33 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -53,6 +53,7 @@ internal OutboxProcessor( ArgumentNullException.ThrowIfNull(owner); ArgumentNullException.ThrowIfNull(grainFactory); ArgumentNullException.ThrowIfNull(options); + ArgumentNullException.ThrowIfNull(options.TimeProvider); ArgumentNullException.ThrowIfNull(logger); if (options.ProcessingTimeout <= TimeSpan.Zero) @@ -71,7 +72,7 @@ internal OutboxProcessor( this.logger = logger; grainType = owner.GetType().Name; reminderName = ReminderPrefix + typeof(TOutbox).FullName; - dispatcher = new OutboxDispatcher(postmen, logger, grainType); + dispatcher = new OutboxDispatcher(postmen, logger, grainType, options.TimeProvider); reconciler = new OutboxReconciler( options.AcknowledgePostedAsync, options.ReconcileFailedAsync); @@ -331,15 +332,26 @@ private async Task ReconcileInBackgroundAsync(CancellationToken cancellationToke { await WaitForTurnAsync(cancellationToken); var reconciliation = pendingReconciliation; - pendingReconciliation = null; try { - await RunAsActiveDrainAsync( - () => ReconcileAsync(reconciliation.GetValueOrDefault(), cancellationToken)); + await RunAsActiveDrainAsync(async () => + { + await reconciler.ReconcileAsync( + reconciliation.GetValueOrDefault(), + cancellationToken); + + // Keep ownership visible while user callbacks run. An + // interleaving empty post must not dispose this timer and + // cancel the reconciliation callback that is using it. + pendingReconciliation = null; + await ReconcileRetryStateAsync(); + }); } catch { + pendingReconciliation = null; + // A failed acknowledgement/reconciliation callback skips // ReconcileRetryStateAsync; arm retry so pending items are not // stranded if the activation goes away. @@ -586,8 +598,15 @@ private async Task DisableRetryAsync() { dispatchTimer?.Dispose(); dispatchTimer = null; - reconciliationTimer?.Dispose(); - reconciliationTimer = null; + + // The durable outbox can be cleared while an already-dispatched batch + // still awaits callbacks. Its timer owns that reconciliation turn; + // disposing it here would cancel the callback before it can finish. + if (pendingReconciliation is null) + { + reconciliationTimer?.Dispose(); + reconciliationTimer = null; + } // Look up a leftover reminder from a previous activation at most once; // afterwards the local field is authoritative, so an empty outbox does diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs index 68c6ca3f..09dd412c 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs @@ -74,4 +74,4 @@ public OutboxProcessor RegisterOutboxProcessor(OutboxProcessor return processor; } } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs index a3f5aa03..c4399229 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorOptions.cs @@ -59,6 +59,16 @@ public sealed class OutboxProcessorOptions /// public TimeSpan ProcessingTimeout { get; init; } = TimeSpan.FromSeconds(20); + /// + /// Clock used to enforce . Default: + /// . + /// + /// + /// This provider controls only the processing timeout. Orleans owns the + /// grain timers and reminders used for . + /// + public TimeProvider TimeProvider { get; init; } = global::System.TimeProvider.System; + /// /// Delay before retrying remaining pending items. Cross-activation retry /// is clamped to at least one minute because Orleans reminders do not diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs index 1e0634cd..e459a236 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs @@ -4,6 +4,7 @@ using Egil.Orleans.Messaging.Tests.Streams; using Orleans.Streams; using Orleans.TestingHost; +using TimeProviderExtensions; namespace Egil.Orleans.Messaging.Tests; @@ -13,6 +14,9 @@ public sealed class MessagingTestClusterFixture : IAsyncLifetime, IGrainActivity public GrainActivityCollector Collector { get; } = new(); + public ManualTimeProvider TimeProvider { get; } = + new(new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero)); + public InProcessTestCluster Cluster => cluster ?? throw new InvalidOperationException("Test cluster not initialized."); public IGrainFactory GrainFactory => Cluster.Client; @@ -30,6 +34,7 @@ public async ValueTask InitializeAsync() AddStreamProviders(siloBuilder); siloBuilder.ConfigureServices(services => { + services.AddSingleton(TimeProvider); services .AddOutboxPostman(OutboxProcessorTestPostmanNames.Success) .AddOutboxPostman(OutboxProcessorTestPostmanNames.Failure) @@ -84,4 +89,4 @@ private static void AddStreamProviders(IClientBuilder clientBuilder) clientBuilder.AddMemoryStreams(StreamManagerTestProviderNames.Implicit); clientBuilder.AddMemoryStreams(OutboxProcessorTestProviderNames.Events); } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs index d96c9ce3..5e68a176 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs @@ -39,10 +39,8 @@ public async Task PostAsync_concurrent_call_waits_then_drains_current_outbox() await gate.DispatchStarted.Task.WaitAsync(TestContext.Current.CancellationToken); var secondPost = grain.PublishAndPostAsync("second"); - var completed = await Task.WhenAny( - secondPost, - Task.Delay(TimeSpan.FromMilliseconds(150), TestContext.Current.CancellationToken)); - Assert.NotSame(secondPost, completed); + await gate.ForegroundPostStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + Assert.False(secondPost.IsCompleted); gate.AllowDispatch.SetResult(); await firstPost.WaitAsync(TestContext.Current.CancellationToken); @@ -224,10 +222,8 @@ public async Task PostAsync_coalesces_with_background_dispatch_before_non_interl { await grain.PublishInBackgroundAsync("dispatch", interleaveReconciliation: false); await gate.DispatchStarted.Task.WaitAsync(TestContext.Current.CancellationToken); - using var cancellation = CancellationTokenSource.CreateLinkedTokenSource(TestContext.Current.CancellationToken); - cancellation.CancelAfter(TimeSpan.FromSeconds(2)); - var postAgain = grain.PostAgainAsync(cancellation.Token); + var postAgain = grain.PostAgainAsync(TestContext.Current.CancellationToken); await gate.ForegroundPostStarted.Task.WaitAsync(TestContext.Current.CancellationToken); gate.AllowDispatch.SetResult(); gate.AllowReconciliation.SetResult(); @@ -252,6 +248,40 @@ await fixture.WaitForAssertionAsync( } } + [Fact] + public async Task Empty_background_post_does_not_cancel_active_reconciliation() + { + var grainKey = Guid.NewGuid(); + var grain = fixture.GrainFactory.GetGrain(grainKey); + var gate = OutboxProcessorSchedulingGate.For(grainKey); + + try + { + await grain.PublishInBackgroundAsync("dispatch", interleaveReconciliation: true); + await gate.DispatchStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + gate.AllowDispatch.SetResult(); + + await gate.ReconciliationStarted.Task.WaitAsync(TestContext.Current.CancellationToken); + await grain.ClearAndPostInBackgroundAsync(TestContext.Current.CancellationToken); + gate.AllowReconciliation.SetResult(); + + await fixture.WaitForAssertionAsync( + grain, + async () => + { + var state = await grain.GetSchedulingStateAsync(); + Assert.Equal(1, state.AcknowledgedCount); + }, + ct: TestContext.Current.CancellationToken); + } + finally + { + gate.AllowDispatch.TrySetResult(); + gate.AllowReconciliation.TrySetResult(); + OutboxProcessorSchedulingGate.Remove(grainKey); + } + } + [Fact] public async Task Background_reconciliation_does_not_interleave_with_other_calls_by_default() { @@ -266,8 +296,6 @@ public async Task Background_reconciliation_does_not_interleave_with_other_calls await gate.ReconciliationStarted.Task.WaitAsync(TestContext.Current.CancellationToken); var writeTask = grain.RecordWriteAsync("during-reconciliation"); - var completed = await Task.WhenAny(writeTask, Task.Delay(TimeSpan.FromMilliseconds(150), TestContext.Current.CancellationToken)); - Assert.NotSame(writeTask, completed); gate.AllowReconciliation.SetResult(); await writeTask.WaitAsync(TestContext.Current.CancellationToken); @@ -367,6 +395,8 @@ public interface IOutboxProcessorReconciliationSchedulingGrain : IGrainWithGuidK Task PostAgainAsync(CancellationToken cancellationToken); + Task ClearAndPostInBackgroundAsync(CancellationToken cancellationToken); + Task RecordWriteAsync(string value); Task GetSchedulingStateAsync(); @@ -522,7 +552,9 @@ public async Task PublishAndPostAsync(string value) { state.State.Outbox = EnsureOutbox().Add(new OutboxProcessorTestEvent(value)); await state.WriteStateAsync(); - await processor!.PostAsync(); + var post = processor!.PostAsync().AsTask(); + OutboxProcessorSchedulingGate.For(this.GetPrimaryKey()).ForegroundPostStarted.SetResult(); + await post; return state.State; } @@ -842,6 +874,13 @@ public async Task PostAgainAsync(CancellationToken cancellationToken) await processor!.PostAsync(cancellationToken); } + public async Task ClearAndPostInBackgroundAsync(CancellationToken cancellationToken) + { + state.State.Outbox = EnsureOutbox().Clear(); + await state.WriteStateAsync(cancellationToken); + await processor!.PostInBackgroundAsync(cancellationToken); + } + public Task RecordWriteAsync(string value) { writes = writes.Add(value); diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index b8a3a157..cf2b71b9 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -1,5 +1,6 @@ using System.Collections.Immutable; using Egil.Orleans.Testing; +using TimeProviderExtensions; namespace Egil.Orleans.Messaging.Tests.Outboxes; @@ -117,37 +118,58 @@ await fixture.WaitForAssertionAsync( public async Task PostAsync_propagates_cancellation_to_keyed_postman_and_leaves_item_pending() { var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + var value = $"cancel-{Guid.NewGuid():N}"; + using var gate = OutboxProcessorPostmanGate.Create(value); - var exceptionType = await source.PublishWithCancellationAsync("cancel"); + var publish = source.PublishWithCancellationAsync(value); + await gate.Started.Task.WaitAsync(TestContext.Current.CancellationToken); + fixture.TimeProvider.Advance(TimeSpan.FromHours(1)); + + var exceptionType = await publish.WaitAsync(TestContext.Current.CancellationToken); var state = await source.GetStateAsync(); Assert.Equal(nameof(TaskCanceledException), exceptionType); Assert.Equal(0, state.AcknowledgedCount); Assert.Equal(0, state.FailedCount); Assert.Equal(1, state.Outbox?.Count); + + await source.DiscardPendingAsync(); } [Fact] public async Task PostAsync_timeout_from_keyed_postman_leaves_item_pending() { var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + var value = $"timeout-{Guid.NewGuid():N}"; + using var gate = OutboxProcessorPostmanGate.Create(value); - var exceptionType = await source.PublishWithTimeoutAsync("timeout"); + var publish = source.PublishWithTimeoutAsync(value); + await gate.Started.Task.WaitAsync(TestContext.Current.CancellationToken); + fixture.TimeProvider.Advance(TimeSpan.FromHours(1)); + + var exceptionType = await publish.WaitAsync(TestContext.Current.CancellationToken); var state = await source.GetStateAsync(); Assert.Equal(nameof(TimeoutException), exceptionType); Assert.Equal(0, state.AcknowledgedCount); Assert.Equal(0, state.FailedCount); Assert.Equal(1, state.Outbox?.Count); + + await source.DiscardPendingAsync(); } [Fact] public async Task PostAsync_timeout_arms_retry_and_later_run_delivers_pending_item() { var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + var value = $"timeout-retry-{Guid.NewGuid():N}"; + using var gate = OutboxProcessorPostmanGate.Create(value); - var exceptionType = await source.PublishWithTimeoutAsync("timeout-retry"); + var publish = source.PublishWithTimeoutAsync(value); + await gate.Started.Task.WaitAsync(TestContext.Current.CancellationToken); + fixture.TimeProvider.Advance(TimeSpan.FromHours(1)); + var exceptionType = await publish.WaitAsync(TestContext.Current.CancellationToken); Assert.Equal(nameof(TimeoutException), exceptionType); await fixture.WaitForAssertionAsync( source, @@ -165,8 +187,20 @@ await fixture.WaitForAssertionAsync( public async Task PostAsync_dispatches_items_for_same_postman_sequentially() { var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + var first = $"first-{Guid.NewGuid():N}"; + var second = $"second-{Guid.NewGuid():N}"; + using var firstGate = OutboxProcessorPostmanGate.Create(first); + using var secondGate = OutboxProcessorPostmanGate.Create(second); - await source.PublishTwoAsync("first", "second"); + var publish = source.PublishTwoAsync(first, second); + await firstGate.Started.Task.WaitAsync(TestContext.Current.CancellationToken); + Assert.False(secondGate.Started.Task.IsCompleted); + + firstGate.AllowCompletion.SetResult(); + await secondGate.Started.Task.WaitAsync(TestContext.Current.CancellationToken); + secondGate.AllowCompletion.SetResult(); + + await publish.WaitAsync(TestContext.Current.CancellationToken); var state = await source.GetStateAsync(); Assert.Equal(2, state.AcknowledgedCount); @@ -179,10 +213,20 @@ public async Task PostAsync_dispatches_items_for_same_postman_sequentially() public async Task PostAsync_dispatches_different_postmen_concurrently() { var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); - - var state = await source.PublishTwoPostmenAsync("primary", "secondary"); - - Assert.Equal(["primary", "secondary"], state.PostedValues.Sort()); + var primary = $"primary-{Guid.NewGuid():N}"; + var secondary = $"secondary-{Guid.NewGuid():N}"; + using var primaryGate = OutboxProcessorPostmanGate.Create(primary); + using var secondaryGate = OutboxProcessorPostmanGate.Create(secondary); + + var publish = source.PublishTwoPostmenAsync(primary, secondary); + await Task.WhenAll( + primaryGate.Started.Task.WaitAsync(TestContext.Current.CancellationToken), + secondaryGate.Started.Task.WaitAsync(TestContext.Current.CancellationToken)); + primaryGate.AllowCompletion.SetResult(); + secondaryGate.AllowCompletion.SetResult(); + + var state = await publish.WaitAsync(TestContext.Current.CancellationToken); + Assert.Equal([primary, secondary], state.PostedValues.Sort()); Assert.Empty(state.FailedValues); Assert.Empty(state.PendingValues); Assert.Equal(2, state.MaxConcurrentPostmen); @@ -205,17 +249,25 @@ public async Task PostAsync_blocks_later_items_for_same_postman_after_failure() public async Task PostInBackgroundAsync_allows_other_calls_while_postman_is_awaiting() { var source = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + var value = $"background-{Guid.NewGuid():N}"; + using var gate = OutboxProcessorPostmanGate.Create(value); - await source.PublishInBackgroundAsync("background"); + await source.PublishInBackgroundAsync(value); + await gate.Started.Task.WaitAsync(TestContext.Current.CancellationToken); + var state = await source.GetStateAsync(); + Assert.Equal(1, state.MaxConcurrentPostmen); + Assert.Equal(0, state.AcknowledgedCount); + Assert.Equal(1, state.Outbox?.Count); + + gate.AllowCompletion.SetResult(); await fixture.WaitForAssertionAsync( source, async () => { - var state = await source.GetStateAsync(); - Assert.Equal(1, state.MaxConcurrentPostmen); - Assert.Equal(0, state.AcknowledgedCount); - Assert.Equal(1, state.Outbox?.Count); + var completed = await source.GetStateAsync(); + Assert.Equal(1, completed.AcknowledgedCount); + Assert.Equal(0, completed.Outbox?.Count); }, ct: TestContext.Current.CancellationToken); } @@ -238,6 +290,59 @@ internal static class OutboxProcessorTestPostmanNames public const string Delay = "outbox-processor-delay"; } +internal sealed class OutboxProcessorPostmanGate : IDisposable +{ + private static readonly Dictionary Gates = []; + private readonly string value; + + private OutboxProcessorPostmanGate(string value) + { + this.value = value; + } + + public TaskCompletionSource Started { get; } = + new(TaskCreationOptions.RunContinuationsAsynchronously); + + public TaskCompletionSource AllowCompletion { get; } = + new(TaskCreationOptions.RunContinuationsAsynchronously); + + public static OutboxProcessorPostmanGate Create(string value) + { + var gate = new OutboxProcessorPostmanGate(value); + lock (Gates) + { + Gates.Add(value, gate); + } + + return gate; + } + + public static OutboxProcessorPostmanGate For(string value) + { + lock (Gates) + { + return Gates[value]; + } + } + + public static OutboxProcessorPostmanGate? Find(string value) + { + lock (Gates) + { + return Gates.GetValueOrDefault(value); + } + } + + public void Dispose() + { + AllowCompletion.TrySetResult(); + lock (Gates) + { + Gates.Remove(value); + } + } +} + public interface IOutboxProcessorSourceGrain : IGrainWithGuidKey { Task PublishInBackgroundAsync(string value); @@ -278,6 +383,8 @@ public interface IOutboxProcessorKeyedCancellationPostmanGrain : IGrainWithGuidK Task PublishWithCancellationAsync(string value); Task GetStateAsync(); + + Task DiscardPendingAsync(); } public interface IOutboxProcessorKeyedTimeoutPostmanGrain : IGrainWithGuidKey @@ -285,6 +392,8 @@ public interface IOutboxProcessorKeyedTimeoutPostmanGrain : IGrainWithGuidKey Task PublishWithTimeoutAsync(string value); Task GetStateAsync(); + + Task DiscardPendingAsync(); } public interface IOutboxProcessorTimeoutRetryGrain : IGrainWithGuidKey @@ -698,7 +807,8 @@ private Outbox EnsureOutbox() } public sealed class OutboxProcessorKeyedCancellationPostmanGrain( - [PersistentState("state", "Default")] IPersistentState state) + [PersistentState("state", "Default")] IPersistentState state, + ManualTimeProvider timeProvider) : Grain, IOutboxProcessorKeyedCancellationPostmanGrain, IOutboxGrain { private OutboxProcessor>? processor; @@ -712,7 +822,8 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], AcknowledgePostedAsync = AcknowledgePostedAsync, ReconcileFailedAsync = ReconcileFailedAsync, - ProcessingTimeout = TimeSpan.FromSeconds(5), + ProcessingTimeout = TimeSpan.FromHours(2), + TimeProvider = timeProvider, RetryDelay = TimeSpan.FromMilliseconds(100) }) .AddPostman>(OutboxProcessorTestPostmanNames.Delay); @@ -726,7 +837,7 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) state.State.Outbox = outbox.Add(new OutboxProcessorTestEvent(value)); await state.WriteStateAsync(); - using var cancellation = new CancellationTokenSource(TimeSpan.FromMilliseconds(20)); + using var cancellation = new CancellationTokenSource(TimeSpan.FromHours(1), timeProvider); try { await processor!.PostAsync(cancellation.Token); @@ -740,6 +851,13 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) public Task GetStateAsync() => Task.FromResult(state.State); + public async Task DiscardPendingAsync() + { + state.State.Outbox = EnsureOutbox().Clear(); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + private ValueTask AcknowledgePostedAsync( ImmutableArray> items, CancellationToken cancellationToken) => @@ -757,7 +875,8 @@ private Outbox EnsureOutbox() } public sealed class OutboxProcessorKeyedTimeoutPostmanGrain( - [PersistentState("state", "Default")] IPersistentState state) + [PersistentState("state", "Default")] IPersistentState state, + ManualTimeProvider timeProvider) : Grain, IOutboxProcessorKeyedTimeoutPostmanGrain, IOutboxGrain { private OutboxProcessor>? processor; @@ -771,7 +890,8 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], AcknowledgePostedAsync = AcknowledgePostedAsync, ReconcileFailedAsync = ReconcileFailedAsync, - ProcessingTimeout = TimeSpan.FromMilliseconds(20), + ProcessingTimeout = TimeSpan.FromHours(1), + TimeProvider = timeProvider, RetryDelay = TimeSpan.FromMilliseconds(100) }) .AddPostman>(OutboxProcessorTestPostmanNames.Delay); @@ -798,6 +918,13 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) public Task GetStateAsync() => Task.FromResult(state.State); + public async Task DiscardPendingAsync() + { + state.State.Outbox = EnsureOutbox().Clear(); + await state.WriteStateAsync(); + await processor!.PostInBackgroundAsync(); + } + private ValueTask AcknowledgePostedAsync( ImmutableArray> items, CancellationToken cancellationToken) => @@ -815,7 +942,8 @@ private Outbox EnsureOutbox() } public sealed class OutboxProcessorTimeoutRetryGrain( - [PersistentState("state", "Default")] IPersistentState state) + [PersistentState("state", "Default")] IPersistentState state, + ManualTimeProvider timeProvider) : Grain, IOutboxProcessorTimeoutRetryGrain, IOutboxGrain { private OutboxProcessor>? processor; @@ -830,10 +958,11 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], AcknowledgePostedAsync = AcknowledgePostedAsync, ReconcileFailedAsync = (_, _) => ValueTask.CompletedTask, - ProcessingTimeout = TimeSpan.FromMilliseconds(250), + ProcessingTimeout = TimeSpan.FromHours(1), + TimeProvider = timeProvider, RetryDelay = TimeSpan.FromMilliseconds(100) }) - .AddPostman>((_, ct) => FirstAttemptBlocksAsync(ct)); + .AddPostman>(FirstAttemptBlocksAsync); await base.OnActivateAsync(cancellationToken); } @@ -857,11 +986,15 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) public Task GetStateAsync() => Task.FromResult(state.State); - private async Task FirstAttemptBlocksAsync(CancellationToken cancellationToken) + private async Task FirstAttemptBlocksAsync( + OutboxMessageEnvelope envelope, + CancellationToken cancellationToken) { if (++attempts == 1) { - await Task.Delay(Timeout.InfiniteTimeSpan, cancellationToken); + var gate = OutboxProcessorPostmanGate.For(envelope.Message.Value); + gate.Started.TrySetResult(); + await gate.AllowCompletion.Task.WaitAsync(cancellationToken); } } @@ -904,7 +1037,7 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) ReconcileFailedAsync = ReconcileFailedAsync, RetryDelay = TimeSpan.FromMilliseconds(100) }) - .AddPostman>(DelayAndTrackConcurrencyAsync); + .AddPostman>(PostWithGateAndTrackConcurrencyAsync); await base.OnActivateAsync(cancellationToken); } @@ -930,7 +1063,7 @@ public async Task PublishInBackgroundAsync(string value) public Task GetStateAsync() => Task.FromResult(state.State); - private async Task DelayAndTrackConcurrencyAsync( + private async Task PostWithGateAndTrackConcurrencyAsync( OutboxMessageEnvelope envelope, CancellationToken cancellationToken) { @@ -939,7 +1072,9 @@ private async Task DelayAndTrackConcurrencyAsync( try { - await Task.Delay(TimeSpan.FromMilliseconds(500), cancellationToken); + var gate = OutboxProcessorPostmanGate.For(envelope.Message.Value); + gate.Started.TrySetResult(); + await gate.AllowCompletion.Task.WaitAsync(cancellationToken); } finally { @@ -1065,7 +1200,11 @@ private async Task PostTrackedAsync( throw new InvalidOperationException($"Cannot post {message.Value}."); } - await Task.Delay(TimeSpan.FromMilliseconds(200), cancellationToken); + if (OutboxProcessorPostmanGate.Find(message.Value) is { } gate) + { + gate.Started.TrySetResult(); + await gate.AllowCompletion.Task.WaitAsync(cancellationToken); + } } finally { @@ -1134,7 +1273,9 @@ public async ValueTask PostAsync( OutboxMessageEnvelope message, CancellationToken cancellationToken) { - await Task.Delay(Timeout.InfiniteTimeSpan, cancellationToken); + var gate = OutboxProcessorPostmanGate.For(message.Message.Value); + gate.Started.TrySetResult(); + await gate.AllowCompletion.Task.WaitAsync(cancellationToken); } } From ecc3b102db0677dbf988ec5dc92c708ff263e34f Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 15:27:58 +0000 Subject: [PATCH 64/82] fix(om): reject duplicate outbox processors Fail immediately when a grain activation attempts to register a second processor, preserving the original reminder component and its durable retry path. Document the one-processor invariant and cover different-type registration through the reminder DIM. --- Egil.Orleans.Messaging/README.md | 6 ++- Egil.Orleans.Messaging/api-design.md | 3 ++ .../Outboxes/IOutboxGrain.cs | 7 ++- .../Outboxes/OutboxProcessor.cs | 11 ++++ .../Outboxes/OutboxProcessorExtensions.cs | 16 ++++++ .../Outboxes/OutboxProcessorCoverageTests.cs | 51 +++++++++++++++++++ .../Outboxes/OutboxProcessorTests.cs | 3 ++ 7 files changed, 95 insertions(+), 2 deletions(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index 09c06f0e..bb69e31d 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -162,7 +162,11 @@ back to its token — which is only reliable when payloads are unique — so prefer the envelope form above unless per-message-type postman registration (shown below) is required. -`IOutboxGrain` forwards reminder ticks to the attached processor. The grain remains responsible for its own message contracts, posting target, and dead-letter policy. +`IOutboxGrain` forwards reminder ticks to the single attached processor. +Register exactly one processor per grain activation; a second registration +throws. Add multiple postmen to that processor when item subtypes need +different delivery behavior. The grain remains responsible for its own +message contracts, posting target, and dead-letter policy. Postman matching is first-match-wins: register specific message types before base interfaces or catch-all handlers. diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 09df53ae..3ac2079e 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -1217,6 +1217,9 @@ in the `Clever.PricingEngine` codebase. - **Grain-scoped, not silo-scoped.** Each grain with an outbox gets its own `OutboxProcessor`. No external scan, no registry, no second store. +- **One processor per activation.** A second registration is rejected because + the `IOutboxGrain` reminder bridge has one component slot. Different item + subtypes use multiple postmen on the same processor. - **In-process retry** for fast retry while activated. - **Durable Reminder** for cross-activation recovery. Reactivates the grain if it deactivates with pending items. Retry scheduling is updated on diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs index df6e4533..c3715030 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/IOutboxGrain.cs @@ -31,6 +31,11 @@ namespace Egil.Orleans.Messaging.Outboxes; /// extension method constrains TGrain : IOutboxGrain, IGrainBase, /// so grains that forget to implement this interface get a compile error. /// +/// +/// Register exactly one outbox processor per grain activation. Register +/// multiple postmen on that processor when different item subtypes require +/// different delivery behavior. +/// /// public interface IOutboxGrain : IRemindable { @@ -53,4 +58,4 @@ Task IRemindable.ReceiveReminder(string reminderName, TickStatus status) return component.ReceiveReminderAsync(reminderName, status).AsTask(); } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index 1fd40b33..7b620ed8 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -24,6 +24,9 @@ public sealed partial class OutboxProcessor : IOutboxComponent where TOutbox : notnull { private const string ReminderPrefix = "egil.orleans.messaging.outbox."; + internal const string DuplicateRegistrationMessage = + "Only one outbox processor can be registered per grain activation."; + private static readonly AsyncLocal?> ActiveDrain = new(); private readonly IGrainBase owner; @@ -270,6 +273,14 @@ public ValueTask ReceiveReminderAsync(string reminderName, TickStatus status) /// internal void AttachToGrain() { + if (owner.GrainContext.GetComponent() is not null) + { + // The IOutboxGrain DIM has one component slot through which all + // durable reminders enter. Replacing it would silently orphan the + // first processor's cross-activation retry path. + throw new InvalidOperationException(DuplicateRegistrationMessage); + } + owner.GrainContext.SetComponent(this); } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs index 09dd412c..fb071e9b 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs @@ -26,6 +26,12 @@ public static class OutboxProcessorExtensions /// for reminder forwarding. /// /// + /// Register exactly one processor per grain activation. Add multiple + /// postmen to that processor when different outbox item subtypes need + /// different delivery behavior. A second registration throws + /// . + /// + /// /// Usage: /// /// public override async Task OnActivateAsync(CancellationToken ct) @@ -57,12 +63,22 @@ public static class OutboxProcessorExtensions /// in a grain field for later /// calls. /// + /// + /// An outbox processor is already registered for the current grain + /// activation. + /// public OutboxProcessor RegisterOutboxProcessor(OutboxProcessorOptions options) where TOutbox : notnull { ArgumentNullException.ThrowIfNull(grain); ArgumentNullException.ThrowIfNull(options); + if (grain.GrainContext.GetComponent() is not null) + { + throw new InvalidOperationException( + OutboxProcessor.DuplicateRegistrationMessage); + } + var services = grain.GrainContext.ActivationServices; var processor = new OutboxProcessor( grain, diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs index 5e68a176..e67f53e5 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs @@ -177,6 +177,20 @@ public async Task RegisterOutboxProcessor_rejects_non_positive_retry_delay() Assert.Equal("options", paramName); } + [Fact] + public async Task Registering_second_outbox_processor_throws_without_replacing_first_reminder_component() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var error = await grain.RegisterSecondProcessorAndReceiveFirstReminderAsync(); + var state = await grain.GetStateAsync(); + + Assert.Equal( + "Only one outbox processor can be registered per grain activation.", + error); + Assert.Equal(1, state.FirstProcessorReminderCount); + } + [Fact] public async Task Background_dispatch_allows_other_calls_while_postman_is_awaiting() { @@ -387,6 +401,10 @@ public interface IOutboxProcessorValidationCoverageGrain : IGrainWithGuidKey Task ValidateProcessingTimeoutAsync(); Task ValidateRetryDelayAsync(); + + Task RegisterSecondProcessorAndReceiveFirstReminderAsync(); + + Task GetStateAsync(); } public interface IOutboxProcessorReconciliationSchedulingGrain : IGrainWithGuidKey @@ -814,6 +832,39 @@ public async Task PostPendingThenDefaultAfterAcknowl } } + public async Task RegisterSecondProcessorAndReceiveFirstReminderAsync() + { + var firstProcessor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> + { + PendingItems = () => + { + state.State.FirstProcessorReminderCount++; + return []; + }, + AcknowledgePostedAsync = static (_, _) => ValueTask.CompletedTask, + RetryDelay = TimeSpan.FromMilliseconds(100) + }); + + string? error = null; + try + { + _ = this.RegisterOutboxProcessor(new OutboxProcessorOptions + { + PendingItems = static () => [], + AcknowledgePostedAsync = static (_, _) => ValueTask.CompletedTask + }); + } + catch (InvalidOperationException ex) + { + error = ex.Message; + } + + await ((IRemindable)this).ReceiveReminder(firstProcessor.ReminderName, default); + return error; + } + + public Task GetStateAsync() => Task.FromResult(state.State); + private OutboxProcessorOptions> CreateOptions( Func>> pendingItems, TimeSpan? processingTimeout = null, diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index cf2b71b9..0ec58ffa 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -446,6 +446,9 @@ public sealed class OutboxProcessorSourceState [Id(4)] public int MaxConcurrentPostmen { get; set; } + + [Id(5)] + public int FirstProcessorReminderCount { get; set; } } [GenerateSerializer] From ecf3e5b5f8ca871abbc22ffb3ed4f5c943ad559a Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 15:47:38 +0000 Subject: [PATCH 65/82] fix(om): retry inherited reminder cleanup Retain a reminder handle loaded from a previous activation before unregistering it, so a transient unregister failure can be retried by the next empty outbox drain. --- .../Outboxes/OutboxProcessor.cs | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs index 7b620ed8..6f1ab0fc 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessor.cs @@ -622,16 +622,17 @@ private async Task DisableRetryAsync() // Look up a leftover reminder from a previous activation at most once; // afterwards the local field is authoritative, so an empty outbox does // not pay a reminder-table read on every successful drain. - var activeReminder = reminder; - if (activeReminder is null && !checkedForExistingReminder) + if (reminder is null && !checkedForExistingReminder) { - activeReminder = await owner.GetReminder(reminderName); + // Retain an inherited handle before attempting removal. If removal + // fails, the next empty drain must retry it without another lookup. + reminder = await owner.GetReminder(reminderName); checkedForExistingReminder = true; } - if (activeReminder is not null) + if (reminder is not null) { - await owner.UnregisterReminder(activeReminder); + await owner.UnregisterReminder(reminder); reminder = null; } } From 11693ba21b0a88be8be332691b1631ba94ee69a2 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 16:18:11 +0000 Subject: [PATCH 66/82] fix(om): preserve concurrent state after write failures Adopt durable state after a successful recovery read before rethrowing mismatches and concurrency conflicts. This keeps recovered state paired with the refreshed ETag so retries cannot overwrite concurrent writes. --- .../State/IStateManager.cs | 6 +- .../State/StateManager.cs | 40 +++++------ .../State/StateManagerTests.cs | 67 +++++++++++++++++-- 3 files changed, 85 insertions(+), 28 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs index 054dd7e8..b83de05e 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs @@ -112,7 +112,9 @@ public interface IStateManager /// /// Recovery on failure: Re-reads from storage. If the write actually /// landed (version or equality match), swallows the exception and returns - /// normally. If it did not land, rethrows the original exception. + /// normally. If it did not land, adopts the persisted state before + /// rethrowing the original exception so the state remains paired with the + /// ETag refreshed by the read. /// InconsistentStateException always rethrows /// even if equality matches — a coincidental match must not hide a real /// concurrent write. @@ -131,4 +133,4 @@ public interface IStateManager /// Clears the persisted state, resetting it to default. /// Task ClearAsync(); -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs index 475d7e79..9bc60169 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs @@ -44,7 +44,8 @@ namespace Egil.Orleans.Messaging.State; /// /// /// Write throws, re-read succeeds, state does not match -/// Write genuinely failed. Original exception rethrown. +/// Write genuinely failed. Persisted state adopted and original +/// exception rethrown. /// /// /// Write throws, re-read also throws @@ -131,42 +132,37 @@ public async Task WriteAsync(T newState) throw; } + var recoveryReadSucceeded = false; try { // Unknown outcome: write may have persisted despite the exception. // Read back from storage to distinguish "lost response" from "real failure." await storage.ReadStateAsync(); var persisted = storage.State; + recoveryReadSucceeded = true; + state = persisted; - if (ex is InconsistentStateException) - { - // Concurrency conflicts must be surfaced even if values happen to match. - // A coincidental equality must not hide an optimistic concurrency violation. - storage.State = previousState; - throw; - } - - if (IsEquivalent(persisted, newState)) + if (ex is not InconsistentStateException && IsEquivalent(persisted, newState)) { // Lost-response case: write landed, but acknowledgement failed. - // Advance committed fence to persisted value and swallow write exception. - state = persisted; + // The committed fence already reflects the persisted value. return; } - - // Read-back proved write did not land; restore local snapshot and rethrow. - storage.State = previousState; - throw; } - catch when (ex is not InconsistentStateException) + catch { - // no-op; throw original below + if (!recoveryReadSucceeded) + { + // A failed read yields no trustworthy durable value. Restore + // the pre-write snapshot; the original write error is rethrown below. + state = previousState; + storage.State = previousState; + } } - // Write failed and recovery read also failed: keep committed fence coherent by - // reverting to the pre-write snapshot, then surface the original write error. - state = previousState; - storage.State = previousState; + // A mismatch proves the write did not land. A concurrency conflict + // must be surfaced even if values happen to match. Once read-back + // succeeds, keep the server value paired with its refreshed ETag. throw; } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs index fabce5ce..67110815 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs @@ -1,3 +1,5 @@ +using Orleans.Storage; + namespace Egil.Orleans.Messaging.Tests.State; public sealed class StateManagerTests @@ -44,20 +46,77 @@ public async Task WriteAsync_when_write_fails_but_read_shows_attempted_state_swa } [Fact] - public async Task WriteAsync_when_write_fails_and_read_shows_different_state_rethrows_original_exception() + public async Task WriteAsync_when_write_fails_and_read_shows_different_state_adopts_persisted_state_and_rethrows() { var writeException = new TimeoutException("write timeout"); + var persisted = new TestState("other"); var storage = new FakePersistentState(new TestState("initial")) { WriteException = writeException, - OnRead = state => state.State = new TestState("other") + OnRead = state => + { + state.State = persisted; + state.Etag = "etag-2"; + } }; var manager = new DefaultStateManager(storage); var ex = await Assert.ThrowsAsync(() => manager.WriteAsync(new TestState("next"))); Assert.Same(writeException, ex); - Assert.Equal(new TestState("initial"), manager.State); + Assert.Equal(persisted, manager.State); + Assert.Equal(persisted, storage.State); + Assert.Equal("etag-2", storage.Etag); + } + + [Fact] + public async Task WriteAsync_on_concurrency_conflict_adopts_persisted_state_and_rethrows() + { + var writeException = new InconsistentStateException("write conflict"); + var attempted = new TestState("next"); + var persisted = new TestState("other"); + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = writeException, + OnRead = state => + { + state.State = persisted; + state.Etag = "etag-2"; + } + }; + var manager = new DefaultStateManager(storage); + + var ex = await Assert.ThrowsAsync( + () => manager.WriteAsync(attempted)); + + Assert.Same(writeException, ex); + Assert.Equal(persisted, manager.State); + Assert.Equal(persisted, storage.State); + Assert.Equal("etag-2", storage.Etag); + } + + [Fact] + public async Task WriteAsync_on_concurrency_conflict_rethrows_when_persisted_state_matches_attempt() + { + var writeException = new InconsistentStateException("write conflict"); + var attempted = new TestState("next"); + var storage = new FakePersistentState(new TestState("initial")) + { + WriteException = writeException, + OnRead = state => + { + state.State = new TestState("next"); + state.Etag = "etag-2"; + } + }; + var manager = new DefaultStateManager(storage); + + var ex = await Assert.ThrowsAsync( + () => manager.WriteAsync(attempted)); + + Assert.Same(writeException, ex); + Assert.Equal(new TestState("next"), manager.State); + Assert.Equal("etag-2", storage.Etag); } [Fact] @@ -232,4 +291,4 @@ public Task ClearStateAsync() public Task ClearStateAsync(CancellationToken cancellationToken) => ClearStateAsync(); } -} \ No newline at end of file +} From 695febcbd041c430c1a02232664f4cdbb5e7e446 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 16:39:11 +0000 Subject: [PATCH 67/82] fix(om): tolerate additive stream token JSON Read cursor, discriminator-envelope, built-in token, and Event Hub token properties by name while ignoring unknown fields. Required fields, known value types, and duplicate properties remain strict so rolling upgrades gain forward tolerance without accepting ambiguous payloads. --- Egil.Orleans.Messaging/api-design.md | 8 + ...entHubStreamSequenceTokenJsonConverters.cs | 245 +++++++++++------- .../Streams/StreamCursorJsonConverter.cs | 113 ++++---- .../StreamSequenceTokenJsonConverters.cs | 228 +++++++++++----- ...EventHubSequenceTokenJsonConverterTests.cs | 137 ++++++++++ .../Streams/StreamCursorJsonConverterTests.cs | 86 ++++++ 6 files changed, 616 insertions(+), 201 deletions(-) diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 3ac2079e..a7d21992 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -1066,6 +1066,14 @@ stream providers can register their own converter with `AddStreamSequenceTokenJsonConverter(...)` or `StreamSequenceTokenJsonConverters.Register(...)`. +Cursor, discriminator-envelope, and built-in token readers treat JSON object +property order as insignificant and skip unknown properties. Writers keep a +canonical order. This allows older silos to read additive payloads during +rolling upgrades or rollback; newly added properties must still be optional +because an older silo cannot preserve an unknown value when it rewrites the +payload. Custom token converters are responsible for the same forward-tolerant +read behavior inside their payloads. + ### Event Hub adapter for enriched tokens The library ships `EnrichedEventHubAdapter`, a public unsealed diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EventHubStreamSequenceTokenJsonConverters.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EventHubStreamSequenceTokenJsonConverters.cs index 5e92b07e..e8fe09b2 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EventHubStreamSequenceTokenJsonConverters.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EventHubStreamSequenceTokenJsonConverters.cs @@ -28,13 +28,15 @@ public sealed class EventHubSequenceTokenJsonConverter : JsonConverter public override EventHubSequenceToken Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { - EventHubSequenceTokenJsonConverterHelpers.ExpectStartObject(ref reader, nameof(EventHubSequenceToken)); - var offset = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredStringProperty(ref reader, "EventHubOffset"); - var sequenceNumber = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt64Property(ref reader, "SequenceNumber"); - var eventIndex = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt32Property(ref reader, "EventIndex"); - EventHubSequenceTokenJsonConverterHelpers.ReadEndObject(ref reader); + var properties = EventHubSequenceTokenJsonConverterHelpers.ReadProperties( + ref reader, + nameof(EventHubSequenceToken), + requireEnrichedProperties: false); - return new EventHubSequenceToken(offset, sequenceNumber, eventIndex); + return new EventHubSequenceToken( + properties.EventHubOffset, + properties.SequenceNumber, + properties.EventIndex); } /// @@ -58,13 +60,15 @@ public sealed class EventHubSequenceTokenV2JsonConverter : JsonConverter public override EventHubSequenceTokenV2 Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { - EventHubSequenceTokenJsonConverterHelpers.ExpectStartObject(ref reader, nameof(EventHubSequenceTokenV2)); - var offset = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredStringProperty(ref reader, "EventHubOffset"); - var sequenceNumber = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt64Property(ref reader, "SequenceNumber"); - var eventIndex = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt32Property(ref reader, "EventIndex"); - EventHubSequenceTokenJsonConverterHelpers.ReadEndObject(ref reader); + var properties = EventHubSequenceTokenJsonConverterHelpers.ReadProperties( + ref reader, + nameof(EventHubSequenceTokenV2), + requireEnrichedProperties: false); - return new EventHubSequenceTokenV2(offset, sequenceNumber, eventIndex); + return new EventHubSequenceTokenV2( + properties.EventHubOffset, + properties.SequenceNumber, + properties.EventIndex); } /// @@ -88,22 +92,18 @@ public sealed class EnrichedEventHubSequenceTokenJsonConverter : JsonConverter public override EnrichedEventHubSequenceToken Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) { - EventHubSequenceTokenJsonConverterHelpers.ExpectStartObject(ref reader, nameof(EnrichedEventHubSequenceToken)); - var offset = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredStringProperty(ref reader, "EventHubOffset"); - var sequenceNumber = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt64Property(ref reader, "SequenceNumber"); - var eventIndex = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredInt32Property(ref reader, "EventIndex"); - var enqueuedTime = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredDateTimeOffsetProperty(ref reader, "EnqueuedTime"); - var providerName = EventHubSequenceTokenJsonConverterHelpers.ReadRequiredStringProperty(ref reader, "ProviderName"); - var traceParent = EventHubSequenceTokenJsonConverterHelpers.ReadNullableStringProperty(ref reader, "TraceParent"); - EventHubSequenceTokenJsonConverterHelpers.ReadEndObject(ref reader); + var properties = EventHubSequenceTokenJsonConverterHelpers.ReadProperties( + ref reader, + nameof(EnrichedEventHubSequenceToken), + requireEnrichedProperties: true); return new EnrichedEventHubSequenceToken( - offset, - sequenceNumber, - eventIndex, - enqueuedTime, - providerName, - traceParent); + properties.EventHubOffset, + properties.SequenceNumber, + properties.EventIndex, + properties.EnqueuedTime, + properties.ProviderName!, + properties.TraceParent); } /// @@ -124,14 +124,6 @@ public override void Write(Utf8JsonWriter writer, EnrichedEventHubSequenceToken internal static class EventHubSequenceTokenJsonConverterHelpers { - public static void ExpectStartObject(ref Utf8JsonReader reader, string typeName) - { - if (reader.TokenType is not JsonTokenType.StartObject) - { - throw new JsonException($"Expected {typeName} object, got '{reader.TokenType}'."); - } - } - public static void WriteBaseProperties( Utf8JsonWriter writer, string eventHubOffset, @@ -143,82 +135,153 @@ public static void WriteBaseProperties( writer.WriteNumber("EventIndex", eventIndex); } - public static string ReadRequiredStringProperty(ref Utf8JsonReader reader, string propertyName) + public static Properties ReadProperties( + ref Utf8JsonReader reader, + string typeName, + bool requireEnrichedProperties) { - ReadRequiredPropertyName(ref reader, propertyName); - if (!reader.Read() || reader.TokenType is not JsonTokenType.String) + if (reader.TokenType is not JsonTokenType.StartObject) { - throw new JsonException($"Property '{propertyName}' must be a string."); + throw new JsonException($"Expected {typeName} object, got '{reader.TokenType}'."); } - return reader.GetString() - ?? throw new JsonException($"Property '{propertyName}' must not be null."); - } + string? eventHubOffset = null; + var hasEventHubOffset = false; + long sequenceNumber = 0; + var hasSequenceNumber = false; + var eventIndex = 0; + var hasEventIndex = false; + DateTimeOffset enqueuedTime = default; + var hasEnqueuedTime = false; + string? providerName = null; + var hasProviderName = false; + string? traceParent = null; + var hasTraceParent = false; - public static string? ReadNullableStringProperty(ref Utf8JsonReader reader, string propertyName) - { - ReadRequiredPropertyName(ref reader, propertyName); - if (!reader.Read()) + while (reader.Read()) { - throw new JsonException($"Unexpected end of JSON while reading '{propertyName}'."); - } + if (reader.TokenType is JsonTokenType.EndObject) + { + return new Properties( + hasEventHubOffset + ? eventHubOffset! + : throw new JsonException($"Missing {typeName} property 'EventHubOffset'."), + hasSequenceNumber + ? sequenceNumber + : throw new JsonException($"Missing {typeName} property 'SequenceNumber'."), + hasEventIndex + ? eventIndex + : throw new JsonException($"Missing {typeName} property 'EventIndex'."), + !requireEnrichedProperties || hasEnqueuedTime + ? enqueuedTime + : throw new JsonException($"Missing {typeName} property 'EnqueuedTime'."), + !requireEnrichedProperties || hasProviderName + ? providerName + : throw new JsonException($"Missing {typeName} property 'ProviderName'."), + !requireEnrichedProperties || hasTraceParent + ? traceParent + : throw new JsonException($"Missing {typeName} property 'TraceParent'.")); + } - return reader.TokenType is JsonTokenType.Null - ? null - : reader.GetString(); - } + if (reader.TokenType is not JsonTokenType.PropertyName) + { + throw new JsonException($"Expected {typeName} property, got '{reader.TokenType}'."); + } - public static long ReadRequiredInt64Property(ref Utf8JsonReader reader, string propertyName) - { - ReadRequiredPropertyName(ref reader, propertyName); - if (!reader.Read() || reader.TokenType is not JsonTokenType.Number) - { - throw new JsonException($"Property '{propertyName}' must be a number."); - } + var propertyName = reader.GetString(); + if (!reader.Read()) + { + throw new JsonException($"Unexpected end of {typeName} JSON."); + } - return reader.GetInt64(); - } + switch (propertyName) + { + case "EventHubOffset": + ThrowIfDuplicate(hasEventHubOffset, typeName, propertyName); + if (reader.TokenType is not JsonTokenType.String) + { + throw new JsonException("Property 'EventHubOffset' must be a string."); + } - public static int ReadRequiredInt32Property(ref Utf8JsonReader reader, string propertyName) - { - ReadRequiredPropertyName(ref reader, propertyName); - if (!reader.Read() || reader.TokenType is not JsonTokenType.Number) - { - throw new JsonException($"Property '{propertyName}' must be a number."); - } + eventHubOffset = reader.GetString() + ?? throw new JsonException("Property 'EventHubOffset' must not be null."); + hasEventHubOffset = true; + break; + case "SequenceNumber": + ThrowIfDuplicate(hasSequenceNumber, typeName, propertyName); + if (reader.TokenType is not JsonTokenType.Number + || !reader.TryGetInt64(out sequenceNumber)) + { + throw new JsonException("Property 'SequenceNumber' must be a 64-bit integer."); + } - return reader.GetInt32(); - } + hasSequenceNumber = true; + break; + case "EventIndex": + ThrowIfDuplicate(hasEventIndex, typeName, propertyName); + if (reader.TokenType is not JsonTokenType.Number + || !reader.TryGetInt32(out eventIndex)) + { + throw new JsonException("Property 'EventIndex' must be a 32-bit integer."); + } - public static DateTimeOffset ReadRequiredDateTimeOffsetProperty(ref Utf8JsonReader reader, string propertyName) - { - ReadRequiredPropertyName(ref reader, propertyName); - if (!reader.Read() || reader.TokenType is not JsonTokenType.String) - { - throw new JsonException($"Property '{propertyName}' must be a date-time string."); - } + hasEventIndex = true; + break; + case "EnqueuedTime" when requireEnrichedProperties: + ThrowIfDuplicate(hasEnqueuedTime, typeName, propertyName); + if (reader.TokenType is not JsonTokenType.String + || !reader.TryGetDateTimeOffset(out enqueuedTime)) + { + throw new JsonException("Property 'EnqueuedTime' must be a date-time string."); + } - return reader.GetDateTimeOffset(); - } + hasEnqueuedTime = true; + break; + case "ProviderName" when requireEnrichedProperties: + ThrowIfDuplicate(hasProviderName, typeName, propertyName); + if (reader.TokenType is not JsonTokenType.String) + { + throw new JsonException("Property 'ProviderName' must be a string."); + } - public static void ReadEndObject(ref Utf8JsonReader reader) - { - if (!reader.Read() || reader.TokenType is not JsonTokenType.EndObject) - { - throw new JsonException("Expected end of JSON object."); + providerName = reader.GetString() + ?? throw new JsonException("Property 'ProviderName' must not be null."); + hasProviderName = true; + break; + case "TraceParent" when requireEnrichedProperties: + ThrowIfDuplicate(hasTraceParent, typeName, propertyName); + if (reader.TokenType is not (JsonTokenType.Null or JsonTokenType.String)) + { + throw new JsonException("Property 'TraceParent' must be a string or null."); + } + + traceParent = reader.TokenType is JsonTokenType.Null + ? null + : reader.GetString(); + hasTraceParent = true; + break; + default: + reader.Skip(); + break; + } } + + throw new JsonException($"Unexpected end of {typeName} JSON."); } - private static void ReadRequiredPropertyName(ref Utf8JsonReader reader, string propertyName) + private static void ThrowIfDuplicate(bool hasProperty, string typeName, string? propertyName) { - if (!reader.Read() || reader.TokenType is not JsonTokenType.PropertyName) + if (hasProperty) { - throw new JsonException($"Expected property '{propertyName}'."); - } - - if (!reader.ValueTextEquals(propertyName)) - { - throw new JsonException($"Expected property '{propertyName}', got '{reader.GetString()}'."); + throw new JsonException($"Duplicate {typeName} property '{propertyName}'."); } } + + public readonly record struct Properties( + string EventHubOffset, + long SequenceNumber, + int EventIndex, + DateTimeOffset EnqueuedTime, + string? ProviderName, + string? TraceParent); } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs index c8f5f1b4..f39d9f0e 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamCursorJsonConverter.cs @@ -1,5 +1,6 @@ using System.Text.Json; using System.Text.Json.Serialization; +using Orleans.Streams; namespace Egil.Orleans.Messaging.Streams; @@ -40,52 +41,79 @@ internal sealed class StreamCursorJsonConverter : JsonConverter throw new JsonException($"Expected StreamCursor object, got '{reader.TokenType}'."); } - var streamNamespace = ReadRequiredStringProperty(ref reader, nameof(StreamCursor.StreamNamespace)); - - ReadRequiredPropertyName(ref reader, nameof(StreamCursor.Token)); - if (!reader.Read()) - { - throw new JsonException("Unexpected end of StreamCursor token JSON."); - } - - var token = reader.TokenType is JsonTokenType.Null - ? null - : StreamSequenceTokenJsonConverters.Read(ref reader, options); - + string? streamNamespace = null; + var hasStreamNamespace = false; + StreamSequenceToken? token = null; + var hasToken = false; string? providerName = null; - if (!reader.Read()) - { - throw new JsonException("Unexpected end of StreamCursor JSON."); - } - - if (reader.TokenType is JsonTokenType.PropertyName) + var hasProviderName = false; + while (reader.Read()) { - if (!reader.ValueTextEquals(nameof(StreamCursor.ProviderName))) + if (reader.TokenType is JsonTokenType.EndObject) { - throw new JsonException($"Expected StreamCursor property '{nameof(StreamCursor.ProviderName)}', got '{reader.GetString()}'."); + return new StreamCursor( + hasStreamNamespace + ? streamNamespace! + : throw new JsonException($"Missing StreamCursor property '{nameof(StreamCursor.StreamNamespace)}'."), + hasToken + ? token + : throw new JsonException($"Missing StreamCursor property '{nameof(StreamCursor.Token)}'."), + providerName); } - if (!reader.Read()) + if (reader.TokenType is not JsonTokenType.PropertyName) { - throw new JsonException("Unexpected end of StreamCursor provider name JSON."); + throw new JsonException($"Expected StreamCursor property, got '{reader.TokenType}'."); } - providerName = reader.TokenType is JsonTokenType.Null - ? null - : reader.GetString(); - + var propertyName = reader.GetString(); if (!reader.Read()) { throw new JsonException("Unexpected end of StreamCursor JSON."); } - } - if (reader.TokenType is not JsonTokenType.EndObject) - { - throw new JsonException("Expected end of StreamCursor object."); + switch (propertyName) + { + case nameof(StreamCursor.StreamNamespace): + ThrowIfDuplicate(hasStreamNamespace, nameof(StreamCursor.StreamNamespace)); + if (reader.TokenType is not JsonTokenType.String) + { + throw new JsonException( + $"StreamCursor property '{nameof(StreamCursor.StreamNamespace)}' must be a string."); + } + + streamNamespace = reader.GetString() + ?? throw new JsonException( + $"StreamCursor property '{nameof(StreamCursor.StreamNamespace)}' must not be null."); + hasStreamNamespace = true; + break; + case nameof(StreamCursor.Token): + ThrowIfDuplicate(hasToken, nameof(StreamCursor.Token)); + token = reader.TokenType is JsonTokenType.Null + ? null + : StreamSequenceTokenJsonConverters.Read(ref reader, options); + hasToken = true; + break; + case nameof(StreamCursor.ProviderName): + ThrowIfDuplicate(hasProviderName, nameof(StreamCursor.ProviderName)); + if (reader.TokenType is not (JsonTokenType.Null or JsonTokenType.String)) + { + throw new JsonException( + $"StreamCursor property '{nameof(StreamCursor.ProviderName)}' must be a string or null."); + } + + providerName = reader.TokenType is JsonTokenType.Null + ? null + : reader.GetString(); + hasProviderName = true; + break; + default: + reader.Skip(); + break; + } } - return new StreamCursor(streamNamespace, token, providerName); + throw new JsonException("Unexpected end of StreamCursor JSON."); } /// @@ -111,28 +139,11 @@ public override void Write(Utf8JsonWriter writer, StreamCursor value, JsonSerial writer.WriteEndObject(); } - private static string ReadRequiredStringProperty(ref Utf8JsonReader reader, string propertyName) - { - ReadRequiredPropertyName(ref reader, propertyName); - if (!reader.Read() || reader.TokenType is not JsonTokenType.String) - { - throw new JsonException($"StreamCursor property '{propertyName}' must be a string."); - } - - return reader.GetString() - ?? throw new JsonException($"StreamCursor property '{propertyName}' must not be null."); - } - - private static void ReadRequiredPropertyName(ref Utf8JsonReader reader, string propertyName) + private static void ThrowIfDuplicate(bool hasProperty, string propertyName) { - if (!reader.Read() || reader.TokenType is not JsonTokenType.PropertyName) - { - throw new JsonException($"Expected StreamCursor property '{propertyName}'."); - } - - if (!reader.ValueTextEquals(propertyName)) + if (hasProperty) { - throw new JsonException($"Expected StreamCursor property '{propertyName}', got '{reader.GetString()}'."); + throw new JsonException($"Duplicate StreamCursor property '{propertyName}'."); } } } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverters.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverters.cs index 86834f14..e8ec4884 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverters.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamSequenceTokenJsonConverters.cs @@ -1,4 +1,5 @@ using System.Collections.Immutable; +using System.Text; using System.Text.Json; using System.Text.Json.Serialization; using Orleans.Providers.Streams.Common; @@ -16,6 +17,9 @@ namespace Egil.Orleans.Messaging.Streams; /// token payload directly to the registered . /// Converters should be registered during silo startup before persisted stream /// positions are serialized or deserialized. +/// The envelope reader accepts properties in any order and ignores unknown +/// properties. Registered converters are responsible for providing the same +/// forward-compatible behavior inside their payloads. /// public static class StreamSequenceTokenJsonConverters { @@ -86,27 +90,93 @@ internal static StreamSequenceToken Read(ref Utf8JsonReader reader, JsonSerializ throw new JsonException($"Expected stream sequence token object, got '{reader.TokenType}'."); } - var typeDescriptor = ReadRequiredStringProperty(ref reader, "Kind"); - var registration = registrations.FirstOrDefault(x => x.TypeDescriptor == typeDescriptor); - if (registration is null) - { - throw new JsonException(CreateUnsupportedTokenMessage(typeDescriptor)); - } + Registration? registration = null; + var hasKind = false; + var hasPayload = false; + JsonElement bufferedPayload = default; + StreamSequenceToken? token = null; + var tokenRead = false; - ReadRequiredPropertyName(ref reader, "Payload"); - if (!reader.Read()) + while (reader.Read()) { - throw new JsonException("Unexpected end of stream sequence token payload."); - } + if (reader.TokenType is JsonTokenType.EndObject) + { + if (!hasKind) + { + throw new JsonException("Missing stream sequence token property 'Kind'."); + } + + if (!hasPayload) + { + throw new JsonException("Missing stream sequence token property 'Payload'."); + } + + return tokenRead + ? token! + : ReadBufferedPayload(bufferedPayload, registration!, options); + } - var token = registration.Read(ref reader, options); + if (reader.TokenType is not JsonTokenType.PropertyName) + { + throw new JsonException($"Expected stream sequence token property, got '{reader.TokenType}'."); + } - if (!reader.Read() || reader.TokenType is not JsonTokenType.EndObject) - { - throw new JsonException("Expected end of stream sequence token object."); + var propertyName = reader.GetString(); + if (!reader.Read()) + { + throw new JsonException("Unexpected end of stream sequence token JSON."); + } + + switch (propertyName) + { + case "Kind": + ThrowIfDuplicate(hasKind, "Kind"); + if (reader.TokenType is not JsonTokenType.String) + { + throw new JsonException("Stream sequence token property 'Kind' must be a string."); + } + + var typeDescriptor = reader.GetString(); + if (string.IsNullOrWhiteSpace(typeDescriptor)) + { + throw new JsonException("Stream sequence token property 'Kind' must not be empty."); + } + + registration = registrations.FirstOrDefault(x => x.TypeDescriptor == typeDescriptor); + if (registration is null) + { + throw new JsonException(CreateUnsupportedTokenMessage(typeDescriptor)); + } + + hasKind = true; + break; + case "Payload": + { + ThrowIfDuplicate(hasPayload, "Payload"); + if (registration is null) + { + // JSON properties are unordered, so payload can arrive before + // its discriminator. Buffer only this value until Kind selects + // the registered converter; canonical Kind-first input streams. + using var document = JsonDocument.ParseValue(ref reader); + bufferedPayload = document.RootElement.Clone(); + } + else + { + token = registration.Read(ref reader, options); + tokenRead = true; + } + + hasPayload = true; + break; + } + default: + reader.Skip(); + break; + } } - return token; + throw new JsonException("Unexpected end of stream sequence token JSON."); } internal static NotSupportedException CreateUnsupportedTokenException(string tokenIdentifier) => @@ -135,33 +205,39 @@ private static void Register(Registration registration) } } - private static string ReadRequiredStringProperty(ref Utf8JsonReader reader, string propertyName) + private static StreamSequenceToken ReadBufferedPayload( + JsonElement payload, + Registration registration, + JsonSerializerOptions options) { - ReadRequiredPropertyName(ref reader, propertyName); - if (!reader.Read() || reader.TokenType is not JsonTokenType.String) + var bytes = Encoding.UTF8.GetBytes(payload.GetRawText()); + var payloadReader = new Utf8JsonReader( + bytes, + new JsonReaderOptions + { + AllowTrailingCommas = options.AllowTrailingCommas, + CommentHandling = options.ReadCommentHandling, + MaxDepth = options.MaxDepth + }); + if (!payloadReader.Read()) { - throw new JsonException($"Stream sequence token property '{propertyName}' must be a string."); + throw new JsonException("Unexpected end of stream sequence token payload."); } - var value = reader.GetString(); - if (string.IsNullOrWhiteSpace(value)) + var token = registration.Read(ref payloadReader, options); + if (payloadReader.Read()) { - throw new JsonException($"Stream sequence token property '{propertyName}' must not be empty."); + throw new JsonException("Expected a single stream sequence token payload."); } - return value; + return token; } - private static void ReadRequiredPropertyName(ref Utf8JsonReader reader, string propertyName) + private static void ThrowIfDuplicate(bool hasProperty, string propertyName) { - if (!reader.Read() || reader.TokenType is not JsonTokenType.PropertyName) - { - throw new JsonException($"Expected stream sequence token property '{propertyName}'."); - } - - if (!reader.ValueTextEquals(propertyName)) + if (hasProperty) { - throw new JsonException($"Expected stream sequence token property '{propertyName}', got '{reader.GetString()}'."); + throw new JsonException($"Duplicate stream sequence token property '{propertyName}'."); } } @@ -227,9 +303,9 @@ public override EventSequenceToken Read(ref Utf8JsonReader reader, Type typeToCo throw new JsonException($"Expected {nameof(EventSequenceToken)} object, got '{reader.TokenType}'."); } - var sequenceNumber = ReadRequiredInt64Property(ref reader, "SequenceNumber"); - var eventIndex = ReadRequiredInt32Property(ref reader, "EventIndex"); - ReadEndObject(ref reader); + var (sequenceNumber, eventIndex) = ReadEventSequenceTokenProperties( + ref reader, + nameof(EventSequenceToken)); return new EventSequenceToken(sequenceNumber, eventIndex); } @@ -252,9 +328,9 @@ public override EventSequenceTokenV2 Read(ref Utf8JsonReader reader, Type typeTo throw new JsonException($"Expected {nameof(EventSequenceTokenV2)} object, got '{reader.TokenType}'."); } - var sequenceNumber = ReadRequiredInt64Property(ref reader, "SequenceNumber"); - var eventIndex = ReadRequiredInt32Property(ref reader, "EventIndex"); - ReadEndObject(ref reader); + var (sequenceNumber, eventIndex) = ReadEventSequenceTokenProperties( + ref reader, + nameof(EventSequenceTokenV2)); return new EventSequenceTokenV2(sequenceNumber, eventIndex); } @@ -268,33 +344,67 @@ public override void Write(Utf8JsonWriter writer, EventSequenceTokenV2 value, Js } } - private static long ReadRequiredInt64Property(ref Utf8JsonReader reader, string propertyName) + private static (long SequenceNumber, int EventIndex) ReadEventSequenceTokenProperties( + ref Utf8JsonReader reader, + string typeName) { - ReadRequiredPropertyName(ref reader, propertyName); - if (!reader.Read() || reader.TokenType is not JsonTokenType.Number) - { - throw new JsonException($"Property '{propertyName}' must be a number."); - } - - return reader.GetInt64(); - } + long sequenceNumber = 0; + var hasSequenceNumber = false; + var eventIndex = 0; + var hasEventIndex = false; - private static int ReadRequiredInt32Property(ref Utf8JsonReader reader, string propertyName) - { - ReadRequiredPropertyName(ref reader, propertyName); - if (!reader.Read() || reader.TokenType is not JsonTokenType.Number) + while (reader.Read()) { - throw new JsonException($"Property '{propertyName}' must be a number."); - } + if (reader.TokenType is JsonTokenType.EndObject) + { + return ( + hasSequenceNumber + ? sequenceNumber + : throw new JsonException($"Missing {typeName} property 'SequenceNumber'."), + hasEventIndex + ? eventIndex + : throw new JsonException($"Missing {typeName} property 'EventIndex'.")); + } - return reader.GetInt32(); - } + if (reader.TokenType is not JsonTokenType.PropertyName) + { + throw new JsonException($"Expected {typeName} property, got '{reader.TokenType}'."); + } - private static void ReadEndObject(ref Utf8JsonReader reader) - { - if (!reader.Read() || reader.TokenType is not JsonTokenType.EndObject) - { - throw new JsonException("Expected end of JSON object."); + var propertyName = reader.GetString(); + if (!reader.Read()) + { + throw new JsonException($"Unexpected end of {typeName} JSON."); + } + + switch (propertyName) + { + case "SequenceNumber": + ThrowIfDuplicate(hasSequenceNumber, "SequenceNumber"); + if (reader.TokenType is not JsonTokenType.Number + || !reader.TryGetInt64(out sequenceNumber)) + { + throw new JsonException("Property 'SequenceNumber' must be a 64-bit integer."); + } + + hasSequenceNumber = true; + break; + case "EventIndex": + ThrowIfDuplicate(hasEventIndex, "EventIndex"); + if (reader.TokenType is not JsonTokenType.Number + || !reader.TryGetInt32(out eventIndex)) + { + throw new JsonException("Property 'EventIndex' must be a 32-bit integer."); + } + + hasEventIndex = true; + break; + default: + reader.Skip(); + break; + } } + + throw new JsonException($"Unexpected end of {typeName} JSON."); } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubSequenceTokenJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubSequenceTokenJsonConverterTests.cs index f061db9b..8f934b3f 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubSequenceTokenJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubSequenceTokenJsonConverterTests.cs @@ -1,10 +1,55 @@ using System.Text.Json; using Egil.Orleans.Messaging.Tracking; +using Orleans.Streaming.EventHubs; namespace Egil.Orleans.Messaging.Streams.EventHubs.Tests.EventHubs; public sealed class EnrichedEventHubSequenceTokenJsonConverterTests { + public static TheoryData InvalidEventHubPayloads => new() + { + { + "orleans.event-hubs.sequence-token", + """{"SequenceNumber":42,"EventIndex":2}""", + "EventHubOffset" + }, + { + "orleans.event-hubs.sequence-token", + """{"EventHubOffset":"12345","EventIndex":2}""", + "SequenceNumber" + }, + { + "orleans.event-hubs.sequence-token", + """{"EventHubOffset":"12345","SequenceNumber":42}""", + "EventIndex" + }, + { + "orleans.event-hubs.sequence-token", + """{"EventHubOffset":1,"SequenceNumber":42,"EventIndex":2}""", + "must be a string" + }, + { + EnrichedEventHubSequenceToken.TypeAlias, + """{"EventHubOffset":"12345","SequenceNumber":42,"EventIndex":2,"ProviderName":"event-hubs","TraceParent":null}""", + "EnqueuedTime" + }, + { + EnrichedEventHubSequenceToken.TypeAlias, + """{"EventHubOffset":"12345","SequenceNumber":42,"EventIndex":2,"EnqueuedTime":"2026-05-26T10:15:00+00:00","TraceParent":null}""", + "ProviderName" + }, + { + EnrichedEventHubSequenceToken.TypeAlias, + """{"EventHubOffset":"12345","SequenceNumber":42,"EventIndex":2,"EnqueuedTime":"2026-05-26T10:15:00+00:00","ProviderName":"event-hubs"}""", + "TraceParent" + }, + { + "orleans.event-hubs.sequence-token", + """{"EventHubOffset":"12345","EventHubOffset":"other","SequenceNumber":42,"EventIndex":2}""", + "Duplicate" + } + }; + [Fact] public void StreamCursor_round_trips_enriched_event_hub_token() { @@ -43,6 +88,85 @@ public void MessageTracker_round_trips_enriched_event_hub_token() AssertEnrichedToken(token, Assert.IsType(latest)); } + [Theory] + [InlineData("orleans.event-hubs.sequence-token", typeof(EventHubSequenceToken))] + [InlineData("orleans.event-hubs.sequence-token-v2", typeof(EventHubSequenceTokenV2))] + [InlineData(EnrichedEventHubSequenceToken.TypeAlias, typeof(EnrichedEventHubSequenceToken))] + public void StreamCursor_deserializes_reordered_event_hub_payload_with_unknown_properties( + string kind, + Type expectedType) + { + RegisterConverters(); + var json = $$""" + { + "StreamNamespace": "orders", + "Token": { + "Payload": { + "FuturePayload": { + "Nested": [1, { "Enabled": true }] + }, + "TraceParent": "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01", + "ProviderName": "event-hubs", + "EnqueuedTime": "2026-05-26T10:15:00+00:00", + "EventIndex": 2, + "SequenceNumber": 42, + "EventHubOffset": "12345" + }, + "Kind": "{{kind}}" + } + } + """; + + var cursor = JsonSerializer.Deserialize(json); + + Assert.NotNull(cursor); + Assert.IsType(expectedType, cursor.Token); + Assert.Equal(42, cursor.Token.SequenceNumber); + Assert.Equal(2, cursor.Token.EventIndex); + switch (cursor.Token) + { + case EnrichedEventHubSequenceToken enriched: + Assert.Equal("12345", enriched.EventHubOffset); + Assert.Equal( + new DateTimeOffset(2026, 5, 26, 10, 15, 0, TimeSpan.Zero), + enriched.EnqueuedTime); + Assert.Equal("event-hubs", enriched.ProviderName); + Assert.Equal( + "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01", + enriched.TraceParent); + break; + case EventHubSequenceTokenV2 tokenV2: + Assert.Equal("12345", tokenV2.EventHubOffset); + break; + case EventHubSequenceToken token: + Assert.Equal("12345", token.EventHubOffset); + break; + } + } + + [Theory] + [MemberData(nameof(InvalidEventHubPayloads))] + public void StreamCursor_rejects_missing_invalid_or_duplicate_event_hub_properties( + string kind, + string payload, + string expectedMessage) + { + RegisterConverters(); + var json = $$""" + { + "StreamNamespace": "orders", + "Token": { + "Kind": "{{kind}}", + "Payload": {{payload}} + } + } + """; + + var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); + + Assert.Contains(expectedMessage, exception.Message); + } + private static EnrichedEventHubSequenceToken CreateToken() => new( "12345", @@ -63,4 +187,17 @@ private static void AssertEnrichedToken( Assert.Equal(expected.ProviderName, actual.ProviderName); Assert.Equal(expected.TraceParent, actual.TraceParent); } + + private static void RegisterConverters() + { + StreamSequenceTokenJsonConverters.Register( + "orleans.event-hubs.sequence-token", + new EventHubSequenceTokenJsonConverter()); + StreamSequenceTokenJsonConverters.Register( + "orleans.event-hubs.sequence-token-v2", + new EventHubSequenceTokenV2JsonConverter()); + StreamSequenceTokenJsonConverters.Register( + EnrichedEventHubSequenceToken.TypeAlias, + new EnrichedEventHubSequenceTokenJsonConverter()); + } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs index d5d9d934..c2d2537c 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamCursorJsonConverterTests.cs @@ -7,6 +7,49 @@ namespace Egil.Orleans.Messaging.Tests.Streams; public sealed class StreamCursorJsonConverterTests { + public static TheoryData InvalidCursorJson => new() + { + { """{"Token":null}""", "StreamNamespace" }, + { """{"StreamNamespace":"orders"}""", "Token" }, + { + """{"StreamNamespace":"orders","Token":{"Payload":{"SequenceNumber":1,"EventIndex":0}}}""", + "Kind" + }, + { + """{"StreamNamespace":"orders","Token":{"Kind":"event-sequence"}}""", + "Payload" + }, + { + """{"StreamNamespace":"orders","Token":{"Kind":"event-sequence","Payload":{"EventIndex":0}}}""", + "SequenceNumber" + }, + { + """{"StreamNamespace":"orders","Token":{"Kind":"event-sequence","Payload":{"SequenceNumber":1}}}""", + "EventIndex" + }, + { """{"StreamNamespace":1,"Token":null}""", "must be a string" }, + { + """{"StreamNamespace":"orders","Token":{"Kind":1,"Payload":{"SequenceNumber":1,"EventIndex":0}}}""", + "must be a string" + }, + { + """{"StreamNamespace":"orders","Token":{"Kind":"event-sequence","Payload":{"SequenceNumber":"1","EventIndex":0}}}""", + "must be a 64-bit integer" + }, + { + """{"StreamNamespace":"orders","StreamNamespace":"other","Token":null}""", + "Duplicate" + }, + { + """{"StreamNamespace":"orders","Token":{"Kind":"event-sequence","Kind":"event-sequence","Payload":{"SequenceNumber":1,"EventIndex":0}}}""", + "Duplicate" + }, + { + """{"StreamNamespace":"orders","Token":{"Kind":"event-sequence","Payload":{"SequenceNumber":1,"SequenceNumber":2,"EventIndex":0}}}""", + "Duplicate" + } + }; + [Fact] public void StreamCursor_is_decorated_with_stream_cursor_json_converter() { @@ -54,6 +97,49 @@ public void JsonSerializer_round_trips_stream_cursor_with_event_sequence_token_v Assert.Equal(cursor, roundTripped); } + [Fact] + public void JsonSerializer_deserializes_reordered_cursor_and_token_with_unknown_properties() + { + var json = """ + { + "FutureCursor": { + "Nested": [1, { "Enabled": true }] + }, + "ProviderName": "provider-a", + "Token": { + "Payload": { + "FuturePayload": { + "Nested": ["ignored"] + }, + "EventIndex": 1, + "SequenceNumber": 7 + }, + "FutureEnvelope": [1, 2, 3], + "Kind": "event-sequence" + }, + "StreamNamespace": "orders" + } + """; + + var cursor = JsonSerializer.Deserialize(json); + + Assert.NotNull(cursor); + Assert.Equal("orders", cursor.StreamNamespace); + Assert.Equal("provider-a", cursor.ProviderName); + Assert.Equal(new EventSequenceToken(7, 1), cursor.Token); + } + + [Theory] + [MemberData(nameof(InvalidCursorJson))] + public void JsonSerializer_rejects_missing_invalid_or_duplicate_known_properties( + string json, + string expectedMessage) + { + var exception = Assert.Throws(() => JsonSerializer.Deserialize(json)); + + Assert.Contains(expectedMessage, exception.Message); + } + [Fact] public void JsonSerializer_throws_for_stream_cursor_payload_with_unknown_token_kind() { From 18d25f52d2684d635d0619da243415a5bf41c109 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 17:20:07 +0000 Subject: [PATCH 68/82] fix(om): stabilize message tracker JSON schema Pin every private tracker model property to its canonical wire name so serializer naming policies cannot change persisted state. Require both root collections so mismatched or malformed payloads fail instead of silently clearing deduplication state. --- Egil.Orleans.Messaging/api-design.md | 6 ++ .../Tracking/MessageTrackerJsonConverter.cs | 18 ++++++ .../MessageTrackerJsonConverterTests.cs | 61 +++++++++++++++++++ 3 files changed, 85 insertions(+) diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index a7d21992..8b247bc2 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -1706,6 +1706,12 @@ Orleans token converters and provider packages register their own unknown token types fail loudly instead of being silently downcast to an ancestor token shape. +`MessageTrackerJsonConverter` pins every private model property to its +PascalCase wire name, independent of ambient STJ naming policies. Its +`Streams` and `Outboxes` roots are required so a naming mismatch or malformed +payload fails instead of silently producing empty deduplication state; +explicit `null` roots remain valid empty collections. + **Why custom converters (not `[JsonInclude]` on private fields):** `Outbox` and `MessageTracker` are sealed classes with private diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs index 829b0248..a5a88d9c 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Tracking/MessageTrackerJsonConverter.cs @@ -20,6 +20,11 @@ namespace Egil.Orleans.Messaging.Tracking; /// the exact wire format. /// /// +/// Wire property names are pinned on the private models so ambient naming +/// policies cannot change persisted state. Both root arrays are required; +/// explicit null is still accepted as an empty collection. +/// +/// /// Registered on via [JsonConverter]. /// STJ discovers the attribute automatically - no user-side /// configuration needed. @@ -117,21 +122,34 @@ private static GrainIdJsonModel ToJsonModel(GrainId grainId) => new(grainId.Type.ToString()!, grainId.Key.ToString()!); private sealed record MessageTrackerJsonModel( + [property: JsonPropertyName("Streams")] + [property: JsonRequired] StreamEntryJsonModel[]? Streams, + [property: JsonPropertyName("Outboxes")] + [property: JsonRequired] OutboxEntryJsonModel[]? Outboxes); private sealed record StreamEntryJsonModel( + [property: JsonPropertyName("LastPosition")] StreamCursor LastPosition, + [property: JsonPropertyName("Received")] DateTimeOffset Received); private sealed record OutboxEntryJsonModel( + [property: JsonPropertyName("Sender")] GrainIdJsonModel Sender, + [property: JsonPropertyName("Epoch")] DateTimeOffset Epoch, + [property: JsonPropertyName("LastSequenceNumber")] long LastSequenceNumber, + [property: JsonPropertyName("Received")] DateTimeOffset Received, + [property: JsonPropertyName("LastTimestamp")] DateTimeOffset? LastTimestamp = null); private sealed record GrainIdJsonModel( + [property: JsonPropertyName("Type")] string Type, + [property: JsonPropertyName("Key")] string Key); } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs index e4b5582e..c9472110 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerJsonConverterTests.cs @@ -7,6 +7,13 @@ namespace Egil.Orleans.Messaging.Tests.Tracking; public sealed class MessageTrackerJsonConverterTests { + public static TheoryData MissingRootArraysJson => new() + { + """{"Streams":[]}""", + """{"Outboxes":[]}""", + """{}""" + }; + public static TheoryData SupportedStreamTokens => new() { new EventSequenceToken(7, 1), @@ -81,6 +88,42 @@ public void JsonSerializer_deserializes_null_stream_and_outbox_arrays_as_empty_t Assert.Null(roundTripped.LatestOutbox(GrainId.Create("test/sender", "one"))); } + [Fact] + public void JsonSerializer_round_trips_full_tracker_across_property_naming_policies() + { + var tracker = CreateFullTracker(); + var camelCaseOptions = new JsonSerializerOptions + { + PropertyNamingPolicy = JsonNamingPolicy.CamelCase + }; + + var camelCaseJson = JsonSerializer.Serialize(tracker, camelCaseOptions); + var fromCamelCase = JsonSerializer.Deserialize(camelCaseJson); + var defaultJson = JsonSerializer.Serialize(tracker); + var fromDefault = JsonSerializer.Deserialize(defaultJson, camelCaseOptions); + + Assert.Contains("\"Streams\"", camelCaseJson); + Assert.Contains("\"Outboxes\"", camelCaseJson); + Assert.Contains("\"LastPosition\"", camelCaseJson); + Assert.Contains("\"Received\"", camelCaseJson); + Assert.Contains("\"Sender\"", camelCaseJson); + Assert.Contains("\"Epoch\"", camelCaseJson); + Assert.Contains("\"LastSequenceNumber\"", camelCaseJson); + Assert.Contains("\"LastTimestamp\"", camelCaseJson); + Assert.Contains("\"Type\"", camelCaseJson); + Assert.Contains("\"Key\"", camelCaseJson); + Assert.DoesNotContain("\"streams\"", camelCaseJson); + Assert.Equal(tracker, fromCamelCase); + Assert.Equal(tracker, fromDefault); + } + + [Theory] + [MemberData(nameof(MissingRootArraysJson))] + public void JsonSerializer_throws_when_required_root_array_is_missing(string json) + { + Assert.Throws(() => JsonSerializer.Deserialize(json)); + } + [Fact] public void JsonSerializer_deserializes_tracked_stream_without_sequence_token() { @@ -135,6 +178,24 @@ public void JsonSerializer_throws_for_unknown_stream_token_kind() Assert.Throws(() => JsonSerializer.Deserialize(json)); } + private static MessageTracker CreateFullTracker() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(now)); + tracker.ProcessMessage( + new StreamCursor("orders", new EventSequenceToken(7, 1), "provider-a"), + out tracker); + tracker.ProcessMessage( + new OutboxSequenceToken( + 42, + GrainId.Create("test/sender", "one"), + now, + now), + out tracker); + return tracker; + } + private sealed class UnsupportedSequenceToken(long sequenceNumber, int eventIndex) : StreamSequenceToken { public override long SequenceNumber { get; protected set; } = sequenceNumber; From 679befc5c5c5711df43e82f81c90a0ff7a49ae59 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 17:31:02 +0000 Subject: [PATCH 69/82] build(om): generate package release notes Generate plain-text release notes from Messaging project commits and inject them into all three NuGet packages during Release packing. --- .../scripts/generate-release-notes.ps1 | 173 ++++++++++++++++++ .../src/Directory.Build.targets | 24 +++ 2 files changed, 197 insertions(+) create mode 100644 Egil.Orleans.Messaging/scripts/generate-release-notes.ps1 create mode 100644 Egil.Orleans.Messaging/src/Directory.Build.targets diff --git a/Egil.Orleans.Messaging/scripts/generate-release-notes.ps1 b/Egil.Orleans.Messaging/scripts/generate-release-notes.ps1 new file mode 100644 index 00000000..0d1aa907 --- /dev/null +++ b/Egil.Orleans.Messaging/scripts/generate-release-notes.ps1 @@ -0,0 +1,173 @@ +<# +.SYNOPSIS + Generates plain-text release notes from conventional commits since the last tag. + +.DESCRIPTION + Finds the latest git tag matching the given prefix, collects commits since + that tag (filtered to the project directory), parses conventional commit + messages (subject and body), and outputs categorised plain-text release notes + suitable for NuGet PackageReleaseNotes (which does not support markdown). + + Commit body text is included as indented detail under each entry to provide + richer context in the changelog. BREAKING CHANGE footers in the body are + detected and promoted to the breaking changes section per the Conventional + Commits specification. Git trailers (Co-authored-by, Signed-off-by, etc.) + are stripped from the body before inclusion. + + Individual commits can be excluded by adding [skip notes] anywhere in the + subject or body (useful for build/infra/tooling changes that carry a feat + or fix type but are not relevant to package consumers). + +.PARAMETER TagPrefix + Tag name prefix used to find the latest release tag. + Default: "egil-orleans-messaging/v" + +.PARAMETER ProjectPath + Project directory path relative to the repository root. + Only commits touching files under this path are included. + Default: "Egil.Orleans.Messaging/" + +.EXAMPLE + ./scripts/generate-release-notes.ps1 + ./scripts/generate-release-notes.ps1 -TagPrefix "my-project/v" -ProjectPath "MyProject/" +#> +[CmdletBinding()] +param( + [string]$TagPrefix = "egil-orleans-messaging/v", + [string]$ProjectPath = "Egil.Orleans.Messaging/" +) + +$ErrorActionPreference = 'Stop' + +# Verify we are inside a git repository. +$gitDir = git rev-parse --git-dir 2>&1 +if ($LASTEXITCODE -ne 0) { + # Not a git repo – silently produce no output so the build is not broken. + exit 0 +} + +# Find the latest tag that matches the prefix. +$latestTag = git tag -l "${TagPrefix}*" --sort=-v:refname | Select-Object -First 1 + +if ($latestTag) { + $range = "${latestTag}..HEAD" +} +else { + $range = "HEAD" +} + +# Collect commit subjects and bodies touching the project directory. +# A unique separator delimits each commit so subject and body can be parsed. +# --no-merges skips merge commits (whose messages are noise like "Merge PR #N") +# but still traverses merged branches so feature/fix commits are included. +$commitSeparator = '---commit-8a3f9b2e---' +# Anchor the path at the repository root so callers can invoke the script from any subdirectory. +$projectPathspec = ":(top)$ProjectPath" +$commitArgs = @('log', $range, "--format=${commitSeparator}%n%s%n%b", '--no-merges', '--', $projectPathspec) +$raw = (& git @commitArgs) -join "`n" + +if (-not $raw -or $raw.Trim().Length -eq 0) { + # No commits since last tag – nothing to report. + exit 0 +} + +$commitBlocks = $raw -split [regex]::Escape($commitSeparator) | Where-Object { $_.Trim() } + +if (-not $commitBlocks -or @($commitBlocks).Count -eq 0) { + exit 0 +} + +# Regex to strip common git trailers from the body. +$trailerPattern = '(?m)^(Co-authored-by|Signed-off-by|Reviewed-by|Acked-by|Tested-by|Reported-by|Helped-by|Cc|Change-Id):\s.*$' + +# Parse conventional commit subjects into categorised lists. +# Each entry is a hashtable with Description (subject text) and Body (cleaned body text). +$breaking = [System.Collections.Generic.List[hashtable]]::new() +$features = [System.Collections.Generic.List[hashtable]]::new() +$fixes = [System.Collections.Generic.List[hashtable]]::new() +$perf = [System.Collections.Generic.List[hashtable]]::new() + +# Types that are not interesting for end-user release notes. +$skipTypes = @('release', 'build', 'ci', 'chore', 'docs', 'style', 'refactor', 'test') + +foreach ($block in $commitBlocks) { + $lines = $block.Trim() -split "`n" + $subject = $lines[0].Trim() + $body = '' + if ($lines.Count -gt 1) { + $body = ($lines[1..($lines.Count - 1)] -join "`n").Trim() + } + + # Allow individual commits to opt out of release notes. + $fullMessage = "$subject`n$body" + if ($fullMessage -match '\[skip notes\]') { + continue + } + + # Match: type(scope)!: description or type!: description or type: description + if ($subject -match '^(?\w+)(?:\([^)]*\))?(?!)?:\s*(?.+)$') { + $type = $Matches['type'].ToLowerInvariant() + $desc = $Matches['desc'].Trim() + $isBreaking = $Matches['bang'] -eq '!' + + # Detect BREAKING CHANGE footer in body (per Conventional Commits spec). + if ($body -match '(?m)^BREAKING[- ]CHANGE:\s*(?.+)') { + $isBreaking = $true + } + + # Clean body: strip BREAKING CHANGE footers and git trailers. + $cleanBody = $body -replace '(?m)^BREAKING[- ]CHANGE:\s*.*$', '' + $cleanBody = $cleanBody -replace $trailerPattern, '' + # Collapse runs of blank lines and trim. + $cleanBody = ($cleanBody -replace '(\r?\n){3,}', "`n`n").Trim() + + $entry = @{ Description = $desc; Body = $cleanBody } + + if ($isBreaking) { + $breaking.Add($entry) + } + elseif ($type -eq 'feat') { + $features.Add($entry) + } + elseif ($type -eq 'fix') { + $fixes.Add($entry) + } + elseif ($type -eq 'perf') { + $perf.Add($entry) + } + elseif ($type -in $skipTypes) { + continue + } + # Unknown types are silently skipped to keep notes clean. + } + # Non-conventional commits are silently skipped. +} + +# Build plain-text output (no markdown – NuGet does not render it). +$sections = [System.Collections.Generic.List[string]]::new() + +function Add-Section([string]$heading, [System.Collections.Generic.List[hashtable]]$items) { + if ($items.Count -eq 0) { return } + $sections.Add($heading) + foreach ($item in $items) { + $sections.Add("- $($item.Description)") + if ($item.Body) { + $bodyLines = $item.Body -split "`n" + foreach ($line in $bodyLines) { + $sections.Add(" $line") + } + } + } +} + +Add-Section "BREAKING CHANGES:" $breaking +Add-Section "New Features:" $features +Add-Section "Bug Fixes:" $fixes +Add-Section "Performance:" $perf + +if ($sections.Count -eq 0) { + exit 0 +} + +# Write to stdout – MSBuild ConsoleToMSBuild will capture this. +$sections -join "`n" diff --git a/Egil.Orleans.Messaging/src/Directory.Build.targets b/Egil.Orleans.Messaging/src/Directory.Build.targets new file mode 100644 index 00000000..096717e9 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Directory.Build.targets @@ -0,0 +1,24 @@ + + + + + <_ProjectRoot>$([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)..')) + + + + + + + + + + + + + From 74b6de70a424fa9a712db197adfe0e462e96eab8 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 17:31:30 +0000 Subject: [PATCH 70/82] fix(ci): enable Orleans messaging release preparation [skip notes] Add Egil.Orleans.Messaging to the manual release workflow so its stable release branch can be prepared consistently with sibling projects. --- .github/workflows/prepare-release.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/prepare-release.yml b/.github/workflows/prepare-release.yml index c3a3c237..f9989e12 100644 --- a/.github/workflows/prepare-release.yml +++ b/.github/workflows/prepare-release.yml @@ -9,6 +9,7 @@ on: type: choice required: true options: + - Egil.Orleans.Messaging - Egil.Orleans.Testing - Egil.StronglyTypedPrimitives - Egil.SystemTextJson.Migration From cdfd52c2021e856729f80cf3174d906122b41cb6 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 17:46:51 +0000 Subject: [PATCH 71/82] fix(om): decouple Azure storage provider Keep only Azure.Core as a runtime dependency of the Azure Storage state-manager companion. Move Table and Blob SDK references to tests and leave Orleans provider selection to consumers, avoiding unwanted provider and version coupling. --- Egil.Orleans.Messaging/Directory.Packages.props | 1 - Egil.Orleans.Messaging/README.md | 8 +++++--- .../Egil.Orleans.Messaging.State.AzureStorage.csproj | 3 --- ...Egil.Orleans.Messaging.State.AzureStorage.Tests.csproj | 2 ++ 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Egil.Orleans.Messaging/Directory.Packages.props b/Egil.Orleans.Messaging/Directory.Packages.props index 4f20e325..7e946421 100644 --- a/Egil.Orleans.Messaging/Directory.Packages.props +++ b/Egil.Orleans.Messaging/Directory.Packages.props @@ -19,7 +19,6 @@ - diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index bb69e31d..ca3adf97 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -49,9 +49,11 @@ Register the default state manager factory on the silo: siloBuilder.AddDefaultStateManager("state"); ``` -For Orleans Azure Table or Blob grain storage, install -`Egil.Orleans.Messaging.State.AzureStorage` and register the Azure-aware -factory instead: +For Orleans Azure Table or Blob grain storage, install and configure the +Orleans storage provider separately. The Messaging companion works through +`IPersistentState` and Azure SDK exceptions; it does not select or install +the underlying provider. Install `Egil.Orleans.Messaging.State.AzureStorage` +and register the Azure-aware factory instead: ```csharp siloBuilder.AddAzureStorageStateManager("state"); diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj index f3623fcd..a9e6f7df 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.State.AzureStorage/Egil.Orleans.Messaging.State.AzureStorage.csproj @@ -34,9 +34,6 @@ runtime; build; native; contentfiles; analyzers; buildtransitive - - - diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/Egil.Orleans.Messaging.State.AzureStorage.Tests.csproj b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/Egil.Orleans.Messaging.State.AzureStorage.Tests.csproj index b7a9e8d6..8d2a03c3 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/Egil.Orleans.Messaging.State.AzureStorage.Tests.csproj +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.State.AzureStorage.Tests/Egil.Orleans.Messaging.State.AzureStorage.Tests.csproj @@ -23,6 +23,8 @@ + + all runtime; build; native; contentfiles; analyzers; buildtransitive From 000d5845b4c0f497ea62d14659fd4fc41a50c378 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 18:15:18 +0000 Subject: [PATCH 72/82] fix(om): expose provider state as nullable Annotate IStateManager.State for storage providers that can expose null after reads or clears, and handle null recovery values without dereferencing them. Preserve Orleans activated defaults while documenting and testing both nullable and non-null missing-record shapes. --- Egil.Orleans.Messaging/README.md | 40 ++++++--- Egil.Orleans.Messaging/api-design.md | 21 +++-- .../Outboxes/OutboxProcessorExtensions.cs | 3 +- .../State/IStateManager.cs | 9 +- .../State/StateManager.cs | 45 ++++++---- .../State/StateManagerTests.cs | 85 ++++++++++++++++++- 6 files changed, 157 insertions(+), 46 deletions(-) diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index ca3adf97..ef0a8c3f 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -77,6 +77,9 @@ public sealed class OrderGrain( { private IStateManager state = default!; + private OrderState CurrentState => + state.State ?? throw new InvalidOperationException("Order state is not initialized."); + public override Task OnActivateAsync(CancellationToken cancellationToken) { state = this.RegisterStateManager("state", storage); @@ -85,12 +88,21 @@ public sealed class OrderGrain( public async Task RenameAsync(string name) { - await state.WriteAsync(state.State with { Name = name }); + await state.WriteAsync(CurrentState with { Name = name }); } } ``` -State types must be reference types and implement `IEquatable`. For non-trivial state graphs, inherit from `VersionedState` so the recovery path compares a library-stamped version rather than relying on structural collection equality. +`State` is nullable because persistent storage can expose no value, including +after `ClearAsync`. Initialize missing state or guard it before dereferencing, +as `CurrentState` does above. Later snippets use `state.State!` only to keep +their focus narrow and assume the surrounding grain has already established +that invariant. + +State types must be reference types and implement `IEquatable`. For +non-trivial state graphs, inherit from `VersionedState` so the recovery path +compares a library-stamped version rather than relying on structural +collection equality. ## Outbox @@ -108,9 +120,9 @@ public sealed record OrderState : VersionedState public async Task SubmitAsync() { - var next = state.State with + var next = state.State! with { - Outbox = state.State.Outbox.Add(new OrderSubmitted()) + Outbox = state.State!.Outbox.Add(new OrderSubmitted()) }; await state.WriteAsync(next); @@ -135,12 +147,12 @@ public sealed class OrderGrain : Grain, IOutboxGrain { outboxProcessor = this.RegisterOutboxProcessor(new OutboxProcessorOptions> { - PendingItems = () => [.. state.State.Outbox], + PendingItems = () => [.. state.State!.Outbox], AcknowledgePostedAsync = async (items, ct) => { - await state.WriteAsync(state.State with + await state.WriteAsync(state.State! with { - Outbox = state.State.Outbox.RemoveRange(items.Select(item => item.Token)) + Outbox = state.State!.Outbox.RemoveRange(items.Select(item => item.Token)) }); }, ReconcileFailedAsync = (_, _) => ValueTask.CompletedTask, @@ -255,12 +267,12 @@ outboxProcessor = this.RegisterOutboxProcessor(options) `MessageTracker` accepts a message only when its stream token, stream cursor, or outbox token advances the stored high-water mark: ```csharp -if (!state.State.Tracker.ProcessMessage("prices", token, out var tracker)) +if (!state.State!.Tracker.ProcessMessage("prices", token, out var tracker)) { return; } -await state.WriteAsync(state.State with { Tracker = tracker }); +await state.WriteAsync(state.State! with { Tracker = tracker }); ``` Use `LatestStreamSequenceToken("prices")` when all you need is the previous @@ -275,18 +287,18 @@ The tracker can also evict old sender or stream entries when your retention poli Use `StreamManager` to configure stream subscriptions from `OnActivateAsync`. Pass a tracker snapshot when you want persisted resume tokens, or omit it when the grain does not track stream positions: ```csharp -streamManager = this.RegisterStreamManager(state.State.Tracker) +streamManager = this.RegisterStreamManager(state.State!.Tracker) .ConfigureExplicitSubscription( "StreamProvider", "prices", async (message, cursor) => { - if (!state.State.Tracker.ProcessMessage(cursor, out var tracker)) + if (!state.State!.Tracker.ProcessMessage(cursor, out var tracker)) { return; } - await state.WriteAsync(state.State with { Tracker = tracker }); + await state.WriteAsync(state.State! with { Tracker = tracker }); }); await streamManager.EnsureExplicitSubscriptionsAsync(cancellationToken); @@ -297,7 +309,7 @@ identity. Use the `StreamId` overload for an explicit stream keyed by another application id: ```csharp -streamManager = this.RegisterStreamManager(state.State.Tracker) +streamManager = this.RegisterStreamManager(state.State!.Tracker) .ConfigureExplicitSubscription( "StreamProvider", StreamId.Create("prices", customerId), @@ -309,7 +321,7 @@ the previous token when a tracker snapshot is supplied. Opt out when a subscription should attach without a resume token: ```csharp -streamManager = this.RegisterStreamManager(state.State.Tracker) +streamManager = this.RegisterStreamManager(state.State!.Tracker) .ConfigureExplicitSubscription( "StreamProvider", "prices", diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index 8b247bc2..a06b3ef8 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -117,7 +117,7 @@ stale `storage.State` after a failed write. ```csharp public interface IStateManager where T : class, IEquatable { - T State { get; } + T? State { get; } Task ReadAsync(); Task WriteAsync(T newState); Task ClearAsync(); @@ -154,6 +154,11 @@ point of view: by the time `OnActivateAsync` runs, `State` already reflects what is durably stored. Grain code does **not** call `ReadAsync` during activation. +`State` is nullable because storage can expose no value, including after a +successful `ClearAsync`. The manager cannot construct an arbitrary `T` as a +fallback. Grain code must initialize missing state or guard it before +dereferencing. + This is achieved by composition rather than reimplementation: the default `StateManager` wraps an `IPersistentState` facet that Orleans hydrates during the `SetupState` grain-lifecycle stage. The @@ -177,7 +182,7 @@ otherwise it falls back to `T.Equals(...)`. internal sealed class StateManager(IPersistentState storage) : IStateManager where T : class, IEquatable { - public T State => storage.State; + public T? State => storage.State; public Task ReadAsync() => storage.ReadStateAsync(); @@ -214,10 +219,11 @@ internal sealed class StateManager(IPersistentState storage) : IStateManag } } - private static bool IsEquivalent(T persisted, T attempted) => - persisted is VersionedState pv && attempted is VersionedState av + private static bool IsEquivalent(T? persisted, T attempted) => + persisted is not null + && (persisted is VersionedState pv && attempted is VersionedState av ? pv.Version.Equals(av.Version) - : persisted.Equals(attempted); + : persisted.Equals(attempted)); } ``` @@ -228,6 +234,7 @@ Behaviour matrix: | Success | `State == newState`, returns | | Timeout, write actually persisted | `State == newState`, returns (silent recovery) | | Timeout, write did not persist | `State == server's value`, throws original ex | +| Recovery read exposes `null` | `State == null`, throws original ex | | 5xx / transient | Same as timeout — re-read decides | | `InconsistentStateException` | `State == server's value`, **always rethrows** | | Re-read also fails (double failure)| `storage.State` reverted, throws original ex | @@ -314,8 +321,8 @@ The wrapper was debated — recovery logic is stateless, could be an extension method on `IPersistentState`. But the wrapper does a fourth thing extensions cannot: **it hides `IPersistentState.State`**. -`IStateManager.State` exposes only the **committed** snapshot — the -value after the last successful `WriteAsync`. During an in-flight write, +`IStateManager.State` exposes only the **committed** snapshot, or +`null` when the storage provider exposes no value. During an in-flight write, `IPersistentState.State` already holds the uncommitted value. Read methods marked `[AlwaysInterleave]` that access `storage.State` directly could observe uncommitted state — and if the write fails, they returned diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs index fb071e9b..02b64e46 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Outboxes/OutboxProcessorExtensions.cs @@ -38,7 +38,8 @@ public static class OutboxProcessorExtensions /// { /// outboxProcessor = this.RegisterOutboxProcessor(new OutboxProcessorOptions<IMyEvent> /// { - /// PendingItems = () => stateManager.State.Outbox + /// PendingItems = () => (stateManager.State + /// ?? throw new InvalidOperationException("State is not initialized.")).Outbox /// .Select(e => e.Message).ToImmutableArray(), /// AcknowledgePostedAsync = async (items, ct) => /// { diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs index b83de05e..d7614f24 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs @@ -77,7 +77,9 @@ public interface IStateManager where T : class, IEquatable { /// - /// Gets the last successfully committed state snapshot. + /// Gets the last successfully committed state snapshot, or + /// when the underlying persistent state contains + /// no value. /// /// /// Safe to read from [AlwaysInterleave] methods — returns only @@ -85,7 +87,7 @@ public interface IStateManager /// committed-state fence that justifies the wrapper over raw /// . /// - T State { get; } + T? State { get; } /// /// Re-reads state from durable storage, replacing the current @@ -130,7 +132,8 @@ public interface IStateManager Task WriteAsync(T newState); /// - /// Clears the persisted state, resetting it to default. + /// Clears the persisted state. On success, reflects + /// the storage provider's cleared value and may be . /// Task ClearAsync(); } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs index 9bc60169..114b011d 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs @@ -10,12 +10,12 @@ namespace Egil.Orleans.Messaging.State; /// /// /// -/// Committed-state fence: always returns the -/// last successfully written value. During , the -/// underlying .State is mutated, but the -/// caller's view is updated only after the write succeeds. On failure, the -/// recovery path re-reads from storage to determine whether the write -/// actually landed. +/// Committed-state fence: returns the last +/// committed value, including when no state value +/// exists. During , the underlying +/// .State is mutated, but the caller's view +/// is updated only after the write succeeds. On failure, the recovery path +/// re-reads from storage to determine whether the write actually landed. /// /// /// Version stamping: If derives from @@ -73,7 +73,7 @@ public abstract class StateManagerBase : IStateManager where T : class, IEquatable { private readonly IPersistentState storage; - private T state; + private T? state; /// /// Initializes the manager over the grain's persistent state facet, @@ -88,7 +88,7 @@ protected StateManagerBase(IPersistentState storage) } /// - public T State + public T? State { get => state; } @@ -127,8 +127,7 @@ public async Task WriteAsync(T newState) { // Provider-specific classification says the write never reached durable storage. // Revert our local fence immediately and rethrow the original write error. - state = previousState; - storage.State = previousState; + RestoreState(previousState); throw; } @@ -138,7 +137,7 @@ public async Task WriteAsync(T newState) // Unknown outcome: write may have persisted despite the exception. // Read back from storage to distinguish "lost response" from "real failure." await storage.ReadStateAsync(); - var persisted = storage.State; + T? persisted = storage.State; recoveryReadSucceeded = true; state = persisted; @@ -155,8 +154,7 @@ public async Task WriteAsync(T newState) { // A failed read yields no trustworthy durable value. Restore // the pre-write snapshot; the original write error is rethrown below. - state = previousState; - storage.State = previousState; + RestoreState(previousState); } } @@ -187,8 +185,7 @@ public async Task ClearAsync() // The provider proved the clear was rejected before it could // persist, so the in-memory view must stay at the pre-clear // state and the original error should be surfaced. - state = previousState; - storage.State = previousState; + RestoreState(previousState); throw; } @@ -221,8 +218,7 @@ public async Task ClearAsync() // Without read-back confirmation, keep the local activation // conservative. The next successful read/write can reconcile // against storage. - state = previousState; - storage.State = previousState; + RestoreState(previousState); } throw; @@ -255,8 +251,13 @@ protected virtual StorageFailureKind ClassifyClearFailure(Exception exception) /// and silently lose that data. Outbox<T>.Equals documents /// how its O(1) fingerprint satisfies this contract. /// - private static bool IsEquivalent(T persisted, T attempted) + private static bool IsEquivalent(T? persisted, T attempted) { + if (persisted is null) + { + return false; + } + if (persisted is VersionedState persistedVersioned && attempted is VersionedState attemptedVersioned) { @@ -265,4 +266,12 @@ private static bool IsEquivalent(T persisted, T attempted) return persisted.Equals(attempted); } + + private void RestoreState(T? previousState) + { + state = previousState; + + // Orleans annotates State as non-null even though clearing reference state can make it null. + storage.State = previousState!; + } } diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs index 67110815..5bba3505 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs @@ -4,6 +4,29 @@ namespace Egil.Orleans.Messaging.Tests.State; public sealed class StateManagerTests { + [Fact] + public void State_when_storage_exposes_null_is_null() + { + var storage = new FakePersistentState(null); + var manager = new DefaultStateManager(storage); + + Assert.Null(manager.State); + } + + [Fact] + public void State_when_storage_exposes_value_without_record_exposes_value() + { + var initial = new TestState("default"); + var storage = new FakePersistentState(initial) + { + RecordExists = false + }; + var manager = new DefaultStateManager(storage); + + Assert.Same(initial, manager.State); + Assert.False(storage.RecordExists); + } + [Fact] public async Task ReadAsync_refreshes_state_from_storage() { @@ -16,6 +39,25 @@ public async Task ReadAsync_refreshes_state_from_storage() Assert.Equal(new TestState("fresh"), manager.State); } + [Fact] + public async Task ReadAsync_when_storage_exposes_null_updates_state_to_null() + { + var storage = new FakePersistentState(new TestState("initial")) + { + OnRead = state => + { + state.State = null!; + state.RecordExists = false; + } + }; + var manager = new DefaultStateManager(storage); + + await manager.ReadAsync(); + + Assert.Null(manager.State); + Assert.False(storage.RecordExists); + } + [Fact] public async Task WriteAsync_success_updates_committed_state() { @@ -147,7 +189,7 @@ public async Task WriteAsync_on_versioned_state_stamps_new_version() await manager.WriteAsync(next); Assert.NotEqual(Guid.Empty, next.Version); - Assert.Equal(next.Version, manager.State.Version); + Assert.Equal(next.Version, manager.State!.Version); } [Fact] @@ -162,6 +204,42 @@ public async Task ClearAsync_success_updates_committed_state() Assert.False(storage.RecordExists); } + [Fact] + public async Task WriteAsync_after_clear_can_persist_new_state() + { + var storage = new FakePersistentState(new TestState("initial")); + var manager = new DefaultStateManager(storage); + var next = new TestState("next"); + + await manager.ClearAsync(); + await manager.WriteAsync(next); + + Assert.Equal(next, manager.State); + Assert.Equal(next, storage.State); + } + + [Fact] + public async Task WriteAsync_from_cleared_state_when_recovery_read_exposes_null_keeps_null() + { + var writeException = new TimeoutException("write timeout"); + var storage = new FakePersistentState(new TestState("initial")); + var manager = new DefaultStateManager(storage); + await manager.ClearAsync(); + storage.WriteException = writeException; + storage.OnRead = state => + { + state.State = null!; + state.RecordExists = false; + }; + + var ex = await Assert.ThrowsAsync( + () => manager.WriteAsync(new TestState("next"))); + + Assert.Same(writeException, ex); + Assert.Null(manager.State); + Assert.False(storage.RecordExists); + } + [Fact] public async Task ClearAsync_when_clear_fails_but_read_shows_missing_record_swallows_exception() { @@ -205,9 +283,10 @@ private sealed record VersionedTestState(string Value) : VersionedState, IEquata private sealed class FakePersistentState : IPersistentState { - public FakePersistentState(TestState state) + public FakePersistentState(TestState? state) { - State = state; + State = state!; + RecordExists = state is not null; } public Exception? ReadException { get; set; } From e0a50c13e24c4b31bbcb3ccfb64b5723e00d3d1f Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 18:45:48 +0000 Subject: [PATCH 73/82] test(om): cover malformed outbox token JSON --- .../OutboxSequenceTokenJsonConverterTests.cs | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs index fc0b33fd..1b9e83bc 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxSequenceTokenJsonConverterTests.cs @@ -5,6 +5,16 @@ namespace Egil.Orleans.Messaging.Tests.Outboxes; public sealed class OutboxSequenceTokenJsonConverterTests { + public static TheoryData InvalidKnownPropertyJson => new() + { + { """{"SequenceNumber":"7"}""" }, + { """{"SequenceNumber":1.5}""" }, + { """{"Timestamp":7}""" }, + { """{"Timestamp":"not-a-date"}""" }, + { """{"Epoch":7}""" }, + { """{"Epoch":"not-a-date"}""" } + }; + public static TheoryData GrainIdCases => new() { GrainId.Create("test/type", "plain-string"), @@ -65,4 +75,11 @@ public void JsonSerializer_throws_for_outbox_sequence_token_payload_missing_send Assert.Throws(() => JsonSerializer.Deserialize(json)); } -} \ No newline at end of file + + [Theory] + [MemberData(nameof(InvalidKnownPropertyJson))] + public void JsonSerializer_reports_invalid_known_property_as_json_error(string json) + { + Assert.Throws(() => JsonSerializer.Deserialize(json)); + } +} From 0da0d8ad29a7c16dbd7c1a02320bf04fbf760d44 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 18:49:44 +0000 Subject: [PATCH 74/82] test(om): cover conflict recovery read failure --- .../State/StateManagerTests.cs | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs index 5bba3505..b7f41d6a 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs @@ -178,6 +178,26 @@ public async Task WriteAsync_when_write_and_read_fail_reverts_state_and_rethrows Assert.Equal(new TestState("initial"), manager.State); } + [Fact] + public async Task WriteAsync_when_conflict_and_read_fail_reverts_state_and_rethrows_conflict() + { + var initial = new TestState("initial"); + var writeException = new InconsistentStateException("write conflict"); + var storage = new FakePersistentState(initial) + { + WriteException = writeException, + ReadException = new InvalidOperationException("read failed") + }; + var manager = new DefaultStateManager(storage); + + var ex = await Assert.ThrowsAsync( + () => manager.WriteAsync(new TestState("next"))); + + Assert.Same(writeException, ex); + Assert.Same(initial, manager.State); + Assert.Same(initial, storage.State); + } + [Fact] public async Task WriteAsync_on_versioned_state_stamps_new_version() { From 22d8f8bc3e16aa27e7054c56cbade54216116643 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Thu, 23 Jul 2026 19:01:37 +0000 Subject: [PATCH 75/82] fix(om): preserve clear concurrency conflicts Keep InconsistentStateException observable when clear recovery finds a missing record, while adopting the provider's recovered state. Document the throwing-path state contract and cover it with a deterministic regression test. --- .../State/IStateManager.cs | 15 +++++++++++++ .../State/StateManager.cs | 8 +++---- .../State/StateManagerTests.cs | 22 +++++++++++++++++++ 3 files changed, 41 insertions(+), 4 deletions(-) diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs index d7614f24..a787bc0f 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/IStateManager.cs @@ -135,5 +135,20 @@ public interface IStateManager /// Clears the persisted state. On success, reflects /// the storage provider's cleared value and may be . /// + /// + /// + /// For failures whose outcome is ambiguous, the manager reads storage to + /// determine whether the clear landed. A missing record confirms a + /// non-conflicting clear, but an InconsistentStateException always + /// rethrows because a coincidental deletion must not hide an optimistic + /// concurrency conflict. + /// + /// + /// When recovery successfully reads a provider value before rethrowing, + /// adopts that value and can therefore change even + /// though this method throws. If recovery also fails, + /// reverts to its pre-clear value. + /// + /// Task ClearAsync(); } diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs index 114b011d..5a418e84 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/State/StateManager.cs @@ -196,14 +196,14 @@ public async Task ClearAsync() // local state should represent "cleared" or the previous value. await storage.ReadStateAsync(); recoveryReadSucceeded = true; + state = storage.State; - if (!storage.RecordExists) + // A missing record confirms an ambiguous clear, but it cannot + // hide a real concurrency conflict caused by another writer. + if (ex is not InconsistentStateException && !storage.RecordExists) { - state = storage.State; return; } - - state = storage.State; } catch { diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs index b7f41d6a..958cc87d 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs @@ -280,6 +280,28 @@ public async Task ClearAsync_when_clear_fails_but_read_shows_missing_record_swal Assert.False(storage.RecordExists); } + [Fact] + public async Task ClearAsync_on_conflict_when_read_shows_missing_record_adopts_state_and_rethrows_conflict() + { + var clearException = new InconsistentStateException("clear conflict"); + var storage = new FakePersistentState(new TestState("initial")) + { + ClearException = clearException, + OnRead = state => + { + state.State = null!; + state.RecordExists = false; + } + }; + var manager = new DefaultStateManager(storage); + + var ex = await Assert.ThrowsAsync(() => manager.ClearAsync()); + + Assert.Same(clearException, ex); + Assert.Null(manager.State); + Assert.False(storage.RecordExists); + } + [Fact] public async Task ClearAsync_when_clear_and_read_fail_reverts_state_and_rethrows_original_exception() { From 1ea815287775a54aa343f01141fa329753298eb5 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Sat, 25 Jul 2026 13:40:34 +0000 Subject: [PATCH 76/82] fix(om): deliver enriched Event Hub tokens Wrap Event Hubs batch containers so stream consumers receive broker enqueue time, provider name, and trace context on batch and per-event sequence tokens. Cover the cached-message and Orleans serialization pipeline deterministically. --- .../EventHubs/EnrichedEventHubAdapter.cs | 72 +++++++++++++------ .../EnrichedEventHubBatchContainer.cs | 59 +++++++++++++++ .../EnrichedEventHubSequenceToken.cs | 10 +-- .../EventHubs/EnrichedEventHubAdapterTests.cs | 70 ++++++++++++++++-- 4 files changed, 181 insertions(+), 30 deletions(-) create mode 100644 Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubBatchContainer.cs diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapter.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapter.cs index 5a205177..4280526b 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapter.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubAdapter.cs @@ -25,20 +25,24 @@ namespace Egil.Orleans.Messaging.Streams.EventHubs; /// /// /// -/// Overrides: This adapter overrides three methods: +/// Overrides: This adapter overrides the Event Hubs conversion points +/// that create messages and sequence tokens: /// /// and /// /// — to return instead of /// . +/// GetBatchContainer(EventHubMessage) — to expose enriched +/// per-event tokens to stream consumers while preserving the base batch +/// container behavior. /// — to stamp /// Activity.Current?.Id into the outgoing /// EventData.Properties["traceparent"] before the event hits /// Event Hub, enabling cross-queue trace correlation via /// s on the consumer side. /// -/// All other adapter behavior (batch container, partition key) is -/// inherited unchanged from . +/// All other adapter behavior, including partition-key selection, is inherited +/// unchanged from . /// /// /// OTel trace correlation: Orleans streams lose @@ -48,8 +52,8 @@ namespace Egil.Orleans.Messaging.Streams.EventHubs; /// Producer side (): /// stashes Activity.Current?.Id into /// EventData.Properties["traceparent"]. -/// Consumer side (): -/// extracts the traceparent property and stores it in +/// Consumer side (GetBatchContainer(EventHubMessage)): +/// extracts the traceparent property and stores it in each delivered /// . /// StreamManager: reads /// and creates an @@ -121,8 +125,8 @@ public EnrichedEventHubAdapter(string providerName, Serializer serializer) /// This is the producer side of the OTel trace correlation pattern. /// The current (W3C format) is captured at /// publish time and stored as an Event Hub application property. On the - /// consumer side, extracts it into the - /// property. + /// consumer side, the batch-container conversion extracts it into each + /// delivered . /// /// /// If no is active at publish time, no property @@ -166,11 +170,11 @@ public override Azure.Messaging.EventHubs.EventData ToQueueMessage( /// cached message — no additional broker round-trip is needed. /// /// - /// Note: The - /// is not available on tokens reconstructed from cache — the - /// struct does not carry Event Hub application - /// properties. Trace correlation uses the token produced by - /// at initial ingest. + /// Note: This cache-cursor token cannot include + /// because + /// does not expose Event Hub application + /// properties. Consumer delivery reconstructs those properties and adds + /// the traceparent through GetBatchContainer(EventHubMessage). /// /// /// The cached Event Hub message. @@ -185,21 +189,35 @@ public override StreamSequenceToken GetSequenceToken(ref CachedMessage cachedMes ? string.Empty : GetOffset(cachedMessage) ?? string.Empty; - var enqueueTimeUtc = cachedMessage.EnqueueTimeUtc.Kind switch - { - DateTimeKind.Unspecified => DateTime.SpecifyKind(cachedMessage.EnqueueTimeUtc, DateTimeKind.Utc), - DateTimeKind.Local => cachedMessage.EnqueueTimeUtc.ToUniversalTime(), - _ => cachedMessage.EnqueueTimeUtc - }; - return new EnrichedEventHubSequenceToken( offset, cachedMessage.SequenceNumber, cachedMessage.EventIndex, - new DateTimeOffset(enqueueTimeUtc), + NormalizeUtc(cachedMessage.EnqueueTimeUtc), ProviderName); } + /// + /// Wraps the base batch container so the tokens observed by stream + /// consumers carry the Event Hub metadata reconstructed from cache. + /// + protected override IBatchContainer GetBatchContainer(EventHubMessage eventHubMessage) + { + var inner = base.GetBatchContainer(eventHubMessage); + var traceParent = eventHubMessage.Properties.TryGetValue(TraceParentPropertyKey, out var value) + ? value as string ?? value?.ToString() + : null; + var sequenceToken = new EnrichedEventHubSequenceToken( + eventHubMessage.Offset, + eventHubMessage.SequenceNumber, + inner.SequenceToken.EventIndex, + NormalizeUtc(eventHubMessage.EnqueueTimeUtc), + ProviderName, + traceParent); + + return new EnrichedEventHubBatchContainer(inner, sequenceToken); + } + /// /// Overrides the base to produce an /// when the adapter first sees a raw EventData from the Event Hub @@ -252,4 +270,16 @@ public override StreamPosition GetStreamPosition( return new StreamPosition(streamPosition.StreamId, enrichedToken); } -} \ No newline at end of file + + private static DateTimeOffset NormalizeUtc(DateTime value) + { + var utc = value.Kind switch + { + DateTimeKind.Unspecified => DateTime.SpecifyKind(value, DateTimeKind.Utc), + DateTimeKind.Local => value.ToUniversalTime(), + _ => value + }; + + return new DateTimeOffset(utc); + } +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubBatchContainer.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubBatchContainer.cs new file mode 100644 index 00000000..8e4504f9 --- /dev/null +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubBatchContainer.cs @@ -0,0 +1,59 @@ +using Orleans.Streams; + +namespace Egil.Orleans.Messaging.Streams.EventHubs; + +/// +/// Preserves the base Event Hubs batch payload and request-context behavior +/// while exposing enriched sequence tokens to stream consumers. +/// +[GenerateSerializer] +[Alias("egil.orleans.messaging.EnrichedEventHubBatchContainer")] +internal sealed class EnrichedEventHubBatchContainer : IBatchContainer +{ + [Id(0)] + private IBatchContainer inner = default!; + + [Id(1)] + private EnrichedEventHubSequenceToken sequenceToken = default!; + + public EnrichedEventHubBatchContainer( + IBatchContainer inner, + EnrichedEventHubSequenceToken sequenceToken) + { + this.inner = inner; + this.sequenceToken = sequenceToken; + } + + /// + /// Creates an instance for Orleans serialization. + /// + public EnrichedEventHubBatchContainer() + { + } + + /// + public StreamId StreamId => inner.StreamId; + + /// + public StreamSequenceToken SequenceToken => sequenceToken; + + /// + public IEnumerable> GetEvents() + { + foreach (var item in inner.GetEvents()) + { + yield return Tuple.Create( + item.Item1, + (StreamSequenceToken)new EnrichedEventHubSequenceToken( + sequenceToken.EventHubOffset, + sequenceToken.SequenceNumber, + item.Item2.EventIndex, + sequenceToken.EnqueuedTime, + sequenceToken.ProviderName, + sequenceToken.TraceParent)); + } + } + + /// + public bool ImportRequestContext() => inner.ImportRequestContext(); +} diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs index 25622611..b30dd73d 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging.Streams.EventHubs/EventHubs/EnrichedEventHubSequenceToken.cs @@ -14,9 +14,9 @@ namespace Egil.Orleans.Messaging.Streams.EventHubs; /// Registration: Call /// /// on the Event Hub stream configurator during silo setup. This registers -/// the library's internal EnrichedEventHubAdapter that overrides -/// GetStreamPosition and GetSequenceToken to produce -/// instances automatically. +/// the library's , which produces +/// instances for cache positions +/// and consumer delivery. /// /// /// Transparent to StreamManager: The is @@ -81,8 +81,8 @@ public class EnrichedEventHubSequenceToken : EventHubSequenceTokenV2, IStreamSeq /// /// /// Stamped by in its - /// ToQueueMessage override (producer side) and extracted in - /// GetStreamPosition (consumer side). + /// ToQueueMessage override (producer side) and extracted during + /// batch-container conversion (consumer side). /// uses this to create /// s — correlating consumer /// spans to producer spans without creating multi-hour parent-child traces. diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubAdapterTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubAdapterTests.cs index 776a16ba..1ec543d5 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubAdapterTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Streams.EventHubs.Tests/EventHubs/EnrichedEventHubAdapterTests.cs @@ -3,6 +3,7 @@ using Orleans.Providers.Streams.Common; using Orleans.Serialization; using Orleans.Streaming.EventHubs; +using Orleans.Streams; namespace Egil.Orleans.Messaging.Tests.Streams.EventHubs; @@ -122,13 +123,74 @@ public void GetStreamPosition_extracts_traceparent_from_event_data_properties() Assert.Equal("provider-a", enriched.ProviderName); } - private static EnrichedEventHubAdapter CreateAdapter() + [Fact] + public void GetBatchContainer_delivers_enriched_tokens_from_cached_message() + { + var adapter = CreateAdapter(out var serializer); + var streamId = StreamId.Create("orders", "one"); + var traceParent = "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01"; + var enqueuedTime = new DateTimeOffset(2026, 5, 24, 19, 0, 0, TimeSpan.Zero); + var outgoing = adapter.ToQueueMessage( + streamId, + ["event-1", "event-2"], + null!, + []); + var properties = new Dictionary(outgoing.Properties) + { + ["traceparent"] = traceParent + }; +#pragma warning disable CS0618 // Test-only factory path for broker-owned EventData fields in the installed Event Hubs package. + var received = EventHubsModelFactory.EventData( + outgoing.EventBody, + properties, + systemProperties: new Dictionary(), + partitionKey: "one", + sequenceNumber: 42, + offset: 123, + enqueuedTime: enqueuedTime); +#pragma warning restore CS0618 + var position = adapter.GetStreamPosition("0", received); + var cachedMessage = adapter.FromQueueMessage( + position, + received, + new DateTime(2026, 5, 24, 19, 0, 1, DateTimeKind.Utc), + static size => new ArraySegment(new byte[size])); + + var serialized = serializer.SerializeToArray( + adapter.GetBatchContainer(ref cachedMessage)); + var container = serializer.Deserialize(serialized); + var delivered = container.GetEvents().ToArray(); + + Assert.Equal(["event-1", "event-2"], delivered.Select(item => item.Item1)); + AssertEnrichedToken(container.SequenceToken, 0, enqueuedTime, traceParent); + AssertEnrichedToken(delivered[0].Item2, 0, enqueuedTime, traceParent); + AssertEnrichedToken(delivered[1].Item2, 1, enqueuedTime, traceParent); + } + + private static EnrichedEventHubAdapter CreateAdapter() => CreateAdapter(out _); + + private static EnrichedEventHubAdapter CreateAdapter(out Serializer serializer) { var services = new ServiceCollection(); - services.AddSerializer(static _ => { }); + services.AddSerializer(builder => builder.AddAssembly(typeof(EnrichedEventHubAdapter).Assembly)); var provider = services.BuildServiceProvider(); - var serializer = provider.GetRequiredService(); + serializer = provider.GetRequiredService(); return new EnrichedEventHubAdapter("provider-a", serializer); } -} \ No newline at end of file + + private static void AssertEnrichedToken( + StreamSequenceToken token, + int expectedEventIndex, + DateTimeOffset expectedEnqueuedTime, + string expectedTraceParent) + { + var enriched = Assert.IsType(token); + Assert.Equal("123", enriched.EventHubOffset); + Assert.Equal(42, enriched.SequenceNumber); + Assert.Equal(expectedEventIndex, enriched.EventIndex); + Assert.Equal(expectedEnqueuedTime, enriched.EnqueuedTime); + Assert.Equal("provider-a", enriched.ProviderName); + Assert.Equal(expectedTraceParent, enriched.TraceParent); + } +} From 6724c6aa3c2a60e0e028a42be1e2eac72b9df052 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 27 Jul 2026 13:21:51 +0000 Subject: [PATCH 77/82] fix(om)!: prevent grain-derived stream ID collisions Derive namespace-based explicit subscription IDs from the complete Orleans GrainId and expose the same helper to producers. Reject textual identities that do not round-trip instead of silently collapsing them. Cover compound keys, hex-parseable strings, cross-kind collisions, Orleans binary serialization, and StreamId text parsing. Document producer wiring and migration. BREAKING CHANGE: Namespace-based explicit subscriptions now use the complete GrainId as the StreamId key. Publishers must call StreamManager.CreateStreamId with the receiving GrainId, and existing durable key-only subscriptions must be recreated or configured through the explicit StreamId overload. --- Egil.Orleans.Messaging/README.md | 21 +++- Egil.Orleans.Messaging/api-design.md | 33 +++++- .../Streams/StreamManager.cs | 68 +++++++---- .../MessagingTestClusterFixture.cs | 6 + .../Outboxes/OutboxPostmanHelperTests.cs | 14 ++- .../Outboxes/OutboxProcessorTests.cs | 6 +- .../Streams/StreamManagerStreamIdTests.cs | 106 ++++++++++++++++++ .../Streams/StreamManagerTests.cs | 12 +- 8 files changed, 227 insertions(+), 39 deletions(-) create mode 100644 Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerStreamIdTests.cs diff --git a/Egil.Orleans.Messaging/README.md b/Egil.Orleans.Messaging/README.md index ef0a8c3f..00267c5f 100644 --- a/Egil.Orleans.Messaging/README.md +++ b/Egil.Orleans.Messaging/README.md @@ -304,9 +304,20 @@ streamManager = this.RegisterStreamManager(state.State!.Tracker) await streamManager.EnsureExplicitSubscriptionsAsync(cancellationToken); ``` -The string namespace overload derives a stream id from the receiving grain -identity. Use the `StreamId` overload for an explicit stream keyed by another -application id: +The string namespace overload derives a stream id from the complete receiving +`GrainId`, including its grain type and compound-key extension. Publishers must +use the same helper with the target grain identity: + +```csharp +var customer = grainFactory.GetGrain(customerId); +var streamId = StreamManager.CreateStreamId("prices", customer.GetGrainId()); +var stream = streamProvider.GetStream(streamId); +``` + +This convention follows the grain type, so renaming that type changes the +derived stream id. Use the `StreamId` overload for an application-owned id that +must survive grain-type changes, or when a custom grain identity cannot +round-trip through Orleans' textual `GrainId` representation: ```csharp streamManager = this.RegisterStreamManager(state.State!.Tracker) @@ -316,6 +327,10 @@ streamManager = this.RegisterStreamManager(state.State!.Tracker) HandlePriceChangedAsync); ``` +The previous key-only convention is not compatible with these full-identity +stream ids. Recreate existing durable subscriptions and update publishers +together, or preserve the previous id through the explicit `StreamId` overload. + Tracked resume tokens are a per-subscription choice. The default is to pass the previous token when a tracker snapshot is supplied. Opt out when a subscription should attach without a resume token: diff --git a/Egil.Orleans.Messaging/api-design.md b/Egil.Orleans.Messaging/api-design.md index a06b3ef8..a80d0a46 100644 --- a/Egil.Orleans.Messaging/api-design.md +++ b/Egil.Orleans.Messaging/api-design.md @@ -821,6 +821,10 @@ public interface IImplicitStreamGrain : IStreamSubscriptionObserver public sealed class StreamManager { + public static StreamId CreateStreamId( + string streamNamespace, + GrainId grainId); + public StreamManager ConfigureImplicitSubscription( string streamNamespace, Func onNextAsync, @@ -885,8 +889,11 @@ parameter. Orleans provides the concrete provider and stream id through `ConfigureExplicitSubscription` requires a provider name because the library must ask Orleans for the stream and durable subscription handles itself. The string namespace overload follows the library's grain-keyed convention and -derives the stream id from the receiving grain identity. Use the `StreamId` -overload when subscribing to an arbitrary Orleans explicit stream. +derives the stream id from the complete receiving `GrainId`. Producers call +`StreamManager.CreateStreamId(streamNamespace, target.GetGrainId())` to derive +the same id. The helper uses Orleans' textual grain identity and rejects any +custom identity which does not round-trip through `GrainId.TryParse`. Use the +`StreamId` overload for an arbitrary or application-owned stream identity. `RegisterStreamManager()` can be called without a `MessageTracker` when the grain does not persist stream high-water marks. In that mode the manager @@ -954,8 +961,18 @@ public override async Task OnActivateAsync(CancellationToken ct) The `TEvent` generic argument is usually inferred from the handler method group. Users only specify it for inline lambdas or ambiguous method groups. -Use the `StreamId` overload when the target explicit stream is not keyed by -the receiving grain: +Publishers targeting the grain-keyed convention derive the stream id from the +target grain reference: + +```csharp +var target = grainFactory.GetGrain(customerId); +var streamId = StreamManager.CreateStreamId("tariff-events", target.GetGrainId()); +var stream = streamProvider.GetStream(streamId); +``` + +Because the derived key contains the grain type, a grain-type rename changes +the stream id. Use the `StreamId` overload when the target stream needs an +application-owned identity or is not keyed by the receiving grain: ```csharp streamManager = this.RegisterStreamManager(state.Tracker) @@ -965,6 +982,10 @@ streamManager = this.RegisterStreamManager(state.Tracker) HandleTariffChangedAsync); ``` +The full-identity convention is incompatible with the previous key-only +derivation. Existing durable handles must be recreated while publishers move +to `CreateStreamId`, or retained by configuring their previous ids explicitly. + ### Implementation changes from the previous design - Rename `AddSubscription(...)` to the more explicit @@ -980,6 +1001,8 @@ streamManager = this.RegisterStreamManager(state.Tracker) component. - Change stream handlers to receive `StreamCursor` instead of raw `StreamSequenceToken?`. +- Derive grain-keyed stream ids from the complete, round-trippable `GrainId` + and expose `CreateStreamId(...)` for producers. - Keep explicit subscription unsubscribe orchestration out of the initial API. The first version only resumes or ensures explicit subscriptions. @@ -1008,7 +1031,7 @@ streamManager = this.RegisterStreamManager(state.Tracker) - `ConfigureExplicitSubscription(...)` records the provider, stream identity, event type, handler, and error callback. The namespace overload derives the - stream id from the receiving grain identity; the `StreamId` overload uses + stream id from the complete receiving `GrainId`; the `StreamId` overload uses the caller-provided stream identity directly. - `ResumeExplicitSubscriptionsAsync(...)` resumes all existing durable handles for each configured explicit stream from the activation-time diff --git a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs index 43ae04b6..2adda100 100644 --- a/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs +++ b/Egil.Orleans.Messaging/src/Egil.Orleans.Messaging/Streams/StreamManager.cs @@ -56,7 +56,7 @@ internal static StreamManager Create( owner, trackerSnapshot, services.GetRequiredKeyedService, - streamNamespace => CreateStreamId(owner, streamNamespace), + streamNamespace => CreateStreamId(streamNamespace, owner.GrainContext.GrainId), logger); manager.AttachToGrain(); @@ -123,9 +123,12 @@ public StreamManager ConfigureImplicitSubscription( /// library's grain-keyed stream id convention. /// /// - /// The stream id is derived from the receiving grain identity and - /// . Use the - /// overload when the explicit stream is keyed by another application id. + /// The stream id is derived from the complete receiving grain identity and + /// . Publishers targeting this + /// subscription must use with + /// that receiving grain's identity. Use the overload + /// for an application-owned identity which must not change with the grain + /// type. /// public StreamManager ConfigureExplicitSubscription( string streamProviderName, @@ -175,6 +178,45 @@ public StreamManager ConfigureExplicitSubscription( useTrackedResumeToken); } + /// + /// Creates the stream identity used by grain-keyed explicit subscriptions. + /// + /// + /// Uses Orleans' textual representation so producers + /// and receiving grains can independently derive the same durable stream + /// identity. This identity includes the grain type and therefore changes if + /// that type changes. Grain identities which do not round-trip through the + /// Orleans representation are rejected instead of risking a collision; use + /// an explicit for those identities. + /// + /// The Orleans stream namespace. + /// The identity of the receiving grain. + /// The stream identity derived from . + /// + /// is blank, + /// is the default value, or the grain identity cannot round-trip through + /// Orleans' textual representation. + /// + public static StreamId CreateStreamId(string streamNamespace, GrainId grainId) + { + ArgumentException.ThrowIfNullOrWhiteSpace(streamNamespace); + if (grainId.IsDefault) + { + throw new ArgumentException("GrainId must not be the default value.", nameof(grainId)); + } + + var textualGrainId = grainId.ToString(); + if (!GrainId.TryParse(textualGrainId, out var parsed) || parsed != grainId) + { + throw new ArgumentException( + "GrainId cannot be represented unambiguously using Orleans' textual grain identity format. " + + "Provide an explicit StreamId instead.", + nameof(grainId)); + } + + return StreamId.Create(streamNamespace, textualGrainId); + } + /// /// Configures a durable Orleans explicit stream subscription for the /// specified stream identity. @@ -497,22 +539,6 @@ private void EnsureCanConfigure() } } - private static StreamId CreateStreamId(IGrainBase owner, string streamNamespace) - { - var grainId = owner.GrainContext.GrainId; - if (GrainIdKeyExtensions.TryGetGuidKey(grainId, out var guidKey, out _)) - { - return StreamId.Create(streamNamespace, guidKey); - } - - if (GrainIdKeyExtensions.TryGetIntegerKey(grainId, out var longKey, out _)) - { - return StreamId.Create(streamNamespace, longKey); - } - - return StreamId.Create(streamNamespace, grainId.Key.ToString()); - } - private interface IImplicitSubscription { Task ResumeAsync(StreamManager manager, IStreamSubscriptionHandleFactory handleFactory); @@ -592,4 +618,4 @@ public Task OnNextAsync(TEvent item, StreamSequenceToken? token = null) => public Task OnErrorAsync(Exception ex) => manager.OnErrorAsync(streamNamespace, ex, onError); } -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs index e459a236..d73af892 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/MessagingTestClusterFixture.cs @@ -69,6 +69,12 @@ public IAsyncStream GetStream(string providerName, string streamNamespace, return provider.GetStream(StreamId.Create(streamNamespace, key)); } + public IAsyncStream GetStream(string providerName, string streamNamespace, GrainId grainId) + { + var provider = Cluster.Client.GetStreamProvider(providerName); + return provider.GetStream(StreamManager.CreateStreamId(streamNamespace, grainId)); + } + Task IGrainActivityWaiter.WaitForAssertionAsync( Func> assertion, Predicate? filter, diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanHelperTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanHelperTests.cs index d67445f4..8c0890ef 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanHelperTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxPostmanHelperTests.cs @@ -139,11 +139,15 @@ public sealed class OutboxProcessorStreamPostmanGrain( public override async Task OnActivateAsync(CancellationToken cancellationToken) { EnsureOutbox(); + var sink = GrainFactory.GetGrain(this.GetPrimaryKey()); + var streamId = StreamManager.CreateStreamId( + OutboxProcessorTestNamespaces.Events, + sink.GetGrainId()); processor = this.RegisterOutboxProcessor(CreateOptions()) .AddStreamPostman>( OutboxProcessorTestProviderNames.Events, - envelope => StreamId.Create(OutboxProcessorTestNamespaces.Events, this.GetPrimaryKey())); + _ => streamId); await base.OnActivateAsync(cancellationToken); } @@ -210,11 +214,15 @@ public sealed class OutboxProcessorProjectedStreamPostmanGrain( public override async Task OnActivateAsync(CancellationToken cancellationToken) { EnsureOutbox(); + var sink = GrainFactory.GetGrain(this.GetPrimaryKey()); + var streamId = StreamManager.CreateStreamId( + OutboxProcessorTestNamespaces.Events, + sink.GetGrainId()); processor = this.RegisterOutboxProcessor(CreateOptions()) .AddStreamPostman, OutboxProcessorTestEvent>( OutboxProcessorTestProviderNames.Events, - envelope => StreamId.Create(OutboxProcessorTestNamespaces.Events, this.GetPrimaryKey()), + _ => streamId, envelope => envelope.Message); await base.OnActivateAsync(cancellationToken); @@ -392,4 +400,4 @@ public async Task ReceiveAsync(string value) } public Task GetStateAsync() => Task.FromResult(state.State); -} \ No newline at end of file +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs index 0ec58ffa..a9b7d2fa 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorTests.cs @@ -514,9 +514,13 @@ private Task PublishEnvelopeAsync( OutboxMessageEnvelope envelope, CancellationToken cancellationToken) { + var sink = GrainFactory.GetGrain(this.GetPrimaryKey()); + var streamId = StreamManager.CreateStreamId( + OutboxProcessorTestNamespaces.Events, + sink.GetGrainId()); var stream = this.GetStreamProvider(OutboxProcessorTestProviderNames.Events) .GetStream>( - StreamId.Create(OutboxProcessorTestNamespaces.Events, this.GetPrimaryKey())); + streamId); _ = stream.OnNextAsync(envelope); return Task.CompletedTask; diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerStreamIdTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerStreamIdTests.cs new file mode 100644 index 00000000..3939ca77 --- /dev/null +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerStreamIdTests.cs @@ -0,0 +1,106 @@ +using System.Text; +using Orleans.Serialization; + +namespace Egil.Orleans.Messaging.Tests.Streams; + +public sealed class StreamManagerStreamIdTests +{ + [Fact] + public void Derived_stream_ids_preserve_compound_grain_extensions() + { + var grainType = GrainType.Create("test.compound-grain"); + var grainKey = Guid.Parse("01234567-89ab-cdef-0123-456789abcdef"); + var tenantA = GrainId.Create( + grainType, + GrainIdKeyExtensions.CreateGuidKey(grainKey, "tenant-a")); + var tenantB = GrainId.Create( + grainType, + GrainIdKeyExtensions.CreateGuidKey(grainKey, "tenant-b")); + + var tenantAStream = StreamManager.CreateStreamId("orders", tenantA); + var tenantBStream = StreamManager.CreateStreamId("orders", tenantB); + + Assert.Equal(tenantA.ToString(), tenantAStream.GetKeyAsString()); + Assert.Equal(tenantB.ToString(), tenantBStream.GetKeyAsString()); + Assert.NotEqual(tenantAStream, tenantBStream); + } + + [Fact] + public void Derived_stream_ids_preserve_case_sensitive_string_keys() + { + var grainType = GrainType.Create("test.string-grain"); + var lowerCase = GrainId.Create(grainType, "a"); + var upperCase = GrainId.Create(grainType, "A"); + + var lowerCaseStream = StreamManager.CreateStreamId("orders", lowerCase); + var upperCaseStream = StreamManager.CreateStreamId("orders", upperCase); + + Assert.Equal(lowerCase.ToString(), lowerCaseStream.GetKeyAsString()); + Assert.Equal(upperCase.ToString(), upperCaseStream.GetKeyAsString()); + Assert.NotEqual(lowerCaseStream, upperCaseStream); + } + + [Fact] + public void Derived_stream_ids_distinguish_integer_and_hex_parseable_string_grains() + { + var integerGrain = GrainId.Create( + GrainType.Create("test.integer-grain"), + GrainIdKeyExtensions.CreateIntegerKey(10)); + var stringGrain = GrainId.Create(GrainType.Create("test.string-grain"), "A"); + + var integerStream = StreamManager.CreateStreamId("orders", integerGrain); + var stringStream = StreamManager.CreateStreamId("orders", stringGrain); + + Assert.Equal(integerGrain.ToString(), integerStream.GetKeyAsString()); + Assert.Equal(stringGrain.ToString(), stringStream.GetKeyAsString()); + Assert.NotEqual(integerStream, stringStream); + } + + [Fact] + public void Derived_stream_id_survives_Orleans_serialization() + { + var grainId = GrainId.Create( + GrainType.Create("test.compound-grain"), + GrainIdKeyExtensions.CreateIntegerKey(42, "tenant-a")); + var streamId = StreamManager.CreateStreamId("orders", grainId); + var services = new ServiceCollection(); + services.AddSerializer(builder => builder.AddAssembly(typeof(StreamManager).Assembly)); + using var serviceProvider = services.BuildServiceProvider(); + var serializer = serviceProvider.GetRequiredService(); + + var serialized = serializer.SerializeToArray(streamId); + var deserialized = serializer.Deserialize(serialized); + + Assert.Equal(streamId, deserialized); + Assert.Equal("orders", deserialized.GetNamespace()); + Assert.True(GrainId.TryParse(deserialized.GetKeyAsString(), out var parsed)); + Assert.Equal(grainId, parsed); + } + + [Fact] + public void Derived_stream_id_survives_Orleans_text_round_trip() + { + var grainId = GrainId.Create( + GrainType.Create("test.compound-grain"), + GrainIdKeyExtensions.CreateGuidKey( + Guid.Parse("01234567-89ab-cdef-0123-456789abcdef"), + "tenant-a")); + var streamId = StreamManager.CreateStreamId("orders", grainId); + + var parsed = StreamId.Parse(Encoding.UTF8.GetBytes(streamId.ToString())); + + Assert.Equal(streamId, parsed); + } + + [Fact] + public void Derived_stream_id_rejects_grain_identity_that_cannot_round_trip_as_text() + { + var grainId = GrainId.Create(GrainType.Create("custom/type"), "one"); + + var exception = Assert.Throws( + () => StreamManager.CreateStreamId("orders", grainId)); + + Assert.Equal("grainId", exception.ParamName); + Assert.Contains("Provide an explicit StreamId instead.", exception.Message); + } +} diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs index d47d68e1..d42509c4 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Streams/StreamManagerTests.cs @@ -13,7 +13,7 @@ public async Task ConfigureExplicitSubscription_with_ValueTask_handler_delivers_ var stream = fixture.GetStream( StreamManagerTestProviderNames.Explicit, StreamManagerTestNamespaces.ValueTask, - grainKey); + grain.GetGrainId()); await stream.OnNextAsync("value-task"); @@ -95,7 +95,7 @@ public async Task Explicit_subscription_does_not_duplicate_after_reactivation() var stream = fixture.GetStream( StreamManagerTestProviderNames.Explicit, StreamManagerTestNamespaces.ValueTask, - grainKey); + grain.GetGrainId()); await stream.OnNextAsync("after-reactivation"); @@ -113,7 +113,7 @@ public async Task ConfigureExplicitSubscription_with_Task_handler_delivers_strea var stream = fixture.GetStream( StreamManagerTestProviderNames.Explicit, StreamManagerTestNamespaces.Task, - grainKey); + grain.GetGrainId()); await stream.OnNextAsync("task"); @@ -131,7 +131,7 @@ public async Task Default_error_handler_does_not_break_subscription_when_handler var stream = fixture.GetStream( StreamManagerTestProviderNames.Explicit, StreamManagerTestNamespaces.Failure, - grainKey); + grain.GetGrainId()); await stream.OnNextAsync("fail"); await stream.OnNextAsync("after-failure"); @@ -150,7 +150,7 @@ public async Task EnsureExplicitSubscriptionsAsync_is_idempotent() var stream = fixture.GetStream( StreamManagerTestProviderNames.Explicit, StreamManagerTestNamespaces.ValueTask, - grainKey); + grain.GetGrainId()); var exceptionType = await grain.EnsureExplicitSubscriptionsAgainAsync(); @@ -379,4 +379,4 @@ public sealed class ImplicitStreamManagerTestState [Id(2)] public int DeliveryCount { get; set; } -} \ No newline at end of file +} From 40d2ccb56038f3e6653c6b238132546cc5e3ffdd Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 27 Jul 2026 13:33:45 +0000 Subject: [PATCH 78/82] test(om): cover steady-state outbox advancement [skip notes] Pin same-epoch higher-sequence acceptance and subsequent duplicate rejection so an inverted comparison fails deterministically. --- .../Tracking/MessageTrackerTests.cs | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs index 3433bbba..67724568 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Tracking/MessageTrackerTests.cs @@ -266,6 +266,26 @@ public void ProcessMessage_rejects_outbox_token_when_same_epoch_and_not_newer_se Assert.Same(tracker, next); } + [Fact] + public void ProcessMessage_accepts_outbox_token_when_same_epoch_and_sequence_is_newer() + { + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var sender = GrainId.Create("test/sender", "one"); + var first = new OutboxSequenceToken(1, sender, now, now); + var second = new OutboxSequenceToken(2, sender, now, now); + var tracker = new MessageTracker(); + tracker.RegisterTimeProvider(new ManualTimeProvider(now)); + tracker.ProcessMessage(first, out tracker); + + var accepted = tracker.ProcessMessage(second, out var next); + var acceptedAgain = next.ProcessMessage(second, out var unchanged); + + Assert.True(accepted); + Assert.Equal(second, next.LatestOutbox(sender)); + Assert.False(acceptedAgain); + Assert.Same(next, unchanged); + } + [Fact] public void ProcessMessage_rejects_outbox_token_when_epoch_is_stale() { From a2432dc5d914b8cafa395ca3041452555c62a08e Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 27 Jul 2026 14:09:58 +0000 Subject: [PATCH 79/82] test(om): pin removal-only outbox equivalence [skip notes] Document the intentional O(1) recovery fingerprint with the exact removal-gap collision from review. Removal-only divergence can cause duplicate delivery but cannot lose an item which still required delivery. --- .../Outboxes/OutboxTests.cs | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs index 42a64cbc..05edcbde 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxTests.cs @@ -182,6 +182,26 @@ public void Equals_treats_matching_sequence_window_as_same_pending_items() Assert.Equal(left, right); } + [Fact] + public void Equals_treats_removal_only_divergence_as_same_recovery_fingerprint() + { + var sender = GrainId.Create("test/sender", "one"); + var now = new DateTimeOffset(2026, 5, 23, 12, 30, 0, TimeSpan.Zero); + var baseline = Outbox.Create(sender) + .Add("first", now) + .Add("second", now) + .Add("third", now) + .Add("fourth", now); + + var left = baseline.RemoveRange([baseline[2].Token]); + var right = baseline.RemoveRange([baseline[1].Token]); + + Assert.Equal([1L, 2L, 4L], left.Select(item => item.Token.SequenceNumber).ToArray()); + Assert.Equal([1L, 3L, 4L], right.Select(item => item.Token.SequenceNumber).ToArray()); + Assert.Equal(left, right); + Assert.Equal(left.GetHashCode(), right.GetHashCode()); + } + [Fact] public void Equals_returns_true_when_sender_sequence_epoch_and_envelopes_match() { From 3a44a766a3e22c1bbc2c78fbffe60fa7286f425c Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 27 Jul 2026 14:26:22 +0000 Subject: [PATCH 80/82] test(om): cover failed clear retaining record [skip notes] --- .../State/StateManagerTests.cs | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs index 958cc87d..8d155591 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/State/StateManagerTests.cs @@ -280,6 +280,31 @@ public async Task ClearAsync_when_clear_fails_but_read_shows_missing_record_swal Assert.False(storage.RecordExists); } + [Fact] + public async Task ClearAsync_when_clear_fails_and_read_shows_record_still_exists_adopts_state_and_rethrows() + { + var clearException = new TimeoutException("clear timeout"); + var persisted = new TestState("persisted"); + var storage = new FakePersistentState(new TestState("initial")) + { + ClearException = clearException, + OnRead = state => + { + state.State = persisted; + state.RecordExists = true; + state.Etag = "etag-2"; + } + }; + var manager = new DefaultStateManager(storage); + + var ex = await Assert.ThrowsAsync(() => manager.ClearAsync()); + + Assert.Same(clearException, ex); + Assert.Same(persisted, manager.State); + Assert.True(storage.RecordExists); + Assert.Equal("etag-2", storage.Etag); + } + [Fact] public async Task ClearAsync_on_conflict_when_read_shows_missing_record_adopts_state_and_rethrows_conflict() { From 41ca92812defac7dccc141006a43007ece727106 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 27 Jul 2026 14:47:39 +0000 Subject: [PATCH 81/82] test(om): verify durable reminder lifecycle [skip notes] --- .../Outboxes/OutboxProcessorCoverageTests.cs | 70 +++++++++++++++++-- 1 file changed, 63 insertions(+), 7 deletions(-) diff --git a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs index e67f53e5..a9b5f673 100644 --- a/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs +++ b/Egil.Orleans.Messaging/test/Egil.Orleans.Messaging.Tests/Outboxes/OutboxProcessorCoverageTests.cs @@ -73,6 +73,24 @@ await fixture.WaitForAssertionAsync( ct: TestContext.Current.CancellationToken); } + [Fact] + public async Task Failed_dispatch_registers_reminder_and_successful_retry_unregisters_it() + { + var grain = fixture.GrainFactory.GetGrain(Guid.NewGuid()); + + var failedState = await grain.PublishAndFailAsync("retry"); + + Assert.Equal(1, failedState.FailedCount); + Assert.Equal(1, failedState.Outbox?.Count); + Assert.True(await grain.HasReminderAsync()); + + var recoveredState = await grain.RetrySuccessfullyAsync(); + + Assert.Equal(1, recoveredState.AcknowledgedCount); + Assert.Equal(0, recoveredState.Outbox?.Count); + Assert.False(await grain.HasReminderAsync()); + } + [Fact] public async Task ReceiveReminderAsync_ignores_unrelated_reminder_name() { @@ -374,6 +392,12 @@ public interface IOutboxProcessorRetryPendingGrain : IGrainWithGuidKey { Task PublishInBackgroundAsync(string value); + Task PublishAndFailAsync(string value); + + Task RetrySuccessfullyAsync(); + + Task HasReminderAsync(); + Task GetStateAsync(); } @@ -619,6 +643,7 @@ public sealed class OutboxProcessorRetryPendingGrain( : Grain, IOutboxProcessorRetryPendingGrain, IOutboxGrain { private OutboxProcessor>? processor; + private bool failPosting = true; public override async Task OnActivateAsync(CancellationToken cancellationToken) { @@ -627,11 +652,9 @@ public override async Task OnActivateAsync(CancellationToken cancellationToken) PendingItems = () => state.State.Outbox?.ToImmutableArray() ?? [], AcknowledgePostedAsync = AcknowledgePostedAsync, ReconcileFailedAsync = ReconcileFailedAsync, - RetryDelay = TimeSpan.FromMilliseconds(100) + RetryDelay = TimeSpan.FromMinutes(2) }) - .AddPostman>( - static _ => new ValueTask(Task.FromException( - new InvalidOperationException("Keep pending for retry.")))); + .AddPostman>(PostItemAsync); await base.OnActivateAsync(cancellationToken); } @@ -643,12 +666,45 @@ public async Task PublishInBackgroundAsync(string value) await processor!.PostInBackgroundAsync(); } + public async Task PublishAndFailAsync(string value) + { + state.State.Outbox = EnsureOutbox().Add(new OutboxProcessorTestEvent(value)); + await state.WriteStateAsync(); + await processor!.PostAsync(); + return state.State; + } + + public async Task RetrySuccessfullyAsync() + { + failPosting = false; + await processor!.PostAsync(); + return state.State; + } + + public async Task HasReminderAsync() => + await this.GetReminder(processor!.ReminderName) is not null; + public Task GetStateAsync() => Task.FromResult(state.State); - private ValueTask AcknowledgePostedAsync( + private ValueTask PostItemAsync(OutboxMessageEnvelope _) => + failPosting + ? new ValueTask(Task.FromException(new InvalidOperationException("Keep pending for retry."))) + : ValueTask.CompletedTask; + + private async ValueTask AcknowledgePostedAsync( ImmutableArray> items, - CancellationToken cancellationToken) => - ValueTask.CompletedTask; + CancellationToken cancellationToken) + { + var outbox = EnsureOutbox(); + foreach (var item in items) + { + outbox = outbox.Remove(item.Token); + } + + state.State.Outbox = outbox; + state.State.AcknowledgedCount += items.Length; + await state.WriteStateAsync(cancellationToken); + } private async ValueTask ReconcileFailedAsync( ImmutableArray<(OutboxMessageEnvelope Item, Exception Error, int Attempt)> failures, From 51b489366e5f2d6acb5eb6fb2dac1c9a6d874265 Mon Sep 17 00:00:00 2001 From: Egil Hansen Date: Mon, 27 Jul 2026 15:10:51 +0000 Subject: [PATCH 82/82] fix(stp): pin patched OpenAPI dependency Pin Microsoft.OpenApi 2.7.5 for .NET 10 so restores avoid the high-severity circular-reference parser vulnerability while .NET 9 remains on its unaffected 1.x API line. --- Egil.StronglyTypedPrimitives/Directory.Packages.props | 3 ++- .../Egil.StronglyTypedPrimitives.Abstractions.csproj | 4 ++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/Egil.StronglyTypedPrimitives/Directory.Packages.props b/Egil.StronglyTypedPrimitives/Directory.Packages.props index 2335ea4e..b0595ba7 100644 --- a/Egil.StronglyTypedPrimitives/Directory.Packages.props +++ b/Egil.StronglyTypedPrimitives/Directory.Packages.props @@ -27,8 +27,9 @@ + - \ No newline at end of file + diff --git a/Egil.StronglyTypedPrimitives/src/Egil.StronglyTypedPrimitives.Abstractions/Egil.StronglyTypedPrimitives.Abstractions.csproj b/Egil.StronglyTypedPrimitives/src/Egil.StronglyTypedPrimitives.Abstractions/Egil.StronglyTypedPrimitives.Abstractions.csproj index 1714ead4..3bb09239 100644 --- a/Egil.StronglyTypedPrimitives/src/Egil.StronglyTypedPrimitives.Abstractions/Egil.StronglyTypedPrimitives.Abstractions.csproj +++ b/Egil.StronglyTypedPrimitives/src/Egil.StronglyTypedPrimitives.Abstractions/Egil.StronglyTypedPrimitives.Abstractions.csproj @@ -28,5 +28,9 @@ + + + +