[alanshaw]

The second step of #328, stacked on #330: the scheduling features the task runner RFC describes for async tasks. Like #330, nothing in the server uses them yet.

• Replacing. A delayed task can be submitted as a replacement. If one with the same subject and action is waiting and has not started, whichever runs later is kept and the other is dropped. Ingot will use this to push a cached file's eviction back each time it is read. A replacement that has already started is left alone, and the new one is added beside it.
• Cancelling. A task that has not started can be cancelled, from a handler or in a caller's own transaction, so the cancel commits with the writes that make it necessary. Ingot will use this to cancel an upload's cleanup once the write that claims its shards commits.
• Batching. An action can take a run of consecutive tasks on one subject in a single go, up to a limit, stopping at the first task of a different action so the order still holds. Ingot will use this to register a space's queued objects with the network in one request.
• Retries. An action can register a backoff and an attempt limit for its async tasks, and a handler can mark an error as one no retry will fix. Sync tasks are never retried: their error goes to the caller.

The package also gets a README with an example of every feature, including the earlier ones.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

Retry scheduling breaks cancellation guarantees and same-subject ordering, while non-atomic batch failure updates violate the fail-together contract.

3 open issues

• Commit batch failure and retry outcomes atomically · New
• Preserve the started marker when scheduling retries · New
• Prevent later same-subject actions from overtaking retries · New

What changed in this PR

Adds scheduling features to the standalone Postgres task runner, building on #330 toward #328’s replacement of specialized locks and sweepers. The runner remains unwired from the server.

Changes:

• Adds pending-task replacement and cancellation, async batching, and configurable retries.
• Introduces a second schema migration for scheduling state.
• Adds live database tests and usage documentation.

File	Description
taskrunner/​worker.go	Claims individual tasks or batches before execution.
taskrunner/​tree.go	Executes batches and integrates retries and cancellation notifications.
taskrunner/​schema.go	Adds scheduling columns, cancellation state, and a replacement index.
taskrunner/​scheduling.go	Implements replacement, cancellation, batch claiming, and retries.
taskrunner/​scheduling_live_test.go	Tests scheduling features against Postgres.
taskrunner/​runner.go	Exposes scheduling options and validates async-only batching.
taskrunner/​README.md	Documents runner behavior and feature usage.
CLAUDE.md	Updates the task runner package description.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

A critical finding remains in taskrunner/tree.go: handlerless batch failures can commit task outcomes individually, leaving the batch partially failed instead of enforcing the documented fail-together behavior.

1 open issue

• Fail handlerless batch tasks in one transaction · New

3 resolved since last review

• Prevent later same-subject actions from overtaking retries
• Preserve the started marker when scheduling retries
• Commit batch failure and retry outcomes atomically

_{Review effort: Lite}

[Copilot]

🟡 Changes recommended

A critical active issue remains because batch expiry is not atomic, allowing some tasks to be committed as expired while others remain pending and later run independently, violating the fail-together contract.

1 open issue

• Batch expiry updates are not atomic · New

1 resolved since last review

• Fail handlerless batch tasks in one transaction

_{Review effort: Lite}

[Copilot]

🟡 Changes recommended

Active critical and moderate findings remain: mixed batch outcomes can violate the fail-together contract when expiry is handled separately, and DrainLimit can be exceeded by batched claims.

2 open issues

• Expired task breaks batch fail-together behavior · New
• DrainLimit ignores tasks claimed in each batch · New

1 resolved since last review

• Batch expiry updates are not atomic

_{Review effort: Lite}