Skip to content

feat(node): follow mode — follow the chain from remote RPC sources - #482

Draft
samlaf wants to merge 1 commit into
mainfrom
sl/sei-1086-featsummit-follow-mode-following-the-chain-from-remote
Draft

samlaf wants to merge 1 commit into
mainfrom
sl/sei-1086-featsummit-follow-mode-following-the-chain-from-remote

Conversation

@samlaf

@samlaf samlaf commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Closes SEI-1086

Have been trying to figure out how to do rpc nodes in TEE world (see SEI-1055) and there are some issues with the current observer mode (mainly that all observers share the same ed25519 keys with the main node, so would need a transport). Sharing this draft PR for a "follow-mode" which is similar but doesn't require any consensus awareness. IMO this is simpler and would be easier to setup for rpc nodes since they can just setup their topology however they want, and create caching services, etc. without having to join the actual commonware p2p mesh.

Looks like a lot of code but the idea is quite simple, and represented by this diagram:
image
Most of the code is just plumbing and wiring. commonware's marshal (which we fork in sync) recent version even already has a follow-up which we could use in our fork. And the code could be made more generic between the node, observer, and follower modes, but have opted to keep this PR as small as possible just to get feedback, and have mentioned those possible cleanups in #481

LLM Summary

A follower runs the finalizer and syncer, so it executes every finalized block against its own execution client and tracks validator-set changes, but it runs no Simplex, joins no P2P peer set, and reads no keystore. It fetches finalized blocks from other nodes over JSON-RPC instead. Sources are untrusted: every reply goes through the syncer's handle_deliver, the same block/certificate checks a P2P peer's reply gets.

Serving side:

  • --follower-rpc-port / --follower-rpc-ip start an opt-in listener (loopback by default) serving getResolverValue and getLatestHeight. getResolverValue hands the key to the syncer's resolver handler (Handler::try_produce), so RPC and P2P serve identical bytes.
  • The listener has its own limits: no batching, 16 KiB requests, 100 connections, 1024 requests waiting on the syncer, and responses sized from genesis max_message_size_bytes. A full listener or mailbox returns 3008.
  • The resolver handler is now created in Engine::new and passed into resolver::p2p::init, so the RPC server can share it.

Following side (summit run --follow <url>..., conflicts with --observer):

  • syncer::resolver::opaque wraps commonware's opaque resolver behind a transport-free Source trait. Each attempt goes to a random source, bounded by the engine's fetch timeout.
  • node::follow: RpcSource (JSON-RPC Source), a follow Driver that runs in place of the orchestrator (registers epoch verifiers, polls source tips, hints the syncer at most 256 heights ahead and never past the entered epoch's last height), and a null network/oracle for the actors that still expect P2P.
  • Engine::start_follow wires these together. The application and orchestrator are built but never started. The engine runs verifier-only with throwaway keys.
  • The follower's RPC returns 4005 for getPublicKeys and doesn't start the admin listener.

Follow-ups:

  • refactor: remove follow-mode workarounds (unbuffered syncer, per-epoch engine in orchestrator) #481 removes the workarounds follow mode ships with, because the syncer and orchestrator still assume P2P and Simplex: the null broadcast network, keeping the unstarted application alive for its cancel-on-drop, and the driver copying the orchestrator's verifier registration. Step 1 ports marshal's unbuffered start into the syncer. Step 2 makes the orchestrator generic over the per-epoch engine, so followers run an epoch fetcher in it and follow/driver.rs goes away.
  • Separately, the node's mode is derived in several places today: Option<KeyStore> (None means follower), flags.observer, and a follower: bool passed to the RPC server. run_node_inner and run_node_local_inner also each duplicate the validator/observer setup. Parsing the flags once into a RunMode { Validator, Observer(index), Follower(sources) } and matching on it would remove those proxies and the duplication.

…EI-1086)

A follower runs the finalizer and syncer, so it executes every finalized
block against its own execution client and tracks validator-set changes,
but it runs no Simplex, joins no P2P peer set, and reads no keystore. It
fetches finalized blocks from other nodes over JSON-RPC instead. Sources
are untrusted: every reply goes through the syncer's `handle_deliver`,
the same block/certificate checks a P2P peer's reply gets.

Serving side:
- `--follower-rpc-port` / `--follower-rpc-ip` start an opt-in listener
  (loopback by default) serving `getResolverValue` and `getLatestHeight`.
  `getResolverValue` hands the key to the syncer's resolver handler
  (`Handler::try_produce`), so RPC and P2P serve identical bytes.
- The listener has its own limits: no batching, 16 KiB requests, 100
  connections, 1024 requests waiting on the syncer, and responses sized
  from genesis `max_message_size_bytes`. A full listener or mailbox
  returns 3008.
- The resolver handler is now created in `Engine::new` and passed into
  `resolver::p2p::init`, so the RPC server can share it.

Following side (`summit run --follow <url>...`, conflicts with
`--observer`):
- `syncer::resolver::opaque` wraps commonware's opaque resolver behind a
  transport-free `Source` trait. Each attempt goes to a random source,
  bounded by the engine's fetch timeout.
- `node::follow`: `RpcSource` (JSON-RPC `Source`), a follow `Driver` that
  runs in place of the orchestrator (registers epoch verifiers, polls
  source tips, hints the syncer at most 256 heights ahead and never past
  the entered epoch's last height), and a null network/oracle for the
  actors that still expect P2P.
- `Engine::start_follow` wires these together. The application and
  orchestrator are built but never started. The engine runs
  verifier-only with throwaway keys.
- The follower's RPC returns 4005 for `getPublicKeys` and doesn't start
  the admin listener.

Follow-ups:
- #481 removes the workarounds follow mode ships with, because the
  syncer and orchestrator still assume P2P and Simplex: the null
  broadcast network, keeping the unstarted application alive for its
  cancel-on-drop, and the driver copying the orchestrator's verifier
  registration. Step 1 ports marshal's unbuffered start into the syncer.
  Step 2 makes the orchestrator generic over the per-epoch engine, so
  followers run an epoch fetcher in it and `follow/driver.rs` goes away.
- Separately, the node's mode is derived in several places today:
  `Option<KeyStore>` (None means follower), `flags.observer`, and a
  `follower: bool` passed to the RPC server. `run_node_inner` and
  `run_node_local_inner` also each duplicate the validator/observer
  setup. Parsing the flags once into a
  `RunMode { Validator, Observer(index), Follower(sources) }` and
  matching on it would remove those proxies and the duplication.
@linear-code

linear-code Bot commented Oct 29, 2026

Copy link
Copy Markdown

SEI-1086

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant