Repository navigation
Feature Request for Vehicle Service Orchestrator #2597
Description
Change Request Type
Feature Request
Description of the Change Request
Vehicle Service Orchestrator (VSO) is a Multi-node Scenario Evidence Layer for Eclipse SDV S-CORE.
VSO defines multi-node scenario contracts (pipeline chains) and subscribes to runtime/diagnostic/platform signals.
monitors pipeline execution determinism, detects policy violations at scenario level, and correlates evidence using scenario_run_id.
VSO does NOT own diagnostic fault lifecycle, does NOT decide Function Group State, and does NOT execute application lifecycle.
VSO hands off determinism evidence packages and conditionRefs to Safety Managers for final safety decisions and approvals.
Execution is performed by S-CORE Lifecycle.
Core Proposal:
- VSO does not execute
- VSO does not decide
- VSO observes multi-node scenario pipelines
- VSO detects determinism policy violations
- VSO generates structured evidence
- VSO delivers evidence packages for decisions and statemanagement.
Execution remains in S-CORE Lifecycle. Safety Decisions remain external to S-CORE.
The intent of this feature request is to define the architecture and interfaces for the Vehicle Service Orchestrator
as a scenario evidence layer that complements existing S-CORE Diagnostics, Lifecycle and Orchestrator components.
Motivation
Problem Statement
Modern SDV scenario pipelines (L3 Highway, Parking/Valet, ADAS services) span HPC, Zonal Controllers and ECUs.
Policy based decisions become ineffective without VSO Evidences, such as decisions to switch Autonomous mode to Manual mode, or degrade to a lower ASIL level.
Pipeline stages have timing, resource, dependency and diagnostic constraints.
What goes wrong without VSO:
- Raw diagnostic data is available, but scenario-level operational evidence is not standardized
- Policy violation data is often lost, late, or not correlated by scenario_run_id
- Orchestrators may accidentally become state decision makers
- Diagnostic fault lifecycle and scenario policy lifecycle may be mixed
- Debug escalation is ad-hoc and not contract-driven
- Lack of structured evidence for safety decsisions and learning
- Fleet/cloud analysis loses context and root-cause evidence
- No standard for multi-node scenario evidence layer
- Missing determinism monitoring layer
Gap Analysis:
- S-CORE Diagnostics → "what happened" (raw signals, faults, logs)
- VSO → "did multi-node pipeline break determinism ?" (scenario-level evidence)
- OEM State Manager → "what should we do" (decision-making)
- S-CORE Lifecycle → "execute decision" (lifecycle actions)
S-CORE already has Diagnostics, Lifecycle, Orchestrator, Logging, and FEO-related building blocks.
VSO is proposed as a scenario evidence policy layer that consumes those signals without duplicating their ownership.
Why VSO Fits S-CORE
S-CORE Direction:
- Common open-source SDV runtime foundation
- Modular, extensible, safety-ready stack
- Feature requests can propose structural changes and new functionality
Diagnostics Direction:
- S-CORE Diagnostics proposes SOVD-based diagnostics
- Real-time fault access and advanced data handling
- Diagnostic feature scope
VSO Alignment:
VSO aligns with S-CORE's feature request process by introducing a bounded, scenario operational feature
that does not bypass safety or execution authority. VSO complements existing S-CORE components:
- Diagnostics: VSO consumes diagnostic outputs; it does not redefine diagnostics
- VSO: VSO generates evidence; safety managers make decisions
- Lifecycle: VSO observes; Lifecycle executes
Rationale
Multi-node Scenario Awareness
What "true multi-node awareness" actually means:
A system is multi-node aware only if a control entity can answer all four questions at runtime:
- Which nodes exist right now?
- What is the role and capability of each node?
- What is the real-time health of each node (not just apps)?
- Can workloads be safely placed / moved / stopped across nodes?
S-CORE today can only partially answer #4 inside a single node.
Health signals S-CORE already provides (baseline):
- Component alive / heartbeat
- Lifecycle state (Init, Running, Error, Terminated)
- Local application failures
- Controlled shutdown / restart hooks
These are necessary, but not sufficient for multi-node decision.
Additional signal categories required for true multi-node awareness:
- CATEGORY A: Node Identity & Topology Signals
- CATEGORY B: Node Liveness & Reachability Signals
- CATEGORY C: Resource Availability (Node-Level)
- CATEGORY D: Safety Capability & ASIL Compatibility Signals
- CATEGORY E: Workload Placement Feedback (Cross-Node)
- CATEGORY F: Fault Domain & Failure Propagation Signals
VSO Solution: Multi-node Scenario Evidence Layer
VSO addresses these gaps by:
- Defining and identifying pipeline chains per Node
- Monitoring end-to-end determinism
- Detecting and correlating violations
- Generating structured evidence packages
OEM State / Safety Manager subscribes and receives:
- Violation type
- Affected nodes
- Confidence level
- conditionRefs for execution adaptation
- evidence quality metrics (freshness, completeness)
Determinism Monitoring
VSO defines and monitors determinism contracts enabling multi-node scenarios.
Key Concepts:
- Multi-node Scenario: A defined pipeline chain and execution dependencies (e.g., L3 Highway: Perception → Fusion → Planning across HPC and zone nodes)
- Determinism Contract: Constraints that ensure timing, ordering, and resource guarantees for the entire pipeline (e.g., end-to-end latency ≤ budget, stage ordering preserved, CPU < 90%)
- Determinism Evidence: Scenario-level evidence capturing violations of pipeline determinism across nodes
- Scenario Policy: Rules for classifying evidence as OK, WATCH, WARN, VIOLATED, or RECOVERED
- Evidence Package: Structured package delivered to OEM State Manager / Safety Manager (scenario_run_id, pipeline_id, affected_nodes, violation_type, confidence)
Clear Boundaries
VSO maintains strict boundaries to avoid conflicting with existing S-CORE components:
VSO Is:
- Multi-node scenario evidence layer
- Pipeline determinism monitoring layer
- Scenario contract manager
- Debug/focused collection response coordinator
- Evidence package generator
- Diagnostic visibility publisher
- Operator/dashboard notification source
VSO Is Not:
- ASIL safety decision layer
- Diagnostic Fault Manager
- DTC lifecycle owner
- OEM Decision Manager replacement
- Lifecycle/Execution replacement
- Container/process/resource controller
- Actuator/trajectory/emergency braking controller
Specification
Overview
VSO is a Multi-node Scenario Evidence Layer that monitors pipeline chain determinism per node and generates evidence for state and Safety Management.
Input:
- Runtime timing (multi-node)
- Diagnostic data / fault status
- Platform resource metrics (per node)
VSO Processing:
- Multi-node pipeline monitoring
- Determinism contract evaluation
- Per-node evidence correlation
Output:
- Determinism violation evidence package
- conditionRefs for execution adaptation
- Diagnostic/decisions event notification
System Architecture
S-CORE Platform Architecture with VSO Integration
VSO is integrated as a Multi-node Evidence Layer within the Eclipse SDV S-CORE platform architecture, positioned between
the Diagnostics module and OEM/Vehicle components.
VSO Architecture Details
The system follows a modular architecture integrated within the S-CORE ecosystem:
- Data Inputs Layer: Per-node signals (Runtime Plane, Diagnostics Module, Platform Resources)
- VSO Core Modules: Scenario Contract Management, Evidence Aggregation, Evidence Package Generation, Response Management
- Output Layer: OEM State Manager / Safety Manager integration, S-CORE Lifecycle handoff
Backwards Compatibility
- VSO operates as a passive observer consuming existing diagnostic/runtime signals
- OEM State Manager integration is optional and OEM-specific
- VSO can operate as PerNode in virtualization scenario as well as multi node scenario
- Existing fault management and lifecycle workflows remain fully preserved
VSO can be enabled or disabled via the feature flag without affecting existing S-CORE functionality.
The image-delivery pipeline, data formats, and OS initialization procedures (Linux/QNX) remain unchanged.
Furthermore, safety and security features complement—rather than replace—current mechanisms,
ensuring full backward compatibility for the entire platform and all existing applications.
Estimates for realization
Will be provided Based on the Next Course of Action
Affects work products
- Requirements
- Architecture
- Safety/Security Analysis
- Detailed Design
Impact analysis
Security Impact
Vehicle Service Orchestrator shall:
- Implement mutual TLS authentication for all inter-component communication
- Validate and sanitize all Manifest inputs
- Audit all orchestrator operations for security monitoring
Since the orchestrator manages both QM and ASIL workloads, a security breach must not violate safety guarantees.
Safety Impact
The Vehicle Service Orchestrator is classified as ASIL_B due to its role in managing safety-critical workloads.
While the orchestrator does not directly implement safety functions (e.g., braking, steering),
it ensure the runtime decisions and resource guarantees necessary for safety-critical applications to meet their timing and reliability requirements.
A failure in the orchestrator's resource allocation, timing enforcement, or automatic recovery mechanisms could indirectly impact safety by:
- Causing timing violations in ASIL-D applications (e.g., delayed object detection in AEB)
- Allowing resource contention between QM and ASIL workloads
- Failing to restart safety-critical containers after crashes
To address these risks:
- ASIL-level workloads are allocated dedicated, isolated resources
- Timing constraints are continuously validated with millisecond-level precision
- Automatic recovery mechanisms ensure continuity of critical functions
- Health checks and monitoring detect failures immediately
Safety analysis (FMEA, DFA) will be conducted to identify and mitigate potential failure modes.
Safety or Security relevance
- none
- Safety relevant
- Security relevant
ASIL classification
ASIL_B
Expected Implementation Version
1.5
Activity
anmittag commentedon Feb 12, 2026
@basheerFZ Thank you for your request, adhoc two questions:
a) QM for ADAS?
b) how does it fit with the other already "available" frameworks/ochestrators? or extend or replace? => open for me, maybe you have some architecture design documents etc.?
basheerFZ commentedon Feb 13, 2026
@anmittag
a) QM for ADAS? -- My Bad i understand its wrong, i have modified.
b) how does it fit with the other already "available" frameworks/ochestrators? or extend or replace? => open for me, maybe you have some architecture design documents etc.?
We will Attach the Architecture image as png in the PR with respective rst files.
vso: create initial draft feature request [eclipse-score#2597]
fixed docsascode issues [eclipse-score#2597]
fix doc issues [eclipse-score#2597]
fixed docs issue[eclipse-score#2597]
basheerFZ commentedon Feb 13, 2026
Diffferences: VSO provides operational flexibility that is not available in FEO, Orchestration, or Lifecycle & Health. Since VSO includes a Manifest that defines scenarios for specific conditions, it can behave adaptively based on the vehicle’s situation and status.
In addition, our goal is to containerize the existing frameworks or orchestrators and operate them as VSO. While the existing frameworks or orchestrators manage workloads at the process, thread, task, or activity level, VSO handles workloads at the container level. As a result, it can offer several advantages described in the FR.
JanStaschulat commentedon Feb 17, 2026
How would the cloud-native application in the container communicate data to the rest of the S-CORE-based system? I assume, that the cloud-native application in the container reads some input data and creates some output data, which comes from/is used by some other S-CORE component. If no framework, like FEO/Orchestrator/DAAL, is used, how does the application communicate with other S-CORE parts?
Do you want to directly use the Platform API of S-CORE? If this is the case, I guess, at least that part of your cloud-native application would have to be adapted.
What are your thoughts on this?
basheerFZ commentedon Feb 18, 2026
Thanks for the question — your understanding is correct.
To communicate with the rest of the S‑CORE–based system, any application (including a cloud‑native application running in a container) must use a communication method that aligns with S‑CORE. We fully agree on this point.
However, we would like to clarify that this requirement is not unique to cloud‑native applications.
Even a traditional/non‑containerized application must also be adapted if it needs to interact with S‑CORE components. In other words, the need for adaptation is a general requirement when integrating with S‑CORE, regardless of how the application is packaged or deployed.
On the other hand, if the cloud‑native application does not rely on other S‑CORE components (e.g., FEO / Orchestrator / DAAL) and does not need to exchange data with the S‑CORE platform, then it can typically run without any modifications. This is one of the notable advantages of enabling containerized cloud‑native workloads within the S‑CORE environment.
In summary:
Yes, we agree that proper adaptation is required to communicate with S‑CORE.
But this is equally true for non–cloud-native applications.
When no interaction with S‑CORE components is needed, existing cloud‑native applications can run as-is, without modification.
We hope this clarifies our perspective. Happy to discuss further integration options if needed.
Chulhee1Lee commentedon Feb 18, 2026
Thanks for your question!! 👍
To add to the previous comment, as you know, packaging an application into a container does not mean creating a special or different type of application. Typical vehicle services — for example, those using SOME/IP, DDS, D‑BUS, or Shared‑Memory IPC — can also be packaged into containers just like cloud‑native applications. Therefore, no additional modifications are required specifically for the proposed Vehicle Service Orchestrator. We suggest developing applications in the same way as before, using the S‑CORE Platform API, FEO, Orchestrator, or DDAL as needed, and then packaging and deploying them as containers.
FScholPer commentedon Feb 20, 2026
@basheerFZ we already have parts of your proposal in lifecycle and health: https://eclipse-score.github.io/score/main/features/lifecycle/index.html (OCI compliance, sandboxing). I think it would make sense to do a feature_modification request to lifecycle and health
13 remaining items
RolandJentschETAS commentedon Mar 9, 2026
I'm completely struggling with how this can be safety qualified. Also any static and dynamic connection to the S-CORE environment is unclear for me. How communicate the applications inside a container one to another ? How the apps inside a container communicate to the applications in another container ? How is the connection of the apps and the container to lifecycleManager, to logging, to persistency, to diagnosis, to health, to feo, to ... . How is this dynamicly handled if the container framework starts a new container ? How is this handled under QNX ? What are the safety concepts, if this container management start and stops containers ? How is FFI is ensured ?
JanStaschulat commentedon Mar 9, 2026
I have some questions regarding the terminology and contents:
daeyoung-jeong-lge commentedon Mar 13, 2026
Thank you @RolandJentschETAS for your detailed questions regarding safety qualification, communication, and integration within the S-CORE environment. I’d like to address your points regarding Communication, S-CORE Module Integration, QNX Support, and Safety Concepts (FFI).
1. Inter-Application Communication
Within a Single Container:
Applications running inside the same container can communicate using the same IPC and networking mechanisms they would use on a native OS. A container provides process and namespace isolation, but within that single container boundary, processes share the same user space. Therefore, standard IPC mechanisms work without special changes.
Between Different Containers:
For inter-container communication, applications simply utilize the standard communication resources provided by the underlying host system.
2. Connection to S-CORE Modules
Based on the standard behavior of Linux containers and their interaction model with the host framework, we expect that existing S-CORE modules (Lifecycle, Logging, Persistency, Diagnosis, FEO, etc.) can be integrated without major architectural changes.
We are currently planning an internal Integration PoC to verify that these services connect naturally without requiring intrusive modifications to the existing modules. Through this validation process, we will define concrete interoperability standards between the VSO and existing S-CORE modules.
3. QNX Support
We are actively collaborating with the QNX team regarding container technology support, starting with QNX SDP 8. Since QNX’s container support is in an early stage and has architectural differences from Linux-based implementations, we are working to ensure consistency.
Our goal is to ensure that applications can run using the same APIs and manifest/spec definitions on both Linux and QNX. While the QNX container runtime is still evolving, we are aligning our development and operational models to support both environments seamlessly.
4. Safety Concepts & FFI
FFI is ensured through a combination of hardware partitioning, OS-level resource isolation, and orchestration-level enforcement provided by Pullpiri and Timpani.
Spatial Interference (Memory)
Temporal Interference (CPU/Timing)
Communication Interference
Lifecycle Interference & Supervision
We hope this clarifies the architecture and safety strategies. We welcome further discussion as we refine these implementations.
daeyoung-jeong-lge commentedon Mar 13, 2026
Thank you @JanStaschulat for your valuable feedback regarding the terminology and concepts. We acknowledge that some terms were not clearly defined in the initial proposal. We plan to update the documentation in the near future to clarify these definitions.
Here are the answers to your specific questions:
Terminology & Definitions
Q1: Is the term "workload" equivalent to "container"?
A1: Yes, in the current version of Pullpiri, we use the term "workload" equivalently to "container".
Q2: How does "node" relate to "workload" and "container"? Or is this a term for hardware resource?
A2: A "node" refers to a computing resource unit, such as an ECU or a VM (Virtual Machine), where the workloads/containers are executed.
Q3: What does "resource management per node" mean?
A3: It encompasses managing both hardware-centric computing resources (e.g., CPU, Memory of the ECU/VM) and the software containers running on top of them.
Q4: What are "services" (Abstract, Req 3.2)?
A4: In the context of Requirement 3.2, a "service" refers to the target object managed by the Orchestrator. Effectively, it implies the containerized application being deployed.
Q5: On which level is "Resource Management" (Req 3.3)?
A5: Resource management is performed at the Service level managed by the VSO.
To clarify the terminology used in our feature request:
Monitoring & Execution
Q6: Which entities are being monitored (Req 3.4)?
A6: The Services operated by the VSO has the entities being monitored. This includes not only container computing resources, but also ECU-level constraints such as power, total CPU capacity, total memory, and thermal limits.
Q7: Difference between "process status" and "application-level health"? Is monitoring application-level and resource management container-level?
A7: Yes, your understanding is correct.
Q8: Is the application aware of its platform (ECU, cores) or deployment changes?
A8: The application inside is not aware of these details. Only the VSO handles these configurations. The application remains agnostic to the underlying infrastructure changes.
Migration Strategy
Q9: If a container is migrated, is it copied/created or just started/stopped?
A9: We assume that the container images are available on the relevant ECUs (or pulled from a registry). Migration in this context means stopping the container on the source node and starting the instance on a target ECU where the container corresponds to a deployable unit.
We hope this clarifies the concepts. We will incorporate these definitions into the specification to avoid further confusion.
updated the revised VSO Proposal with Feature request[eclipse-score#2597
basheerFZ commentedon Jun 12, 2026
@qor-lb @FScholPer --> we have updated the feature request from the old Proposal to the new Proposal , We are ready for the discussions with Technical Council
anmittag commentedon Jul 10, 2026
@qor-lb any final feedback from Architecture Community?
qor-lb commentedon Aug 24, 2026
closed as discssed in https://github.com/orgs/eclipse-score/discussions/2457?sort=new#discussioncomment-17645226