depthai-rs

Safe Rust for Luxonis depthai-core v3 (OAK cameras), with no C++ visible to Rust and no bindgen/autocxx/cxx at build time.

Crate	Role
depthai-sys	Raw FFI. A hand-written pure-C shim (csrc/depthai_c.h/.cpp, ~120 functions) over depthai-core, compiled by build.rs; links = "depthai-core". Enum values and struct layouts are static_assert-pinned against the pinned depthai-core tag.
depthai	The safe wrapper: Device, Pipeline, nodes (Camera, Sync, StereoDepth, VideoEncoder, Imu), typed Output<M> / OutputQueue<M>, messages (ImgFrame, MessageGroup, ImuData, EncodedFrame), CalibrationHandler, DeviceBootloader.

Pinned depthai-core: v3.7.1.

depthai-sys is raw: one C function per depthai-core member, named after it; std::optional/overloads collapse into sentinels that select the C++ default. The wrapper is deliberately faithful and unopinionated: it does not read environment variables, convert timestamps to wall-clock, clamp IMU rates, pick stereo presets, repack strides, or default the unit / spec-translation choices in the calibration getters. Those decisions belong to the driver built on top (see kornia/sensor-rt's sensor-oak).

use std::time::Duration;
use depthai::node::{Camera, Sync};
use depthai::{CameraBoardSocket, Device, ImgFrame, ImgFrameType, ImgResizeMode, Message, Pipeline};

let dev = Device::open(None, None)?;                 // first available OAK
let pipeline = Pipeline::new(&dev)?;
let left = pipeline.create::<Camera>()?;  left.build(CameraBoardSocket::CamB)?;
let right = pipeline.create::<Camera>()?; right.build(CameraBoardSocket::CamC)?;
let lo = left.request_output((640, 400), Some(ImgFrameType::Gray8), ImgResizeMode::Crop, Some(30.0), Some(false))?;
let ro = right.request_output((640, 400), Some(ImgFrameType::Gray8), ImgResizeMode::Crop, Some(30.0), Some(false))?;
let sync = pipeline.create::<Sync>()?;
sync.set_sync_threshold(Duration::from_millis(16))?;
lo.link(&sync.input("left")?)?;
ro.link(&sync.input("right")?)?;
let q = sync.out()?.create_output_queue(4, false)?;
pipeline.start()?;
while let Some(group) = q.get(Duration::from_secs(1))? {
    let l: ImgFrame = group.get("left")?.expect("left");
    let r: ImgFrame = group.get("right")?.expect("right");
    // l.data() / r.data(): zero-copy GRAY8; clone the frame to keep it past the next poll
}

Messages are refcounted handles: Clone is a refcount bump, data() is zero-copy, and a clone stays valid across later polls and threads (Send + Sync).

Building

depthai-sys needs a depthai-core install prefix (lib/cmake/depthai, include/depthai, lib/libdepthai-core.so). depthai-core is not packaged for Ubuntu/Debian; Luxonis ships only pip wheels and source. The ROS apt repos carry ros-kilted-depthai (3.9.0, Ubuntu 24.04 only) — ros-humble-depthai on 22.04 is v2 and unusable here. So: build it once from the pinned submodule.

git submodule update --init --recursive
DEPTHAI_CMAKE_EXTRA="-D DEPTHAI_OPENCV_SUPPORT=OFF" bash depthai-sys/scripts/build_depthai.sh

• Needs cmake ≥ 3.20, ninja, a C++17 compiler and ~4 GB of free disk for the vcpkg build trees (prune depthai-sys/vendor/depthai-core/build/vcpkg/buildtrees if space is tight). 30–60 min on a Jetson Orin (-j2, RAM-capped).
• DEPTHAI_OPENCV_SUPPORT=OFF skips the OpenCV/ffmpeg vcpkg build (hours) — this ABI needs none of it. Upstream then leaves three ImageFilters/Rectification symbols undefined inside libdepthai-core.so; csrc/depthai_nocv_stub.cpp provides weak throwing definitions so it links. depthai-core itself is never patched.
• The script copies vcpkg's libusb-1.0.so into the prefix (depthai-core's DT_NEEDED names the unversioned file), so the prefix is self-contained.
• pixi run depthai-build does the same inside a conda env with cmake/ninja/ pkg-config, for machines without them.

Resolution order in build.rs:

1. DEPTHAI_PREFIX=/path/to/prefix
2. a vendor/depthai directory found by walking up from the crate — what the script above produces
3. with --features vendored: the same build, done by build.rs into ~/.cache/kornia-depthai/<tag>/<target> (DEPTHAI_RS_CACHE_DIR, DEPTHAI_JOBS to override).

build.rs bakes an absolute rpath into depthai-sys's own targets only (cargo does not propagate link args). A crate that ships binaries adds two lines to its own build.rs, using the metadata depthai-sys exports (needs a direct dependency on depthai-sys):

if let Ok(rpath) = std::env::var("DEP_DEPTHAI_CORE_RPATH") {
    for dir in rpath.split(':') { println!("cargo:rustc-link-arg=-Wl,-rpath,{dir}"); }
}

or set LD_LIBRARY_PATH=<prefix>/lib at run time.

Check-only builds (CI, laptops without depthai-core): DEPTHAI_SYS_SKIP_NATIVE=1 cargo test links an error-only stub so cargo check/clippy and the pure-Rust tests run anywhere. Never ship that.

On a Jetson Orin cap cargo with CARGO_BUILD_JOBS=2.

Tests

• cargo test — unit tests (layout pins, enums, conversions) + host-only graph tests (Pipeline::host_only(), no camera; auto-skip on the stub).
• DEPTHAI_HIT=1 cargo test -p depthai --test hit -- --ignored --test-threads=1 — hardware tests with an OAK attached.
• cargo run --example list_devices | stereo_sync | imu_dump | rgbd_depth.
• gated_h264: camera → Gate → VideoEncoder; the host opens/closes the stream at runtime with GateControl (nothing crosses the link while closed).
• yolo_detect: DetectionNetwork + the zoo's YOLOv6-nano on an OAK-D (RVC2), decoded on the device into ImgDetections.
• rfdetr_segment: NeuralNetwork + RF-DETR nano instance segmentation (RVC4 only), decoded on the host from NnData tensors, with the H.264 stream gated to bursts after a detection.

Adding to the ABI

1. Declare the function in depthai-sys/csrc/depthai_c.h, implement it in depthai_c.cpp (wrap the body in DAI_GUARD).
2. python3 depthai-sys/scripts/gen_ffi.py && python3 depthai-sys/scripts/gen_stub.py (regenerates src/ffi.rs and the stub; CI checks they are committed).
3. Wrap it in depthai/.

Symbol prefix is dai_; if you ever link this next to another depthai binding that also uses dai_, expect duplicate symbols.

License

Apache-2.0.