Repository navigation
esp32: Add ESP-NOW - #2472
esp32: Add ESP-NOW#2472gabrielmancini wants to merge 10 commits into
Conversation
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
|
The failing |
|
The second red job, |
|
LLM review, seems like there is v2 of esp-now which allows more data: Review: ESP-NOW supportScope and recommendationRequest changes. Reviewed the combined diff
The second commit only reformats the The requested oracle review was performed, and its principal findings were cross-checked against the runtime, scheduler, and local ESP-IDF 5.5.5 sources. This is a source review, not a hardware qualification: no firmware build, allocation-fault test, or two-device radio test was run. Proposed diffs below are review suggestions, not applied or tested fixes. Severity: P1 = high-impact correctness issue to fix before merging; P2 = correctness issue under a narrower configuration or failure condition. 1. P1 — Abnormal port termination leaves live references to freed stateLocation: The port stores Trigger: Open ESP-NOW, terminate the port with Required fix: Tie teardown to the port's lifetime, including kill/linked-owner termination, through a runtime-supported resource monitor or a native-context cleanup facility. Make cleanup run exactly once and coordinate it with in-flight callbacks. A global driver-destroy callback alone would not cover individual port death. This is not safely fixed by a one-line change in the close branch. Missing regression: Terminate an open port without 2. P1 — Radio ownership and singleton transitions race on SMPLocation: The new shared flags, holder callback, and Two concrete failing interleavings:
Concurrent opens can also pass both singleton checks before either publishes ownership. Required fix: Serialize complete radio ownership transitions and singleton acquisition/release, including their SDK operations. Atomics on individual flags or a local copy of the callback do not preserve the ownership invariant. Choose one lock/serialized owner and document its order relative to listener and callback synchronization. Missing regression: On a dual-core target, repeatedly overlap open/open, close/network-start, and close/network-stop. Assert only one open succeeds and that the remaining radio user can still transmit after each transition. A single-core-only test cannot cover this finding. 3. P1 — AP-only networking disables ESP-NOW's selected interfaceLocation: Every peer, including the automatically added broadcast peer, uses Trigger: Either startup order of ESP-NOW plus an AP-only network. ESP-NOW cannot reliably send through its configured interface, contrary to the documented coexistence contract. Required fix: Derive the Wi-Fi mode from both owners. AP-only network plus ESP-NOW needs APSTA, whether the holder or the network starts first. Keep the extra STA unassociated. Simply selecting APSTA is insufficient: Missing regression: For both startup orders, start an AP-only network and exchange actual broadcast and acknowledged unicast frames with a second device. Verify the AP remains usable and the extra STA does not attempt an unintended connection. Repeat with STA and APSTA as controls. 4. P2 — Failed network startup does not restore a held radioLocation: Starting a network stops the existing bare STA, but Trigger: Open ESP-NOW successfully, then inject a failure after network startup stops the bare STA. Even subsequent Suggested narrow recovery change, after fixing the ownership synchronization: --- a/src/platforms/esp32/components/avm_builtins/network_driver.c
+++ b/src/platforms/esp32/components/avm_builtins/network_driver.c
@@
// A holder (ESP-NOW) keeps the radio on, as a bare STA again.
if (wifi_holder_restarted != NULL) {
- if (network_had_wifi && wifi_start_bare_sta() == ESP_OK) {
+ if (!wifi_started && wifi_start_bare_sta() == ESP_OK) {
wifi_holder_restarted();
}
} else {This repairs the skipped restart when cleanup runs; it does not replace the need for a defined startup-error rollback path or error handling if the recovery restart itself fails. Missing regression: Fault-inject failure after the held radio is stopped but before 5. P2 — RX silently truncates valid ESP-NOW v2 datagramsLocation: ESP-IDF 5.5.5 defines a 1470-byte v2 maximum alongside the 250-byte v1 maximum. The receive callback clamps The public API intentionally supports only 250 bytes; reject unsupported sizes rather than silently changing datagram contents. Suggested small fix: --- a/src/platforms/esp32/components/avm_builtins/espnow_driver.c
+++ b/src/platforms/esp32/components/avm_builtins/espnow_driver.c
@@
static void espnow_recv_cb(const esp_now_recv_info_t *info, const uint8_t *payload, int len)
{
+ if (len > ESP_NOW_MAX_DATA_LEN) {
+ // Use the synchronized drop-counter operation introduced for stats.
+ struct EspNowData *data = espnow_data;
+ if (data != NULL) {
+ data->dropped++;
+ }
+ return;
+ }
struct EspNowEvent event = { .kind = EspNowReceived };
memcpy(event.mac, info->src_addr, MAC_LEN);
event.rssi = info->rx_ctrl->rssi;
- event.len = len < ESP_NOW_MAX_DATA_LEN ? len : ESP_NOW_MAX_DATA_LEN;
+ event.len = len;The counter increment above illustrates the rejection path; it must use the synchronized counter and callback-lifetime scheme described below, rather than introduce another unsynchronized increment. Missing regression: Send distinct 249-, 250-, 251-, and 1470-byte frames from a v2-capable sender. Compare every accepted byte; require oversized frames to be absent from RX messages and counted as dropped. A send-side rejection test alone does not exercise this callback. 6. P2 — Allocation failures can crash open/RX or abandon calls foreverLocation: There are three separate exhaustion paths:
Suggested small fix for context allocation: --- a/src/platforms/esp32/components/avm_builtins/espnow_driver.c
+++ b/src/platforms/esp32/components/avm_builtins/espnow_driver.c
@@
Context *ctx = context_new(global);
+ if (IS_NULL_PTR(ctx)) {
+ teardown(data);
+ return NULL;
+ }
ctx->native_handler = espnow_consume_mailbox;
ctx->platform_data = data;This addresses only the NULL dereference and depends on making teardown safe. For RX, use checked binary construction before accessing the buffer, destroy the temporary heap on failure, and count the drop. For calls, use the runtime's established OOM/error or termination path instead of silently discarding a synchronous request. Missing regression: Inject context-allocation, refcounted-binary-allocation, and reply-heap failures separately. Assert failed open leaves no singleton/radio hold; failed RX is dropped without a crash; synchronous calls resolve or fail rather than hang. Additional issues and small suggestionsPending queue teardown violates the FreeRTOS queue-set contract
This is a confirmed API-precondition violation, but not independently proven to cause an immediate crash: AtomVM compares unmatched sender handles rather than dereferencing them. Queue-handle reuse and close/reopen under traffic need targeted testing. Treat this as part of the teardown fix, not as evidence that SDK callback unregistration necessarily waits for every in-flight callback. Dropped statistics are neither atomic nor valid across the full counter range
Use atomic updates/a synchronized snapshot and the existing boxed-integer construction API, reserving the additional heap words. Test concurrent increments and the values immediately below, at, and above the small-integer boundary. Do not create a new integer-conversion wrapper if an existing helper suffices. Describe the actual queue limit, not receiver-side backpressureThe eight-entry queue bounds Wi-Fi-task-to-VM backlog. Once the listener runs, it forwards events into the receiver's mailbox without checking whether that receiver consumes them. The receiver mailbox is not bounded by eight events. Documentation currently attributes drops to a slow receiver and can imply a stronger memory bound than implemented. --- a/libs/avm_esp32/src/espnow.erl
+++ b/libs/avm_esp32/src/espnow.erl
@@
-%% @doc `dropped': frames and send results lost because the receiver did not
-%% keep up (the driver queues 8).
+%% @doc `dropped': frames and send results lost by the driver, including
+%% callback-queue overflow. The driver queues 8 events before VM delivery;
+%% this does not bound the receiver's mailbox.Update the equivalent paragraph in Compile both callback signatures in CIThe only ESP-NOW-enabled CI configuration uses ESP32/IDF 5.5.5. The pre-5.5 send-callback branch is never compiled. Add a small enabled configuration on a pre-5.5 SDK and a single-core target, preserving existing disabled builds: --- a/.github/workflows/esp32-build.yaml
+++ b/.github/workflows/esp32-build.yaml
@@
include:
+ - esp-idf-target: "esp32c3"
+ idf-version: 'v5.2.8'
+ espnow: 'ON'
+ jit: false
- esp-idf-target: "esp32p4"Check the GitHub matrix expansion when applying this: the added entry augments the matching base combination; it need not create a duplicate build. Suggested missing public-API smoke testNo tests were added by these commits. The following is a small proposed ESP32-only test module, to register in the appropriate opt-in device harness once ESP-NOW is enabled. It checks outbound size validation and explicit close/reopen without relying on receipt of radio acknowledgments. It does not replace the lifecycle, SMP, AP coexistence, RX, and fault-injection scenarios above. --- /dev/null
+++ b/tests/libs/eavmlib/test_espnow.erl
@@
+-module(test_espnow).
+-export([test/0]).
+
+test() ->
+ {ok, Port} = espnow:open([{channel, 1}]),
+ try
+ {error, badarg} = espnow:send(Port, broadcast, <<>>),
+ {error, badarg} = espnow:send(Port, broadcast, binary:copy(<<16#A5>>, 251)),
+ {error, badarg} = espnow:set_channel(Port, 0),
+ ok = espnow:send(Port, broadcast, binary:copy(<<16#5A>>, 250)),
+ {ok, 1} = espnow:get_channel(Port)
+ after
+ ok = espnow:close(Port)
+ end,
+ {ok, Reopened} = espnow:open([{channel, 6}]),
+ try
+ {ok, 6} = espnow:get_channel(Reopened)
+ after
+ ok = espnow:close(Reopened)
+ end,
+ ok.This test sketch needs the standard source license header and device-harness integration before inclusion; it has not been compiled or executed. The asymmetric channel choices ensure reopening cannot pass by accidentally reporting the previous port's channel. Evidence and merge gate
|
bettio
left a comment
There was a problem hiding this comment.
I still have to do an API review. Some initial findings.
bettio
left a comment
There was a problem hiding this comment.
Few additional comments:
- Port death: see the inline comment at espnow_driver.c:327 for a fix that needs no runtime change.
- AP-only network: every peer is on WIFI_IF_STA while an
ap-only config runs WIFI_MODE_AP, so every send fails with ESP_ERR_ESPNOW_IF in both start orders. Either run APSTA while a holder exists and keep the STA unassociated (event_handler must then not call esp_wifi_connect on STA_START for an AP-only config), or refuse the combination with a clear error and document it. - RX frames above the limit (espnow_driver.c:133): drop and count them in
droppedinstead of truncating. - Check
context_newfor NULL (espnow_driver.c:325) andteardownon failure. - Drain the queue after unregistering the callbacks and before
xQueueRemoveFromSet(espnow_driver.c:241), and check its result: it fails on a non-empty queue. - Drops are not caused by a slow receiver: they happen when the 8-slot queue between the Wi-Fi task and the scheduler fills, and the receiver's mailbox is unbounded. Reword espnow.erl:121 and the guide at line 560.
- CI: only the 5.5.5 build compiles the driver, so the pre-5.5
espnow_send_cbis never built. Addespnow: 'ON'to a pre-5.5 matrix entry too, for example esp32c3 v5.2.8.
Two questions on the hardware tests, since the code relies on them: did espnow:set_channel/2 fail while the STA was connected (inline comment at espnow_driver.c:387), and did encrypted peers keep working across network:start/1 and network:stop/0?
|
This can be a follow-up commit / change, so an additional commit on this PR history. This supersedes my inline comment about the v2 frame size. Two parts: a different way to stage frames between the Wi-Fi task and the VM, which removes the fixed queue and the need for a slot-size option, and a teardown handshake that the driver needs regardless of which staging it uses. WhyOn a 32-bit target every binary of 32 bytes or more is a refc binary, so
One helper in libAtomVM(Let's implement this as a separated commit, to make review easier). The non-const branch of term term_from_refc_binary(struct RefcBinary *refc, Heap *heap, GlobalContext *glb)
{
term *boxed_value = memory_heap_alloc(heap, TERM_BOXED_REFC_BINARY_SIZE);
boxed_value[0] = ((TERM_BOXED_REFC_BINARY_SIZE - 1) << 6) | TERM_BOXED_REFC_BINARY;
boxed_value[1] = (term) refc->size;
boxed_value[2] = RefcNoFlags;
boxed_value[3] = (term) refc;
term ret = ((term) boxed_value) | TERM_PRIMARY_BOXED;
refc->ref_count = 1; // this heap's MSO list owns it from here
heap->root->mso_list = term_list_init_prepend(boxed_value + REFC_BINARY_CONS_OFFSET, ret, heap->root->mso_list);
synclist_append(&glb->refc_binaries, &refc->head);
return ret;
}Driver state#include <stdatomic.h>
#ifdef ESP_NOW_MAX_DATA_LEN_V2
#define ESPNOW_MAX_FRAME ESP_NOW_MAX_DATA_LEN_V2
#else
#define ESPNOW_MAX_FRAME ESP_NOW_MAX_DATA_LEN
#endif
#define ESPNOW_QUEUE_LEN 16
struct EspNowEvent
{
uint8_t kind;
uint8_t mac[MAC_LEN];
int8_t rssi;
uint8_t ok;
struct RefcBinary *frame; // EspNowReceived only; never listed while staged
};
struct EspNowData
{
QueueHandle_t queue;
EventListener listener;
GlobalContext *global;
int32_t receiver;
_Atomic uint32_t rx_bytes; // staged bytes, callback to delivery
_Atomic uint32_t dropped;
};
static _Atomic(struct EspNowData *) espnow_data; // stored before the callbacks are registered
static _Atomic uint32_t espnow_in_callback;
static _Atomic uint8_t espnow_channel; // read by espnow_wifi_restarted, which must not touch dataCallbacks, in the Wi-Fi taskThey take no VM lock: malloc, memcpy, atomics and a zero-timeout queue send. // NULL means the driver is gone: do nothing.
static struct EspNowData *callback_enter(void)
{
atomic_fetch_add(&espnow_in_callback, 1); // seq_cst, ordered before the load
struct EspNowData *data = atomic_load(&espnow_data);
if (data == NULL) {
atomic_fetch_sub(&espnow_in_callback, 1);
}
return data;
}
static void callback_leave(void)
{
atomic_fetch_sub(&espnow_in_callback, 1);
}
static void espnow_recv_cb(const esp_now_recv_info_t *info, const uint8_t *payload, int len)
{
struct EspNowData *data = callback_enter();
if (data == NULL) {
return;
}
if (len <= 0 || len > ESPNOW_MAX_FRAME) {
atomic_fetch_add(&data->dropped, 1);
goto leave;
}
if (atomic_fetch_add(&data->rx_bytes, len) + len > CONFIG_AVM_ESPNOW_RX_BUDGET) {
goto drop;
}
struct RefcBinary *frame = refc_binary_create_refc(len);
if (UNLIKELY(frame == NULL)) {
goto drop;
}
memcpy(frame->data, payload, len);
struct EspNowEvent event = { .kind = EspNowReceived, .rssi = info->rx_ctrl->rssi, .frame = frame };
memcpy(event.mac, info->src_addr, MAC_LEN);
if (LIKELY(xQueueSend(data->queue, &event, 0) == pdTRUE)) {
goto leave;
}
free(frame); // never listed, count still 0
drop:
atomic_fetch_sub(&data->rx_bytes, len);
atomic_fetch_add(&data->dropped, 1);
leave:
callback_leave();
}
Listener, in the schedulerstatic EventListener *espnow_event_callback(GlobalContext *glb, EventListener *listener)
{
struct EspNowData *data = GET_LIST_ENTRY(listener, struct EspNowData, listener);
struct EspNowEvent event;
if (xQueueReceive(data->queue, &event, 0) != pdTRUE) {
return listener;
}
bool received = event.kind == EspNowReceived;
size_t staged = received ? event.frame->size : 0; // read now: the frame may be freed below
size_t size = term_binary_heap_size(MAC_LEN)
+ (received ? TUPLE_SIZE(5) + TERM_BOXED_REFC_BINARY_SIZE + term_map_size_in_terms(1) : TUPLE_SIZE(4));
Heap heap;
if (UNLIKELY(memory_init_heap(&heap, size) != MEMORY_GC_OK)) {
if (received) {
free(event.frame); // still unlisted
atomic_fetch_sub(&data->rx_bytes, staged);
}
atomic_fetch_add(&data->dropped, 1);
return listener;
}
term message;
if (received) {
term payload = term_from_refc_binary(event.frame, &heap, glb); // listed, count 1
// {espnow, rx, Mac, Payload, #{rssi => Rssi}} built as today, with payload in place of the copy
} else {
// {espnow, sent, Mac, ok | fail} as today
}
globalcontext_send_message(glb, data->receiver, message); // the copy takes the count to 2
memory_destroy_heap(&heap, glb); // the sweep takes it back to 1, the receiver's
if (received) {
atomic_fetch_sub(&data->rx_bytes, staged);
}
return listener;
}If the receiver no longer exists the send copies nothing and the sweep frees the frame, so a dead receiver leaks nothing. TeardownThis order is needed with the current staging too. On a dual-core target the Wi-Fi task can be inside a callback on the other core while static void teardown(struct EspNowData *data)
{
esp_now_unregister_recv_cb();
esp_now_unregister_send_cb();
esp_now_deinit();
atomic_store(&espnow_data, NULL); // no new callback touches data from here
while (atomic_load(&espnow_in_callback) != 0) { // one already running finishes first
vTaskDelay(1);
}
atomic_store(&espnow_channel, 0);
sys_unregister_listener(data->global, &data->listener); // returns only after a running dispatch
struct EspNowEvent event;
while (xQueueReceive(data->queue, &event, 0) == pdTRUE) {
if (event.kind == EspNowReceived) {
free(event.frame); // staged only, never listed
}
}
xQueueRemoveFromSet(data->queue, event_set); // succeeds: the queue is empty
vQueueDelete(data->queue);
network_wifi_release();
free(data);
}
|
|
Thanks to both of you for the reviews. Pushed six commits on top, one per area, and tested them on two ESP32 boards (ESP-IDF 5.5.5).
Hardware answers.
What the hardware test covered (one board driving, one answering): unicast with and without an ack ( I could not run erlfmt locally this time; CI will tell if |
An espnow port driver, behind CONFIG_AVM_ENABLE_ESPNOW (off by default),
and the espnow module: frames of up to 250 bytes to a peer or broadcast,
received as {espnow, rx, Mac, Data, #{rssi := Rssi}}, with the send result
as {espnow, sent, Mac, ok | fail}.
The network driver becomes the one owner of the Wi-Fi radio. ESP-NOW holds
it (network_wifi_hold/1): with no network the radio comes up as a bare
STA, and network:stop/0 leaves it on while ESP-NOW is open, so ESP-NOW
survives the network going down and up again.
Events go through the VM listener, queued 8 deep; a frame that does not
fit is dropped and counted (stats/1).
The API follows harmon25/atomvm_espnow (Apache-2.0).
Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
From the review: - Replies are built on a stack heap and always sent: a request whose reply did not fit was dropped, and its caller waited in port:call/2 forever. - ESP-NOW errors a caller can act on come as atoms (not_found, peer_exists, peer_list_full, interface_mismatch, channel_mismatch, no_mem), others through esp_err_to_term/2. - add_peer/3 and del_peer/2 refuse broadcast: the broadcast peer is the driver's, and deleting it broke every later broadcast. - set_channel/2 is refused with sta_connected while the STA is associated, instead of relying on esp_wifi_set_channel to fail; open leaves the channel of an associated STA alone too. - A received frame longer than 250 bytes (ESP-NOW v2) is dropped and counted, not truncated. - A failed context_new tears the driver down; the queue is emptied before it leaves the event set, which fails on a non-empty queue. - The docs say where drops happen: the driver's queue, not the receiver's mailbox. - CI builds ESP-NOW on ESP-IDF 5.2 too (esp32c3), for the pre-5.5 send callback. Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
…h ESP-NOW
From the review:
- network:start/1 returns before its port starts Wi-Fi, so on a
dual-core target espnow:open/1 could run network_wifi_hold while
start_wifi ran, both on unsynchronized flags. One mutex (a no-op
without SMP) now covers network_wifi_hold, network_wifi_release,
start_wifi and the radio part of stop_network.
- stop_network brings a held radio back as a bare STA whenever it is
off, not only when the network used it: start_wifi stops the held
bare STA before it marks the radio as the network's, and a failure in
between left ESP-NOW open with the radio off.
- An AP-only network runs the radio in WIFI_MODE_AP, where ESP-NOW,
whose peers are on the STA interface, cannot send. network:start/1
returns {error, ap_only_held} for one while the radio is held, and
network_wifi_hold refuses while one runs (ESP_ERR_NOT_SUPPORTED).
Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
From the review:
- espnow:open/1 creates the port cheaply and opens ESP-NOW in its first
call, {open, Options}, whose reply says why it did not:
{error, already_open}, {error, ap_only}, badarg for a bad option, the
ESP-IDF error otherwise. Before, every failure was badarg from
open_port. A port that fails to open terminates.
- The port's context does not point at the driver's state: a port killed
(exit/2, a linked owner that crashed) is destroyed without its handler
running, and context_destroy freed the state the Wi-Fi task, the
listener and the radio hold still used. The state keeps the port's
process id; only that port drives ESP-NOW ({error, not_open} for
others), and an open that finds the port gone tears the old state
down and opens again.
- espnow:open/1 links the port to its caller, so a supervised owner
that crashes takes the port down and can open ESP-NOW again.
- The channel restored after network:stop/0 is kept apart from the
state, which the network driver's callback no longer reads.
Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
The non-const branch of term_alloc_refc_binary without the allocation: wraps a refc binary created and filled elsewhere, unlisted and with a count of 0, into a term of a heap, whose MSO list takes it. A driver can then fill the binary a receiver will see in its own callback, outside of the scheduler, and copy each payload once (ESP-NOW, next commit). Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
…lbacks
From the review:
- Received frames: the Wi-Fi task fills, with one copy, the refc binary
the receiver gets (term_from_refc_binary), and queues a small item.
The fixed slots of 250 bytes (12 KB idle with v2 frames) are gone, and
a frame of any length ESP-IDF delivers is kept: up to 1470 bytes with
ESP-NOW v2. CONFIG_AVM_ESPNOW_RX_BUDGET (8192 bytes) bounds what waits
for the scheduler; past it frames are dropped and counted.
- send/3 waits for the send callback and returns ok or {error, fail}:
the driver gives the radio one frame at a time, as the ESP-NOW guide
recommends, since nothing in the callback says which frame it is
about, and answers each caller from its frame's callback. The
{espnow, sent, Mac, Result} message is gone. Frames up to 1470 bytes
on ESP-IDF 5.4 or newer.
- Teardown: the callbacks are unregistered and the state unpublished
first, then teardown waits for a callback already running (counted on
entry), then the listener, the queue and its staged frames, then the
radio. The channel restored after network:stop/0 is an atomic that
teardown does not free.
Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
Answering the review's question with a hardware test: encrypted peers did
not survive network:start/1 and network:stop/0. Stopping the radio clears
the keys ESP-NOW installed in it; the peer list stays, so frames went out
unencrypted and the receiver dropped them, while the sender saw the MAC
acknowledgement and got ok.
The network driver now calls the holder each time it stops and starts a
held radio (a network starting, and stopping back to a bare STA), and
ESP-NOW sets its PMK and its encrypted peers' LMKs again. It keeps them
for that, apart from the state teardown frees.
The guide says how an AP-only network is refused while ESP-NOW is open:
network:start/1 returns before the driver starts Wi-Fi, so the network
process stops with {start_port_failed, ap_only_held}.
Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
b21536a to
b6ed4e5
Compare
|
Rebased on |
|
AMP code: Latest review: ESP-NOW — PR #2472Scope and recommendationRequest changes. Reassessed PR #2472 at b6ed4e56 on 2026-11-17. Oracle checked all seven findings from the preceding review at b21536a5 against the latest source and relevant runtime and ESP-IDF 5.5.5 contracts. All seven remain applicable; none was fixed or disproven. F3 is configuration-dependent, and the runtime consequences described below have not been independently reproduced on hardware. The ESP-NOW driver and Erlang wrapper are unchanged between those heads. Intervening PPP, resource/select-event, and term changes do not resolve the findings. The original review also covered the feature updates since Two P1 concurrency defects remain. The smaller corrections below include proposed diffs where a local change is appropriate. These are review suggestions, not applied or tested fixes. The concurrency and teardown changes need a coherent ownership design rather than isolated one-line patches. All What the update resolves
F1. P1 — Singleton readers race with reclamation and closeLocations: After the owner port dies, two schedulers can both load the retained Atomic publication protects the pointer value, not the object's lifetime. The callback counter covers Wi-Fi callbacks, not scheduler-side readers. The radio mutex is acquired too late to protect this lookup/reclamation path. The scheduler releases its scheduling lock before invoking native handlers ( Required fix: Serialize scheduler-side singleton lookup/use, stale-state reclamation, initialization, and close under a lifecycle mutex or equivalent ownership protocol. A CAS claiming teardown alone does not protect readers already holding the pointer. Preserve callback/listener synchronization and define the lock order with radio ownership. Missing regression: On a dual-core target, kill the owner and synchronize two concurrent reopen attempts. Require exactly one success and no double cleanup. Separately race open against explicit close, including uninitialized ports. Repeat with memory-error detection where practical. F2. P1 — Restart can restore an obsolete LMK over a successful re-keyLocations: The restart callback snapshots encrypted-peer keys under
Both public peer operations can return Required fix: Serialize each complete SDK peer mutation plus cache update against the radio stop/start/restore transaction. Include init/deinit and sends in the ownership design so they cannot enter halfway through restoration. Use a sleeping mutex; do not hold a FreeRTOS critical section across SDK calls. Missing regression: Pause restart after its snapshot, delete/re-add a peer using an asymmetric new LMK, then resume. Check the final SDK key and an encrypted exchange. Also pause between SDK add and cache update. The stale-key overwrite follows from the source; actual on-air behavior still needs device validation. F3. P2 — Accepted encrypted peers can exceed restoration-cache capacityLocations:
Suggested correction: Use the SDK's total peer bound for the cache, leaving the SDK to enforce its configured encrypted-peer limit. This avoids relying on the contradictory six-peer macro: --- a/src/platforms/esp32/components/avm_builtins/espnow_driver.c
+++ b/src/platforms/esp32/components/avm_builtins/espnow_driver.c
@@
-static struct EspNowKey espnow_lmks[ESP_NOW_MAX_ENCRYPT_PEER_NUM];
+static struct EspNowKey espnow_lmks[ESP_NOW_MAX_TOTAL_PEER_NUM];
@@
- if (espnow_lmk_count < ESP_NOW_MAX_ENCRYPT_PEER_NUM) {
+ if (espnow_lmk_count < ESP_NOW_MAX_TOTAL_PEER_NUM) {
@@
- struct EspNowKey lmks[ESP_NOW_MAX_ENCRYPT_PEER_NUM];
+ struct EspNowKey lmks[ESP_NOW_MAX_TOTAL_PEER_NUM];This increases static and callback-stack storage; check the target's stack allowance. Alternatively size from the configured capacity or enforce a documented six-peer limit before accepting another encrypted peer. None of these alternatives replaces F2's synchronization fix. Missing regression: On default ESP32, verify SDK acceptance of seven encrypted peers, restart the radio, and verify all seven keys and encrypted exchanges. Verify acceptance on hardware because the SDK header macro and configured capacity disagree. Also test a higher configured capacity where supported. F4. P2 — A normal mailbox message can fabricate send completionLocations: Any sender holding the port can send This is a completion-correlation defect reachable through the port mailbox, not evidence of a remote-radio attack. Required fix: Make the mailbox message only a wakeup and consume an actual synchronized driver-owned completion, or use private correlated completion tokens. Validating only Missing regression: Inject the lookalike tuple while A is outstanding and B waits. Neither caller may finish, nor B reach the SDK, before the actual completion of A. Keep the legitimate callback path as a control. F5. P2 — The caller can die before its ownership link is installedLocation: The wrapper links the port only after initialization returns. AtomVM's Also, a normal linked exit does not kill an ordinary non-trapping peer, contrary to the broad documentation promise that caller death closes the port. Small suggested change for the initialization window: --- a/libs/avm_esp32/src/espnow.erl
+++ b/libs/avm_esp32/src/espnow.erl
@@
Port ->
+ link(Port),
Opts = [{receiver, Receiver} | proplists:delete(receiver, Options)],
@@
- link(Port),
{ok, Port};This narrows the initialization window, not the entire lifetime problem: the caller can still die between port creation and linking. Handle ownership-installation failure and review normal-exit behavior. If every owner exit must close the port, implement an ownership mechanism that observes every exit; otherwise narrow the promise. Monitoring is not a drop-in native-port fix: the current scheduler ignores native Missing regression: Kill the caller while initialization is paused, resume, and verify another caller can open. Test normal exit separately from crash/kill. F6. P2 — Member draining leaves queue-set notifications behindLocation: Teardown now drains AtomVM compares unmatched handles rather than immediately dereferencing them, so an immediate use-after-free from this alone is not established. FreeRTOS removal does not purge the corresponding set notifications. Handle reuse and temporary stale-notification capacity consumption remain risks; lost events or persistent capacity exhaustion have not been established. Required fix: Coordinate final draining and deletion with the event-poll owner, consuming both notifications and payloads without discarding unrelated drivers' events. Do not reset or blindly drain the shared set as a local workaround. Missing regression: Repeatedly close/reopen with RX pending. Verify no stale notifications or lost set notifications, and verify another driver sharing the set continues to receive events. F7. P2 — Dropped statistics still overflow the small-integer representationLocations: Atomic updates fix lost increments, but --- a/src/platforms/esp32/components/avm_builtins/espnow_driver.c
+++ b/src/platforms/esp32/components/avm_builtins/espnow_driver.c
@@
- term_set_map_assoc(map, 0, globalcontext_make_atom(glb, ATOM_STR("\x7", "dropped")), term_from_int(atomic_load(&data->dropped)));
+ term dropped = term_make_maybe_boxed_int64(atomic_load(&data->dropped), heap);
+ term_set_map_assoc(map, 0, globalcontext_make_atom(glb, ATOM_STR("\x7", "dropped")), dropped);
@@
- BEGIN_WITH_STACK_HEAP(PORT_REPLY_SIZE + TUPLE_SIZE(2) + term_map_size_in_terms(1), heap);
+ BEGIN_WITH_STACK_HEAP(PORT_REPLY_SIZE + TUPLE_SIZE(2) + term_map_size_in_terms(1) + BOXED_INT64_SIZE, heap);Missing regression: Initialize the counter under controlled test conditions and check the public stats result at 134,217,727, 134,217,728, and UINT32_MAX. Assert exact nonnegative values. Documentation suggestionThe PR description still describes the original 250-byte, asynchronous-send, eight-slot API. Current code and documentation implement SDK-dependent 250/1470-byte frame limits, synchronous sends, and a 16-slot event queue with one slot reserved for send completion and a separate RX byte budget. Refresh the PR description before merge. If ownership remains link-based, clarify the normal-exit limitation in both --- a/libs/avm_esp32/src/espnow.erl
+++ b/libs/avm_esp32/src/espnow.erl
@@
-%% to the caller: if the caller dies the port closes, and ESP-NOW can be
-%% opened again.
+%% to the caller: an abnormal caller exit terminates the port, and a later
+%% open reclaims its ESP-NOW state. A normal caller exit does not close it;
+%% call close/1 explicitly before returning normally.Verification and merge gate
|
|
Thanks for the reassessment. All seven findings are addressed in one commit on top, Ownership design (F1, F2). Rather than adding a second lifecycle mutex next to the radio's, ESP-NOW now uses the network driver's radio lock for everything it does in the scheduler. It is exposed as
F3. The cache is sized F4. The listener stores the result in an atomic F5. F6. Teardown no longer drains the member queue. After the callbacks and the radio are released, and with the listeners lock held so no listener runs, it marks the state
The listener is now the first member of the state, because F7. Evidence so far: source and build only. The ESP32/IDF 5.5.5 build with ESP-NOW and Ethernet enabled is clean, with no warnings, and erlfmt passes. The hardware regressions you listed come next on two boards:
I will post the results here. Some of these need a test hook to pause at the right point, and I will keep such hooks out of this PR unless you would like them in. |
|
Hardware regressions for
Notes:
The test app is about 400 lines of Erlang plus two start modules. It needs two boards, so it does not fit the QEMU suite. I can push it as an example or attach it if that would help. |
From the review of 2026-11-17: - The radio lock (network_wifi_lock/0, the network driver's wifi_mutex) is taken around everything the scheduler does with ESP-NOW: open, close, reclaiming a killed port's state, peers and their keys, sends. The network driver holds it while it restarts the radio and calls espnow_wifi_restarted, so two ports can no longer both reclaim the same state, an open cannot use a state a close is freeing, and a peer added again with a new LMK cannot be overwritten by a restart with the old one (F1, F2). network_wifi_hold/1 and network_wifi_release/0 are called with it taken. Lock order: the radio lock, then the listeners lock. - The LMK cache has room for every peer (ESP_NOW_MAX_TOTAL_PEER_NUM): the SDK takes as many encrypted peers as CONFIG_ESP_WIFI_ESPNOW_MAX_ENCRYPT_NUM (7 by default), not the 6 of ESP_NOW_MAX_ENCRYPT_PEER_NUM (F3). - The send result is left by the listener in the driver state, and the atom espnow_sent only wakes the port up: a lookalike message from any process finds no result and completes nothing (F4). - espnow:open/1 links the port before ESP-NOW starts, and the docs say a normal caller exit does not close it (F5). - Close no longer empties the queue itself, which left its notifications in the shared event set. Under the listeners lock, an empty queue goes at once; one with events left goes from the listener, after it takes the last one through the event set as usual (F6). The listener is the first member of the state, which sys_listener_destroy frees. - stats/1 boxes a dropped count past the small integers (F7). Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
a83da48 to
d217abb
Compare
|
Formatting-only amend: clang-format 16 removed one blank line left in |
Adds an
espnowport driver on ESP32, behindCONFIG_AVM_ENABLE_ESPNOW(off by default), and theespnowmodule inavm_esp32.{espnow, rx, Mac, Data, #{rssi := Rssi}}, each one's binary copied once, in the Wi-Fi task.send/3is synchronous. It returnsokonce the frame is out (for a unicast frame, the peer acknowledged it) or{error, fail}. The driver gives the radio one frame at a time, as the ESP-NOW guide recommends, so concurrent senders wait their turn.open/1(receiver,channel0..14,pmk),close/1,add_peer/2,3(optionalchannelandlmk),del_peer/2,send/3(a MAC orbroadcast),set_channel/2,get_channel/1,stats/1.CONFIG_AVM_ESPNOW_RX_BUDGET(8192 by default). A frame past either limit is dropped and counted instats/1.open/1reclaims ESP-NOW. A normal exit does not close it, so callclose/1.The Wi-Fi radio gets one owner, the network driver. ESP-NOW holds it through
network_wifi_hold/1andnetwork_wifi_release/0(include/network_wifi.h), and does not callesp_wifi_initoresp_wifi_startitself.network_wifi_lock/0), serializes the network's start and stop with everything ESP-NOW does in the scheduler: open, close, peers and keys, sends.network:start/1andnetwork:stop/0can be called while ESP-NOW is open. Each radio restart installs the PMK and the encrypted peers' LMKs again.network:stop/0leaves the radio on and back on the ESP-NOW channel.{error, ap_only}), and the other way round.The API follows harmon25/atomvm_espnow (Apache-2.0) by @harmon25:
open,send,add_peer,del_peerand the{espnow, rx, Mac, Data}message. Its driver is what our first range tests ran on. This one is new code, because ESP-NOW here has to share the Wi-Fi radio with thenetworkdriver and survivenetwork:start/1andnetwork:stop/0. Thanks to @harmon25 for it.The ESP32 v5.5.5 PPP build in the CI matrix also enables ESP-NOW, so the two network options are built together, and an ESP32-C3/IDF 5.2.8 build covers the older send-callback signature. The network programming guide gets a section.
Tested on hardware: ESP32 (T-PCIE, WROVER with the VM not using PSRAM) and ESP32 without PSRAM (T-Call). The numbers below were measured side by side (−26 to −30 dBm, 250-byte frames) with an earlier asynchronous version of
send; they are kept as a range reference:The tests also covered:
network:start/1andnetwork:stop/0cycles with ESP-NOW open;The review-driven changes since then (radio lock, deferred queue teardown, private send completion) still need their hardware regression run. No QEMU test: QEMU has no Wi-Fi radio.
These changes are made under both the "Apache 2.0" and the "GNU Lesser General Public License 2.1 or later" license terms (dual license).
SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later