Skip to content
jacopoabramoPublic

About

Python inter-process containers with shared memory backend.

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

sharedbox

PyPI CI CodSpeed Ruff clang-format Checked with mypy

Warning

This project is a work in progress; be patient or feel free to contribute.

sharedbox lets Python processes on one machine share data in shared memory, without a server and without pickling. It gives you two things to share:

  • a box, one record that every process reads and writes, as if they all held the same dataclass;
  • a stream, a flow of typed items that one process sends and each reader receives once.

You declare the fields once, and every process that opens the box or the stream works on the same bytes in shared memory, so nothing passes through a pipe or a socket between the processes. That fits wherever separate processes need the same data at once: web workers sharing state with a dashboard, a GUI and the backend process that does its work, the stages of a data pipeline, the state of a game or a simulation, or lab instruments and the programs that show them.

Installation

sharedbox needs CPython 3.11 or newer on Windows x64 or Linux x86_64. PyPI has compiled wheels, so you don't need a compiler:

pip install sharedbox

or, in a project that uv manages:

uv add sharedbox

Quick start

import multiprocessing as mp
from typing import Annotated

from sharedbox import Capacity, SharedBox


class Mailbox(SharedBox):
    letters: int
    flag_up: bool
    owner: Annotated[str, Capacity(32)]


def deliver() -> None:
    mailbox = Mailbox.attach()  # finds the box by its class
    mailbox.letters = 1
    mailbox.close()


if __name__ == "__main__":
    with Mailbox(0, False, "Ada") as mailbox:
        mailbox.events.letters.connect(lambda new, old: print(old, "->", new))
        carrier = mp.Process(target=deliver)
        carrier.start()
        carrier.join()  # prints: 0 -> 1
    Mailbox.unlink()

The carrier process finds the box by its class alone, and the parent prints the change as soon as the carrier makes it.

Documentation

The documentation site starts with three tutorials, then has how-to guides for single tasks, such as sending items through a stream. After them come explanations of how a box works and the API reference. C and C++ code can use a box too, either handed over from Python or opened by name; the C and C++ guides show how.

Development

To change sharedbox itself you need git, uv, CMake 3.30 or newer and a C++20 compiler (MSVC or GCC):

git clone https://github.com/jacopoabramo/sharedbox.git
cd sharedbox
uv sync --dev
uv run pytest

How to set up a development environment has the details.

License

Licensed under Apache 2.0

About

Python inter-process containers with shared memory backend.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages