-
Notifications
You must be signed in to change notification settings - Fork 0
Describe the low-level API #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
griffinmilsap
wants to merge
2
commits into
main
Choose a base branch
from
low-level-api
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| Low-level API | ||
| ##################### | ||
|
|
||
| `ezmsg` exposes two layers of API that build on the same pub/sub core: | ||
|
|
||
| A **low-level API** that lets you create publishers and subscribers directly, and a **high-level API** that lets you describe a processing graph (a dataflow pipeline) and have ezmsg manage execution for you. | ||
|
|
||
| This page explains the low-level model and how it differs from the high-level API so you can choose the right tool for the job. | ||
|
|
||
|
|
||
| |ezmsg_logo_small| The Low-level Model | ||
| **************************************** | ||
|
|
||
| At its core, `ezmsg` is a publish/subscribe messaging system. | ||
|
|
||
| - A **publisher** can send messages to **many subscribers**. | ||
| - A **subscriber** can listen to **many publishers**. | ||
| - **Channels** route messages between publishers and subscribers; they are **created and managed automatically** when you connect endpoints. | ||
| - Messages can be any Python object. `AxisArray` is optional, not required. | ||
|
|
||
| The low-level API gives you direct access to these primitives. You decide **when** to publish, **how** to receive, and **how** to schedule your own control flow. This makes the low-level API a good fit when you want to integrate messaging into an existing application structure instead of adopting the full `ezmsg` pipeline runtime. | ||
|
|
||
|
|
||
| |ezmsg_logo_small| Relationship to the High-level API | ||
| ****************************************************** | ||
|
|
||
| The high-level API is a **stream-based dataflow runtime**. You define a graph of processing nodes (`Unit` and `Collection`) and connect their input/output streams. The graph is then executed by `ezmsg` (typically via `ez.run()`), which wraps your main entry point and manages scheduling, process layout, and message flow. | ||
|
|
||
| Under the hood, the high-level API is **built on the same publisher/subscriber primitives**. Input/output streams are convenience wrappers around those low-level endpoints. | ||
|
|
||
|
|
||
| |ezmsg_logo_small| Choosing the Right API | ||
| ******************************************* | ||
|
|
||
| The **low-level API** is a good fit when: | ||
|
|
||
| - You already have an application loop or framework you want to keep. | ||
| - You want explicit control over concurrency and scheduling. | ||
| - You only need a small number of pub/sub links instead of a full graph. | ||
| - You are integrating `ezmsg` into a larger system that already manages process lifetime. | ||
|
|
||
| The **high-level API** is a good fit when: | ||
|
|
||
| - You want to express a graph of processing steps and let `ezmsg` execute it. | ||
| - You want consistent dataflow semantics and standardized stream connections. | ||
| - You benefit from the pipeline tooling (graph visualization, CLI integration, etc.). | ||
| - You want a structured way to scale across threads/processes without managing it yourself. | ||
|
|
||
|
|
||
| |ezmsg_logo_small| Examples | ||
| ******************************** | ||
|
|
||
| The ezmsg repository includes example scripts that demonstrate the low-level API in both synchronous and asynchronous styles: | ||
|
|
||
| - `ezmsg/examples/simple_publisher.py` | ||
| - `ezmsg/examples/simple_subscriber.py` | ||
| - `ezmsg/examples/simple_async_publisher.py` | ||
| - `ezmsg/examples/simple_async_subscriber.py` | ||
KonradPilch marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| .. important:: The synchronous low-level API is a **convenience wrapper** around the async implementation. Internally it uses a threaded `asyncio` loop and calls like ``run_coroutine_threadsafe`` to bridge into the async backend. This is ergonomic (and familiar to users coming from ROS2-style semantics), but it comes with **throughput overhead**. For high data-rate paths, the async low-level API is recommended. | ||
|
|
||
| .. note:: For a full introduction to the high-level API, see the :doc:`Tutorials <../tutorials/content-tutorials>` and the :doc:`Pipeline how-to <../how-tos/pipeline/content-pipeline>`. | ||
|
|
||
| .. |ezmsg_logo_small| image:: ../_static/_images/ezmsg_logo.png | ||
| :width: 40 | ||
| :alt: ezmsg logo | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.