System Architecture

This document describes the system architecture of “Palmimo DevKit,” a desktop-sized hexapod robot.

This document covers the palmimo_sdk robot.py facade + engine.py computation layer + ServoDriver abstraction: the current structure and the design intent behind it (why this layering).

Overall System Architecture

The overview is shown below.

Power is external and compute is a Raspberry Pi: the DevKit carries a Raspberry Pi 5 (16 GB) as its control host, the servos run from a 5 V DC adapter, and the Pi from its own USB PD supply — there is no onboard battery. During development a PC can also act as the host (see below).

Execution Model

The execution host is assumed to be a Raspberry Pi by default. The control loop (gait generation → servo writes, 60 Hz) runs entirely on the Pi, which connects directly to the OpenRB-150 (USB–DYNAMIXEL bridge) over USB serial. Because servo round trips never cross a network, the loop is unaffected by network latency or jitter. During development, a PC connects over SSH etc., but is not involved in control.

Layer	Component	Role
Development machine	PC (SSH, etc.)	Editing, launching, log viewing. Not involved in control
Execution host	Raspberry Pi (default assumption)	Runs palmimo_sdk (Palmimo + MotionEngine + ServoDriver) and user code
Interface board	OpenRB-150	USB-connected to the Pi. Ships pre-flashed as a USB–DYNAMIXEL bridge, relaying servo commands onto the bus
Actuators	21× serial-bus smart servos	18 leg + 3 neck. Protocol 2.0 / baud 1,000,000

Software Architecture

Data Flow

1. User calls robot.forward() -> sets Motion enum
2. Each robot.step() polls the neck and leg guards and applies their verdicts
3. The flight recorder consumes the guards’ completed sweeps and polls latched hardware-error status when due, before the next goal is computed
4. MotionEngine computes Dict[str, int] (servo_name -> tick_value); ServoDriver sends positions to the bus when connected and permitted by the guards
5. A due incident is written synchronously after that frame’s goal write; the callback then receives the positions
6. Repeat at target fps (default 60); without a driver, computation still runs

The recorder observes the last successfully submitted facade goal alongside telemetry, so a reading is compared with the goal that preceded it. Diagnostic persistence lives in _flight_recorder.py; _flight_ride.py connects it to the existing sweep producers without opening a hardware backend. During disconnect, parking steps defer incident persistence until after the driver’s disconnect attempt, keeping disk latency out of the torque-off path. See Servo Flight Recorder for the design rationale and its API contract for the public controls and file format.

Key Design Principles

• Computation != Communication: engine.py never touches serial ports
• Neutral Contract: Every motion starts and ends at neutral stance — raw servo neutral (2048) on every joint except neck_pitch1, which settles at the engine’s rest-trimmed center (see neck_rest_pitch_deg tuning in api-reference.md); neck_pitch2 has no trim of its own and settles at raw neutral
• Frame-based: Each step() produces one frame of servo positions
• Deterministic: Same motion + same phase = same output (testable without hardware)
• Lean core: pyserial is the SDK’s only base dependency; every heavier import stays behind an extra, so import palmimo_sdk needs nothing beyond pyserial. TTS synthesis is pluggable (TtsEngine, default PiperEngine over piper-plus) and lives in palmimo_sdk.io.tts behind the speech extra; palmimo_sdk.io.speaker.Speaker itself is engine-agnostic and only owns queueing/playback. Shared mic capture, GTCRN denoise, and DTLN-AEC echo cancellation (palmimo_sdk.audio) live behind the voice extra. Any further voice-command processing (VAD / KWS / ASR) is the caller’s responsibility, built on top of these — see the examples/agents/wakeword app for a concrete instance
• palmimo_sdk.io owns I/O resources: every hardware boundary is consolidated under io/ — servos are ServoDriver / DynamixelDriver, camera capture is HeadCamera, and mic/speaker/display are Microphone / MicStream / Speaker / FaceDisplay. Upper layers only consume these; they never open a backend themselves. palmimo_sdk.io is an internal layer — consumers import from the palmimo_sdk root (direct reach-in is blocked by ruff TID251). Heavy backends (dynamixel_sdk / cv2 / piper / pyserial / sounddevice) are imported lazily, so importing the module itself stays hardware-independent. This avoids duplicating the same capture logic across consumers (vision processing lives in the app, capture lives in the SDK). MicStream is the resource that shares one physical mic across several consumers (an always-on VAD, on-demand ASR capture, a denoiser, …): it opens a single sounddevice.InputStream on a background thread and fans each chunk out to every subscribe()d consumer. That a Microphone and a MicStream point at the same physical mic is expressed only by the convention of giving both the same device_key string (there is no mechanism that matches device identities)
• palmimo_sdk.audio is a pure processing layer: where io owns hardware and returns raw bytes/arrays, audio only transforms sample streams and holds no I/O — the same separation as engine.py holding no servo I/O. GTCRN noise removal (SpeechDenoiser / StreamingDenoiser, in denoise.py) and DTLN-AEC acoustic echo cancellation (DtlnResidual / GatedDtlnCanceller, in dtln.py) both live here, lazily importing sherpa_onnx / ai-edge-litert / numpy. AudioProcessor (a WAV-bytes-in/mono-WAV-bytes-out Protocol) is the DI seam MicStream / Microphone route captured audio through, and any number of processors can be cascaded — only the first may require more than one input channel (its capture_channels convention). Denoiser wraps StreamingDenoiser behind that contract; EchoCanceller (in aec.py) wraps GatedDtlnCanceller the same way, consuming a multi-channel WAV (microphone + loudspeaker reference) and producing mono. MicStream defaults to processors=[EchoCanceller()] ([] for raw audio)
• The facade bundles peripherals: peripherals are owned symmetrically with the driver by the facade, as Palmimo(display=..., speaker=..., camera=..., mic=...); connect() / disconnect() open and close them together with the servo bus — connect() also auto-wakes a connected driver, and disconnect() parks it (neutral return + neck soft-release) before closing. Every peripheral is optional — with no device present, set_expression() / say() become no-ops returning None (upper layers never need to know whether the physical layer exists). If one peripheral fails to open after another has already opened, the ones already open are closed before the error is re-raised, so a partial failure never leaves a port open. Unlike set_expression() / say(), HeadCamera / Microphone don’t get a read method added on the facade — since a single physical device can have multiple readers (for a camera: VLM observe / wave watch), consumers read/record directly via robot.camera / robot.mic (the same shape as robot.driver.write_positions)

Peripheral Ownership

Palmimo optionally owns the driver plus FaceDisplay / Speaker / HeadCamera / Microphone, and connect() / disconnect() open and close them together. Data flows in two directions — servo / face / speech send commands, camera / mic ingest data — but direction does not decide ownership. Palmimo owns all of them regardless.

Microphone is used by apps whose interaction completes within a single recording. The SDK also ships MicStream (shared streaming capture) and palmimo_sdk.audio (GTCRN noise removal, DTLN-AEC echo cancellation), so raw chunk capture, denoising, and echo cancellation can all ride on the SDK. VAD / KWS / ASR themselves remain the caller’s responsibility — the SDK stops short of them. An app that runs its own streaming VAD + ASR stack and doesn’t use the SDK’s mic simply keeps mic=None — the facade “can own it, but doesn’t force it.” This is exactly the same treatment as every other peripheral; mic is not a special case.

Resource reservation boundary

The backend that opens a device also owns its logical resource reservation: DynamixelBus owns servo_bus, HeadCamera owns camera, Microphone and MicStream own microphone, Speaker owns speaker, and FaceDisplay owns display. The facade delegates to these owners and uses its existing rollback to close earlier resources when a later open fails. Apps, diagnostics, and MCP consumers therefore share the same arbitration without taking a second set of locks at the facade layer.

palmimo_sdk.reservation uses non-blocking advisory flock locks. OS process exit releases them, including after SIGKILL. Process-local reference counts allow several backend instances to express intent for the same resource; other processes remain excluded until the last reference is released. This coordination does not replace the mic registry’s in-process stream delegation or make multiple hardware handles safe to operate concurrently. Code opening devices directly outside the SDK does not participate, and a forked child can extend an inherited lock’s lifetime.

A reservation lasts across gaps in playback or recording. Teardown releases it only after the device handle closes, so a bounded camera or mic close that abandons a blocked read preserves exclusivity until deferred cleanup succeeds. The lock file is retained rather than unlinked, keeping all waiters on the same inode. Shared storage under /run/lock/palmimo lets local users coordinate without group membership or image-specific setup. Newly created directories are sticky and world-writable (1777); files are world-readable and writable (0666). Directories are published after their shared permissions are set, so peers never see a partially configured directory at the reservation path. Lock opens reject symlinks and validate the descriptor as a regular file before locking or writing metadata. Opening existing files without O_CREAT permits cooperation under Linux’s fs.protected_regular. An unusable shared directory fails rather than silently splitting reservations into private temporary storage. See resource reservations for API, storage, and error contracts.

Package Structure

A uv-managed workspace holding the SDK core; palmimo_sdk is the only member. The examples under examples/ each consume it through the Palmimo facade, but as their own standalone uv projects depending on the published palmimo-sdk package, not as workspace members — see examples/README.md. The dependency only ever runs one way — the core never imports a package built on top of it:

pyproject.toml                          # uv workspace root
packages/
  palmimo_sdk/                             # Core SDK — the single user-facing window
    palmimo_sdk/
      __init__.py                       # Re-exports Palmimo / Motion / RoutineStep / MotionEngine / ServoDriver / DynamixelDriver / kinematics / palmimo_motor_ids / SUPPORTED_MOTOR_MODELS
      robot.py                          # Palmimo facade — public API + connection lifecycle
      _flight_recorder.py               # Servo incident rings and JSONL persistence, no bus I/O
      _flight_ride.py                    # Guard-sweep reuse and hardware-error polling for the recorder
      _leg_safety.py                    # LegSafety — one leg telemetry sweep judged by the overload and rail guards
      engine.py                         # MotionEngine — pure gait/IK computation, no I/O
      hardware_error.py                 # Public descriptions of latched servo hardware error bits
      kinematics.py                     # Shared IK/FK (leg_ik / servo ticks / body-frame foot position)
      io/__init__.py                    # Re-exports ServoDriver / HardwareErrorRead / DynamixelDriver / HeadCamera / Microphone / MicStream / Speaker / FaceDisplay / find_servo_port
      io/base.py                        # ServoDriver ABC (I/O boundary) + ServoTelemetry / ServoPositions / HardwareErrorRead
      io/dynamixel.py                   # DynamixelDriver — concrete ServoDriver over the Dynamixel bus (+ find_servo_port auto-detection)
      io/camera.py                      # HeadCamera — head-camera capture resource (full-FOV MJPG -> downscale -> rotate)
      io/microphone.py                  # Microphone — USB mic capture (arecord/rec → WAV bytes)
      io/mic_stream.py                  # MicStream — shared streaming mic capture (background thread, fan-out to subscribers)
      io/_mic_registry.py               # Process-local device_key registry coordinating Microphone <-> MicStream ownership
      io/speaker.py                     # Speaker — non-blocking TTS orchestration (queue/worker/barge-in), delegates synthesis to a TtsEngine
      io/tts/base.py                    # TtsEngine ABC / TtsVoice protocol — the synthesis-engine boundary
      io/tts/piper.py                   # PiperEngine — the default TtsEngine, over piper-plus (MIT)
      io/display.py                     # FaceDisplay — host-side client for the RP2350 face display (USB-CDC)
      audio/denoise.py                  # SpeechDenoiser / StreamingDenoiser — GTCRN (sherpa-onnx) noise removal, pure sample transforms
      audio/dtln.py                     # DtlnResidual / GatedDtlnCanceller — DTLN-AEC (ai-edge-litert) echo cancellation, pure sample transforms
      audio/aec.py                      # EchoCanceller — AudioProcessor implementation wrapping GatedDtlnCanceller
      audio/processor.py                # AudioProcessor protocol (WAV in/out, DI/cascade) + Denoiser/ClipDenoiser + int16_to_wav/wav_to_int16/wav_to_int16_multi
    tests/                              # engine / robot / driver / kinematics tests
examples/
  agents/
    wakeword/                              # Wake-word voice agent example (Silero VAD + Whisper STT + LLM tool-calling)
    companion/                             # Always-on companion agent example (idle loop + speech/vision-driven responses, LLM tool-calling)
    openclaw/                              # Connection kit for driving Palmimo from OpenClaw (self-hosted AI assistant) over the MCP server
scripts/                                # Supported user diagnostics (diagnose_servos: scan / power / errors / joints / recover / oscillate)

Dependencies

• pyserial>=3.5 — the SDK’s only base dependency (serial port communication)
• Everything else lives behind an extra in packages/palmimo_sdk/pyproject.toml: hardware (dynamixel-sdk, the servo communication protocol, via DynamixelBus), face (pyserial, for FaceDisplay), speech (piper-plus TTS), voice (mic streaming plus denoise/AEC — sounddevice, numpy, sherpa-onnx, ai-edge-litert), agent (pydantic, the LLM tool-calling layer), vision (opencv-python, for HeadCamera), and mcp (mcp, mcp-types, starlette, uvicorn, plus agent)
• tqdm>=4.66.0 — a dependency of this repository’s own scripts/, not of the SDK package

Design Intent (Why This Layering)

1. Hardware-independent SDK entry point — reaching for a motor library directly couples callers to internal implementation. palmimo_sdk is the single window, so users only need to know the Palmimo facade
2. Separating motion computation from I/O — gait generation, IK, and other motion computation is confined to MotionEngine, which only passes computed results (servo positions) to the Driver. The computation layer is testable without hardware
3. Driver abstraction — I/O is encapsulated behind the ServoDriver ABC, in preparation for future backends
4. Lean core — heavy optional dependencies (voice / tts) stay behind extras. import palmimo_sdk costs one base dependency (pyserial), and app-level processing built on top of them lives in the examples/ apps (this is also a premise of distributing it as an OSS library)

As shown in the software architecture diagram above, the entry-point class Palmimo delegates motion computation to MotionEngine, and I/O is encapsulated behind ServoDriver. The backbone policy is to keep computation (gait / IK) and communication (serial) in separate layers.

• robot.py (Palmimo) = state and orchestration — holds the user-facing API and the connection lifecycle (with-block control): connect() opens resources and, with a connected driver, auto-runs the wake() glide (limp -> neutral) unless built with auto_wake=False; disconnect() parks the robot first (return-to-neutral, then a neck soft-release ramp) before closing. __exit__ delegates to disconnect() — a clean exit parks fully, but after an exception only the neck soft-release runs (motion commands are skipped so they can’t mask the error / act on a possibly-bad state). Computation is delegated to MotionEngine, I/O to ServoDriver
• engine.py (MotionEngine) = pure computation — computes and returns servo positions from commands/frames only. Holds no state and does no I/O (testable without hardware)
• io/ (ServoDriver + concrete drivers) = the I/O boundary — streams computed servo positions to the backend. dynamixel_sdk (DynamixelBus) is imported lazily via an optional extra (palmimo-sdk[hardware]), so the core stays hardware-independent

Separating motion selection (forward(), etc.) from execution (run(seconds=) / run(steps=)) lets you specify “how many seconds / how many steps” without counting control frames yourself, and the control rate (fps) is centralized in Palmimo.

run() is also the one place a caller running on another thread can interrupt the robot: Palmimo.cancel() increments an internal counter under a small lock and is documented as the single facade method safe to call cross-thread (it touches no other state, so it cannot race the motion loop’s own reads/writes). Each paced public method (run() / perform_dance() / play_realtime()) snapshots that counter the instant it is entered, and _pace() — the shared pacing loop all three use — compares the live counter against that snapshot once per frame, raising MotionCancelled there if it increased, unwinding the blocking call within roughly one frame period instead of only once it finishes on its own. perform_dance()’s own non-paced end-hold and glide-home sections poll the same counter on a short interval, and guarantee stop() (IDLE) runs before a cancellation propagates out of it. Taking the snapshot at entry (rather than clearing a threading.Event at the pacing loop’s own start, the earlier design) means a stray cancel() delivered while idle never carries over into the next call, without the clear()/set() race a clearable Event has at that same entry point.

A caller that DISPATCHES a paced call onto another thread (rather than calling it inline) has one more window to close: a cancel() landing after dispatch but before the worker thread has actually reached its paced method’s own entry would otherwise be absorbed by that method’s entry snapshot. Palmimo._arm_cancel_scope() / _disarm_cancel_scope() close this structurally — arm records the live counter as the armed baseline right before dispatch, and whichever paced method the worker reaches first consumes it (via _take_cancel_snapshot()) as ITS OWN entry snapshot instead of a fresh read of the counter, so a cancel() delivered anywhere from arming onward is guaranteed to raise MotionCancelled. AgentToolSet.call() arms before handing a tool’s execute() to its worker thread and disarms in a finally once that worker finishes, closing the window end-to-end for anything going through the agent toolset.

Safety Design

This section makes the Hardware Safety section of AGENTS.md concrete from the SDK design perspective. The guiding principle is to fail safe even if control stops or an SSH session drops.

• Safe servo range — strictly enforce 200–3900 ticks, staying away from the mechanical limits (0 / 4095) (clamped in io/base.py’s ServoDriver.write_positions, so every backend gets it, not just the Dynamixel one)

• Neutral stance at startup — connection and script startup always begin smoothly from the neutral stance; abrupt jumps are prohibited

• Continuous leg yaw — kinematics.leg_ik normalizes each yaw into (-180°, 180°], moving atan2’s ±180° branch cut to each leg’s antipode (a folded-backwards pose outside the reachable envelope), so no reachable foot position commands a near-full-turn jump

• Servo telemetry — ServoDriver.read_telemetry() reads present current, input voltage and temperature in a single bus sweep (io/base.py, implemented over a span read in io/_dynamixel_bus.py). One sweep rather than one read per signal is what lets a guard sample inside the control loop’s frame budget. A motor that does not answer is reported as unread, never as a zero — an invented reading would be taken for evidence of health. Present position rides in the same span at no extra bus time. Both guards read through telemetry.py’s TelemetryPoller, which owns the poll interval and what a read that cannot be made means, so the two cannot drift apart

• Stopping a motion, not cutting torque — when a guard does act, the intended response is to end the motion (motion → IDLE) and leave torque on. Cutting torque on a servo makes that joint go limp, and a limp joint on a standing robot drops it — worse than the fault being guarded against. The servo’s own firmware remains the breaker of last resort. The neck thermal guard follows the same rule: its HOT response forces a motion (the neck to center) rather than cutting anything (see below). The one deliberate exception is the overload guard’s reaction on a gripped arm, described next

• Leg current IS acted on (overload.py’s OverloadGuard, polled at up to 30 Hz from Palmimo.step() through _leg_safety.py’s LegSafety) — a leg axis that is held or blocked draws current until the servo latches its own Overload error and goes limp, which drops the body unpredictably; stopping first lets the robot choose where it ends up. Three consecutive samples at or above a per-channel threshold drop the motion to IDLE. Samples 0.2 s or more apart do not count as consecutive, so a streak left over from before a pause in stepping cannot combine with a fresh one; the rail guard below follows the same rule. LegSafety reads and judges, including which arm axes go soft and under which motions; robot.py keeps the reaction (dropping the motion, writing the soft-return gain). The raised arm of a wave is the exception to “leave torque on”: someone holding it fights a stiff servo, so on an arm trip that arm alone is lowered to a soft gain and eased back to neutral, and only if the gain cannot be written is its torque cut. The soft gain is one more state of the gesture-tuning sync in robot.py, because the release of the wave’s own tuning on the next frame would otherwise overwrite it. A leg trip touches no gain. Position, not current, decides when the arm is home: a gripped arm draws little current at the soft gain but stays away from neutral. See the API reference for thresholds, the trip API and the scope, which is that of the neck guard

• Rail voltage IS acted on (rail.py’s RailGuard, fed by the same 30 Hz leg telemetry sweep as the overload guard — no extra bus read) — resistance in the supply path pulls every axis’s input voltage down together when the summed current is high. A sag that persists makes the servos latch an Input Voltage error and drop torque, while each axis’s own current stays inside its limit, so the per-axis overload guard cannot see it. Every axis measures the same rail, so the guard treats the sweep’s lowest voltage as one signal for the whole body, not one per axis. Six consecutive sweeps below 3.8 V (200 ms, up to about 300 ms when polls slip a frame at 60 fps) drop the motion to IDLE; torque stays on, and no gain is touched. The streak belongs to the rail rather than to a motion, so it survives a motion change. Across a full-motion measurement of an assembled robot, the longest stretch below 3.8 V during normal walking was 67 ms, and the servo’s Min Voltage Limit is 3.5 V. How quickly a servo latches under a sag has not been measured, so the guard is not claimed to fire before the latch. See the API reference

• Leg temperature is reported, not acted on — a leg servo that is already hot does not cool at the speed a guard could react, so ending the motion buys little; and over an eight-hour exhibition day these servos measured around 50 °C, well inside their rating. The reading is surfaced and the decision to stop is left to the operator, who can see the room and the schedule; nothing acts on it

• Neck temperature IS acted on (thermal.py’s NeckThermalGuard, polled once a second from Palmimo.step()) — unlike a leg, which only bears load during its stance phase, the neck holds the head up continuously and has been observed latching an Overheating HW error (bit 2) and cutting torque at 71 °C after a sustained holding pose (2026-09), which the operator-in-the-loop reasoning above does not cover: nobody is watching a held look angle the way they watch a walking gait. At HOT (>= 62 °C) the neck is forced to center (every neck-writing path, not just look() — see MotionEngine.neck_hold_center) and NOD/HEAD_SHAKE are downgraded to IDLE — motion changes, torque stays on. This holds until the neck cools to <= 55 °C (hysteresis, so a neck hovering near the boundary doesn’t flap the lockout every poll). A telemetry sweep missing a neck motor keeps the last judgement rather than guessing, until it goes stale (NECK_STALE_S), at which point neck_thermal_state reports UNMONITORED instead of trusting a reading that may no longer reflect reality — but the lockout itself (neck_lock_active) stays engaged if it was HOT going into that stale period, since an unobserved neck may still be hot; it only releases on disconnect or a fresh sweep at or below 55 °C. See the API reference for the full state table. The legs keep the reported-only behavior above. Scope: the guard only reaches callers that go through Palmimo.step() (run()/play()/play_keyframes(), the MCP server, the agent tool layer) — the LeRobot teleop integration drives the engine directly and is not covered. A software glide that writes the driver directly instead of going through step() (Palmimo.wake(), _timed_return_to_neutral()) is outside the guard’s reach too, but only ever moves the neck toward its neutral/center target, never away from it, so that gap carries no hazard the guard exists to close.

  An earlier design also had the guard soft-release the neck (gain ramped down) at a CRITICAL threshold above HOT. A second adversarial review found it unsafe as specified — any full-gain write elsewhere in the same connection (wave(), dance(), perform_dance()) re-armed every motor’s gain and silently undid the release, the release-then-full-gain-restore sequence produced a large single-step jump back to the held pose (up to ~25°) instead of a graceful recovery, and there was no measured data on how fast the neck actually heats to justify the extra state — so it was removed rather than shipped. HOT’s stop-the-motion response above is the whole guard.

• Losing control is a different case from guarding against a reading — when the bus disconnects, status packets go wrong, or the loop stalls, there is no motion left to end, and the fallback is torque-off (limp) or a safe hold depending on whether the robot is bearing load. That is not in tension with leaving torque on above: a guard acts while control still works, this is what happens when it does not

I/O Runtime Environment Requirements

palmimo_sdk’s core is hardware-independent, but the execution host that drives real I/O is assumed to be a Raspberry Pi by default, with the following prerequisites.

• OS — 64-bit Linux (aarch64, e.g. Raspberry Pi OS). uv and the dynamixel_sdk dependency must resolve
• Runtime — uv (software setup must succeed)
• Serial permissions — the execution user belongs to the dialout group
• Device naming — a stable OpenRB-150 port name via udev (fixed so the number doesn’t change across USB unplug/replug)