Skip to content

feat(go2): unitree-go2-joystick blueprint, operator input recorded and plotted - #4242

Open
KrishnaH96 wants to merge 3 commits into
mainfrom
krishna/feat/go2-joystick-blueprint
Open

KrishnaH96 wants to merge 3 commits into
mainfrom
krishna/feat/go2-joystick-blueprint

Conversation

@KrishnaH96

@KrishnaH96 KrishnaH96 commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Problem

The viewer's WASD input is converted to tele_cmd_vel right away and never published in raw form, so --record cannot store it, and no Go2 blueprint shows the operator's input next to the velocity it turns into.

Solution

  • RerunWebSocketServer publishes the viewer's twist message as joystick: Out[Joy] before converting it to tele_cmd_vel. 6 added lines, the existing conversion untouched.
  • unitree-go2-joystick: unitree-go2 with its own viewer window adding joystick, tele_cmd_vel and cmd_vel plots under the camera and odom, one color per axis, step interpolation. Run with --record to store all three. The default unitree-go2 window is unchanged.
  • unitree-go2-joystick-replay: the same window on the replay module from feat: replay blueprint - dimos --replay-db <memory.db> run replay #3738, for recordings without a run folder such as an LFS dataset.
  • LFS dataset go2_teleop_sf_office_2026-09-18 (379 MB): 3 min real Go2 run, WASD driving then two click-to-go goals.

+237 lines, 0 deletions. A recording made with unitree-go2-joystick lands in a run folder named after it, so plain run replay finds this window automatically.

Note: dimos-viewer converts keys to velocities itself, so joystick equals tele_cmd_vel today. tele_cmd_vel vs cmd_vel is the pair that diverges, whenever nav drives (2:22 and 2:43 in the clip).

How to Test

Replay the shared recording, no robot needed. The dataset is pulled from LFS the first time:

dimos --replay-db go2_teleop_sf_office_2026-09-18 run unitree-go2-joystick-replay
Screencast_2026-09-18_14-50-29.mp4

To run the blueprint yourself and collect data on hardware: dimos --record run unitree-go2-joystick --robot-ip <ip>.

Alternative

#4224 does the same thing by adding the plots to the default unitree-go2 window, so every Go2 user sees them, plus a unitree-go2-replay blueprint for LFS datasets. Both are hardware-tested and mergeable; @spomichter pick one and I will close the other.

AI assistance

Claude Code (Claude Opus 5) helped survey the code, draft the change and run the checks. I reviewed and understand every line.

Checklist

  • I have read and approved the CLA.

RerunWebSocketServer publishes the viewer's raw twist message as
joystick: Out[Joy], six axes in the viewer's field order, before it
converts it to tele_cmd_vel, so --record stores both.
…lotted

unitree-go2 with its own viewer window: joystick, tele_cmd_vel and
cmd_vel time series under the camera and the odom plot, one color per
axis and step interpolation, since command streams go silent between
bursts. Run it with --record to store all three streams. The default
unitree-go2 window is unchanged.

unitree-go2-joystick-replay plays a recording back with the same window:
run replay finds the window from the recording's run folder name, which
a dataset pulled from LFS does not have.
Three minute Go2 run recorded with dimos --record run unitree-go2: WASD
driving from the viewer, then two click-to-go nav goals. Carries
joystick, tele_cmd_vel, cmd_vel, nav_cmd_vel, lidar, global_map,
color_image, camera_info, odom and tf.
@KrishnaH96 KrishnaH96 changed the title Krishna/feat/go2 joystick blueprint feat(go2): unitree-go2-joystick blueprint, operator input recorded and plotted Sep 19, 2026
@codecov

codecov Bot commented Sep 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.29630% with 3 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...itree/go2/blueprints/smart/unitree_go2_joystick.py 90.90% 2 Missing and 1 partial ⚠️
@@           Coverage Diff           @@
##             main    #4242   +/-   ##
=======================================
  Coverage   79.72%   79.72%           
=======================================
  Files        1508     1510    +2     
  Lines      142607   142688   +81     
  Branches    12209    12211    +2     
=======================================
+ Hits       113689   113763   +74     
- Misses      25478    25485    +7     
  Partials     3440     3440           
Components Coverage Δ
Tests 95.42% <100.00%> (+<0.01%) ⬆️
Flag Coverage Δ
OS-ubuntu-24.04-arm 75.36% <96.29%> (+<0.01%) ⬆️
OS-ubuntu-latest 76.18% <96.29%> (+0.01%) ⬆️
Py-3.10 75.95% <96.29%> (+0.01%) ⬆️
Py-3.11 76.15% <96.29%> (+<0.01%) ⬆️
Py-3.12 76.16% <96.29%> (+0.01%) ⬆️
SelfHosted-Large 30.63% <50.61%> (+0.01%) ⬆️
SelfHosted-Linux 36.66% <50.61%> (+0.01%) ⬆️
SelfHosted-macOS 36.99% <50.61%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/robot/all_blueprints.py 100.00% <ø> (ø)
.../go2/blueprints/smart/test_unitree_go2_joystick.py 100.00% <100.00%> (ø)
dimos/visualization/rerun/test_websocket_server.py 98.49% <100.00%> (+0.12%) ⬆️
dimos/visualization/rerun/websocket_server.py 86.36% <100.00%> (+0.53%) ⬆️
...itree/go2/blueprints/smart/unitree_go2_joystick.py 90.90% <90.90%> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@greptile-apps

greptile-apps Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

The verified test-coverage gap is non-blocking, but it should be fixed so Linux SocketCAN behavior remains covered without optional macOS-only dependencies.

What we checked:

  • T-Rex produced a proof for a posted P2 finding. T-Rex
  • T-Rex authored a focused harness for the Go2 joystick blueprint entries and attempted to run it with the Unitree dependency extra, but the build stopped due to missing portaudio.h. T-Rex
  • T-Rex documented the blocker caused by missing portaudio.h when pyaudio==0.2.14 is pulled for dimos[unitree], noting that no source code was modified and only the evidence harness was authored. T-Rex

Summary

  • Adds Go2 joystick telemetry publication plus live and replay viewer layouts for comparing operator input with teleoperation and final velocity commands.
  • Registers the new Go2 blueprints and adds a replay dataset pointer.
  • Linux CAN command coverage is accidentally disabled whenever the optional control dependency is not installed.

T-Rex validation blocked

  • The Go2 joystick blueprint could not be imported or rendered because the Unitree dependency requires unavailable PortAudio headers. The replay dataset also could not be fetched because Git LFS is unavailable.

Reviews (1) · Last reviewed commit: "data: go2_teleop_sf_office_2026-09-18 re..."

@greptile-apps

greptile-apps Bot commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Comments Outside Diff

These findings sit on lines the diff does not cover, so they could not be posted inline. Each one leaves this list once its file changes.

  • P2 Linux CAN tests skipped dimos/cli/test_can.py:22

    The module-level optional-dependency skip prevents every CAN CLI test from running when the control extra is absent, including Linux SocketCAN paths that do not import that dependency. This is non-blocking, but it removes regression coverage for Linux device discovery, setup, status, up, and down commands in the default environment.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Sep 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant