Tidybot2 on Franka Research 3

Phone-based teleoperation for Franka Research 3 — a lighter, faster pipeline for collecting manipulation demonstrations.

Summary

Building Tidybot2, a phone-based teleoperation pipeline for the Franka Research 3 arm. The goal: lower the friction of demonstration collection so the lab can record more diverse, more naturalistic manipulation data — without specialized VR or leader–follower rigs.

Motivation

State-of-the-art manipulation models are data-hungry. Existing teleoperation setups — VR headsets, spacemouse, leader–follower rigs — each have ergonomic, cost, or latency drawbacks that cap the variety of demonstrations one operator can record in a session. A browser-based phone controller, something every researcher already owns, is a practical alternative if the latency and stability can be tamed.

Approach

  • Web frontend on the phone streams pose deltas to a Python backend over WebSocket.
  • Backend fuses incoming deltas with a safety filter, then issues joint-space setpoints to the Franka Research 3.
  • Joint trajectories are logged at a fixed rate for downstream imitation learning.

Highlights

  • 5× faster teleoperation. Increased the motion-rate factor from 0.01 → 0.05 while keeping trajectories smooth — operators complete full pick-and-place cycles in a fraction of the time.
  • ~90% fewer teleoperation interruptions during long sessions, through buffering, deadband tuning, and a reconnection protocol that survives transient network drops.
  • Implemented gripper open/close control end-to-end so the recorded trajectories capture full manipulation primitives, not just arm motion.

My role

Collaborator. Focused on the stability and control-rate improvements; integrated gripper actuation into the recorded trajectory stream.

Stack

Python · WebSocket · Web frontend (JS) · Franka ROS driver

Status

Completed (Sep 2025 – Dec 2025).

Collaborators

Tidybot2 team at SYSU HCP Lab — names to be added.

  • Hardware: Franka Research 3.
  • Goal: pipeline feeds downstream manipulation-learning projects.