Robot Learning Starts Before the Robot: Designing Better Demonstrations

Pollen Robotics introduced Grabette in a July 21 Hugging Face article as an open handheld system for recording manipulation demonstrations. The release combines a gripper with observation and tracking sensors, provides a processing workflow for LeRobot-format datasets, and describes sharing through the Hugging Face Hub. A corresponding robotic gripper, Gripette, connects the demonstration approach to execution on a robot arm.

The attraction is straightforward: collecting a useful example need not begin with a person teleoperating an expensive robot for every movement. But making capture easier does not automatically make the resulting dataset diverse or reliable. The collection plan still determines which situations the learner sees and which assumptions remain hidden in the room where recording happens.

Define the task as an observable outcome

Consider an illustrative task of placing a cup on a marked area of a table. A vague label such as move cup leaves several questions unanswered. Does the cup need to remain upright? Must it be entirely inside the area? Is touching another object acceptable? Define success so that another reviewer can apply the same standard to a recording.

Record the initial conditions along with the outcome. Cup position, orientation and surrounding clutter may change the difficulty. A dataset containing only one convenient starting pose could produce impressive training results while providing little evidence about performance in a different setup.

Keep the task boundary manageable. Combining opening a cupboard, selecting an object and placing it somewhere else may be a useful eventual goal, but it introduces several opportunities for failure. Shorter, clearly labelled episodes can help identify which part of the behaviour needs additional examples before longer sequences are attempted.

Collect variation deliberately

Create a collection matrix before recording many demonstrations. Vary object position, lighting, background and operator where those differences are relevant to the target deployment. The aim is not random complexity. It is to represent the conditions under which the eventual system is expected to work.

Avoid letting one easy configuration dominate simply because it is fast to record. Fifty near-identical demonstrations do not necessarily teach more useful variation than a smaller collection covering several meaningful situations. Track counts by condition so that gaps are visible while there is still time to collect missing examples.

Keep unusual episodes rather than quietly deleting every awkward attempt. A slip, interrupted motion or failed grasp can reveal a weakness in the capture process or task definition. Label these outcomes separately. Whether they belong in a particular training objective is a later decision; losing the record prevents that decision from being made thoughtfully.

Inspect geometry and timing before training

A video can look smooth while associated trajectory data contains a discontinuity. Review motion traces alongside the visual recording. Sudden jumps, impossible positions or a gripper state that changes at the wrong moment are reasons to investigate the episode rather than assume the learner will compensate.

Check alignment across recorded signals. If the visual frame and action information refer to different moments, an otherwise good demonstration can teach the wrong relationship. Establish a repeatable quality-control procedure with a few known movements before beginning a large collection session.

Record the capture configuration and processing version. If calibration or preprocessing changes halfway through collection, that change should remain visible in the dataset metadata. Reprocessing may be necessary, but it should be reproducible. Preserve raw recordings where appropriate so that a later improvement does not require repeating every demonstration.

Hold out environments, not only frames

Training and evaluation become misleading when nearly identical material appears on both sides of a split. Dividing neighbouring frames from the same episode is particularly weak evidence of generalisation. Keep complete episodes together and consider holding out entire recording sessions, object arrangements or environments.

Evaluate actual task completion separately from resemblance to a demonstration. A trajectory can look similar yet leave the cup outside the target area. Conversely, a different but valid motion may accomplish the task safely. The outcome definition established at the beginning should guide the evaluation instead of visual similarity alone.

Physical execution also introduces constraints that handheld recording does not remove. Reachability, robot geometry and the surrounding workspace need their own checks. Begin with supervised trials in a controlled setup, and expand only when results justify it. The purpose of accessible capture is to improve the supply of useful evidence, not to bypass the work of validating a robot's behaviour.