Version the data contract.
The dataset format version and the software package version are different things. Record both when a training environment is working. If a collaborator says “LeRobot format,” ask which dataset version and loader revision they mean.
Three layers to inspect.
| Layer | What to inspect | A useful check |
|---|---|---|
| Tabular records | Feature columns, indices, timestamps and any state/action values. | Are dimensions, types and units documented? |
| Visual streams | Camera names, encoded videos or images and timing. | Does each sampled frame match the intended observation? |
| Metadata | Episode boundaries, tasks, feature definitions and statistics. | Can the loader find the complete intended episode? |
A state vector is not an action vector.
Observed state describes the system at a point in time. An action describes the command or target used by a controller. They may have different dimensions and units. Similar-looking arrays do not establish the same physical meaning.
Check the temporal relationship.
Choose several episodes and inspect synchronized video and records around a visible event, such as a grasp closing or an object leaving a surface. Look at the start and end as well as the middle. A successful file load does not prove that observations and actions line up.
Keep conversions honest.
Moving human video into a dataset layout does not create robot state or action measurements. If a feature was inferred, label it as inferred and document the method. If it was not captured, do not silently fill it with values that imply a physical measurement.
LeRobot, HDF5 and RLDS solve different parts of the problem.
HDF5 is a storage container. The groups, datasets and attributes inside it still need an agreed schema. RLDS describes episodes and steps for sequential decision-making data. LeRobot provides an organization and tooling designed around robot learning. None of these names alone specifies the content of a particular collection.
A delivery checklist for your project.
- Name the dataset format version and the loader revision.
- List required cameras, observations, state and action fields.
- Define episode boundaries, timestamp units and clock relationships.
- Document task labels, quality flags and train/evaluation splits.
- Check representative episodes through the real training loader.
- Agree how missing signals and conversion failures are reported.
Questions, answered.
Is LeRobot a video format?
No. It is a dataset organization for robot learning that connects visual streams, time-series records and metadata. The video encoding is one component of that organization.
Does converting to LeRobot make human video action-labeled?
No. File conversion cannot create measurements that were never recorded. Robot actions must be captured separately or derived through an explicitly documented method.
What should I validate first?
Check the format and loader versions, feature meanings, episode boundaries and observation/action timing. Then load representative episodes through the preprocessing used for training.
Sources and further reading
Primary references checked October 26, 2026.