---
title: "LeRobot dataset format: what buyers should check | Wake"
description: "A practical guide to LeRobot dataset structure, episode metadata, video alignment and validation, with distinctions from HDF5 and RLDS."
url: https://wake.inc/blog/lerobot-dataset-format/
---

[Home](https://wake.inc/) [Blog](https://wake.inc/blog/) LeRobot dataset format

Wake guides

# LeRobot dataset format: what buyers should check.

A LeRobot dataset organizes visual streams, time-series records and metadata for robot learning. In the v3 design, several episodes can share storage files, while metadata identifies their boundaries. A compatible layout is only one part of usability: feature meanings, timing and recorded signals must also match the training pipeline.

By [Wake](https://wake.inc/about/) · October 26, 2026

On this page

- [Version the data contract.](https://wake.inc/blog/lerobot-dataset-format/#version-the-data-contract)
- [Three layers to inspect.](https://wake.inc/blog/lerobot-dataset-format/#three-layers-to-inspect)
- [A state vector is not an action vector.](https://wake.inc/blog/lerobot-dataset-format/#a-state-vector-is-not-an-action-vector)
- [Check the temporal relationship.](https://wake.inc/blog/lerobot-dataset-format/#check-the-temporal-relationship)
- [Keep conversions honest.](https://wake.inc/blog/lerobot-dataset-format/#keep-conversions-honest)
- [LeRobot, HDF5 and RLDS solve different parts of the problem.](https://wake.inc/blog/lerobot-dataset-format/#lerobot-hdf5-and-rlds-solve-different-parts-of-the-problem)
- [A delivery checklist for your project.](https://wake.inc/blog/lerobot-dataset-format/#a-delivery-checklist-for-your-project)
- [Questions, answered](https://wake.inc/blog/lerobot-dataset-format/#faq-title)

## Version the data contract.

The dataset format version and the software package version are different things. Record both when a training environment is working. If a collaborator says “LeRobot format,” ask which dataset version and loader revision they mean.

## Three layers to inspect.

| Layer | What to inspect | A useful check |
| --- | --- | --- |
| Tabular records | Feature columns, indices, timestamps and any state/action values. | Are dimensions, types and units documented? |
| Visual streams | Camera names, encoded videos or images and timing. | Does each sampled frame match the intended observation? |
| Metadata | Episode boundaries, tasks, feature definitions and statistics. | Can the loader find the complete intended episode? |

## A state vector is not an action vector.

Observed state describes the system at a point in time. An action describes the command or target used by a controller. They may have different dimensions and units. Similar-looking arrays do not establish the same physical meaning.

## Check the temporal relationship.

Choose several episodes and inspect synchronized video and records around a visible event, such as a grasp closing or an object leaving a surface. Look at the start and end as well as the middle. A successful file load does not prove that observations and actions line up.

## Keep conversions honest.

Moving human video into a dataset layout does not create robot state or action measurements. If a feature was inferred, label it as inferred and document the method. If it was not captured, do not silently fill it with values that imply a physical measurement.

## LeRobot, HDF5 and RLDS solve different parts of the problem.

HDF5 is a storage container. The groups, datasets and attributes inside it still need an agreed schema. RLDS describes episodes and steps for sequential decision-making data. LeRobot provides an organization and tooling designed around robot learning. None of these names alone specifies the content of a particular collection.

## A delivery checklist for your project.

- Name the dataset format version and the loader revision.
- List required cameras, observations, state and action fields.
- Define episode boundaries, timestamp units and clock relationships.
- Document task labels, quality flags and train/evaluation splits.
- Check representative episodes through the real training loader.
- Agree how missing signals and conversion failures are reported.

## Questions, answered.

Is LeRobot a video format?

No. It is a dataset organization for robot learning that connects visual streams, time-series records and metadata. The video encoding is one component of that organization.

Does converting to LeRobot make human video action-labeled?

No. File conversion cannot create measurements that were never recorded. Robot actions must be captured separately or derived through an explicitly documented method.

What should I validate first?

Check the format and loader versions, feature meanings, episode boundaries and observation/action timing. Then load representative episodes through the preprocessing used for training.

## Sources and further reading

Primary references checked October 26, 2026.

- [Hugging Face: LeRobotDataset v3.0](https://huggingface.co/docs/lerobot/en/lerobot-dataset-v3)
- [Google Research: RLDS data model](https://github.com/google-research/rlds)
- [HDF Group: HDF5 data model](https://support.hdfgroup.org/documentation/hdf5/latest/_h5_d_m__u_g.html)

## Keep exploring.

[Robot training data grounded in real work.](https://wake.inc/robot-training-data/) [Teleoperation vs egocentric human video.](https://wake.inc/blog/teleoperation-vs-egocentric-video/) [Wake vs Ego4D: collection partner or research dataset?](https://wake.inc/compare/wake-vs-ego4d/)

## What should your robot learn next?

Bring the task, setting and signals you need. We’ll work through the collection and annotation requirements with you.

[Build with Wake](https://wake.inc/contact/)