Skip to main content
LIBERO (Lifelong robot learning benchmark) provides multiple task suites for studying knowledge transfer in robot learning, including spatial reasoning, object generalization, goal-conditioned learning, and long-horizon multi-step tasks. For more information, see the official website.

​
Benchmark results

GR00T N1.6 achieves state-of-the-art performance across all LIBERO suites:
SuiteSuccess rateTraining stepsBatch size
Spatial195/200 (97.65%)20K640
Goal195/200 (97.5%)20K640
Object197/200 (98.45%)20K640
10 (Long)189/200 (94.35%)20K640

​
Fine-tuning

Remember to set WANDB_API_KEY since --use-wandb is enabled by default. If you don’t have a Weights & Biases account, remove this argument from the training scripts.

​
LIBERO spatial

1

Download dataset

huggingface-cli download \
    --repo-type dataset IPEC-COMMUNITY/libero_spatial_no_noops_1.0.0_lerobot \
    --local-dir examples/LIBERO/libero_spatial_no_noops_1.0.0_lerobot/
2

Copy modality configuration

cp -r examples/LIBERO/modality.json examples/LIBERO/libero_spatial_no_noops_1.0.0_lerobot/meta/
3

Run fine-tuning

uv run bash examples/LIBERO/finetune_libero_spatial.sh

​
LIBERO goal

1

Download dataset

huggingface-cli download \
    --repo-type dataset IPEC-COMMUNITY/libero_goal_no_noops_1.0.0_lerobot \
    --local-dir examples/LIBERO/libero_goal_no_noops_1.0.0_lerobot/
2

Apply patches

# Copy modality configuration
cp -r examples/LIBERO/modality.json examples/LIBERO/libero_goal_no_noops_1.0.0_lerobot/meta/

# Patch corrupted episode
cp examples/LIBERO/patches/episode_000082.mp4 \
   examples/LIBERO/libero_goal_no_noops_1.0.0_lerobot/videos/chunk-000/observation.images.wrist_image/
3

Run fine-tuning

uv run bash examples/LIBERO/finetune_libero_goal.sh

​
LIBERO object

1

Download dataset

huggingface-cli download \
    --repo-type dataset IPEC-COMMUNITY/libero_object_no_noops_1.0.0_lerobot \
    --local-dir examples/LIBERO/libero_object_no_noops_1.0.0_lerobot/
2

Copy modality configuration

cp -r examples/LIBERO/modality.json examples/LIBERO/libero_object_no_noops_1.0.0_lerobot/meta/
3

Run fine-tuning

uv run bash examples/LIBERO/finetune_libero_object.sh

​
LIBERO 10 (long)

1

Download dataset

huggingface-cli download \
    --repo-type dataset IPEC-COMMUNITY/libero_10_no_noops_1.0.0_lerobot \
    --local-dir examples/LIBERO/libero_10_no_noops_1.0.0_lerobot/
2

Copy modality configuration

cp -r examples/LIBERO/modality.json examples/LIBERO/libero_10_no_noops_1.0.0_lerobot/meta/
3

Run fine-tuning

uv run bash examples/LIBERO/finetune_libero_10.sh

​
Evaluation

​
Setup environment

Install the required dependencies (only needs to be done once): 
sudo apt update
sudo apt install libegl1-mesa-dev libglu1-mesa
bash gr00t/eval/sim/LIBERO/setup_libero.sh

​
Run evaluation

1

Start policy server

In Terminal 1:
uv run python gr00t/eval/run_gr00t_server.py \
    --model-path /tmp/libero_spatial/checkpoint-20000/ \
    --embodiment-tag LIBERO_PANDA \
    --use-sim-policy-wrapper
2

Start evaluation client

In Terminal 2:
gr00t/eval/sim/LIBERO/libero_uv/.venv/bin/python gr00t/eval/rollout_policy.py \
    --n_episodes 10 \
    --policy_client_host 127.0.0.1 \
    --policy_client_port 5555 \
    --max_episode_steps=720 \
    --env_name libero_sim/KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it \
    --n_action_steps 8 \
    --n_envs 5

​
Task lists

​
Spatial suite (10 tasks)

Spatial reasoning tasks with varying object locations:
  • libero_sim/pick_up_the_black_bowl_between_the_plate_and_the_ramekin_and_place_it_on_the_plate
  • libero_sim/pick_up_the_black_bowl_next_to_the_ramekin_and_place_it_on_the_plate
  • libero_sim/pick_up_the_black_bowl_from_table_center_and_place_it_on_the_plate
  • libero_sim/pick_up_the_black_bowl_on_the_cookie_box_and_place_it_on_the_plate
  • libero_sim/pick_up_the_black_bowl_in_the_top_drawer_of_the_wooden_cabinet_and_place_it_on_the_plate
  • libero_sim/pick_up_the_black_bowl_on_the_ramekin_and_place_it_on_the_plate
  • libero_sim/pick_up_the_black_bowl_next_to_the_cookie_box_and_place_it_on_the_plate
  • libero_sim/pick_up_the_black_bowl_on_the_stove_and_place_it_on_the_plate
  • libero_sim/pick_up_the_black_bowl_next_to_the_plate_and_place_it_on_the_plate
  • libero_sim/pick_up_the_black_bowl_on_the_wooden_cabinet_and_place_it_on_the_plate

​
Goal suite (10 tasks)

Goal-conditioned manipulation tasks:
  • libero_sim/open_the_middle_drawer_of_the_cabinet
  • libero_sim/put_the_bowl_on_the_stove
  • libero_sim/put_the_wine_bottle_on_top_of_the_cabinet
  • libero_sim/open_the_top_drawer_and_put_the_bowl_inside
  • libero_sim/put_the_bowl_on_top_of_the_cabinet
  • libero_sim/push_the_plate_to_the_front_of_the_stove
  • libero_sim/put_the_cream_cheese_in_the_bowl
  • libero_sim/turn_on_the_stove
  • libero_sim/put_the_bowl_on_the_plate
  • libero_sim/put_the_wine_bottle_on_the_rack

​
Object suite (10 tasks)

Object generalization with different items:
  • libero_sim/pick_up_the_alphabet_soup_and_place_it_in_the_basket
  • libero_sim/pick_up_the_cream_cheese_and_place_it_in_the_basket
  • libero_sim/pick_up_the_salad_dressing_and_place_it_in_the_basket
  • libero_sim/pick_up_the_bbq_sauce_and_place_it_in_the_basket
  • libero_sim/pick_up_the_ketchup_and_place_it_in_the_basket
  • libero_sim/pick_up_the_tomato_sauce_and_place_it_in_the_basket
  • libero_sim/pick_up_the_butter_and_place_it_in_the_basket
  • libero_sim/pick_up_the_milk_and_place_it_in_the_basket
  • libero_sim/pick_up_the_chocolate_pudding_and_place_it_in_the_basket
  • libero_sim/pick_up_the_orange_juice_and_place_it_in_the_basket

​
Long suite (10 tasks)

Long-horizon multi-step tasks:
  • libero_sim/LIVING_ROOM_SCENE2_put_both_the_alphabet_soup_and_the_tomato_sauce_in_the_basket
  • libero_sim/LIVING_ROOM_SCENE2_put_both_the_cream_cheese_box_and_the_butter_in_the_basket
  • libero_sim/KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it
  • libero_sim/KITCHEN_SCENE4_put_the_black_bowl_in_the_bottom_drawer_of_the_cabinet_and_close_it
  • libero_sim/LIVING_ROOM_SCENE5_put_the_white_mug_on_the_left_plate_and_put_the_yellow_and_white_mug_on_the_right_plate
  • libero_sim/STUDY_SCENE1_pick_up_the_book_and_place_it_in_the_back_compartment_of_the_caddy
  • libero_sim/LIVING_ROOM_SCENE6_put_the_white_mug_on_the_plate_and_put_the_chocolate_pudding_to_the_right_of_the_plate
  • libero_sim/LIVING_ROOM_SCENE1_put_both_the_alphabet_soup_and_the_cream_cheese_box_in_the_basket
  • libero_sim/KITCHEN_SCENE8_put_both_moka_pots_on_the_stove
  • libero_sim/KITCHEN_SCENE6_put_the_yellow_and_white_mug_in_the_microwave_and_close_it

Build docs developers (and LLMs) love

Get started for freeTalk to us