Robotics
LeRobot
English
OpenRAL
rskill
smolvla
vision-language-action
franka_panda
vla
maniskill
maniskill3
manipulation
Instructions to use OpenRAL/rskill-smolvla-franka_panda-maniskill3-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use OpenRAL/rskill-smolvla-franka_panda-maniskill3-bf16 with LeRobot:
# See https://github.com/huggingface/lerobot?tab=readme-ov-file#installation for more details git clone https://github.com/huggingface/lerobot.git cd lerobot pip install -e .[smolvla]
# Launch finetuning on your dataset python lerobot/scripts/train.py \ --policy.path=OpenRAL/rskill-smolvla-franka_panda-maniskill3-bf16 \ --dataset.repo_id=lerobot/svla_so101_pickplace \ --batch_size=64 \ --steps=20000 \ --output_dir=outputs/train/my_smolvla \ --job_name=my_smolvla_training \ --policy.device=cuda \ --wandb.enable=true
# Run the policy using the record function python -m lerobot.record \ --robot.type=so101_follower \ --robot.port=/dev/ttyACM0 \ # <- Use your port --robot.id=my_blue_follower_arm \ # <- Use your robot id --robot.cameras="{ front: {type: opencv, index_or_path: 8, width: 640, height: 480, fps: 30}}" \ # <- Use your cameras --dataset.single_task="Grasp a lego block and put it in the bin." \ # <- Use the same task description you used in your dataset recording --dataset.repo_id=HF_USER/dataset_name \ # <- This will be the dataset name on HF Hub --dataset.episode_time_s=50 \ --dataset.num_episodes=10 \ --policy.path=OpenRAL/rskill-smolvla-franka_panda-maniskill3-bf16 - Notebooks
- Google Colab
- Kaggle
chore: publish rSkill OpenRAL/rskill-smolvla-maniskill-franka v0.1.0
Browse files- README.md +8 -8
- eval/maniskill3_franka_pick_cube.json +50 -0
- rskill.yaml +9 -5
README.md
CHANGED
|
@@ -18,7 +18,7 @@ language:
|
|
| 18 |
|
| 19 |
> **OpenRAL rSkill** β SmolVLA (0.45 B) finetuned on a 1000-demo Franka
|
| 20 |
> LiftCube dataset in ManiSkill3 SAPIEN, packaged for use with the
|
| 21 |
-
> [OpenRAL](https://github.com/
|
| 22 |
|
| 23 |
This package wraps
|
| 24 |
[`Calvert0921/smolvla_franka_liftcube_1000`](https://huggingface.co/Calvert0921/smolvla_franka_liftcube_1000)
|
|
@@ -73,7 +73,7 @@ this layout.
|
|
| 73 |
|
| 74 |
| Robot | Embodiment tag | Status | Notes |
|
| 75 |
| --- | --- | --- | --- |
|
| 76 |
-
| Franka Panda (ManiSkill3 SAPIEN) | `franka_panda` | β end-to-end | Manifest validates and `
|
| 77 |
|
| 78 |
## Sensors required
|
| 79 |
|
|
@@ -91,7 +91,7 @@ Full schema: [`openral_core.schemas.RSkillManifest`](../../python/core/src/openr
|
|
| 91 |
|
| 92 |
| Field | Value |
|
| 93 |
| --- | --- |
|
| 94 |
-
| `name` | `
|
| 95 |
| `version` | `0.1.0` |
|
| 96 |
| `license` | `apache-2.0` |
|
| 97 |
| `role` | `s1` (fast visuomotor policy) |
|
|
@@ -122,7 +122,7 @@ just bootstrap && uv sync --all-packages --group sim --group maniskill3
|
|
| 122 |
|
| 123 |
# End-to-end rollout (live SAPIEN window via --view).
|
| 124 |
DISPLAY=:1 uv run --group sim --group maniskill3 \
|
| 125 |
-
|
| 126 |
--rskill rskill://rskills/smolvla-maniskill-franka \
|
| 127 |
--view
|
| 128 |
```
|
|
@@ -135,11 +135,11 @@ viewer. On a warm cache the policy lifts the cube within ~200 steps
|
|
| 135 |
## Evaluation
|
| 136 |
|
| 137 |
No benchmarks shipped yet (`eval/.gitkeep` only). The headline LiftCube
|
| 138 |
-
success rate will be populated by `
|
| 139 |
benchmark suite lands in `benchmarks/`:
|
| 140 |
|
| 141 |
```bash
|
| 142 |
-
|
| 143 |
--suite maniskill3_pick_place \
|
| 144 |
--vla smolvla:rskill://rskills/smolvla-maniskill-franka
|
| 145 |
```
|
|
@@ -185,6 +185,6 @@ should be updated to match.
|
|
| 185 |
## See also
|
| 186 |
|
| 187 |
- [`robots/franka_panda/robot.yaml`](../../robots/franka_panda/robot.yaml) β RobotDescription manifest.
|
| 188 |
-
- [`
|
| 189 |
-
- [`
|
| 190 |
- [CLAUDE.md Β§6.4](../../CLAUDE.md) β rSkill packaging contract.
|
|
|
|
| 18 |
|
| 19 |
> **OpenRAL rSkill** β SmolVLA (0.45 B) finetuned on a 1000-demo Franka
|
| 20 |
> LiftCube dataset in ManiSkill3 SAPIEN, packaged for use with the
|
| 21 |
+
> [OpenRAL](https://github.com/OpenRAL/openral) robot agent framework.
|
| 22 |
|
| 23 |
This package wraps
|
| 24 |
[`Calvert0921/smolvla_franka_liftcube_1000`](https://huggingface.co/Calvert0921/smolvla_franka_liftcube_1000)
|
|
|
|
| 73 |
|
| 74 |
| Robot | Embodiment tag | Status | Notes |
|
| 75 |
| --- | --- | --- | --- |
|
| 76 |
+
| Franka Panda (ManiSkill3 SAPIEN) | `franka_panda` | β end-to-end | Manifest validates and `openral sim run --view` produces a live SAPIEN window of the policy lifting the cube; processors are auto-synthesized from the training dataset's `meta/episodes_stats.jsonl` because the upstream model repo doesn't ship `policy_*processor.json`. |
|
| 77 |
|
| 78 |
## Sensors required
|
| 79 |
|
|
|
|
| 91 |
|
| 92 |
| Field | Value |
|
| 93 |
| --- | --- |
|
| 94 |
+
| `name` | `OpenRAL/rskill-smolvla-maniskill-franka` |
|
| 95 |
| `version` | `0.1.0` |
|
| 96 |
| `license` | `apache-2.0` |
|
| 97 |
| `role` | `s1` (fast visuomotor policy) |
|
|
|
|
| 122 |
|
| 123 |
# End-to-end rollout (live SAPIEN window via --view).
|
| 124 |
DISPLAY=:1 uv run --group sim --group maniskill3 \
|
| 125 |
+
openral sim run --config scenes/benchmarks/smolvla_maniskill_pick_cube.yaml \
|
| 126 |
--rskill rskill://rskills/smolvla-maniskill-franka \
|
| 127 |
--view
|
| 128 |
```
|
|
|
|
| 135 |
## Evaluation
|
| 136 |
|
| 137 |
No benchmarks shipped yet (`eval/.gitkeep` only). The headline LiftCube
|
| 138 |
+
success rate will be populated by `openral benchmark run` once a paired
|
| 139 |
benchmark suite lands in `benchmarks/`:
|
| 140 |
|
| 141 |
```bash
|
| 142 |
+
openral benchmark run \
|
| 143 |
--suite maniskill3_pick_place \
|
| 144 |
--vla smolvla:rskill://rskills/smolvla-maniskill-franka
|
| 145 |
```
|
|
|
|
| 185 |
## See also
|
| 186 |
|
| 187 |
- [`robots/franka_panda/robot.yaml`](../../robots/franka_panda/robot.yaml) β RobotDescription manifest.
|
| 188 |
+
- [`scenes/benchmarks/smolvla_maniskill_pick_cube.yaml`](../../scenes/benchmarks/smolvla_maniskill_pick_cube.yaml) β paired SimEnvironment config.
|
| 189 |
+
- [`scenes/benchmarks/maniskill3_pick_cube.yaml`](../../scenes/benchmarks/maniskill3_pick_cube.yaml) β bare-backend MS3 PickCube wiring test.
|
| 190 |
- [CLAUDE.md Β§6.4](../../CLAUDE.md) β rSkill packaging contract.
|
eval/maniskill3_franka_pick_cube.json
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema_version": "0.1",
|
| 3 |
+
"source": {
|
| 4 |
+
"paper": "https://arxiv.org/abs/2410.00425",
|
| 5 |
+
"arxiv": "https://arxiv.org/abs/2410.00425",
|
| 6 |
+
"model_variant": "smolvla",
|
| 7 |
+
"evaluated_by": "OpenRAL:openral benchmark run",
|
| 8 |
+
"reproduced_locally": true,
|
| 9 |
+
"reproduction_planned": null,
|
| 10 |
+
"reproduction_cli": "openral benchmark run --suite maniskill3_franka_pick_cube --rskill rskill://smolvla-maniskill-franka",
|
| 11 |
+
"table": null,
|
| 12 |
+
"status": "reproduced"
|
| 13 |
+
},
|
| 14 |
+
"benchmark": {
|
| 15 |
+
"name": "ManiSkill3 PickCube (Franka Panda)",
|
| 16 |
+
"dataset": null,
|
| 17 |
+
"protocol": "10 episodes per task, success_key=success, max_steps=100",
|
| 18 |
+
"robot": "franka_panda",
|
| 19 |
+
"simulator": "ManiSkill3 (SAPIEN, GPU)"
|
| 20 |
+
},
|
| 21 |
+
"eval_config": {
|
| 22 |
+
"n_episodes": 10,
|
| 23 |
+
"seeds": [
|
| 24 |
+
0,
|
| 25 |
+
1,
|
| 26 |
+
2,
|
| 27 |
+
3,
|
| 28 |
+
4,
|
| 29 |
+
5,
|
| 30 |
+
6,
|
| 31 |
+
7,
|
| 32 |
+
8,
|
| 33 |
+
9
|
| 34 |
+
],
|
| 35 |
+
"success_key": "success",
|
| 36 |
+
"max_steps": 100,
|
| 37 |
+
"vla_id": "smolvla",
|
| 38 |
+
"weights_uri": "rskill://smolvla-maniskill-franka"
|
| 39 |
+
},
|
| 40 |
+
"results": {
|
| 41 |
+
"maniskill3/PickCube-v1_success_rate": 0.1,
|
| 42 |
+
"avg_success_rate": 0.1,
|
| 43 |
+
"n_tasks": 1,
|
| 44 |
+
"n_episodes_per_task": 10,
|
| 45 |
+
"n_episodes_total": 10,
|
| 46 |
+
"mean_step_latency_ms_avg": 23.350579773366917
|
| 47 |
+
},
|
| 48 |
+
"baselines": {},
|
| 49 |
+
"trace_id": null
|
| 50 |
+
}
|
rskill.yaml
CHANGED
|
@@ -40,11 +40,12 @@
|
|
| 40 |
# processors.
|
| 41 |
|
| 42 |
# ββ Identity βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 43 |
-
schema_version: "1"
|
| 44 |
-
name: "
|
| 45 |
version: "0.1.0"
|
| 46 |
license: "apache-2.0"
|
| 47 |
role: "s1"
|
|
|
|
| 48 |
|
| 49 |
# ββ Policy identity ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 50 |
model_family: "smolvla"
|
|
@@ -116,7 +117,7 @@ latency_budget:
|
|
| 116 |
|
| 117 |
# ββ Provenance βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 118 |
# No benchmarks shipped yet β see eval/.gitkeep. The headline LiftCube
|
| 119 |
-
# success rate will be populated by `
|
| 120 |
# backend supports the wrist-camera surface (see header comment).
|
| 121 |
paper_url: "https://arxiv.org/abs/2506.01844"
|
| 122 |
dataset_uri: "hf://Calvert0921/SmolVLA_LiftCube_Franka_1000"
|
|
@@ -128,9 +129,9 @@ description: >
|
|
| 128 |
Panda lifting a cube in ManiSkill3 SAPIEN). Action chunks of length
|
| 129 |
50 across overhead + wrist RGB views and a 9-D Franka qpos state.
|
| 130 |
Runs end-to-end on ManiSkill3 PickCube-v1 with a live SAPIEN viewer
|
| 131 |
-
via `
|
| 132 |
|
| 133 |
-
# ADR-
|
| 134 |
# "lift" is not in the RSkillAction enum; the lift motion is expressed
|
| 135 |
# as a pick (the post-grasp upward motion is implicit in the policy's
|
| 136 |
# closed-loop control).
|
|
@@ -146,3 +147,6 @@ scenes:
|
|
| 146 |
# to bind the LeRobot v3 `action` feature shape).
|
| 147 |
action_contract:
|
| 148 |
dim: 8
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
# processors.
|
| 41 |
|
| 42 |
# ββ Identity βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 43 |
+
schema_version: "0.1"
|
| 44 |
+
name: "OpenRAL/rskill-smolvla-maniskill-franka"
|
| 45 |
version: "0.1.0"
|
| 46 |
license: "apache-2.0"
|
| 47 |
role: "s1"
|
| 48 |
+
kind: "vla" # ADR-00XX: rSkill kind discriminator. "vla" = learnable Vision-Language-Action policy.
|
| 49 |
|
| 50 |
# ββ Policy identity ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 51 |
model_family: "smolvla"
|
|
|
|
| 117 |
|
| 118 |
# ββ Provenance βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 119 |
# No benchmarks shipped yet β see eval/.gitkeep. The headline LiftCube
|
| 120 |
+
# success rate will be populated by `openral benchmark run` once the MS3
|
| 121 |
# backend supports the wrist-camera surface (see header comment).
|
| 122 |
paper_url: "https://arxiv.org/abs/2506.01844"
|
| 123 |
dataset_uri: "hf://Calvert0921/SmolVLA_LiftCube_Franka_1000"
|
|
|
|
| 129 |
Panda lifting a cube in ManiSkill3 SAPIEN). Action chunks of length
|
| 130 |
50 across overhead + wrist RGB views and a 9-D Franka qpos state.
|
| 131 |
Runs end-to-end on ManiSkill3 PickCube-v1 with a live SAPIEN viewer
|
| 132 |
+
via `openral sim run --view`.
|
| 133 |
|
| 134 |
+
# ADR-0022 β action vocabulary surfaced to the reasoner LLM tool palette.
|
| 135 |
# "lift" is not in the RSkillAction enum; the lift motion is expressed
|
| 136 |
# as a pick (the post-grasp upward motion is implicit in the policy's
|
| 137 |
# closed-loop control).
|
|
|
|
| 147 |
# to bind the LeRobot v3 `action` feature shape).
|
| 148 |
action_contract:
|
| 149 |
dim: 8
|
| 150 |
+
|
| 151 |
+
benchmarks:
|
| 152 |
+
maniskill3_franka_pick_cube: 0.1
|