eckartal/asimov-flex

Asimov Flex: base walk v0.10

Velocity-commanded walking policy for Asimov 1, trained in Isaac Lab with PPO. Like v0.9, it is a base walk for later skills to build on. It walks forward, backward and sideways, turns while walking or on the spot, and stands still.

What changed

Version 0.10 is walk v0.9 trained further with the same robot actuation in the simulator (the robot's ankle, its action filter and joint limits, motor torque limits that include the rated torques, sole-shaped feet, extra torso mass, a sometimes slower control loop). One thing is new:

• A reward for following the commanded turn rate. v0.9 turned more slowly than commanded (76% of the command while walking). v0.10 reaches about 86%.

During training it stayed close to v0.9 (the training pulls the policy toward v0.9's actions), so most other results are about the same as v0.9 (see the known limits for the exceptions). Sideways walking with the robot's actuation also got a little closer to the command.

We also tried more practice on 10-12 cm stairs at the rated torques, in two further training runs. It did not make 10 cm stairs better, so it is not part of this version.

The inputs, the outputs, the gait clock and the timing are the same as in v0.8 and v0.9.

It needs a firmware that sends the 2-value gait clock. The policy has 80 inputs: the standard 78 followed by the gait clock. A firmware that only sends the 78 standard inputs cannot run it.

Tested in simulation only (Isaac Lab, and MuJoCo with the public Asimov 1 model). It has not been run on the real robot.

• policy.onnx is the exported policy for inference, producing joint-position actions.
• agent.yaml records the actor-critic and AMP training settings.
• env.yaml records the simulation and task settings. Some robot actuator details are marked "withheld".
• Training and evaluation code: menloresearch/isaac_asimov.

Inputs and outputs

• Control rate 50 Hz. Actions: 23 joint-position offsets, target = default_pose + 0.25 * action. On the robot this target then goes through the robot's action filter and joint limits (the policy was trained with them). The actions input is the policy's previous raw output, before the filter.
• 80 inputs: the same 78 as the standard Asimov 1 walking policy, in the same order, followed by a 2-value gait clock.
• Gait clock: (sin(2 pi phi), cos(2 pi phi)), with phi in [0, 1).
  • Each policy step while walking: phi += elapsed / period, where elapsed is the measured real time since the previous policy step (0.02 s at 50 Hz). Use the measured time, not a fixed 0.02 s, if the loop runs slower.
  • period is 0.8 s at command speeds up to 0.2 m/s and 0.7 s at 0.8 m/s and above, linear in between.
  • When |v_xy| + |w_z| <= 0.1 the robot stands: the clock outputs (0, 0) and phi stays at 0.
  • Walking starts from phi = 0 (both feet down); the right foot lifts first.
• Ankle roll uses the same 0.25 action scale as every other joint. A robot that scales ankle roll differently needs this policy re-exported for that scale.

Results in simulation

Measured in MuJoCo with the public Asimov 1 model, at the robot's timing (50 Hz, 35 ms command delay), with the robot's ankle, action filter and joint limits in the loop. "With motor limits" means the leg motors are held to their rated torques and each ankle motor to 12 N m. "Without motor limits" uses the peak torques. Falls are the share of tries. The brackets give the 95% range for the fall rate. Every row below comes from test runs on fresh random seeds that were set aside before training and were not used to pick this checkpoint.

Test (MuJoCo, robot actuation)	v0.10	v0.9	v0.8
Turn rate while walking (0.5 m/s, 0.5 rad/s), share of the command (64 tries)	86%	76%	79%
Sideways speed (0.2 m/s, both directions), share of the command (128 tries)	81%	78%	79%
Stairs up 6 / 8 / 10 cm, with motor limits (256 tries each)	1.2% / 2.3% / 57.4% falls	1.2% / 2.3% / 58.6%	2.7% / 13.3% / 60.5%
Stairs up 8 cm, with motor limits, plus two fixed batches (344 tries)	2.0% falls (1.0-4.1%)	2.9%	12.5%
Stairs up 6 / 8 / 10 cm, without motor limits (256 tries each)	0% / 1.6% / 6.2% falls	0% / 0.8% / 2.7%	0% / 1.6% / 4.7%
Pushes of about 10 and 20 N s, standing and walking, 4 directions, with / without motor limits (512 tries each)	2 / 1 falls	3 / 0	3 / 1
Chest push 215 N while standing, from behind / from the front, with motor limits (128 tries)	28% / 100% falls	22% / 100%	35% / 100%
Chest push 215 N while standing, from behind / from the front, without motor limits (128 tries)	10% / 38% falls	12% / 38%	16% / 34%
Forward speed at a 0.5 / 0.8 m/s command, with motor limits	0.44 / 0.73 m/s	0.44 / 0.73 m/s	0.41 / 0.70 m/s
Falls on flat ground, all walking tests	0%	0%	0%

The turn rate gain is clear: every one of the 64 tries turned faster than the v0.9 average. Sideways walking is a little faster than v0.9 in both directions. The 10 cm stairs with the motor limits are about the same as v0.9 and v0.8. The other differences in the table are within noise. The 10 cm stairs without motor limits are a little worse in this run (16 against 7 falls in 256, p = 0.09), which is not a clear difference either way.

The project's standard test set (Isaac Lab, and MuJoCo with a plain-ankle model; these tests do not use the robot's ankle, action filter or joint limits; one run each, 64-576 tries per row):

Test (standard set)	v0.10	v0.9	v0.8
Turning on the spot, shortfall against the command (Isaac)	2.1%	23.3%	25.9%
Sideways speed tracking (Isaac / MuJoCo)	67.2% / 79.7%	78.4% / 78.9%	76.2% / 77.9%
Joint jitter, lower is smoother (Isaac)	0.099	0.096	0.093
Forward speed at a 0.8 m/s command (Isaac)	0.754 m/s	0.748 m/s	0.735 m/s
10 cm stairs down on 0.35 m treads (MuJoCo)	10.9% falls	11.7%	20.3%
6 cm stairs up on 0.35 m treads (MuJoCo, 640 tries)	1.9% falls	0.9%	0.3%
Standing push 0.8 m/s, worst direction (MuJoCo)	22.7% falls	29.7%	34.4%
Chest push 215 N from the front while standing (MuJoCo plain-ankle model, ankle motors 12 / 18 / 36 N m)	62.5% / 61.7% / 63.3% falls	57.8% / 61.7% / 64.8%	48.4% / 46.1% / 47.7%
12 cm stairs up (Isaac Lab)	13.9% falls	16.7%	12.7%
Slowest control loop before falls pass 10% (Isaac Lab stress test; lower is better)	about 36 Hz	about 36 Hz	about 36 Hz

Known limits

• When it steps sideways, the leading leg lifts its knee higher than the trailing leg: in MuJoCo with the robot's actuation the leading knee swings through about 0.49 rad and the trailing knee about 0.36 rad (v0.9: 0.45 and 0.38; v0.8: 0.46 and 0.42). This flips with the direction, so it is a side-stepping style, not a difference between the left and right leg. Step lengths are the same on both sides and nothing falls. Walking forward and turning are symmetric.
• In the Isaac Lab sideways test of the standard set (plain-ankle model, without the robot's action filter), v0.10 walks sideways at 67% of the command, against 76-78% for v0.8 and v0.9, with the same uneven knee lift, stronger there. In MuJoCo it reaches 80-81%, with the robot's actuation and with the plain-ankle model. We expect the robot to behave like the MuJoCo robot-actuation result, but this has not been checked on the real robot.
• Joint motion is a little less smooth in the Isaac Lab tests: jitter 0.099 against 0.096 for v0.9 and 0.093 for v0.8.
• 6 cm stairs up on narrow (0.35 m) treads in the plain-ankle MuJoCo test: 1.9% falls against 0.3% for v0.8 (12 against 2 in 640 tries). Small, but measurable.
• A hard chest push from the front while standing (215 N) knocks it over more often than v0.8 in the plain-ankle test model (about 62% against 46-48%), the same as v0.9. With the robot's own ankle and actuation the same push is about the same as v0.8 (38% against 34% without motor limits; with the motor limits all three versions fall every time).
• 10 cm stairs are still hard when the motors are held to their rated torque: about 57% of tries fall, about the same as v0.9 (59%) and v0.8 (61%).
• 10 cm stairs without the motor limits: 6.2% falls against 2.7% for v0.9 on the seeds set aside for reporting (16 against 7 in 256 tries, p = 0.09). That is not a clear difference either way.
• Turning while walking still falls a little short of the command (86%).
• It needs a firmware that sends the 2-value gait clock (80 inputs). It does not run on a 78-input firmware.
• The action filter, joint limits and ankle linkage details are not published here. The policy was trained with the robot's values; a robot with different values may behave differently.
• It has not been run on the real robot yet.