docs: HF model card — best-of-both front-matter derived from manifest

README.md

@@ -1,85 +1,184 @@
 ---
+language:
+- en
 license: apache-2.0
-base_model: robometer/Robometer-4B
-tags:
-  - reward-model
-  - robotics
-  - vision-language-action
-  - openral
-  - bitsandbytes
-  - nf4
-library_name: openral
 pipeline_tag: robotics
+tags:
+- OpenRAL
+- rskill
+- nf4
+- 4-bit
+- any
+- reward
+- reward-model
+- robot-learning
+- progress-estimation
+- success-detection
+- qwen3-vl
+- bitsandbytes
+base_model:
+- robometer/Robometer-4B
+base_model_relation: quantized
+inference: false
 ---
 
 # rskill-robometer-4b-nf4
 
-Pre-quantized **NF4** build of [`robometer/Robometer-4B`](https://huggingface.co/robometer/Robometer-4B)
-(a Qwen3-VL-4B robotic **reward foundation model**, arXiv 2603.02115), packaged as an
-[OpenRAL](https://github.com/OpenRAL/openral) **`kind: reward`** rSkill (ADR-0057).
-
-It runs **in parallel with a VLA policy** and scores the live rollout: given the
-robot's camera frames + the task instruction, it emits **per-frame normalized
-progress (0–1)** and **per-frame success probability**. The OpenRAL reasoner polls
-it on demand (read-only `query_task_progress` tool) to decide whether to continue,
-advance, or replan — **advisory only**, never on the control path.
-
-## What's in this repo
-
-A self-contained checkpoint that the OpenRAL reward sidecar loads **directly as
-4-bit** — no bf16 materialization, no requantize:
-
-- `model.safetensors` — 236 `Linear` modules packed to bitsandbytes NF4 (~3.32 GB
-  resident), plus the folded non-persistent rotary `inv_freq` buffers.
-- `config.json` — model config (resized vocab 151674).
-- `config.yaml` — the `robometer` `ExperimentConfig` (lets the sidecar rebuild the
-  `RBM` graph offline).
-- tokenizer / processor files (incl. `added_tokens.json` — the model's added
-  progress token).
-- `quantization_metadata.json` — provenance.
-
-> The model **class is `RBM`** (`robometer.models.rbm`) — the upstream
-> `config.json` advertises `architectures: ["RFM"]` with **no `auto_map`**, so
-> vanilla `transformers.AutoModel` cannot load it. The OpenRAL sidecar installs the
-> pinned `robometer` package (commit `a669dffc`) with **`transformers==4.57.1`** in
-> an isolated venv and builds the skeleton on the `meta` device, then installs these
-> packed NF4 weights via `Params4bit.from_prequantized`.
-
-## Provenance & verification
-
-- **Source:** `robometer/Robometer-4B` @ `beef63bc914c5c189329d49c6d712d96d632aa34` (Apache-2.0).
-- **Quantization:** bitsandbytes NF4 (double-quant), compute dtype bf16, the OpenRAL
-  rule `nn.Linear.numel ≥ 4e6 → Linear4bit`. Built by
-  [`tools/build_robometer_nf4_checkpoint.py`](https://github.com/OpenRAL/openral/blob/master/tools/build_robometer_nf4_checkpoint.py).
-- **Bit-identical** to loading the upstream bf16 weights and quantizing in place:
-  same-process forward `max|Δ| = 0`; 4-bit dequant round-trip `0`. For a byte-stable
-  reward ramp across process launches, the sidecar pins the math SDP kernel +
-  `use_deterministic_algorithms(True)` + `CUBLAS_WORKSPACE_CONFIG=:4096:8` +
-  `cudnn.allow_tf32=False`.
-- **Footprint:** ~3.32 GB resident on an 8 GB GPU; co-resident with the sim (and a
-  small NF4 VLA). The reward forward subsamples the frame window to bound activation.
-
-## Usage (OpenRAL)
-
-This is consumed by OpenRAL, not loaded standalone. The `kind: reward` manifest
-points `weights_uri` here:
-
-```yaml
-weights_uri: "hf://OpenRAL/rskill-robometer-4b-nf4"
-```
+> **OpenRAL rSkill** — Robometer-4B (Qwen3-VL-4B robotic **reward foundation
+> model**) packaged as an NF4 bitsandbytes `reward` rSkill (ADR-0057). Given a
+> rollout's RGB frames plus the task instruction, it emits **per-frame
+> normalized progress (0–1)** and **per-frame success probability**, queried on
+> demand by the Reasoner. **No actuators. Advisory-only.** Apache-2.0.
 
-and in deploy-sim:
+## Quick Start
 
 ```bash
-openral deploy sim --config scenes/deploy/<scene>.yaml --enable-reward-monitor
+ral skill install hf://OpenRAL/rskill-robometer-4b-nf4
+```
+
+```python
+from openral_core.schemas import RSkillManifest
+
+manifest = RSkillManifest.from_yaml("rskills/robometer-4b/rskill.yaml")
+assert manifest.kind == "reward"
+assert manifest.role == "s2"
+assert manifest.reward.progress_range == (0.0, 1.0)
+assert manifest.quantization.extra["scheme"] == "nf4"
+assert manifest.is_commercial_use_allowed is True
 ```
 
-brings up the reward monitor parallel to the VLA and lets the reasoner poll
-`/openral/perception/query_task_progress`. See
-[ADR-0057](https://github.com/OpenRAL/openral/blob/master/docs/adr/0057-robometer-reward-rskill.md).
+## What It Does
+
+Robometer is a general-purpose robotic reward model trained on RBM-1M (>1M
+trajectories across diverse embodiments, including failures) with a dual
+objective: a frame-level **progress** loss anchored on expert data and a
+trajectory-comparison **preference** loss for global ordering. Given a task
+instruction and a rollout video, it predicts per-frame progress (continuous
+values over time) and per-frame success probability.
+
+This rSkill declares `kind: reward` and `role: s2`: it is a pure perception
+**consumer** operating at S2 (slow-reasoning) rate (~0.2–1 Hz), not an S1 fast
+policy. It runs **in parallel with a `kind: vla` policy**, continuously
+ingesting the VLA's camera frames into a rolling window, and the Reasoner
+queries it on demand — *"how is success doing now / over the last X seconds?"* —
+to decide whether to continue, escalate to a scene VLM (`query_scene`), advance
+to the next subgoal, or enter the replanning ladder. It **never drives
+`ros2_control` joints** and never gates motors (CLAUDE.md §1.1).
+
+## Why a reward model alongside the VLA
+
+A VLA policy emits actions but has no notion of whether it is *succeeding*.
+Robometer closes that loop: it turns the camera stream into a normalized
+per-frame progress + success signal the Reasoner can act on, so a stalled or
+failing rollout triggers replanning instead of running to a timeout.
+
+## Architecture
+
+Robometer-4B finetunes `Qwen/Qwen3-VL-4B-Instruct` (`model_type: qwen3_vl`)
+with three prediction heads — `progress_head`, `success_head`, `preference_head`
+— on top of a frame-pooled attention readout (`frame_pool_attn`). The on-disk
+HF `config.json` advertises `architectures: ["RFM"]`, but the actual model
+class is `RBM` (in the upstream `robometer` package). **It has no `auto_map` and
+ships no Hub-side modeling code, so vanilla `transformers.AutoModel` cannot load
+it** — the sidecar loads it via the pinned `robometer` package
+(`robometer.utils.save.load_model_from_hf`).
+
+## Runtime
+
+The `kind: reward` runtime is implemented as a read-only Reasoner tool
+(`QueryTaskProgressTool`), **not** an `ExecuteSkill` (a reward monitor produces
+scalars, not actions):
+
+- **Sidecar**: an out-of-process ZMQ REQ/REP + msgpack server boots the NF4
+  model in its own isolated venv, maintains a rolling time-indexed frame buffer
+  (`frame_window_s`), and answers windowed progress/success queries. It loads
+  via `robometer.utils.save.load_model_from_hf` with **`transformers` pinned to
+  `4.57.1`** (5.x changes the processor `__call__` kwargs and drops `input_ids`)
+  and the `robometer` package pinned to commit `a669dffc`.
+- **Frame source**: abstracted for **sim and real**. The sidecar consumes the
+  same `sensor_msgs/Image` camera topic the co-active VLA uses — fed by the
+  GStreamer perception tee on real hardware, or by the sim HAL camera publisher
+  in `deploy-sim` (which has no GStreamer). In `deploy-sim` only camera-rendering
+  robots expose frames; absent frames surface as `ROSPerceptionStale`.
+- **Reasoner tool**: the LLM sees the read-only `query_task_progress` tool when
+  a reward rSkill is co-active with a VLA. It asks for the windowed assessment
+  (`progress_now`, `success_now`, trends, `stalled`) and the answer feeds the
+  next reasoning tick / the replanning ladder.
+
+### Inference contract
+
+Discrete (binned) mode yields the normalized signal OpenRAL consumes:
+`compute_batch_outputs(..., sample_type="progress", is_discrete_mode=True,
+num_bins=100)` returns `progress_pred` (per-frame ∈ [0,1]) and
+`outputs_success["success_probs"]` (per-frame ∈ [0,1]). Continuous mode returns
+raw, unnormalized regression values instead. Default sampling is 3 fps.
+
+### Validated live
+
+End-to-end on an **NVIDIA RTX 4070 Laptop (8 GB)** (ADR-0057 Phases 0/2/3):
+
+- **NF4 quantization**: 236 `Linear` modules → `Linear4bit`; **8.91 GB bf16 →
+  3.33 GB resident**, **3.56 GB peak** including an 8-frame forward — **4.44 GB
+  headroom** for a co-resident small NF4 VLA.
+- **Working sidecar**: streaming a real rollout video ("Put green stick in
+  brown bowl") through the ZMQ sidecar, **progress ramped 0.21 → 0.88** and
+  **success spiked to 0.90 exactly at task completion**, then eased — exactly
+  the Reasoner signal intended.
+
+Run with `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`. The model loads via
+the `robometer` package (not `AutoModel`); the sidecar venv pins
+`transformers==4.57.1`.
+
+## Benchmark Numbers
+
+Paper-reported (Robometer team, March 2026, arXiv 2603.02115);
+`reproduced_locally: false`. Robometer reports more generalizable reward
+functions than prior methods (GVL, VLAC, RoboDopamine, TOPReward) across
+benchmarks and real-world evaluations, improving downstream robot-learning
+performance. See the paper for the full tables.
+
+## Supported robots and embodiments
+
+This reward monitor is **embodiment-agnostic** — it scores camera frames + a
+task instruction and emits scalars, never actuator commands, so it imposes no
+kinematic requirement. The only hardware dependency is an RGB camera stream of
+at least 224×224. It pairs with any S1 VLA policy: the VLA acts, this model
+reports whether the task is progressing / has succeeded.
+
+## Sensors and Observation Contract
+
+| Direction | Key | Modality | Shape / format | Notes |
+|---|---|---|---|---|
+| in | any RGB camera | RGB video frames | min 224 × 224 | the same topic the co-active VLA consumes |
+| in | task instruction | text | natural language | required (`instruction_required: true`) |
+| out | progress | float per frame | ∈ `progress_range` (`[0,1]`) | normalized task progress |
+| out | success | float per frame | ∈ `[0,1]` | per-frame success probability |
+
+The model emits no action chunks and has no proprioception contract.
+
+## Manifest Summary
+
+| Field | Value |
+|---|---|
+| `name` | `OpenRAL/rskill-robometer-4b-nf4` |
+| `version` | `0.1.0` |
+| `license` | `apache-2.0` |
+| `role` / `kind` | `s2` / `reward` |
+| `runtime` | `pytorch` |
+| `quantization.dtype` / `scheme` | `int4` / `nf4` |
+| `weights_uri` | `hf://OpenRAL/rskill-robometer-4b-nf4` (pre-quantized NF4, meta-loadable; built from the SHA-pinned upstream `source_repo`) |
+| `min_vram_gb.bf16` | 9.0 GB |
+| `min_vram_gb.int4` | 3.6 GB |
+| `reward.frame_window_s` / `target_fps` | 40.0 s / 3.0 fps (ADR-0074 amendment — scores the whole attempt start→now, not an 8 s trailing slice) |
+| `reward.progress_range` / `success_threshold` | `[0,1]` / 0.5 |
+| `latency_budget.per_chunk_ms` | 3000 ms |
+| `actions` | `monitor` |
 
 ## License
 
-Apache-2.0, inherited from the upstream `robometer/Robometer-4B`. See `LICENSE`.
-The upstream `robometer` package is pinned by commit and executed only in an
-isolated sidecar venv (it is not an OpenRAL-trusted org).
+The rSkill package metadata and README are OpenRAL project files under
+Apache-2.0. The wrapped Robometer-4B weights are released under **Apache-2.0**,
+permitting commercial use. No `OPENRAL_ALLOW_NONCOMMERCIAL=1` flag is needed.
+The upstream `robometer` code (loaded by the sidecar) is governed by its own
+repository license; it is executed in an isolated, pinned sidecar venv and is
+**not** an OpenRAL-trusted org (see `_vendor/PROVENANCE.md`).