PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation
Abstract
PINT unifies vision-language reasoning, video world modeling, and action generation via learnable predictive intent tokens to improve robot manipulation generalization without embodied pretraining.
Generalist robot manipulation can benefit from pretrained vision-language models (VLMs) for semantic reasoning and video generation models (VGMs) for world modeling. However, existing unified models that directly fuse them through shared attention at multiple layers can make it harder to draw on their specialized capabilities. We introduce PINT, a unified model that coordinates semantic reasoning, world modeling, and action generation through a shared chunk-level Predictive Intent while preserving expert-specific pathways. PINT inserts learnable Intent Tokens into the VLM sequence to form this representation, conditioning both the VGM and an Action Expert (AE). It replaces only the VGM's prompt conditioning and retains the VGM at inference, where the AE reads its representations through one-way attention. Rather than prescribing what Predictive Intent should encode, PINT learns it jointly from the native prediction objectives of the VLM, VGM, and AE, which provide complementary views of the same upcoming task execution. Training uses two stages: Stage 1 freezes the VGM so that future video prediction shapes Predictive Intent with task-relevant future dynamics, while Stage 2 fixes the learned Intent and adapts the VGM and AE. Without embodied pretraining, PINT achieves strong in-distribution (ID) performance and substantially improves out-of-distribution (OOD) generalization, reaching 15.5% on RoboTwin 2.0-Hard compared with 1.8% for Fast-WAM and 3.2% for StarVLA, and 79.5% on LIBERO-Plus compared with 51.5% and 75.0%, respectively. Across five real-world bimanual tasks, PINT reaches 68% ID and 24% OOD success, compared with 35%/9% for pretrained Motus and 67%/35% for pretrained π_{0.5}.
Community
Multiple experts, one intent.
PINT explores how to unify semantic reasoning, world modeling, and action generation while preserving expert specialization. It introduces a shared, chunk-level Predictive Intent that coordinates a pretrained vision-language model, video generation model, and action expert. Learned through complementary video and action prediction objectives, this shared representation enables effective cross-expert coordination and improved OOD generalization without embodied pretraining.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction (2026)
- Rewiring Semantics, Dynamics, and Control: A Simple yet Effective Action-Centric Tri-Stream Transformer (2026)
- GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation (2026)
- Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models (2026)
- Acting in Meters: Learning Metric Interactions for Precise Robotic Manipulation (2026)
- PLaW-VLA: Predictive Latent World Modeling for Vision-Language-Action Policies (2026)
- Forward Dynamics Models (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2610.16355 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper