TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs
Abstract
Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs. We study generalist video temporal grounding, in which one model predicts a variable-cardinality set of evidence intervals across video lengths, domains, query forms, and viewpoints. Existing training strategies are misaligned with this set-valued task: long-video labels often rely on brittle one-pass annotation, while reinforcement-learning rewards either fail to distinguish non-overlapping predictions or require fragile segment matching. TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization. TimeLens2-93K constructs reliable multi-span supervision through caption-derived proposals, independent localization, cross-agent consensus, semantic verification, and boundary refinement. Our temporal Wasserstein reward computes exact one-dimensional \(W_1\) between uniform distributions over merged interval supports, providing dense, matching-free feedback under unequal cardinalities and equivalent fragmentation; temporal IoU complements it with precise-overlap feedback. Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines on every benchmark, while the 4B and 8B variants achieve state-of-the-art performance, surpassing open-source models with up to 397B parameters. The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.
Community
Hi everyone! We are excited to share TimeLens2, a generalist video temporal grounding MLLM that identifies not only what happens in a video, but also when the supporting evidence occurs.
TimeLens2 treats temporal evidence as a variable-cardinality set of intervals across short and long videos, single- and multi-span events, declarative and question-form queries, and third- and first-person viewpoints.
Our two key contributions are:
- TimeLens2-93K: 93,232 verified grounding instances from 23,793 diverse videos, constructed through caption-derived proposals, dual-agent localization, temporal consensus, semantic verification, and boundary refinement.
- Temporal Wasserstein reward: dense, matching-free feedback over merged interval supports, complementing temporal IoU when predictions do not overlap or contain different numbers of intervals.
Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines, while the 4B and 8B variants achieve state-of-the-art performance. Notably, TimeLens2-4B surpasses Qwen3.5-397B-A17B on every benchmark by 7.5 average mIoU points.
š¤ Models and resources
We would love to hear your thoughts on generalist temporal grounding, interval-set-aware optimization, and applications that require traceable video evidence!
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- SER: Learning to Ground Video Reasoning with Semantic Evidence Rewards (2026)
- TimeThink: Reasoning with Time for Video LLMs (2026)
- DynFrame: Adaptive Reasoning-Driven Multimodal Framework with Dynamic Frame Augmentation for Complex Video Understanding (2026)
- Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QA (2026)
- Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding (2026)
- Efficient Spatio-Temporal Grounding with Multimodal Large Models via Second-Level Tracking and RL Verification (2026)
- Reasoning as Intersection: Consensus-Frame Alignment for Visual Focus in Video-MLLMs (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2607.17423 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash