Nymbo PRO

huh

Checked both. It's not the commit rule. I ran the raw decoder output (no caption pipeline at all) on every scene, and Parakeet-unified scores identically: same WER, same deletions, all 16 scenes.

The deletions don't cluster before pauses either. Over the 6.5 uncut hours, 92% of Parakeet's extra deletions are mid-phrase. What they are is disfluencies: about two-thirds are repeated words and fillers, and most of the rest are false starts. Stuff like "I was [I was] out in", "until we [until we] moved", "[you know]". The references are verbatim, Nemotron keeps the stutters, Parakeet-unified cleans them up.

If you excuse those deletions for both engines, the uncut WER is 10.12 vs 10.19. So the whole gap is transcription style, not accuracy. For live captions I'd call dropping "I was, I was" a feature.

Thanks for digging into this, you found a real bug. That China call didn't make sense, so I looked at its errors: almost all of Nemotron's extra words were numbers. It spells them out ("twenty twenty one", "two point three") while the reference uses digits, and my scorer wasn't normalizing them.

I've switched the benchmark to Whisper's English normalizer (same as the Open ASR Leaderboard) and re-scored everything:

• Real-world mean: Parakeet-unified 12.1% vs Nemotron 560 11.4%
• China call: 12.5 vs 12.5, a tie
• Nemotron now wins 7 of 11 real scenes, Parakeet-unified the 3 podcasts
• Synthetic still goes to Nemotron, 15.0 vs 16.3, on the same loud-effects and quiet scenes
• Scoring the full uncut recordings (for WideChaos's question) gives 11.7 vs 10.8, so it isn't the excerpt cuts

On the leveller: Nemotron gets 33.5 without it vs 33.7 with it on the old scoring. Normalized, it's neutral for both engines. Parakeet-unified's 22.0 to 16.2 was the same number artifact, just with the leveller nudging how it formatted numbers. It only helps a little on very quiet speech now.

So it's not a WER win. Parakeet-unified is the GPU default for its punctuation (sentence F1 0.60 vs 0.40), lower p90 delay, and near-idle CPU. README has the corrected numbers.

Ran it. These transcripts have no timestamps, so fixed time cuts would still need an alignment to know where the reference text ends. Instead I scored the nine cut recordings whole, against their whole transcripts, with no cuts at all. That's about 6.5 hours of audio.

Same picture as the excerpts: Parakeet-unified 11.7% vs Nemotron 10.8%. On those same nine recordings the excerpts gave 11.2 vs 10.6, so if anything the cuts were slightly kind to Parakeet. Nemotron is ahead on everything except the Crimetown podcast, and most of Parakeet's extra errors are dropped words.

For the record, the cuts came from Parakeet TDT v3 offline, not the model being tested, but your question was the right one to ask. Uncut mode is in the repo now (bench.realset build --full).