Post
86
A transcript can read fine and still get your phone number wrong. So I stopped scoring words and started scoring slots.
VoxEntity-EU is an open German/French benchmark: does a speech model capture a dictated name, phone number, email, date, address or reference code exactly? One wrong character fails the slot.
Whisper large-v3-turbo 34.2% · Voxtral Mini 3B 37.1% · correct-text ceiling 90.8%. 240 items per model, 95% CIs in the report. Emails: 0 to 20%.
My first extractor version produced a wrong headline: "ASR is the bottleneck." It was my parser. I retracted it in public commits and shipped what survived.
Read with care: synthetic speech (Qwen3-TTS), simulated phone line, provisional. A second TTS scores 10 points higher, so both results are published.
Dataset
Gjusev/voxentity-eu
Demo
Gjusev/voxentity-eu-results
Full record, retraction included https://github.com/Gjusev/voxentity-eu
VoxEntity-EU is an open German/French benchmark: does a speech model capture a dictated name, phone number, email, date, address or reference code exactly? One wrong character fails the slot.
Whisper large-v3-turbo 34.2% · Voxtral Mini 3B 37.1% · correct-text ceiling 90.8%. 240 items per model, 95% CIs in the report. Emails: 0 to 20%.
My first extractor version produced a wrong headline: "ASR is the bottleneck." It was my parser. I retracted it in public commits and shipped what survived.
Read with care: synthetic speech (Qwen3-TTS), simulated phone line, provisional. A second TTS scores 10 points higher, so both results are published.
Dataset
Gjusev/voxentity-eu
Demo
Gjusev/voxentity-eu-results
Full record, retraction included https://github.com/Gjusev/voxentity-eu