Hugging Face's logo
R's picture

R PRO

Twu31
13
masterset-ai's profile picture dipankarsarkar's profile picture emreclskn's profile picture
·

AI & ML interests

None yet

Recent Activity

repliedto aufklarer's post about 12 hours ago
We’ve published Soniqo’s voice-agent leaderboard and an article explaining what we measured. https://huggingface.co/spaces/aufklarer/full-duplex-leaderboard The board covers three parts of a speech system: - Turn-taking · TurnBench: detecting turn endings, false alarms and decision delay. - Commands · SLURP: choosing the correct tool and matching its argument values. - Voice style · InstructTTSEval: following voice descriptions, delivery directions and role-play instructions, with separate scores for each. Models include VAP, p99, Smart Turn, Qwen3-TTS, VoxCPM2, FireRedTTS3 and MOSS-VoiceGenerator, alongside cloud configurations from OpenAI, Google, AWS and ElevenLabs. Each table shows run sizes and links to the protocol. We also published https://huggingface.co/aufklarer/P99-Turn-1-Mini-CoreML, https://huggingface.co/aufklarer/VAP-OTO-CoreML and https://huggingface.co/soniqo/VAP-OTO-ONNX exports. Both models have native Swift APIs; VAP also has C++ and Android support. Explore the leaderboard https://huggingface.co/spaces/aufklarer/full-duplex-leaderboard · Read the article https://soniqo.audio/blog/voice-agent-benchmarks
repliedto eaddario's post about 12 hours ago
Experimental global target bits‑per‑weight quantization of IFM/K2-Horizon-7B and IFM/K2-Horizon-MoVA-36B-A4B. Unlike standard llama.cpp quantization that rely on fixed type heuristics (e.g., Q4_K_M), the Target BPW approach automatically optimizes per-tensor precision where it matters the most, and produces high quality models that meet a precise global size target. Key Advantages: - VRAM Maximization: Can generate high quality models sized exactly to fit hardware constraints (e.g., fitting the model into exactly 24GB VRAM). - Data-Driven Precision: Quantization mix is determined by actual weight error sensitivity rather than hardcoded rules, often yielding better PPL/KLD size trade-offs. Full benchmarks (PPL, KLD, ARC, GPQA, MMLU, etc.) and methodology in the model's card. https://huggingface.co/eaddario/K2-Horizon-7B-GGUF https://huggingface.co/eaddario/K2-Horizon-MoVA-36B-A4B-GGUF
repliedto jiteshkprasad's post 1 day ago
We tested 27 small open language models, all under 400M parameters, in the same way. Same harness, same settings, and the model versions pinned. I think small models are really hard to compare from their own cards, so we put the whole thing out. Board: https://huggingface.co/spaces/udgamlabs/small-model-board Dataset: https://huggingface.co/datasets/udgamlabs/small-model-board-scores Write-up on the Udgam Labs blog: https://udgamlabs.com/blog/small-model-board The most surprising part for me was MMLU. In the usual A, B, C, D format, just answering "D" every time gets 26.9%, and 23 of these 27 models score at or below that. So at this size the letter format basically tests if a model can pick a letter, not what it knows. When we asked the same questions in cloze format, where the answer text itself is scored, all 23 did better. Our own models are in there like everyone else. Laghu-Base is 5th of 19 models up to 200M, and Gargi-Base is 7.3 points behind SmolLM2-360M. So we are not on top, and that is fine. What I wanted was a fair comparison. Do you compare small models on letter or cloze MMLU?
View all activity

Organizations

None yet