Saga

Right! The observation that the recovered checkpoint is a worse teacher, and that you should skip it and go to the original, is correct and useful. But fundamentally, your QAH is QAT: LLM-QAT was doing data-free distillation from the full-precision model years ago.

Hi there! From how I'm reading this, I think you might have missed a method error in your experiment that invalidates your headline.

The headline comparison looks like it has no control. The bf16 60B got one round of distillation and the MXFP4 60B got that round plus a second round from a better teacher. The obvious missing comparison is to give the bf16 60B the identical second distillation pass in bf16, same data, same steps, same teacher. Without that comparison you cannot separate "QAH is a good recipe" from "we trained one checkpoint longer against a teacher than the other."

Results table labels the teacher "120B teacher (MXFP4)", which is right since gpt-oss-120b ships with MXFP4 MoE weights natively. But the approach section says the teacher is full-size and full-precision, and the title claims the student beats its full-precision original. Those contradict.

Unless I'm wrong, I'd retract this and consdier republishing when you can show the significant difference on the comparable arms without extending one further than the other with more compute.