Songhao Wu
shwu
AI & ML interests
Mixture-of-Experts Model, Language Model Pretraining
Recent Activity
upvoted a paper 18 days ago
LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts authored a paper about 1 month ago
Redesign Mixture-of-Experts Routers with Manifold Power Iteration upvoted a paper about 1 month ago
Redesign Mixture-of-Experts Routers with Manifold Power IterationOrganizations
None yet