Humaneval score?
#1
by rombodawg - opened
What are the coding benchmark scores? Humaneval and humaneval+
Id like to ask the bloke to quantize this model but I wanna see if its even worth it considering this is llama based, and llama generally doesnt do that well in coding unless finetuned correctly. But if you can surprise me with good coding benchmark scores Ill ask
I just realized the base model is deepseekcoder, but I would still like to see how much the model improved with another humaneval bench after your finetune
Human Eval is now in progress.
I've quantized it 5-bit with GGUF already if you want to test that out
senseable/33x-coder-gguf
thank you, im actually testing it right now