RuntimeError: Detected that you are using FX to symbolically trace a dynamo-optimized function. This is not supported at the moment.
Anybody is getting this error, trying to finetune it (in my case, using a similar approach to SapBERT): RuntimeError: Detected that you are using FX to symbolically trace a dynamo-optimized function. This is not supported at the moment.
You should be able to specify reference_compile=False in the from_pretrained("answerdotai/ModernBERT-base", reference_compile=False). In short, the reference implementation uses torch.compile for some components of the model. This is not compatible with FX, so we also give you the option to disable the compile.
- Tom Aarsen
Hey! To add to @tomaarsen 's comment, if you are using multi-GPU with a HuggingFace trainer, this can sometimes be resolved by using accelerate to handle the multi-GPU aspect.
Thank you, Tom. Now it is working perfectly.
Jumping in a bit late, but this issue only seem to arise if you are launching multi-gpu training with DP training.
Using accelerate or torchrunfixes the issue while allowing you to get the sweet speed-up from compiling.
Please see this issue for more information.
Just use:.from_pretrained(model_path,attn_implementation="eager",reference_compile=False)
I encountered this doing inference. aidayy's suggestion worked, but without that, using with torch.no_grad() also worked.
When using infinity I met the problem, any ideas?