Instructions to use MachinePulseAI/karpo-decision-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MachinePulseAI/karpo-decision-12B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="MachinePulseAI/karpo-decision-12B")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("MachinePulseAI/karpo-decision-12B") model = AutoModelForMultimodalLM.from_pretrained("MachinePulseAI/karpo-decision-12B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Running this model on Apple Silicon Macs (unofficial port, MPS and MLX)
Thank you for releasing Karpo Decision. We wanted to run it on Macs and found that the first decide() call fails there, because the sliding-window layers use a flash-attention call that PyTorch only provides for CUDA (aten::_flash_attention_forward is not implemented for MPS or CPU).
We wrote a small unofficial port and are sharing it in case it helps other Mac users:
- Code (Apache-2.0, no weights): https://github.com/fluxcodestudio/karpo-apple-silicon
karpo_apple_silicon.install()replaces that one function at run time with ordinary masked attention. Nothing in the installed karpo-decision package is edited.
The 12B loads and answers on a Mac with this patch (about 0.95 s per short request as shipped, about 0.44 s in float16 batches of 16, on the machine we used). Its answers were not compared with published 12B results; the comparison we ran was on the 31B, where 1 of 1,606 top answers changed against your published results. Details and limits are in the README.
The port is not affiliated with or endorsed by MachinePulse; if you would like it named or described differently, tell us and we will change it.