juliannunezb/crisp3b-v8-ckpts

Text Generation • Updated Sep 16

Note Start here: 2.67B base, custom tokenizer, 38h on one B200

Note The corpus: 165B tokens, 11-stage cleaning pipeline, nested sampling

Note Crisp retokenized with the v8 single-digit tokenizer (stable + anneal phases)

juliannunezb/crisp3b-v7-ckpts

Text Generation • Updated Sep 16

Note 2.67B with GPT-2 tokenizer; base of the post-training chain

juliannunezb/crisp3b-sft-ckpts

Updated Jul 10

Note SFT on 330k ChatML conversations

juliannunezb/crisp3b-reason-ckpts

Updated Jul 10

Note Reasoning cold-start () on 35k short verified traces

Note SFT corpus

Note Reasoning cold-start corpus

juliannunezb/llm-training-v1-checkpoints

Text Generation • Updated Apr 20

Note Where it started: 90M GPT-2 on OpenWebText, hardware/optimizer ablations

juliannunezb/transformer-lm-500m

Updated Apr 21

Note 500M stage (v1)

juliannunezb/transformer-lm-500m-v2

Updated Apr 22

Note 500M stage (v2)

juliannunezb/transformer-lm-500m-v3

Updated Apr 24

Note 500M stage (v3)

juliannunezb/llm-500m-pretrain-v1

Updated May 15

Note 500M pretrain on the 100B mix

juliannunezb/llm-500m-sft-v1

Updated Jun 6

Note 500M SFT

juliannunezb/local50m-combo3-v1

Updated May 2

Note 50M pipeline on a Mac, step 1: pretrain

juliannunezb/local50m-midtrain-v2

Updated May 14

Note 50M pipeline, step 2: midtrain

juliannunezb/local50m-combo3-sft-v1

Updated May 14

Note 50M pipeline, step 3: SFT

juliannunezb/local50m-combo3-dpo-v1

Updated May 14

Note 50M pipeline, step 4: DPO

juliannunezb/local50m-combo3-grpo-bootstrap-v1

Updated May 14

Note 50M pipeline, step 5: GRPO bootstrap

Note 100B-token pretrain mix (GPT-2 BPE)

Note 10B-token pretrain mix (GPT-2 BPE)

Note Midtrain / anneal mix

Note SFT data (SmolTalk, GPT-2 BPE)

Note DPO data (UltraFeedback, GPT-2 BPE)