Abhinav-jigsawstack commited on
Commit
cd391f9
·
verified ·
1 Parent(s): f1b230a

Card: badges and links on one line; padded, rounded, 2x charts; vertical diagram

Browse files
.gitattributes CHANGED
@@ -35,3 +35,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  assets/leaderboard.png filter=lfs diff=lfs merge=lfs -text
37
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
 
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  assets/leaderboard.png filter=lfs diff=lfs merge=lfs -text
37
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
38
+ assets/accuracy-per-parameter.png filter=lfs diff=lfs merge=lfs -text
39
+ assets/hero.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -24,29 +24,13 @@ tags:
24
 
25
  lev answers typed questions about a piece of context in a single forward pass. You give it a **state** (text, a ticket, an email, or JSON) and a set of yes/no, choice, and score questions. It reads each answer from the logits it already computed and returns calibrated probabilities over exactly the options you supplied. It is a LoRA adapter on Qwen3.5-4B, and it speaks TypeSafe's `/v1/systemone` protocol, so code written for the TypeSafe SDK works against it once you change the base URL.
26
 
27
- <p align="center">
28
- <img src="https://img.shields.io/badge/license-Apache--2.0-2a78d6?style=flat-square" alt="Apache-2.0">
29
- <img src="https://img.shields.io/badge/base-Qwen3.5--4B-2a78d6?style=flat-square" alt="Qwen3.5-4B">
30
- <img src="https://img.shields.io/badge/output%20tokens-0-2a78d6?style=flat-square" alt="Zero output tokens">
31
- <img src="https://img.shields.io/badge/API-%2Fv1%2Fsystemone-2a78d6?style=flat-square" alt="/v1/systemone compatible">
32
- <a href="https://github.com/Abhinavexists/lev"><img src="https://img.shields.io/badge/code-GitHub-14181f?style=flat-square&logo=github" alt="GitHub"></a>
33
- </p>
34
 
35
  <h2 align="center">72.5% on S1Bench. 69 ms of compute. 4B parameters.</h2>
36
 
37
- <p align="center"><strong>Qwen3.5-4B + LoRA · one H100 · six S1Bench subsets, 1,999 items.</strong><br>
38
- Sixth of 22 on the S1Bench board over these subsets, ahead of every model its size or smaller.<br>
39
- 69 ms is compute inside the container. End to end from a laptop it is 414–463 ms.<br>
40
- <a href="#benchmarks">Benchmarks</a> · <a href="#speed">Speed</a> · <a href="#boundaries-worth-understanding">Boundaries</a></p>
41
 
42
- <p align="center">
43
- <a href="#quickstart"><strong>Quickstart</strong></a> ·
44
- <a href="#self-hosting-a-jev-compatible-http-server">Self-hosting</a> ·
45
- <a href="#benchmarks">Benchmarks</a> ·
46
- <a href="#why-it-works">Why it works</a> ·
47
- <a href="#training">Training</a> ·
48
- <a href="https://github.com/Abhinavexists/lev">GitHub</a>
49
- </p>
50
 
51
  **No generated tokens. No JSON to parse. No retries until it validates.**
52
 
@@ -205,7 +189,7 @@ Most of lev's round trip from a laptop is network and Modal's ingress; its compu
205
 
206
  ## Why it works
207
 
208
- <p align="center"><img src="assets/how-it-works.png" alt="State and questions go through one forward pass, then label-token readout or the candidate-path head, then per-bucket temperatures, then typed answers" width="100%"></p>
209
 
210
  - **Label-token readout.** Each option gets a short code, and the answer is read from the next-token logits over those codes. Codes that would split into two tokens are skipped, so every option stays one token. This handles yes/no, scores, and choice lists up to several hundred options. Choices are read in two option orders and averaged, which cancels position bias.
211
  - **Candidate-path head.** Past that point, a small learned head matches the state against each option's text, so the number of options has no fixed ceiling.
 
24
 
25
  lev answers typed questions about a piece of context in a single forward pass. You give it a **state** (text, a ticket, an email, or JSON) and a set of yes/no, choice, and score questions. It reads each answer from the logits it already computed and returns calibrated probabilities over exactly the options you supplied. It is a LoRA adapter on Qwen3.5-4B, and it speaks TypeSafe's `/v1/systemone` protocol, so code written for the TypeSafe SDK works against it once you change the base URL.
26
 
27
+ <p align="center"><img src="https://img.shields.io/badge/license-Apache--2.0-2a78d6?style=flat-square" alt="Apache-2.0"> <img src="https://img.shields.io/badge/base-Qwen3.5--4B-2a78d6?style=flat-square" alt="Qwen3.5-4B"> <img src="https://img.shields.io/badge/output%20tokens-0-2a78d6?style=flat-square" alt="Zero output tokens"> <img src="https://img.shields.io/badge/API-%2Fv1%2Fsystemone-2a78d6?style=flat-square" alt="/v1/systemone compatible"> <a href="https://github.com/Abhinavexists/lev"><img src="https://img.shields.io/badge/code-GitHub-14181f?style=flat-square&logo=github" alt="GitHub"></a></p>
 
 
 
 
 
 
28
 
29
  <h2 align="center">72.5% on S1Bench. 69 ms of compute. 4B parameters.</h2>
30
 
31
+ <p align="center"><strong>Qwen3.5-4B + LoRA · one H100 · six S1Bench subsets, 1,999 items.</strong><br>Sixth of 22 on the S1Bench board over these subsets, ahead of every model its size or smaller.<br>69 ms is compute inside the container. End to end from a laptop it is 414–463 ms.<br><a href="#benchmarks">Benchmarks</a> · <a href="#speed">Speed</a> · <a href="#boundaries-worth-understanding">Boundaries</a></p>
 
 
 
32
 
33
+ <p align="center"><a href="#quickstart"><strong>Quickstart</strong></a> · <a href="#self-hosting-a-jev-compatible-http-server">Self-hosting</a> · <a href="#benchmarks">Benchmarks</a> · <a href="#why-it-works">Why it works</a> · <a href="#training">Training</a> · <a href="https://github.com/Abhinavexists/lev">GitHub</a></p>
 
 
 
 
 
 
 
34
 
35
  **No generated tokens. No JSON to parse. No retries until it validates.**
36
 
 
189
 
190
  ## Why it works
191
 
192
+ <p align="center"><img src="assets/how-it-works.png" alt="State and questions go through one forward pass, then label-token readout or the candidate-path head, then per-bucket temperatures, then typed answers" width="460"></p>
193
 
194
  - **Label-token readout.** Each option gets a short code, and the answer is read from the next-token logits over those codes. Codes that would split into two tokens are skipped, so every option stays one token. This handles yes/no, scores, and choice lists up to several hundred options. Choices are read in two option orders and averaged, which cancels position bias.
195
  - **Candidate-path head.** Past that point, a small learned head matches the state against each option's text, so the number of options has no fixed ceiling.
assets/accuracy-by-subset.png CHANGED
assets/accuracy-per-parameter.png CHANGED

Git LFS Details

  • SHA256: dac47e9c68b2aadeaff10217b037c226f5b0d4a0fed7311bb97bb7dcc7a2dab6
  • Pointer size: 131 Bytes
  • Size of remote file: 103 kB
assets/compute.png CHANGED
assets/hero.png CHANGED

Git LFS Details

  • SHA256: a8d0d817ad2fd121347f51e61b268ed176f37a13c8e4487f4e17c0a60ef2c6c8
  • Pointer size: 131 Bytes
  • Size of remote file: 151 kB
assets/how-it-works.png CHANGED
assets/latency.png CHANGED
assets/leaderboard.png CHANGED

Git LFS Details

  • SHA256: 550c4fa229b4394b6568e4a6face54414cf53658b4944c4307e89661d21b4230
  • Pointer size: 131 Bytes
  • Size of remote file: 113 kB

Git LFS Details

  • SHA256: 23735f3ea5bd43fd39e22996b3ff14919209c669760eed930239a4ba7910a10e
  • Pointer size: 131 Bytes
  • Size of remote file: 241 kB