ChanLumerico commited on
Commit
facb666
·
verified ·
1 Parent(s): 8c1b494

Update model card

Browse files
Files changed (1) hide show
  1. README.md +60 -0
README.md ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: lucid
3
+ license: mit
4
+ tags:
5
+ - base
6
+ - gpt2
7
+ - lucid
8
+ datasets:
9
+ - webtext
10
+ pipeline_tag: feature-extraction
11
+ ---
12
+
13
+ # GPT-2 XL
14
+
15
+ > https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
16
+
17
+ [Lucid](https://github.com/ChanLumerico/lucid) port of `transformers/gpt2-xl`,
18
+ converted to Lucid-native safetensors.
19
+
20
+ ## Available weights
21
+
22
+ | Tag | Params | GFLOPs | Size | Source |
23
+ |---|---|---|---|---|
24
+ | `WEBTEXT` *(default)* | 1557.6M | — | 5941.87 MB | transformers |
25
+
26
+ ## Usage
27
+
28
+ ```python
29
+ import lucid
30
+ import lucid.models as models
31
+ from lucid.models.weights import GPT2XLargeWeights
32
+
33
+ # default tag
34
+ model = models.gpt2_xlarge(pretrained=True)
35
+
36
+ # explicit tag (enum or string)
37
+ model = models.gpt2_xlarge(weights=GPT2XLargeWeights.WEBTEXT)
38
+ model = models.gpt2_xlarge(pretrained="WEBTEXT")
39
+
40
+ # feed token ids (tokenize with the matching lucid.utils.tokenizer)
41
+ input_ids = lucid.tensor([[101, 7592, 2088, 102]], dtype=lucid.int64)
42
+ out = model(input_ids)
43
+ hidden = out.last_hidden_state # (B, T, hidden_size)
44
+ ```
45
+
46
+ ## Conversion
47
+
48
+ Converted from `transformers/gpt2-xl` via
49
+ `python -m tools.convert_weights gpt2_xlarge --tag WEBTEXT`.
50
+ Key mapping + numerical parity verified against the source.
51
+
52
+ ## License
53
+
54
+ `mit` — inherited from the original weights.
55
+
56
+ ## Citation
57
+
58
+ ```
59
+ Radford et al., "Language Models are Unsupervised Multitask Learners", 2019 (GPT-2).
60
+ ```