harpertoken commited on
Commit
c64d507
·
verified ·
1 Parent(s): 1d8b417

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -14,13 +14,13 @@ tags:
14
 
15
  # chat
16
 
17
- `openai-community/gpt2` further fine-tuned on DailyDialog, a corpus of short multi-turn English conversations about everyday topics. The base model is the 124M-parameter GPT-2 — `config.json` records `_name_or_path: gpt2`, twelve layers, 768 embedding dimensions. The previous version of this card declared its base model as `gpt2`, an identifier that has since moved; `openai-community/gpt2` is the current one.
18
 
19
  This is a continuation, not a chat model. Nothing here establishes turn-taking, instruction following or any conversational protocol: there is no chat template in `tokenizer_config.json`, and the only token added beyond GPT-2's vocabulary is a `[PAD]` entry at id 50257, which takes `vocab_size` to 50258 against GPT-2's 50257. It produces fluent English that resembles dialogue because that is what it was trained to imitate, which is a weaker property than it sounds.
20
 
21
  One detail about the checkpoint itself. It carries twelve `transformer.h.*.attn.masked_bias` tensors that current Transformers does not define: the key was removed from `GPT2Attention` years ago, and the upstream `gpt2` weights contain no such tensor. Loading this repository therefore prints an `UNEXPECTED` warning for those twelve keys on every run. They are inert, since nothing reads them, and generation is unaffected. They are left in place rather than silently stripped, so the file matches whatever produced it.
22
 
23
- The card this replaces documented a `generate_response.py` script and a FastAPI service on `localhost:8000` with `/status`, `/chat` and interactive docs endpoints. None of that is in the repository — it contains weights and tokenizer files only — so those sections have been removed. The declared perplexity, BLEU and F1 metrics likewise had no values behind them; no evaluation is recorded for this checkpoint.
24
 
25
  ## Usage
26
 
@@ -35,7 +35,7 @@ A `pad_token` is not configured, so batching prompts together will warn. Set `to
35
 
36
  ## Limitations
37
 
38
- DailyDialog is scripted, crowd-sourced, and narrow — 13k conversations of polite small talk, annotated for emotion and communication acts. A model trained on it will handle greetings and farewells far better than disagreement, and will reproduce the register of that corpus rather than anything wider. Because GPT-2 is a 124M-parameter base and the fine-tuning is small, output is frequently fluent and wrong. The biases of both GPT-2 and DailyDialog carry through unchanged.
39
 
40
  ## Attribution
41
 
 
14
 
15
  # chat
16
 
17
+ `openai-community/gpt2` further fine-tuned on DailyDialog, a corpus of short multi-turn English conversations about everyday topics. The base model is the 124M-parameter GPT-2: `config.json` records `_name_or_path: gpt2`, twelve layers, 768 embedding dimensions. The previous version of this card declared its base model as `gpt2`, an identifier that has since moved; `openai-community/gpt2` is the current one.
18
 
19
  This is a continuation, not a chat model. Nothing here establishes turn-taking, instruction following or any conversational protocol: there is no chat template in `tokenizer_config.json`, and the only token added beyond GPT-2's vocabulary is a `[PAD]` entry at id 50257, which takes `vocab_size` to 50258 against GPT-2's 50257. It produces fluent English that resembles dialogue because that is what it was trained to imitate, which is a weaker property than it sounds.
20
 
21
  One detail about the checkpoint itself. It carries twelve `transformer.h.*.attn.masked_bias` tensors that current Transformers does not define: the key was removed from `GPT2Attention` years ago, and the upstream `gpt2` weights contain no such tensor. Loading this repository therefore prints an `UNEXPECTED` warning for those twelve keys on every run. They are inert, since nothing reads them, and generation is unaffected. They are left in place rather than silently stripped, so the file matches whatever produced it.
22
 
23
+ The card this replaces documented a `generate_response.py` script and a FastAPI service on `localhost:8000` with `/status`, `/chat` and interactive docs endpoints. None of that is in the repository, which contains weights and tokenizer files only, so those sections have been removed. The declared perplexity, BLEU and F1 metrics likewise had no values behind them; no evaluation is recorded for this checkpoint.
24
 
25
  ## Usage
26
 
 
35
 
36
  ## Limitations
37
 
38
+ DailyDialog is scripted, crowd-sourced, and narrow: 13k conversations of polite small talk, annotated for emotion and communication acts. A model trained on it will handle greetings and farewells far better than disagreement, and will reproduce the register of that corpus rather than anything wider. Because GPT-2 is a 124M-parameter base and the fine-tuning is small, output is frequently fluent and wrong. The biases of both GPT-2 and DailyDialog carry through unchanged.
39
 
40
  ## Attribution
41