Initial private release: Fleck-M-500K

README.md

@@ -0,0 +1,151 @@
+---
+language:
+- en
+- ja
+license: mit
+tags:
+- fleck
+- causal-language-model
+- custom-architecture
+- text-generation
+- conversational
+- research
+base_model: ML-is-Fun/Fleck-M-500K-Base
+---
+
+# [Fleck-M-500K](https://huggingface.co/ML-is-Fun/Fleck-M-500K)
+
+**[Fleck-M-500K Instruct](https://huggingface.co/ML-is-Fun/Fleck-M-500K)** — the independently instruction-tuned child of [`Fleck-M-500K-Base`](https://huggingface.co/ML-is-Fun/Fleck-M-500K-Base).
+
+- **497,288 parameters**
+- **BF16 SafeTensors weights**
+- Fine-tuned from the Base-100M model on Dolly-15k
+- Intended for local conversational experiments on Apple Silicon
+
+## Model Details
+
+|  |  |
+| --- | --- |
+| Architecture | Decoder-only Transformer |
+| Parameters | 497,288 |
+| Hidden size | 128 |
+| FFN size | 336 |
+| Physical blocks | 2 |
+| Effective depth | 4 (`A → B → A → B`) |
+| Attention | GQA — 4 query heads, 2 KV heads, head dimension 32 |
+| Normalization | RMSNorm |
+| Embedding | Factorized tied embedding, rank 64 |
+| Vocabulary | 2,048 |
+| Context length | 2,048 tokens |
+| Canonical dtype | BF16 |
+| Base model | [`ML-is-Fun/Fleck-M-500K-Base`](https://huggingface.co/ML-is-Fun/Fleck-M-500K-Base) |
+
+## Training
+
+### Pretraining
+
+The Base parent was initialized from scratch and pretrained for exactly 100,000,000 real tokenizer tokens on a 70/30 FineWeb-Edu/FineWeb mixture.
+
+### Instruction Tuning
+
+|  |  |
+| --- | --- |
+| Base model | [`Fleck-M-500K-Base`](https://huggingface.co/ML-is-Fun/Fleck-M-500K-Base) |
+| Dataset | [`databricks/databricks-dolly-15k`](https://huggingface.co/datasets/databricks/databricks-dolly-15k) |
+| Dataset license | CC BY-SA 3.0 |
+| Requested budget | 250K supervised tokens |
+| Realized supervised tokens | 216,070 |
+| Optimizer | AdamW |
+| Instruction tuning | Assistant-response supervision |
+| Hardware | Apple M2 (10-core GPU) |
+
+## Benchmark Results
+
+Corrected zero-shot evaluation, no chat template, FP32 evaluation, and the same benchmark aggregation protocol were used for both variants.
+
+| Task | Metric | Shots | Base | **Instruct** | Δ |
+| --- | --- | ---: | ---: | ---: | ---: |
+| HellaSwag | `acc_norm` | 0 | 25.04% | **24.67%** | -0.38pp |
+| PIQA | `acc_norm` | 0 | 51.31% | 50.98% | -0.33pp |
+| ARC-Easy | `acc_norm` | 0 | 26.05% | **26.09%** | +0.04pp |
+| ARC-Challenge | `acc_norm` | 0 | 24.15% | **24.23%** | +0.09pp |
+| LAMBADA OpenAI | `acc` | 0 | 0.04% | **0.08%** | +0.04pp |
+| WinoGrande | `acc` | 0 | 51.30% | 50.99% | -0.32pp |
+| BoolQ | `acc` | 0 | 37.83% | 37.83% | 0.00pp |
+| MMLU (57-subject macro) | `acc` | 0 | 23.14% | **23.20%** | +0.06pp |
+| **Eight-task mean** | — | 0 | 29.86% | **29.76%** | -0.10pp |
+
+## Tokenizer and Chat Format
+
+- `Fleck-Tokenizer-2048`
+- Byte-level BPE
+- Vocabulary size: 2,048
+
+The training chat format is:
+
+```text
+<bos><|user|>{user}<|eot|><|assistant|>{response}<|eot|><eos>
+```
+
+### Special Tokens
+
+| Token | ID | Role |
+| --- | ---: | --- |
+| `<bos>` | 0 | sequence start |
+| `<eos>` | 1 | sequence end |
+| `<pad>` | 2 | padding |
+| `<unk>` | 3 | unknown token |
+| `<\|system\|>` | 4 | system turn |
+| `<\|user\|>` | 5 | user turn |
+| `<\|assistant\|>` | 6 | assistant turn |
+| `<\|eot\|>` | 7 | end of turn |
+
+## Usage
+
+The bundle includes a self-contained `inference.py`; it does not import the Fleck-LM checkout. The accompanying `config.json`, `generation_config.json`, and `tokenizer_config.json` describe the custom architecture and generation/tokenizer defaults; standard `transformers.AutoModel` loading is not supported. The `chat_template.jinja` file contains the Instruct chat template used by compatible HF tooling. Install the three runtime dependencies:
+
+```bash
+python -m pip install torch safetensors tokenizers
+```
+
+By default the CLI starts an interactive chat. `/exit` quits and `/clear` resets the conversation history:
+
+```bash
+python inference.py --device cpu --max-tokens 32
+```
+
+For one prompt without interactive mode, pass `--no-chat` and `--prompt`:
+
+```bash
+python inference.py \
+  --ckpt model.safetensors \
+  --tokenizer tokenizer.json \
+  --no-chat \
+  --prompt "Explain what a tokenizer does." \
+  --max-tokens 32 \
+  --device cpu
+```
+
+The script reads and runs the BF16 checkpoint without an FP32 model copy, validates every SafeTensors key, shape, and dtype, and uses FP32 only for attention score/softmax and tied-logit accumulation. It reproduces the factorized tied embedding/logits, effective-depth execution `A → B → A → B`, half-split RoPE, GQA, physical KV caches, RMSNorms, and greedy generation without repository-local imports. Generation stops on `<|eot|>` or `<eos>`.
+
+## Limitations
+
+This model is extremely small and is intended for research and local experimentation rather than reliable general-purpose assistance. It may produce repetitions, malformed text, weak factual answers, or incoherent responses. Instruction tuning improves conversational behavior but does not overcome the limits of a 497,288 parameter model.
+
+## License
+
+MIT License.
+
+## Files
+
+The public bundle contains these files:
+
+- `README.md` — model card and usage documentation
+- `inference.py` — standalone strict loader and interactive/single-prompt inference CLI
+- `model.safetensors` — BF16 model weights
+- `tokenizer.json` — standalone tokenizer
+- `config.json` — custom architecture configuration
+- `generation_config.json` — greedy generation defaults
+- `tokenizer_config.json` — tokenizer defaults and special-token mapping
+- `chat_template.jinja` — Instruct chat template
+No training data, optimizer state, or other training outputs are included.

chat_template.jinja

@@ -0,0 +1,9 @@
+{{ bos_token }}
+{%- for message in messages %}
+{%- if message['role'] == 'system' %}{{ '<|system|>' }}{{ message['content'] }}{{ '<|eot|>' }}
+{%- elif message['role'] == 'user' %}{{ '<|user|>' }}{{ message['content'] }}{{ '<|eot|>' }}
+{%- elif message['role'] == 'assistant' %}{{ '<|assistant|>' }}{{ message['content'] }}{{ '<|eot|>' }}
+{%- else %}{{ raise_exception('Unsupported role: ' ~ message['role']) }}
+{%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}{{ '<|assistant|>' }}{%- else %}{{ eos_token }}{%- endif %}

config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "FleckModel"
+  ],
+  "attention_type": "GQA",
+  "bos_token_id": 0,
+  "candidate_id": "Fleck-M-500K",
+  "custom_architecture": true,
+  "embedding_mode": "factorized_tied",
+  "embedding_rank": 64,
+  "eos_token_id": 1,
+  "execution_order": [
+    "A",
+    "B",
+    "A",
+    "B"
+  ],
+  "head_dim": 32,
+  "hidden_size": 128,
+  "implementation": "inference.py",
+  "intermediate_size": 336,
+  "max_position_embeddings": 2048,
+  "model_type": "fleck",
+  "num_attention_heads": 4,
+  "num_hidden_layers": 4,
+  "num_key_value_heads": 2,
+  "num_physical_blocks": 2,
+  "pad_token_id": 2,
+  "tie_word_embeddings": true,
+  "torch_dtype": "bfloat16",
+  "unk_token_id": 3,
+  "use_cache": true,
+  "vocab_size": 2048
+}

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "bos_token_id": 0,
+  "eos_token_id": [1, 7],
+  "pad_token_id": 2,
+  "do_sample": false,
+  "max_new_tokens": 32,
+  "use_cache": true
+}

inference.py

@@ -0,0 +1,378 @@
+#!/usr/bin/env python3
+"""Self-contained CPU/MPS greedy inference for the Fleck-M-500K Instruct export.
+
+This file intentionally has no import from the Fleck-LM source tree.  The model
+architecture, chat template, and SafeTensors contract are reproduced here so
+this directory can be copied from Hugging Face and used on its own.
+"""
+from __future__ import annotations
+
+import argparse
+import math
+from collections.abc import Iterable
+from pathlib import Path
+from typing import cast
+
+import torch
+from safetensors import safe_open
+from safetensors.torch import load_file
+from torch import Tensor, nn
+
+from tokenizers import Tokenizer
+
+VOCAB_SIZE = 2048
+CONTEXT_LENGTH = 2048
+HIDDEN_SIZE = 128
+INTERMEDIATE_SIZE = 336
+PHYSICAL_BLOCKS = 2
+EFFECTIVE_DEPTH = 4
+QUERY_HEADS = 4
+KV_HEADS = 2
+HEAD_DIM = 32
+EMBEDDING_RANK = 64
+ROPE_THETA = 10000.0
+BOS_ID, EOS_ID, PAD_ID, UNK_ID = 0, 1, 2, 3
+SYSTEM_ID, USER_ID, ASSISTANT_ID, EOT_ID = 4, 5, 6, 7
+EXECUTION_ORDER = (0, 1, 0, 1)
+TOKENIZER_NAME = "Fleck-Tokenizer-2048"
+MODEL_METADATA = {
+    "candidate_id": "Fleck-M-500K",
+    "canonical_dtype": "bfloat16",
+    "context_length": "2048",
+    "effective_depth": "4",
+    "embedding_rank": "64",
+    "head_dim": "32",
+    "kv_heads": "2",
+    "logical_execution_order": '["A", "B", "A", "B"]',
+    "parameter_count": "497288",
+    "physical_blocks": "2",
+    "query_heads": "4",
+    "tokenizer_name": TOKENIZER_NAME,
+}
+
+
+class RMSNorm(nn.Module):
+    def __init__(self, size: int, eps: float = 1e-5) -> None:
+        super().__init__()
+        self.weight = nn.Parameter(torch.ones(size))
+        self.eps = eps
+
+    def forward(self, hidden: Tensor) -> Tensor:
+        variance = hidden.float().pow(2).mean(dim=-1, keepdim=True)
+        scale = torch.rsqrt(variance + self.eps).to(dtype=hidden.dtype)
+        return hidden * scale * self.weight
+
+
+class Attention(nn.Module):
+    def __init__(self) -> None:
+        super().__init__()
+        self.q_proj = nn.Linear(HIDDEN_SIZE, QUERY_HEADS * HEAD_DIM, bias=False)
+        self.k_proj = nn.Linear(HIDDEN_SIZE, KV_HEADS * HEAD_DIM, bias=False)
+        self.v_proj = nn.Linear(HIDDEN_SIZE, KV_HEADS * HEAD_DIM, bias=False)
+        self.o_proj = nn.Linear(HIDDEN_SIZE, HIDDEN_SIZE, bias=False)
+
+    @staticmethod
+    def _rope(value: Tensor, positions: Tensor) -> Tensor:
+        half = HEAD_DIM // 2
+        frequencies = torch.arange(half, device=value.device, dtype=torch.float32)
+        frequencies = ROPE_THETA ** (-2 * frequencies / HEAD_DIM)
+        angles = positions.float().unsqueeze(-1) * frequencies
+        cos = angles.cos().to(dtype=value.dtype)[None, None, :, :]
+        sin = angles.sin().to(dtype=value.dtype)[None, None, :, :]
+        first, second = value[..., :half], value[..., half:]
+        return torch.cat((first * cos - second * sin, first * sin + second * cos), dim=-1)
+
+    def forward(
+        self,
+        hidden: Tensor,
+        positions: Tensor,
+        past: tuple[Tensor, Tensor] | None = None,
+        attention_mask: Tensor | None = None,
+    ) -> tuple[Tensor, tuple[Tensor, Tensor]]:
+        batch, length, _ = hidden.shape
+        query = self.q_proj(hidden).view(batch, length, QUERY_HEADS, HEAD_DIM).transpose(1, 2)
+        key = self.k_proj(hidden).view(batch, length, KV_HEADS, HEAD_DIM).transpose(1, 2)
+        value = self.v_proj(hidden).view(batch, length, KV_HEADS, HEAD_DIM).transpose(1, 2)
+        query = self._rope(query, positions)
+        key = self._rope(key, positions)
+        past_length = 0 if past is None else past[0].shape[2]
+        if past is not None:
+            key = torch.cat((past[0], key), dim=2)
+            value = torch.cat((past[1], value), dim=2)
+        total_length = key.shape[2]
+        repeats = QUERY_HEADS // KV_HEADS
+        expanded_key = key.repeat_interleave(repeats, dim=1)
+        expanded_value = value.repeat_interleave(repeats, dim=1)
+        scores = torch.matmul(query.float(), expanded_key.float().transpose(-1, -2))
+        scores = scores / math.sqrt(HEAD_DIM)
+        query_positions = torch.arange(
+            past_length, past_length + length, device=hidden.device
+        )[:, None]
+        key_positions = torch.arange(total_length, device=hidden.device)[None, :]
+        causal = key_positions <= query_positions
+        scores = scores.masked_fill(
+            ~causal[None, None, :, :], torch.finfo(torch.float32).min
+        )
+        if attention_mask is not None:
+            if attention_mask.shape != (batch, total_length):
+                raise ValueError("attention_mask must have shape (batch, complete_kv_length)")
+            scores = scores.masked_fill(
+                ~attention_mask.to(torch.bool)[:, None, None, :], torch.finfo(torch.float32).min
+            )
+        probabilities = torch.softmax(scores, dim=-1, dtype=torch.float32)
+        attended = torch.matmul(probabilities, expanded_value.float()).to(hidden.dtype)
+        attended = attended.transpose(1, 2).contiguous().view(batch, length, HIDDEN_SIZE)
+        return self.o_proj(attended), (key, value)
+
+
+class PhysicalBlock(nn.Module):
+    def __init__(self) -> None:
+        super().__init__()
+        self.attention = Attention()
+        self.mlp_gate = nn.Linear(HIDDEN_SIZE, INTERMEDIATE_SIZE, bias=False)
+        self.mlp_up = nn.Linear(HIDDEN_SIZE, INTERMEDIATE_SIZE, bias=False)
+        self.mlp_down = nn.Linear(INTERMEDIATE_SIZE, HIDDEN_SIZE, bias=False)
+
+
+class FleckModel(nn.Module):
+    def __init__(self) -> None:
+        super().__init__()
+        self.token_embedding = nn.Parameter(torch.empty(VOCAB_SIZE, EMBEDDING_RANK))
+        self.embedding_projection = nn.Parameter(torch.empty(EMBEDDING_RANK, HIDDEN_SIZE))
+        self.blocks = nn.ModuleList(PhysicalBlock() for _ in range(PHYSICAL_BLOCKS))
+        self.attention_norms = nn.ModuleList(
+            RMSNorm(HIDDEN_SIZE) for _ in range(EFFECTIVE_DEPTH)
+        )
+        self.mlp_norms = nn.ModuleList(RMSNorm(HIDDEN_SIZE) for _ in range(EFFECTIVE_DEPTH))
+        self.depth_embeddings = nn.Parameter(torch.empty(EFFECTIVE_DEPTH, HIDDEN_SIZE))
+        self.attention_residual_scales = nn.Parameter(torch.empty(EFFECTIVE_DEPTH))
+        self.mlp_residual_scales = nn.Parameter(torch.empty(EFFECTIVE_DEPTH))
+        self.final_norm = RMSNorm(HIDDEN_SIZE)
+
+    def _embed(self, input_ids: Tensor) -> Tensor:
+        return torch.nn.functional.embedding(input_ids, self.token_embedding) @ self.embedding_projection
+
+    def _logits(self, hidden: Tensor) -> Tensor:
+        rank_hidden = hidden.float() @ self.embedding_projection.float().t()
+        return rank_hidden @ self.token_embedding.float().t()
+
+    def forward(
+        self,
+        input_ids: Tensor,
+        *,
+        past_key_values: tuple[tuple[Tensor, Tensor], ...] | None = None,
+        attention_mask: Tensor | None = None,
+        use_cache: bool = False,
+    ) -> tuple[Tensor, tuple[tuple[Tensor, Tensor], ...] | None]:
+        if input_ids.ndim != 2 or input_ids.shape[1] == 0:
+            raise ValueError("input_ids must have shape (batch, non-empty sequence)")
+        batch, length = input_ids.shape
+        if past_key_values is not None and len(past_key_values) != EFFECTIVE_DEPTH:
+            raise ValueError("past_key_values must contain one cache per effective depth")
+        past_length = 0 if past_key_values is None else past_key_values[0][0].shape[2]
+        if past_length + length > CONTEXT_LENGTH:
+            raise ValueError(f"sequence exceeds context_length={CONTEXT_LENGTH}")
+        positions = torch.arange(past_length, past_length + length, device=input_ids.device)
+        hidden = self._embed(input_ids)
+        caches: list[tuple[Tensor, Tensor]] = []
+        for depth, physical_index in enumerate(EXECUTION_ORDER):
+            hidden = hidden + self.depth_embeddings[depth].view(1, 1, -1)
+            block = cast(PhysicalBlock, self.blocks[physical_index])
+            attention_input = self.attention_norms[depth](hidden)
+            past = None if past_key_values is None else past_key_values[depth]
+            attended, cache = block.attention(attention_input, positions, past, attention_mask)
+            hidden = hidden + self.attention_residual_scales[depth] * attended
+            mlp_input = self.mlp_norms[depth](hidden)
+            mlp_output = block.mlp_down(
+                torch.nn.functional.silu(block.mlp_gate(mlp_input)) * block.mlp_up(mlp_input)
+            )
+            hidden = hidden + self.mlp_residual_scales[depth] * mlp_output
+            caches.append(cache)
+        logits = self._logits(self.final_norm(hidden))
+        return logits, tuple(caches) if use_cache else None
+
+
+def _expected_shapes(model: nn.Module) -> dict[str, tuple[int, ...]]:
+    return {name: tuple(parameter.shape) for name, parameter in model.named_parameters()}
+
+
+def load_model(checkpoint: str | Path, device: str = "cpu") -> FleckModel:
+    """Load the public BF16 artifact strictly, then run it as FP32."""
+    if device not in {"cpu", "mps"}:
+        raise ValueError("device must be 'cpu' or 'mps'")
+    if device == "mps" and not torch.backends.mps.is_available():
+        raise RuntimeError("MPS was requested but is not available")
+    source = Path(checkpoint)
+    if not source.is_file():
+        raise FileNotFoundError(source)
+    model = FleckModel()
+    expected = _expected_shapes(model)
+    with safe_open(str(source), framework="pt", device="cpu") as handle:
+        metadata = handle.metadata() or {}
+        for key, expected_value in MODEL_METADATA.items():
+            if metadata.get(key) != expected_value:
+                raise ValueError(
+                    f"SafeTensors metadata mismatch for {key}: "
+                    f"expected {expected_value!r}, got {metadata.get(key)!r}"
+                )
+        names = set(handle.keys())
+        if names != set(expected):
+            raise ValueError(
+                "SafeTensors parameter names mismatch: "
+                f"missing={sorted(set(expected) - names)}, extra={sorted(names - set(expected))}"
+            )
+        for name, shape in expected.items():
+            tensor_slice = handle.get_slice(name)
+            if tensor_slice.get_dtype() != "BF16":
+                raise ValueError(f"{name}: expected BF16, got {tensor_slice.get_dtype()}")
+            if tuple(tensor_slice.get_shape()) != shape:
+                raise ValueError(
+                    f"{name}: expected shape {shape}, got {tuple(tensor_slice.get_shape())}"
+                )
+    tensors = load_file(str(source), device="cpu")
+    for name, shape in expected.items():
+        tensor = tensors[name]
+        if tensor.dtype != torch.bfloat16 or tuple(tensor.shape) != shape:
+            raise ValueError(f"{name}: SafeTensors dtype/shape contract mismatch")
+    model.load_state_dict(tensors, strict=True, assign=True)
+    model = model.to(device=torch.device(device), dtype=torch.bfloat16).eval()
+    if any(parameter.dtype != torch.bfloat16 for parameter in model.parameters()):
+        raise TypeError("model parameters must be BF16 after loading")
+    return model
+
+
+def load_tokenizer(path: str | Path) -> Tokenizer:
+    source = Path(path)
+    tokenizer = Tokenizer.from_file(str(source))
+    if tokenizer.get_vocab_size() != VOCAB_SIZE:
+        raise ValueError(f"tokenizer vocabulary must be {VOCAB_SIZE}")
+    expected = {
+        "<bos>": BOS_ID,
+        "<eos>": EOS_ID,
+        "<pad>": PAD_ID,
+        "<unk>": UNK_ID,
+        "<|system|>": SYSTEM_ID,
+        "<|user|>": USER_ID,
+        "<|assistant|>": ASSISTANT_ID,
+        "<|eot|>": EOT_ID,
+    }
+    if set(tokenizer.get_added_tokens_decoder()) != set(expected.values()):
+        raise ValueError("tokenizer added-token set does not match the strict contract")
+    for token, token_id in expected.items():
+        added_token = tokenizer.get_added_tokens_decoder().get(token_id)
+        if (
+            tokenizer.token_to_id(token) != token_id
+            or added_token is None
+            or getattr(added_token, "content", None) != token
+            or not bool(getattr(added_token, "special", False))
+        ):
+            raise ValueError(f"tokenizer special token contract mismatch for {token}")
+    return tokenizer
+
+
+@torch.inference_mode()
+def generate(
+    model: FleckModel,
+    input_ids: Tensor,
+    max_tokens: int,
+    stop_token_ids: Iterable[int] = (EOS_ID,),
+) -> Tensor:
+    if max_tokens < 0:
+        raise ValueError("max_tokens must be non-negative")
+    if input_ids.shape[1] + max_tokens > CONTEXT_LENGTH:
+        raise ValueError(f"prompt plus generation exceeds context_length={CONTEXT_LENGTH}")
+    stop_ids = set(stop_token_ids)
+    generated = input_ids
+    cache: tuple[tuple[Tensor, Tensor], ...] | None = None
+    for _ in range(max_tokens):
+        current = generated if cache is None else generated[:, -1:]
+        logits, cache = model(current, past_key_values=cache, use_cache=True)
+        next_token = logits[:, -1, :].argmax(dim=-1, keepdim=True)
+        generated = torch.cat((generated, next_token), dim=1)
+        if bool(torch.all(torch.isin(next_token, torch.tensor(list(stop_ids), device=next_token.device)))):
+            break
+    return generated
+
+
+def _default_path(name: str) -> str:
+    return str(Path(__file__).with_name(name))
+
+
+def chat_ids(tokenizer: Tokenizer, user_text: str) -> list[int]:
+    """Encode one training-format user turn and open the assistant turn."""
+    return [BOS_ID, USER_ID, *tokenizer.encode(user_text, add_special_tokens=False).ids, EOT_ID, ASSISTANT_ID]
+
+
+def decode_reply(tokenizer: Tokenizer, token_ids: list[int]) -> str:
+    """Stop at EOT/EOS and decode only ordinary text."""
+    for index, token_id in enumerate(token_ids):
+        if token_id in {EOT_ID, EOS_ID}:
+            token_ids = token_ids[:index]
+            break
+    return tokenizer.decode(token_ids, skip_special_tokens=True).strip()
+
+
+def run_single(args: argparse.Namespace, tokenizer: Tokenizer, model: FleckModel) -> None:
+    prompt = args.prompt if args.prompt is not None else "Hello"
+    ids = chat_ids(tokenizer, prompt)
+    input_ids = torch.tensor([ids], dtype=torch.long, device=args.device)
+    output = generate(model, input_ids, args.max_tokens, (EOT_ID, EOS_ID))
+    reply = decode_reply(tokenizer, output[0, len(ids) :].detach().cpu().tolist())
+    print(reply)
+
+
+def run_chat(args: argparse.Namespace, tokenizer: Tokenizer, model: FleckModel) -> None:
+    history: list[tuple[str, str]] = []
+    print("Fleck chat. Type /exit to quit or /clear to reset history.")
+    while True:
+        try:
+            prompt = input("user> ")
+        except (EOFError, KeyboardInterrupt):
+            print()
+            return
+        if prompt.strip() == "/exit":
+            return
+        if prompt.strip() == "/clear":
+            history.clear()
+            print("(history cleared)")
+            continue
+        if not prompt.strip():
+            continue
+        ids = [BOS_ID]
+        for old_user, old_reply in history:
+            ids.extend([USER_ID, *tokenizer.encode(old_user, add_special_tokens=False).ids, EOT_ID, ASSISTANT_ID])
+            ids.extend([*tokenizer.encode(old_reply, add_special_tokens=False).ids, EOT_ID])
+        ids.extend([USER_ID, *tokenizer.encode(prompt, add_special_tokens=False).ids, EOT_ID, ASSISTANT_ID])
+        if len(ids) + args.max_tokens > CONTEXT_LENGTH:
+            history.clear()
+            ids = chat_ids(tokenizer, prompt)
+        input_ids = torch.tensor([ids], dtype=torch.long, device=args.device)
+        output = generate(model, input_ids, args.max_tokens, (EOT_ID, EOS_ID))
+        reply = decode_reply(tokenizer, output[0, len(ids) :].detach().cpu().tolist())
+        print(f"assistant> {reply}")
+        history.append((prompt, reply))
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--ckpt", default=_default_path("model.safetensors"))
+    parser.add_argument("--tokenizer", default=_default_path("tokenizer.json"))
+    parser.add_argument("--prompt", default=None, help="single prompt; omit for interactive chat")
+    parser.add_argument("--max-tokens", type=int, default=32)
+    parser.add_argument("--device", choices=("cpu", "mps"), default="cpu")
+    parser.add_argument("--no-chat", action="store_true", help="run one prompt instead of interactive chat")
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    tokenizer = load_tokenizer(args.tokenizer)
+    model = load_model(args.ckpt, args.device)
+    if args.no_chat or args.prompt is not None:
+        run_single(args, tokenizer, model)
+    else:
+        run_chat(args, tokenizer, model)
+
+
+if __name__ == "__main__":
+    main()

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cbdae9ca01691fb19ca51871c11727b9f2889eb33e710015dc3273fb7ddd86ab
+size 997440

tokenizer.json

@@ -0,0 +1 @@
+{"version":"1.0","truncation":null,"padding":null,"added_tokens":[{"id":0,"content":"<bos>","single_word":false,"lstrip":false,"rstrip":false,"normalized":false,"special":true},{"id":1,"content":"<eos>","single_word":false,"lstrip":false,"rstrip":false,"normalized":false,"special":true},{"id":2,"content":"<pad>","single_word":false,"lstrip":false,"rstrip":false,"normalized":false,"special":true},{"id":3,"content":"<unk>","single_word":false,"lstrip":false,"rstrip":false,"normalized":false,"special":true},{"id":4,"content":"<|system|>","single_word":false,"lstrip":false,"rstrip":false,"normalized":false,"special":true},{"id":5,"content":"<|user|>","single_word":false,"lstrip":false,"rstrip":false,"normalized":false,"special":true},{"id":6,"content":"<|assistant|>","single_word":false,"lstrip":false,"rstrip":false,"normalized":false,"special":true},{"id":7,"content":"<|eot|>","single_word":false,"lstrip":false,"rstrip":false,"normalized":false,"special":true}],"normalizer":null,"pre_tokenizer":{"type":"ByteLevel","add_prefix_space":false,"trim_offsets":true,"use_regex":true},"post_processor":null,"decoder":{"type":"ByteLevel","add_prefix_space":true,"trim_offsets":true,"use_regex":true},"model":{"type":"BPE","dropout":null,"unk_token":"<unk>","continuing_subword_prefix":null,"end_of_word_suffix":null,"fuse_unk":false,"byte_fallback":false,"ignore_merges":false,"vocab":{"<bos>":0,"<eos>":1,"<pad>":2,"<unk>":3,"<|system|>":4,"<|user|>":5,"<|assistant|>":6,"<|eot|>":7,"!":8,"\"":9,"#":10,"$":11,"%":12,"&":13,"'":14,"(":15,")":16,"*":17,"+":18,",":19,"-":20,".":21,"/":22,"0":23,"1":24,"2":25,"3":26,"4":27,"5":28,"6":29,"7":30,"8":31,"9":32,":":33,";":34,"<":35,"=":36,">":37,"?":38,"@":39,"A":40,"B":41,"C":42,"D":43,"E":44,"F":45,"G":46,"H":47,"I":48,"J":49,"K":50,"L":51,"M":52,"N":53,"O":54,"P":55,"Q":56,"R":57,"S":58,"T":59,"U":60,"V":61,"W":62,"X":63,"Y":64,"Z":65,"[":66,"\\":67,"]":68,"^":69,"_":70,"`":71,"a":72,"b":73,"c":74,"d":75,"e":76,"f":77,"g":78,"h":79,"i":80,"j":81,"k":82,"l":83,"m":84,"n":85,"o":86,"p":87,"q":88,"r":89,"s":90,"t":91,"u":92,"v":93,"w":94,"x":95,"y":96,"z":97,"{":98,"|":99,"}":100,"~":101,"¡":102,"¢":103,"£":104,"¤":105,"¥":106,"¦":107,"§":108,"¨":109,"©":110,"ª":111,"«":112,"¬":113,"®":114,"¯":115,"°":116,"±":117,"²":118,"³":119,"´":120,"µ":121,"¶":122,"·":123,"¸":124,"¹":125,"º":126,"»":127,"¼":128,"½":129,"¾":130,"¿":131,"À":132,"Á":133,"Â":134,"Ã":135,"Ä":136,"Å":137,"Æ":138,"Ç":139,"È":140,"É":141,"Ê":142,"Ë":143,"Ì":144,"Í":145,"Î":146,"Ï":147,"Ð":148,"Ñ":149,"Ò":150,"Ó":151,"Ô":152,"Õ":153,"Ö":154,"×":155,"Ø":156,"Ù":157,"Ú":158,"Û":159,"Ü":160,"Ý":161,"Þ":162,"ß":163,"à":164,"á":165,"â":166,"ã":167,"ä":168,"å":169,"æ":170,"ç":171,"è":172,"é":173,"ê":174,"ë":175,"ì":176,"í":177,"î":178,"ï":179,"ð":180,"ñ":181,"ò":182,"ó":183,"ô":184,"õ":185,"ö":186,"÷":187,"ø":188,"ù":189,"ú":190,"û":191,"ü":192,"ý":193,"þ":194,"ÿ":195,"Ā":196,"ā":197,"Ă":198,"ă":199,"Ą":200,"ą":201,"Ć":202,"ć":203,"Ĉ":204,"ĉ":205,"Ċ":206,"ċ":207,"Č":208,"č":209,"Ď":210,"ď":211,"Đ":212,"đ":213,"Ē":214,"ē":215,"Ĕ":216,"ĕ":217,"Ė":218,"ė":219,"Ę":220,"ę":221,"Ě":222,"ě":223,"Ĝ":224,"ĝ":225,"Ğ":226,"ğ":227,"Ġ":228,"ġ":229,"Ģ":230,"ģ":231,"Ĥ":232,"ĥ":233,"Ħ":234,"ħ":235,"Ĩ":236,"ĩ":237,"Ī":238,"ī":239,"Ĭ":240,"ĭ":241,"Į":242,"į":243,"İ":244,"ı":245,"IJ":246,"ij":247,"Ĵ":248,"ĵ":249,"Ķ":250,"ķ":251,"ĸ":252,"Ĺ":253,"ĺ":254,"Ļ":255,"ļ":256,"Ľ":257,"ľ":258,"Ŀ":259,"ŀ":260,"Ł":261,"ł":262,"Ń":263,"Ġt":264,"Ġa":265,"he":266,"in":267,"re":268,"on":269,"Ġthe":270,"er":271,"Ġs":272,"Ġo":273,"at":274,"en":275,"Ġw":276,"Ġc":277,"es":278,"nd":279,"is":280,"it":281,"or":282,"Ġp":283,"Ġb":284,"Ġf":285,"al":286,"ed":287,"an":288,"ing":289,"Ġof":290,"ou":291,"Ġin":292,"ar":293,"Ġm":294,"Ġto":295,"Ġand":296,"ic":297,"Ġd":298,"ion":299,"le":300,"ro":301,"as":302,"Ġh":303,"Ġth":304,"ent":305,"il":306,"Ġe":307,"Ġre":308,"ct":309,"om":310,"Ġl":311,"ve":312,"st":313,"Ġn":314,"âĢ":315,"ly":316,"Ġbe":317,"Ġis":318,"Ġg":319,"ĠT":320,"se":321,"ol":322,"id":323,"et":324,"ut":325,"ot":326,"im":327,"ation":328,"ur":329,"ce":330,"ow":331,"Ġfor":332,"Ġy":333,"ĠS":334,"ra":335,"ĠA":336,"ch":337,"Ġon":338,"ĠI":339,"ig":340,"Ġthat":341,"Ġu":342,"ĠC":343,"ver":344,"Ġst":345,"ay":346,"ir":347,"am":348,"Ġyou":349,"ith":350,"ad":351,"el":352,"ul":353,"Ġas":354,"Ġwith":355,"her":356,"Ġpro":357,"Ġan":358,"Ġit":359,"Ġare":360,"Ġwh":361,"ĠM":362,"Ġcon":363,"Ġal":364,"if":365,"ter":366,"Ġor":367,"ĠP":368,"od":369,"âĢĻ":370,"ĠThe":371,"ge":372,"ill":373,"ĠB":374,"Ġ(":375,"ers":376,"ment":377,"Ġ1":378,"Ġhe":379,"th":380,"ate":381,"ess":382,"Ġwe":383,"op":384,"ist":385,"Ġha":386,"Ġde":387,"us":388,"Ġcom":389,"ore":390,"Ġex":391,"ri":392,"est":393,"ab":394,"ke":395,"ĠH":396,"ac":397,"ĠW":398,"ld":399,"Ġsu":400,"ect":401,"ity":402,"Ġat":403,"rom":404,"res":405,"pp":406,"Ġv":407,"ĠD":408,"ive":409,"ant":410,"um":411,"ies":412,"igh":413,"qu":414,"os":415,"Ġwas":416,"em":417,"Ġby":418,"nt":419,"ĠR":420,"ain":421,"ĠF":422,"Ġne":423,"ĠE":424,"Ġfrom":425,"Ġ2":426,"ort":427,"oc":428,"Ġse":429,"Ġch":430,"and":431,"ud":432,"Ġcan":433,"Ġr":434,"ĠâĢ":435,"un":436,"Ġnot":437,"iv":438,"ĠG":439,"Ġhave":440,"ĠL":441,"art":442,"ĠN":443,"Ġle":444,"Ġsh":445,"ure":446,"Ġthis":447,"ial":448,"ight":449,"ust":450,"ical":451,"og":452,"Ġsp":453,"Ġen":454,"all":455,"Ġwor":456,"Ġab":457,"ould":458,"ome":459,"out":460,"our":461,"00":462,"red":463,"gh":464,"ost":465,"rou":466,"pt":467,"ard":468,"ich":469,"Ġyour":470,"The":471,"Ġpl":472,"ĠO":473,"ine":474,"ions":475,"per":476,"ag":477,"ide":478,"ther":479,"ast":480,"ell":481,"ear":482,"Ġus":483,"ĠJ":484,"Ġwill":485,"ated":486,"ff":487,"ak":488,"Ġim":489,"Ġk":490,"Ġint":491,"Ġtheir":492,"pl":493,"Ġall":494,"age":495,"act":496,"Ġwhe":497,"end":498,"ans":499,"ie":500,"du":501,"Ġthey":502,"ĠU":503,"Ġad":504,"âĢĿ":505,"ous":506,"Ġhas":507,"Ġwhich":508,"Ġdo":509,"iz":510,"ec":511,"ood":512,"Ġcomp":513,"ong":514,"Ġcont":515,"Ġmore":516,"cc":517,"ip":518,"ap":519,"ok":520,"ĠIn":521,"ib":522,"'s":523,"ru":524,"Ġbut":525,"Ġinc":526,"ult":527,"are":528,"ia":529,"Ġme":530,"Ġj":531,"orm":532,"Ġres":533,"able":534,"Ġdis":535,"so":536,"ations":537,"ere":538,"ry":539,"ary":540,"Ġcl":541,"ally":542,"ĠTh":543,"vel":544,"ind":545,"ack":546,"Ġman":547,"ice":548,"Ġone":549,"ake":550,"Ġout":551,"Ġother":552,"ence":553,"ĠâĢľ":554,"ance":555,"ep":556,"reat":557,"Ġso":558,"one":559,"very":560,"Ġwere":561,"Ġte":562,"Ġper":563,"ose":564,"ach":565,"ime":566,"ild":567,"ign":568,"ass":569,"ase":570,"Ġalso":571,"Ġabout":572,"Ġhis":573,"au":574,"ber":575,"ents":576,"av":577,"ame":578,"Ġup":579,"ĠSt":580,"Ġapp":581,"ater":582,"ite":583,"Ġwho":584,"ress":585,"ook":586,"Ġimp":587,"Ġun":588,"ike":589,"ire":590,"Ġar":591,"ition":592,"ace":593,"ction":594,"ord":595,"ks":596,"Ġev":597,"Ġpart":598,"ount":599,"ew":600,"ree":601,"ĠK":602,"ub":603,"Ġ19":604,"ish":605,"ue":606,"ail":607,"Ġpe":608,"ens":609,"ory":610,"Ġthem":611,"Ġ\"":612,"Ġqu":613,"ays":614,"now":615,"Ġsc":616,"Ġgo":617,"form":618,"ĠIt":619,"ile":620,"Ġyear":621,"ors":622,"ound":623,"Ġ20":624,"Ġsome":625,"ces":626,"Ġcomm":627,"ll":628,"Ġour":629,"ang":630,"ĠCh":631,"Ġbec":632,"Ġbeen":633,"low":634,"Ġpre":635,"Ġthan":636,"Ġstud":637,"Ġthere":638,"ings":639,").":640,"Ġtime":641,"Ġwhen":642,"Ġrec":643,"Ġwork":644,"Ġag":645,"Ġsa":646,"ric":647,"ough":648,"erv":649,"ople":650,"ĠV":651,"Ġover":652,"Ġsy":653,"mer":654,"ĠY":655,"Ġhad":656,"Ġtra":657,"Ġmay":658,"iff":659,"ond":660,"Ġ3":661,"Ġinto":662,"Ġlike":663,"Ġits":664,"ove":665,"ian":666,"ates":667,"rit":668,"nder":669,"int":670,"Ġro":671,"olog":672,"vers":673,"Ġnew":674,"Ġwould":675,"Ġhow":676,"Ġoff":677,"Ġany":678,"rough":679,"irst":680,"Ġpeople":681,"ual":682,"ons":683,"te":684,"ph":685,"Ġknow":686,"lud":687,"Ġind":688,"Ġif":689,"lic":690,"ause":691,"ĠThis":692,"tern":693,"ren":694,"Ġcol":695,"Ġthese":696,"Ġwhat":697,"Ġacc":698,"Ġtw":699,"Ġra":700,"wn":701,"Ġdiff":702,"Ġdes":703,"ob":704,"Ġneed":705,"Ġbet":706,"Ġcons":707,"Ġmost":708,"Ġuse":709,"Ġact":710,"ark":711,"ious":712,"ms":713,"we":714,"ning":715,"Ġbl":716,"ments":717,"Ġunder":718,"ov":719,"les":720,"ific":721,"ities":722,"Ġrep":723,"ys":724,"ject":725,"Ġfe":726,"hed":727,"Ġno":728,"lect":729,"Ġ-":730,"Ġam":731,"Ġtr":732,"Ġrel":733,"Ġmy":734,"Ġonly":735,"Ġfirst":736,"Ġhel":737,"fter":738,"ink":739,"ck":740,"Ġinclud":741,"Ġ201":742,"ool":743,"ple":744,"Ġprov":745,"Ġhelp":746,"Ġget":747,"alth":748,"Ġprodu":749,"ating":750,"Ġph":751,"velop":752,"..":753,"ational":754,"ced":755,"Ġsuch":756,"Ġthrough":757,"stem":758,"ne":759,"Ġeff":760,"own":761,"ative":762,"cess":763,"ful":764,"Ġher":765,"round":766,"ise":767,"Ġtwo":768,"ick":769,"Ġem":770,"Ġmany":771,"old":772,"ient":773,"Ġgen":774,"ular":775,"ts":776,"Ġpo":777,"ĠâĢĵ":778,"ĠHe":779,"iss":780,"ath":781,"ĠUn":782,"eth":783,"Ġadd":784,"oss":785,"oth":786,"Ġexp":787,"Ġreg":788,"Ġchild":789,"),":790,"yp":791,"hen":792,"row":793,"als":794,"rib":795,"Ġused":796,"Ġspec":797,"Ġatt":798,"Ġjust":799,"ics":800,"Ġ4":801,"Ġwell":802,"Ġac":803,"uring":804,"ied":805,"ife":806,"ade":807,"Ġdec":808,"Ġdevelop":809,"fore":810,"ert":811,"Ġdiffere":812,"ible":813,"Ġmake":814,"erm":815,"Ġshould":816,"ange":817,"ased":818,"ility":819,"sel":820,"Ġsupp":821,"ram":822,"Ġass":823,"rop":824,"Ġco":825,"Ġsaid":826,"Ġyears":827,"Ġ5":828,"Ġrem":829,"arch":830,"Ġinv":831,"Ġcould":832,"Ġfl":833,"ures":834,"Ġbu":835,"ble":836,"ng":837,"ft":838,"iet":839,"Ġpr":840,"Ġsystem":841,"ract":842,"oy":843,"aw":844,"xt":845,"Ġob":846,"ower":847,"Ġ200":848,"Ġsm":849,"ily":850,"uch":851,"Ġsur":852,"Ġsub":853,"Ġwhere":854,"ting":855,"ss":856,"In":857,"gan":858,"Ġpers":859,"hes":860,"ten":861,"Ġhigh":862,"ĠWe":863,"Ġpres":864,"ont":865,"ween":866,"ident":867,"its":868,"Ġinf":869,"Ġafter":870,"cy":871,"Ġvery":872,"ited":873,"iew":874,"Ġsee":875,"ution":876,"Ġevery":877,"hat":878,"ience":879,"Ġshe":880,"chool":881,"Ġthen":882,"Ġeven":883,"Ġbecause":884,"urn":885,"Ġway":886,"Ġbetween":887,"Ġinst":888,"Ġtrans":889,"Ġpol":890,"ever":891,"ict":892,"meric":893,"Ġexper":894,"ants":895,"Ġwater":896,"Ġdisc":897,"Ġear":898,"formation":899,"Ġlong":900,"aking":901,"Ġke":902,"chn":903,"Ġmod":904,"Ġcar":905,"Ġback":906,"uc":907,"land":908,"ually":909,"other":910,"Ġnum":911,"ins":912,"get":913,"Ġworld":914,"Ġhealth":915,"Ġinter":916,"Ġform":917,"ĠRe":918,"ines":919,"Ġsim":920,"tle":921,"Ġimport":922,"Ġsec":923,"ology":924,".âĢĿ":925,"Ġfound":926,"Ġrese":927,"ness":928,"Ġdifferent":929,"Ġthose":930,"Ġlook":931,"ivers":932,"cept":933,"Ġcount":934,"ural":935,"ollow":936,"ists":937,"Ġbeing":938,"Ġmed":939,"ock":940,"ward":941,"ork":942,"Ġgreat":943,"omm":944,"Ġcall":945,"Ġown":946,"Ġright":947,"Ġgood":948,"Ġset":949,"Ġmuch":950,"led":951,"Th":952,"ata":953,"Ġent":954,"Ġexam":955,"Ġresp":956,"Ġed":957,"ism":958,"ĠAmeric":959,"Ġprog":960,"Ġlar":961,"Ġref":962,"Ġdet":963,"any":964,"ature":965,"Ġpub":966,"Ġeffect":967,"Ġlear":968,"Ġread":969,"ues":970,"Ġcent":971,"ices":972,"ef":973,"Ġ18":974,"Ġext":975,"hip":976,"Ġgra":977,"Ġsign":978,"Ġ6":979,"Ġend":980,"ically":981,"ps":982,"Ġserv":983,"ved":984,"Ġmon":985,"Ġrequ":986,"Ġlife":987,"gy":988,"ives":989,"Ġmade":990,"Ġeach":991,"day":992,"Ġfam":993,"Ġdoes":994,"Ġdef":995,"Ġart":996,"io":997,"Ġgener":998,"Ġloc":999,"Ġfind":1000,"ĠPro":1001,"vent":1002,"ages":1003,"ather":1004,"ĠAl":1005,"arm":1006,"Ġhim":1007,"Ġinformation":1008,"Ġnow":1009,"Ġincre":1010,"Ġtyp":1011,"und":1012,"Ġfact":1013,"Ġcre":1014,"Ġperson":1015,"Ġvar":1016,"Ġprocess":1017,"Ġcommun":1018,"imes":1019,"cer":1020,"Ġagain":1021,"Ġfollow":1022,"Ġins":1023,"ody":1024,"Ġshow":1025,"Ġlead":1026,"Ġbefore":1027,"ty":1028,"Ġmin":1029,"Ġpat":1030,"ĠIf":1031,"Ġsoc":1032,"Ġresult":1033,"Ġhum":1034,"ven":1035,"||":1036,"gg":1037,"vern":1038,"Ġche":1039,"ĠAs":1040,"Ġdown":1041,"Ġsame":1042,"Ġday":1043,"ale":1044,"Ġpartic":1045,"Ġaut":1046,"Ġsk":1047,"Ġtechn":1048,"ized":1049,"Ġav":1050,"ted":1051,"Ġtake":1052,"Ġduring":1053,"Ġdist":1054,"Ġdid":1055,"ruct":1056,"Ġdep":1057,"ix":1058,"Ġca":1059,"ases":1060,"20":1061,"'t":1062,"Ġcur":1063,"ize":1064,"ince":1065,"ĠAn":1066,"cent":1067,"the":1068,"Ġstart":1069,"ĠThey":1070,"Ġbr":1071,"Ġimportant":1072,"Ġlit":1073,"ds":1074,"ving":1075,"Ġchildren":1076,"Ġboth":1077,"ĠNew":1078,"air":1079,"ĠYou":1080,"orn":1081,"ection":1082,"ĠAr":1083,"ality":1084,"read":1085,"der":1086,"ax":1087,"Ġcomple":1088,"up":1089,"Ġstr":1090,"Ġwant":1091,"inal":1092,"ĠFor":1093,"ex":1094,"ĠSp":1095,"Ġwhile":1096,"Ġschool":1097,"nce":1098,"iel":1099,"iron":1100,"Ġproble":1101,"akes":1102,"com":1103,"con":1104,"Ġ10":1105,"Ġplay":1106,"Ġmem":1107,"Ġhand":1108,"Ġlevel":1109,"ars":1110,"Ġgu":1111,"Ġgl":1112,"Ġstudents":1113,"000":1114,"Ġstate":1115,"ier":1116,"irect":1117,"ĠAnd":1118,"Ġthink":1119,"ash":1120,"ull":1121,"Ġ7":1122,"Ġdata":1123,"ee":1124,"Ġgrou":1125,"Ġfood":1126,"Ġvis":1127,"Ġchar":1128,"ined":1129,"Ġsmall":1130,"Ġusing":1131,"ute":1132,"self":1133,"Ġaround":1134,"oh":1135,"iness":1136,"19":1137,"Ġdesign":1138,"Ġprovid":1139,"ĠHow":1140,"ially":1141,"Ġexpl":1142,"ave":1143,"ah":1144,"ner":1145,"Ġop":1146,"Ġpoint":1147,"Ġmain":1148,"Ġthree":1149,"Ġval":1150,"Ġposs":1151,"ished":1152,"ered":1153,"Ġhist":1154,"âĢľ":1155,"iversity":1156,"ĠBut":1157,"ves":1158,"ters":1159,"stand":1160,"way":1161,"Ġsupport":1162,",âĢĿ":1163,"Ġsol":1164,"Ġconf":1165,"ank":1166,"Ġdon":1167,"Ġpop":1168,"ouse":1169,"Ġwrit":1170,"Ġser":1171,"Ġbel":1172,"omen":1173,"Ġstudy":1174,"Ġ8":1175,".,":1176,"Ġfun":1177,"line":1178,"Ġcle":1179,"Ġgrow":1180,"Ġless":1181,"ivid":1182,"Ġcour":1183,"Ġthough":1184,"ected":1185,"Ġnumber":1186,"Ġorgan":1187,"ien":1188,"Ġtest":1189,"Ġsom":1190,"rol":1191,"This":1192,"Ġret":1193,"ĠInd":1194,"anc":1195,"ason":1196,"ral":1197,"Ġ&":1198,".\"":1199,"Ġbo":1200,"Ġcare":1201,"ĠSc":1202,"ital":1203,"Ġclass":1204,"Ġprot":1205,"Ġdi":1206,"iqu":1207,"Ġstill":1208,"eg":1209,"Ġreport":1210,"Ġmust":1211,"uss":1212,"rent":1213,"Ġpower":1214,"ional":1215,"ron":1216,"Ġbest":1217,"Ġoften":1218,"cl":1219,"alk":1220,"ross":1221,"Ġresearch":1222,"dition":1223,"Ġplan":1224,"viron":1225,"Ġplace":1226,"Ġadv":1227,"Ġ9":1228,"Ġproduct":1229,"Ġhere":1230,"Ġbook":1231,"ger":1232,"ury":1233,"Ġmark":1234,"aj":1235,"Ġbus":1236,"Ġhome":1237,"Ġcontin":1238,"Ġtreat":1239,"man":1240,"Ġeas":1241,"Ġocc":1242,"Ġfew":1243,"imate":1244,"ired":1245,"iety":1246,"ĠThere":1247,"Ġquest":1248,"Ġbody":1249,"âĢĶ":1250,"ĠWh":1251,"Ġcor":1252,"Ġgovern":1253,"ared":1254,"Ġlast":1255,"Ġest":1256,"Ġallow":1257,"els":1258,"oun":1259,"erest":1260,"Ġincluding":1261,"ains":1262,"ium":1263,"bs":1264,"Ġactiv":1265,"ĠâĢĺ":1266,"Ġpur":1267,"Ġmat":1268,"ets":1269,"Ġbuild":1270,"10":1271,"ton":1272,"Ġpublic":1273,"ĠDe":1274,"az":1275,"Ġide":1276,"ĠSh":1277,"bers":1278,"less":1279,"amp":1280,"Ġlarge":1281,"Ġaff":1282,"Ġmight":1283,"Ġris":1284,"vironment":1285,"outh":1286,"ering":1287,"Ġprof":1288,"de":1289,"itions":1290,"ulation":1291,"Ġsl":1292,"Ġhuman":1293,"ense":1294,"Ġconst":1295,"iving":1296,"Ġwithout":1297,"ug":1298,"Ġiss":1299,"hers":1300,"Ġland":1301,"Ġwee":1302,"Ġstand":1303,"reen":1304,"view":1305,"Ġess":1306,"esp":1307,"ĠIs":1308,"It":1309,"Ġequ":1310,"Ġelect":1311,"Ġfin":1312,"Ġdise":1313,"lish":1314,"ries":1315,"ways":1316,"ertain":1317,"ining":1318,"Ġexample":1319,"Ġcommon":1320,"Ġmet":1321,"ris":1322,"Ġappro":1323,"me":1324,"ption":1325,"Ġsay":1326,"osed":1327,"Ġredu":1328,"Ġanother":1329,"Ġpass":1330,"Ġunderstand":1331,"Ġcell":1332,"acter":1333,"Ġdirect":1334,"ration":1335,"Ġ$":1336,"conom":1337,"Ġproject":1338,"tt":1339,"Ġbeh":1340,"Ġtop":1341,"ĠCl":1342,"Ġcalled":1343,"raph":1344,"inc":1345,"Ġtoo":1346,"eral":1347,"Ġpract":1348,"Ġwithin":1349,"Ġpresent":1350,"ĠAd":1351,"Ġlight":1352,"Ġsit":1353,"illion":1354,"Ġrest":1355,"We":1356,"oci":1357,"ik":1358,"Ġinterest":1359,"Ġdem":1360,"Ġreal":1361,"ior":1362,"Ġmus":1363,"Ġchang":1364,"eng":1365,"ization":1366,"ording":1367,"aterial":1368,"ividual":1369,"Ġ199":1370,"Ġeduc":1371,"ids":1372,"Ġpos":1373,"Ġbra":1374,"Ġhapp":1375,"Ġkeep":1376,"Ġbelie":1377,"Ġopp":1378,"ergy":1379,"itive":1380,"Ġbre":1381,"ĠEng":1382,"Ġbetter":1383,"Ġknown":1384,"Ġavail":1385,"Ġchange":1386,"Ġeng":1387,"Ġcap":1388,"Ġsecond":1389,"Ġpri":1390,"ination":1391,"...":1392,"ential":1393,"rist":1394,"gen":1395,"orth":1396,"ield":1397,"ccess":1398,"Ġold":1399,"âĢĵ":1400,"Ġthings":1401,"iod":1402,"ĠCon":1403,"iven":1404,"Ġagainst":1405,"Ġprogram":1406,"Ġarea":1407,"Ġopen":1408,"ision":1409,"Ġfeel":1410,"to":1411,"arent":1412,"sc":1413,"Ġfamily":1414,"Ġfree":1415,"Ġlearn":1416,"ĠAmerican":1417,"ames":1418,"Ġve":1419,"ently":1420,"ĠPh":1421,"Ġleg":1422,"ĠUniversity":1423,"oms":1424,"Ġlaw":1425,"Ġben":1426,"Ġimm":1427,"ources":1428,"Ġair":1429,"Ġbro":1430,"Ġorder":1431,"ified":1432,"Ġbas":1433,"Ġrun":1434,"Ġimpro":1435,"ware":1436,"Ġindividual":1437,"ĠâĢĶ":1438,"Ġbusiness":1439,"Ġlist":1440,"ead":1441,"rodu":1442,"ajor":1443,"here":1444,"hor":1445,"Ġsince":1446,"Ġpost":1447,"Ġproper":1448,"Ġwomen":1449,"ĠThese":1450,"gin":1451,"ats":1452,"Ġsocial":1453,"Ġtem":1454,"ability":1455,"Ġperform":1456,"Ġdevelopment":1457,"ians":1458,"ards":1459,"Ġdel":1460,"ify":1461,"eed":1462,"Ġmaterial":1463,"Ġmonth":1464,"ĠCol":1465,"ention":1466,"ern":1467,"Ġoper":1468,"Ġhead":1469,"ĠEx":1470,"ission":1471,"Ġlittle":1472,"Ġcome":1473,"Ġanal":1474,"Ġpot":1475,"Ġunt":1476,"ends":1477,"ling":1478,"Ġwar":1479,"ained":1480,",\"":1481,"Ġ.":1482,"Ġavailable":1483,"ething":1484,"Ġprovide":1485,"Ġput":1486,"iver":1487,"Ġinclude":1488,"Ġable":1489,"ĠPl":1490,"ills":1491,"ories":1492,"Ġident":1493,"ients":1494,"angu":1495,"ider":1496,"Ġbecome":1497,"Ġcaus":1498,"Ġsever":1499,"ethod":1500,"erc":1501,"ĠQ":1502,"Ġdesc":1503,"Ġgovernment":1504,"Ġenvironment":1505,"Ġeconom":1506,"Ġpar":1507,"Ġpain":1508,"Ġcurrent":1509,"Ġmove":1510,"icle":1511,"omp":1512,"Ġel":1513,"ĠSe":1514,"Ġgoing":1515,"Ġcontrol":1516,"Ġ15":1517,"Ġenergy":1518,"ived":1519,"Ġ[":1520,"__":1521,"Ġter":1522,"atic":1523,"Ġgroup":1524,"ote":1525,"sp":1526,"Ġphys":1527,"Ġmil":1528,"Ġcost":1529,"ploy":1530,"Ġant":1531,"Ġsays":1532,"ĠZ":1533,"Ġrece":1534,"Ġlot":1535,"Ġele":1536,"Ġteam":1537,"Ġearly":1538,"Ġdev":1539,"Ġlow":1540,"agn":1541,"Ġcult":1542,"Ġnear":1543,"use":1544,"ental":1545,"Ġreally":1546,"Ġmar":1547,"ret":1548,"Ġprom":1549,"Ġaddition":1550,"Ġfour":1551,"Ġlet":1552,"rap":1553,"Ġfore":1554,"Ġhard":1555,"Ġfil":1556,"ator":1557,"ource":1558,"ably":1559,"lished":1560,"Ġet":1561,"Ġmeans":1562,"Ġsomething":1563,"oint":1564,"Ġmajor":1565,"Ġtre":1566,"Ġsym":1567,"be":1568,"ĠTo":1569,"ina":1570,"ĠRes":1571,"aper":1572,"ĠAll":1573,"atural":1574,"ĠWhen":1575,"Ġhistory":1576,"Ġseveral":1577,"Ġcase":1578,"Ġmean":1579,"Ġfac":1580,"Ġalways":1581,"Ġpossible":1582,"Ġmen":1583,"ĠCan":1584,"Ġorig":1585,"ĠGod":1586,"Ġtri":1587,"oad":1588,"Ġmeas":1589,"Ġsaf":1590,"Ġparticular":1591,"Ġname":1592,"Ġ12":1593,"ĠComm":1594,"Ġview":1595,"lu":1596,"of":1597,"ĠUnited":1598,"Ġdri":1599,"ĠFr":1600,"ograph":1601,"ane":1602,"lease":1603,"Ġpercent":1604,"ope":1605,"Ġcountry":1606,"Ġcho":1607,"port":1608,"If":1609,"Ġaccess":1610,"ĠBe":1611,"ĠHowever":1612,"Ġnext":1613,"Ġnon":1614,"Ġinvest":1615,"Ġtimes":1616,"Ġ'":1617,"oot":1618,"ortun":1619,"ring":1620,"ological":1621,"thing":1622,"Ġinvol":1623,"ended":1624,"Ġmethod":1625,"Ġturn":1626,"irc":1627,"work":1628,"Ġseem":1629,"por":1630,"Ġlim":1631,"Ġdays":1632,"ĠTe":1633,"Ġfollowing":1634,"ww":1635,"Ġdisease":1636,"Ġothers":1637,"Ġtry":1638,"ocus":1639,"ised":1640,"Ġhig":1641,"Ġ0":1642,"ples":1643,"Ġintern":1644,"Ġperiod":1645,"Ġbenef":1646,"par":1647,"What":1648,"Ġsuccess":1649,"ĠAt":1650,"Ġdra":1651,"atch":1652,"Ġrisk":1653,"urop":1654,"Ġanim":1655,"ĠCent":1656,"Ġuntil":1657,"Ġgr":1658,"Ġnever":1659,"Ġcertain":1660,"Ġmov":1661,"ird":1662,"12":1663,"arc":1664,"Ġconn":1665,"Ġexperience":1666,"Ġbased":1667,"Ġgive":1668,"Ġask":1669,"ĠWhat":1670,"idence":1671,"ged":1672,"Ġamong":1673,"Ġmaking":1674,"Ġlocal":1675,"ĠChrist":1676,"over":1677,"elf":1678,"Ġblood":1679,"ĠUS":1680,"ĠSo":1681,"ann":1682,"though":1683,"ances":1684,"une":1685,"ĠBl":1686,"ogn":1687,"ilar":1688,"Ġshort":1689,"Ġbig":1690,"Ġcreat":1691,"Ġcourse":1692,"Ġfar":1693,"oney":1694,"Ġindust":1695,"Ġsw":1696,"ohn":1697,"pr":1698,"ember":1699,"osp":1700,"ĠOn":1701,"Ġpast":1702,"ately":1703,"ĠOr":1704,"Ġfre":1705,"sy":1706,"Ġoccur":1707,"Ġproblem":1708,"Ġtoday":1709,"Ġcause":1710,"cing":1711,"Ġweek":1712,"urther":1713,"Ġlangu":1714,"ither":1715,"Ġcal":1716,"ours":1717,"Ġ17":1718,"ĠStates":1719,"Ġalong":1720,"oid":1721,"Ġfull":1722,"urope":1723,"Ġsing":1724,"ending":1725,"ole":1726,"50":1727,"app":1728,"br":1729,"ones":1730,"Ġsignific":1731,"not":1732,"Ġareas":1733,"ured":1734,"Ġyoung":1735,"vious":1736,"18":1737,"Ġwhy":1738,"Ġcommunity":1739,"Ġfri":1740,"ides":1741,"resent":1742,"agement":1743,"aster":1744,"Ġspecial":1745,"Ġlearning":1746,"Ġlater":1747,"Ġcr":1748,"Ġfocus":1749,"Ġline":1750,"Ġpress":1751,"Ġmult":1752,"Ġmarket":1753,"Ġsure":1754,"uture":1755,"Ġworking":1756,"Ġgiven":1757,"Ġvol":1758,"As":1759,"ggest":1760,"Ġaway":1761,"ĠShe":1762,"pect":1763,"olution":1764,"aps":1765,"Ġpolit":1766,"Ġlab":1767,"orts":1768,"Ġtoget":1769,"Ġprofess":1770,"Ġapplic":1771,"Ġtogether":1772,"mit":1773,"11":1774,"Ġage":1775,"Ġcond":1776,"by":1777,"hy":1778,"idd":1779,"ley":1780,"Ġplant":1781,"aken":1782,"Ġkind":1783,"Ġside":1784,"Ġrecord":1785,"Ġamount":1786,"Ġclear":1787,"Ġworks":1788,"Ġproblems":1789,"min":1790,"Ġleft":1791,"Ġappear":1792,"Ġpay":1793,"Ġtru":1794,"ox":1795,"ode":1796,"icult":1797,"Ġcomplete":1798,"let":1799,"St":1800,"Ġmillion":1801,"Ġcolle":1802,"Ġ16":1803,"ER":1804,"Ġsuggest":1805,"ĠLe":1806,"Ġspecies":1807,"Ġtype":1808,"Ġtreatment":1809,"ĠEurope":1810,"ledge":1811,"Ġobject":1812,"Ġneeds":1813,"iting":1814,"Ġleast":1815,"Ġcomb":1816,"ĠWor":1817,"Ġscient":1818,"Ġstruct":1819,"ored":1820,"Ġdue":1821,"Ġheart":1822,"Ġauthor":1823,"ication":1824,"Ġwr":1825,"Ġenough":1826,"most":1827,"ression":1828,"ustom":1829,"Ġnatural":1830,"ush":1831,"ĠThat":1832,"âĢ¦":1833,"ĠAf":1834,"Ġey":1835,"Ġsal":1836,"Ġhaving":1837,"ery":1838,"ipp":1839,"Ġprevent":1840,"Ġwords":1841,"aut":1842,"ĠNational":1843,"ream":1844,"Ġcomput":1845,"Ġcreate":1846,"isc":1847,"Ġlove":1848,"af":1849,"alf":1850,"Ġflow":1851,"Ġassoci":1852,"aim":1853,"There":1854,"Ġdiscuss":1855,"ape":1856,"ott":1857,"30":1858,"Ġacross":1859,"ĠGo":1860,"utes":1861,"Ġdr":1862,"cle":1863,"ique":1864,"Ġenc":1865,"Ġstrong":1866,"sw":1867,"ready":1868,"stit":1869,"Ġcirc":1870,"ecially":1871,"ray":1872,"Ġbegin":1873,"Ġaccount":1874,"side":1875,"ler":1876,"Ġresults":1877,"Ġopt":1878,"Ġimpact":1879,"ĠMed":1880,"ots":1881,"Ġphot":1882,"Ġfuture":1883,"Ġsignificant":1884,"ĠJohn":1885,"Ġlikely":1886,"Ġdeath":1887,"Ġcompany":1888,"Ġabove":1889,"ively":1890,"Ġdeterm":1891,"Ġcrit":1892,"Ġbi":1893,"Ġstory":1894,"for":1895,"Ġemploy":1896,"Ġmeet":1897,"Ġprop":1898,"Ġvir":1899,"Ġlevels":1900,"ateg":1901,"Ġtechnology":1902,"Ġmakes":1903,"augh":1904,"Ġfr":1905,"Ġann":1906,"Ġoffic":1907,"Ġhouse":1908,"ĠCar":1909,"Ġevent":1910,"Ġteac":1911,"Ġcharacter":1912,"Ġ30":1913,"Ġrad":1914,"Ġeducation":1915,"rug":1916,"atory":1917,"aring":1918,"Ġparent":1919,"Ġspace":1920,"especially":1921,"Ġexist":1922,"Ġdifficult":1923,"For":1924,"Ġfund":1925,"aining":1926,"Ġkey":1927,"Ġconsum":1928,"enn":1929,"Ġalready":1930,"asing":1931,"Ġsite":1932,"Ġfunction":1933,"selves":1934,"Ġrequire":1935,"ators":1936,"Ġusually":1937,"Ġhowever":1938,"ĠWith":1939,"Ġsimilar":1940,"How":1941,"mber":1942,"Ġmind":1943,"Ġspecific":1944,"Ġincrease":1945,"Ġproducts":1946,"ges":1947,"Ġcity":1948,"Ġrespons":1949,"Ġsum":1950,"ĠSy":1951,"obal":1952,"Ġdone":1953,"ĠMar":1954,"Ġer":1955,"Ġreason":1956,"face":1957,"Ġill":1958,"atures":1959,"Ġnet":1960,"Ġservice":1961,"Ġrelations":1962,"val":1963,"Ġnecess":1964,"ruction":1965,"Ġplac":1966,"ilt":1967,"icro":1968,"ilities":1969,"Ġdescrib":1970,"Ġvarious":1971,"hem":1972,"omes":1973,"Ġquality":1974,"ument":1975,"Ġsystems":1976,"Ġtell":1977,"Ġwin":1978,"Ġservices":1979,"Ġterm":1980,"Ġstep":1981,"Ġonline":1982,"Ġtool":1983,"Ġdam":1984,"ued":1985,"urch":1986,"Ġmot":1987,"ourn":1988,"ffic":1989,"duct":1990,"Ġquestion":1991,"15":1992,"oman":1993,"Ġlive":1994,"Ġlanguage":1995,"acy":1996,"Ġi":1997,"Ġpaper":1998,"ĠEn":1999,"erence":2000,"An":2001,"ĠAm":2002,"Ġhy":2003,"Ġpolic":2004,"iam":2005,"Ġwhether":2006,"ĠBrit":2007,"Ġclose":2008,"ump":2009,"Ġespecially":2010,"Ġconcer":2011,"itor":2012,"Ġchanges":2013,"Ġhigher":2014,"Ġmembers":2015,"Ġjob":2016,"Ġobs":2017,"Ġways":2018,"aching":2019,"Ġimprove":2020,"ysis":2021,"Ġcame":2022,"Ġ14":2023,"ĠMan":2024,"Ġonce":2025,"Ġbelow":2026,"Ġbehav":2027,"ĠTr":2028,"Ġseen":2029,"Ġgeneral":2030,"Ġsubject":2031,"Ġmoney":2032,"Ġestab":2033,"Ġconsider":2034,"utions":2035,"wards":2036,"ĠOne":2037,"Ġconditions":2038,"unt":2039,"ĠMay":2040,"verage":2041,"Ġcells":2042,"Ġz":2043,"Ġla":2044,"ney":2045,"sh":2046,"Ġdetail":2047},"merges":[["Ġ","t"],["Ġ","a"],["h","e"],["i","n"],["r","e"],["o","n"],["Ġt","he"],["e","r"],["Ġ","s"],["Ġ","o"],["a","t"],["e","n"],["Ġ","w"],["Ġ","c"],["e","s"],["n","d"],["i","s"],["i","t"],["o","r"],["Ġ","p"],["Ġ","b"],["Ġ","f"],["a","l"],["e","d"],["a","n"],["in","g"],["Ġo","f"],["o","u"],["Ġ","in"],["a","r"],["Ġ","m"],["Ġt","o"],["Ġa","nd"],["i","c"],["Ġ","d"],["i","on"],["l","e"],["r","o"],["a","s"],["Ġ","h"],["Ġt","h"],["en","t"],["i","l"],["Ġ","e"],["Ġ","re"],["c","t"],["o","m"],["Ġ","l"],["v","e"],["s","t"],["Ġ","n"],["â","Ģ"],["l","y"],["Ġb","e"],["Ġ","is"],["Ġ","g"],["Ġ","T"],["s","e"],["o","l"],["i","d"],["e","t"],["u","t"],["o","t"],["i","m"],["at","ion"],["u","r"],["c","e"],["o","w"],["Ġf","or"],["Ġ","y"],["Ġ","S"],["r","a"],["Ġ","A"],["c","h"],["Ġ","on"],["Ġ","I"],["i","g"],["Ġth","at"],["Ġ","u"],["Ġ","C"],["v","er"],["Ġs","t"],["a","y"],["i","r"],["a","m"],["Ġy","ou"],["it","h"],["a","d"],["e","l"],["u","l"],["Ġa","s"],["Ġw","ith"],["he","r"],["Ġp","ro"],["Ġa","n"],["Ġ","it"],["Ġa","re"],["Ġw","h"],["Ġ","M"],["Ġc","on"],["Ġa","l"],["i","f"],["t","er"],["Ġo","r"],["Ġ","P"],["o","d"],["âĢ","Ļ"],["ĠT","he"],["g","e"],["il","l"],["Ġ","B"],["Ġ","("],["er","s"],["m","ent"],["Ġ","1"],["Ġ","he"],["t","h"],["at","e"],["es","s"],["Ġw","e"],["o","p"],["is","t"],["Ġh","a"],["Ġd","e"],["u","s"],["Ġc","om"],["o","re"],["Ġe","x"],["r","i"],["es","t"],["a","b"],["k","e"],["Ġ","H"],["a","c"],["Ġ","W"],["l","d"],["Ġs","u"],["e","ct"],["it","y"],["Ġa","t"],["ro","m"],["re","s"],["p","p"],["Ġ","v"],["Ġ","D"],["i","ve"],["an","t"],["u","m"],["i","es"],["ig","h"],["q","u"],["o","s"],["Ġw","as"],["e","m"],["Ġb","y"],["n","t"],["Ġ","R"],["a","in"],["Ġ","F"],["Ġn","e"],["Ġ","E"],["Ġf","rom"],["Ġ","2"],["or","t"],["o","c"],["Ġs","e"],["Ġc","h"],["a","nd"],["u","d"],["Ġc","an"],["Ġ","r"],["Ġ","âĢ"],["u","n"],["Ġn","ot"],["i","v"],["Ġ","G"],["Ġha","ve"],["Ġ","L"],["ar","t"],["Ġ","N"],["Ġ","le"],["Ġs","h"],["u","re"],["Ġth","is"],["i","al"],["igh","t"],["u","st"],["ic","al"],["o","g"],["Ġs","p"],["Ġ","en"],["al","l"],["Ġw","or"],["Ġa","b"],["ou","ld"],["om","e"],["ou","t"],["ou","r"],["0","0"],["re","d"],["g","h"],["o","st"],["r","ou"],["p","t"],["ar","d"],["ic","h"],["Ġyou","r"],["T","he"],["Ġp","l"],["Ġ","O"],["in","e"],["ion","s"],["p","er"],["a","g"],["id","e"],["t","her"],["as","t"],["el","l"],["e","ar"],["Ġu","s"],["Ġ","J"],["Ġw","ill"],["at","ed"],["f","f"],["a","k"],["Ġ","im"],["Ġ","k"],["Ġin","t"],["Ġthe","ir"],["p","l"],["Ġal","l"],["a","ge"],["a","ct"],["Ġw","he"],["en","d"],["an","s"],["i","e"],["d","u"],["Ġthe","y"],["Ġ","U"],["Ġa","d"],["âĢ","Ŀ"],["ou","s"],["Ġh","as"],["Ġwh","ich"],["Ġd","o"],["i","z"],["e","c"],["o","od"],["Ġcom","p"],["on","g"],["Ġcon","t"],["Ġm","ore"],["c","c"],["i","p"],["a","p"],["o","k"],["ĠI","n"],["i","b"],["'","s"],["r","u"],["Ġb","ut"],["Ġin","c"],["ul","t"],["a","re"],["i","a"],["Ġm","e"],["Ġ","j"],["or","m"],["Ġre","s"],["ab","le"],["Ġd","is"],["s","o"],["ation","s"],["e","re"],["r","y"],["ar","y"],["Ġc","l"],["al","ly"],["ĠT","h"],["ve","l"],["in","d"],["ac","k"],["Ġm","an"],["ic","e"],["Ġon","e"],["a","ke"],["Ġo","ut"],["Ġo","ther"],["en","ce"],["ĠâĢ","ľ"],["an","ce"],["e","p"],["re","at"],["Ġs","o"],["on","e"],["ver","y"],["Ġwe","re"],["Ġt","e"],["Ġp","er"],["o","se"],["a","ch"],["im","e"],["il","d"],["ig","n"],["as","s"],["as","e"],["Ġal","so"],["Ġab","out"],["Ġh","is"],["a","u"],["b","er"],["ent","s"],["a","v"],["am","e"],["Ġu","p"],["ĠS","t"],["Ġa","pp"],["at","er"],["it","e"],["Ġwh","o"],["res","s"],["o","ok"],["Ġim","p"],["Ġu","n"],["i","ke"],["i","re"],["Ġa","r"],["it","ion"],["a","ce"],["ct","ion"],["or","d"],["k","s"],["Ġe","v"],["Ġp","art"],["ou","nt"],["e","w"],["re","e"],["Ġ","K"],["u","b"],["Ġ1","9"],["is","h"],["u","e"],["a","il"],["Ġp","e"],["en","s"],["or","y"],["Ġthe","m"],["Ġ","\""],["Ġ","qu"],["ay","s"],["n","ow"],["Ġs","c"],["Ġg","o"],["f","orm"],["ĠI","t"],["i","le"],["Ġy","ear"],["or","s"],["ou","nd"],["Ġ2","0"],["Ġs","ome"],["c","es"],["Ġcom","m"],["l","l"],["Ġo","ur"],["an","g"],["ĠC","h"],["Ġbe","c"],["Ġbe","en"],["l","ow"],["Ġp","re"],["Ġth","an"],["Ġst","ud"],["Ġthe","re"],["ing","s"],[")","."],["Ġt","ime"],["Ġwhe","n"],["Ġre","c"],["Ġwor","k"],["Ġa","g"],["Ġs","a"],["r","ic"],["ou","gh"],["er","v"],["op","le"],["Ġ","V"],["Ġo","ver"],["Ġs","y"],["m","er"],["Ġ","Y"],["Ġh","ad"],["Ġt","ra"],["Ġm","ay"],["if","f"],["on","d"],["Ġ","3"],["Ġint","o"],["Ġl","ike"],["Ġit","s"],["o","ve"],["i","an"],["at","es"],["r","it"],["nd","er"],["in","t"],["Ġ","ro"],["ol","og"],["ver","s"],["Ġne","w"],["Ġw","ould"],["Ġh","ow"],["Ġof","f"],["Ġan","y"],["rou","gh"],["ir","st"],["Ġpe","ople"],["u","al"],["on","s"],["t","e"],["p","h"],["Ġk","now"],["l","ud"],["Ġin","d"],["Ġ","if"],["l","ic"],["au","se"],["ĠTh","is"],["ter","n"],["re","n"],["Ġc","ol"],["Ġthe","se"],["Ġwh","at"],["Ġa","cc"],["Ġt","w"],["Ġ","ra"],["w","n"],["Ġd","iff"],["Ġd","es"],["o","b"],["Ġne","ed"],["Ġbe","t"],["Ġcon","s"],["Ġm","ost"],["Ġu","se"],["Ġa","ct"],["ar","k"],["i","ous"],["m","s"],["w","e"],["n","ing"],["Ġb","l"],["ment","s"],["Ġu","nder"],["o","v"],["l","es"],["if","ic"],["it","ies"],["Ġre","p"],["y","s"],["j","ect"],["Ġf","e"],["he","d"],["Ġn","o"],["le","ct"],["Ġ","-"],["Ġa","m"],["Ġt","r"],["Ġre","l"],["Ġm","y"],["Ġon","ly"],["Ġf","irst"],["Ġhe","l"],["f","ter"],["in","k"],["c","k"],["Ġinc","lud"],["Ġ20","1"],["o","ol"],["p","le"],["Ġpro","v"],["Ġhel","p"],["Ġg","et"],["al","th"],["Ġpro","du"],["at","ing"],["Ġp","h"],["vel","op"],[".","."],["ation","al"],["c","ed"],["Ġsu","ch"],["Ġth","rough"],["st","em"],["n","e"],["Ġe","ff"],["ow","n"],["at","ive"],["c","ess"],["f","ul"],["Ġ","her"],["rou","nd"],["is","e"],["Ġtw","o"],["ic","k"],["Ġe","m"],["Ġman","y"],["ol","d"],["i","ent"],["Ġg","en"],["ul","ar"],["t","s"],["Ġp","o"],["ĠâĢ","ĵ"],["ĠH","e"],["is","s"],["at","h"],["ĠU","n"],["et","h"],["Ġad","d"],["os","s"],["ot","h"],["Ġex","p"],["Ġre","g"],["Ġch","ild"],[")",","],["y","p"],["he","n"],["ro","w"],["al","s"],["ri","b"],["Ġus","ed"],["Ġsp","ec"],["Ġat","t"],["Ġj","ust"],["ic","s"],["Ġ","4"],["Ġw","ell"],["Ġa","c"],["ur","ing"],["i","ed"],["if","e"],["ad","e"],["Ġde","c"],["Ġde","velop"],["f","ore"],["er","t"],["Ġdiff","ere"],["ib","le"],["Ġm","ake"],["er","m"],["Ġsh","ould"],["an","ge"],["as","ed"],["il","ity"],["se","l"],["Ġsu","pp"],["ra","m"],["Ġas","s"],["ro","p"],["Ġc","o"],["Ġsa","id"],["Ġyear","s"],["Ġ","5"],["Ġre","m"],["ar","ch"],["Ġin","v"],["Ġc","ould"],["Ġf","l"],["u","res"],["Ġb","u"],["b","le"],["n","g"],["f","t"],["i","et"],["Ġp","r"],["Ġsy","stem"],["ra","ct"],["o","y"],["a","w"],["x","t"],["Ġo","b"],["ow","er"],["Ġ2","00"],["Ġs","m"],["il","y"],["u","ch"],["Ġs","ur"],["Ġsu","b"],["Ġwhe","re"],["t","ing"],["s","s"],["I","n"],["g","an"],["Ġp","ers"],["he","s"],["t","en"],["Ġh","igh"],["ĠW","e"],["Ġp","res"],["on","t"],["we","en"],["id","ent"],["it","s"],["Ġin","f"],["Ġa","fter"],["c","y"],["Ġ","very"],["it","ed"],["ie","w"],["Ġse","e"],["ut","ion"],["Ġe","very"],["h","at"],["i","ence"],["Ġs","he"],["ch","ool"],["Ġthe","n"],["Ġev","en"],["Ġbec","ause"],["ur","n"],["Ġw","ay"],["Ġbet","ween"],["Ġin","st"],["Ġtr","ans"],["Ġp","ol"],["e","ver"],["ic","t"],["mer","ic"],["Ġex","per"],["ant","s"],["Ġw","ater"],["Ġdis","c"],["Ġe","ar"],["form","ation"],["Ġl","ong"],["ak","ing"],["Ġ","ke"],["ch","n"],["Ġm","od"],["Ġc","ar"],["Ġb","ack"],["u","c"],["l","and"],["u","ally"],["ot","her"],["Ġn","um"],["in","s"],["g","et"],["Ġwor","ld"],["Ġhe","alth"],["Ġin","ter"],["Ġfor","m"],["ĠR","e"],["in","es"],["Ġs","im"],["t","le"],["Ġimp","ort"],["Ġse","c"],["olog","y"],[".","âĢĿ"],["Ġf","ound"],["Ġre","se"],["n","ess"],["Ġdiffere","nt"],["Ġth","ose"],["Ġl","ook"],["i","vers"],["ce","pt"],["Ġc","ount"],["ur","al"],["ol","low"],["ist","s"],["Ġbe","ing"],["Ġm","ed"],["oc","k"],["w","ard"],["or","k"],["Ġg","reat"],["om","m"],["Ġc","all"],["Ġo","wn"],["Ġr","ight"],["Ġg","ood"],["Ġs","et"],["Ġm","uch"],["l","ed"],["T","h"],["at","a"],["Ġ","ent"],["Ġex","am"],["Ġres","p"],["Ġ","ed"],["is","m"],["ĠA","meric"],["Ġpro","g"],["Ġl","ar"],["Ġre","f"],["Ġd","et"],["an","y"],["at","ure"],["Ġp","ub"],["Ġeff","ect"],["Ġle","ar"],["Ġre","ad"],["u","es"],["Ġc","ent"],["ic","es"],["e","f"],["Ġ1","8"],["Ġex","t"],["h","ip"],["Ġg","ra"],["Ġs","ign"],["Ġ","6"],["Ġen","d"],["ical","ly"],["p","s"],["Ġs","erv"],["v","ed"],["Ġm","on"],["Ġre","qu"],["Ġl","ife"],["g","y"],["iv","es"],["Ġm","ade"],["Ġe","ach"],["d","ay"],["Ġf","am"],["Ġdo","es"],["Ġde","f"],["Ġar","t"],["i","o"],["Ġgen","er"],["Ġl","oc"],["Ġf","ind"],["ĠP","ro"],["v","ent"],["ag","es"],["at","her"],["ĠA","l"],["ar","m"],["Ġh","im"],["Ġin","formation"],["Ġn","ow"],["Ġinc","re"],["Ġt","yp"],["u","nd"],["Ġf","act"],["Ġc","re"],["Ġpers","on"],["Ġv","ar"],["Ġpro","cess"],["Ġcomm","un"],["im","es"],["c","er"],["Ġag","ain"],["Ġf","ollow"],["Ġin","s"],["od","y"],["Ġsh","ow"],["Ġle","ad"],["Ġbe","fore"],["t","y"],["Ġm","in"],["Ġp","at"],["ĠI","f"],["Ġs","oc"],["Ġres","ult"],["Ġh","um"],["v","en"],["|","|"],["g","g"],["ver","n"],["Ġc","he"],["ĠA","s"],["Ġd","own"],["Ġs","ame"],["Ġd","ay"],["al","e"],["Ġpart","ic"],["Ġa","ut"],["Ġs","k"],["Ġte","chn"],["iz","ed"],["Ġa","v"],["t","ed"],["Ġt","ake"],["Ġd","uring"],["Ġd","ist"],["Ġd","id"],["ru","ct"],["Ġde","p"],["i","x"],["Ġc","a"],["as","es"],["2","0"],["'","t"],["Ġc","ur"],["iz","e"],["in","ce"],["ĠA","n"],["c","ent"],["t","he"],["Ġst","art"],["ĠThe","y"],["Ġb","r"],["Ġimport","ant"],["Ġl","it"],["d","s"],["v","ing"],["Ġchild","ren"],["Ġb","oth"],["ĠN","ew"],["a","ir"],["ĠY","ou"],["or","n"],["ect","ion"],["ĠA","r"],["al","ity"],["re","ad"],["d","er"],["a","x"],["Ġcomp","le"],["u","p"],["Ġst","r"],["Ġw","ant"],["in","al"],["ĠF","or"],["e","x"],["ĠS","p"],["Ġwh","ile"],["Ġs","chool"],["n","ce"],["i","el"],["ir","on"],["Ġpro","ble"],["ak","es"],["c","om"],["c","on"],["Ġ1","0"],["Ġpl","ay"],["Ġm","em"],["Ġha","nd"],["Ġle","vel"],["ar","s"],["Ġg","u"],["Ġg","l"],["Ġstud","ents"],["00","0"],["Ġst","ate"],["i","er"],["ire","ct"],["ĠA","nd"],["Ġth","ink"],["as","h"],["ul","l"],["Ġ","7"],["Ġd","ata"],["e","e"],["Ġg","rou"],["Ġf","ood"],["Ġv","is"],["Ġch","ar"],["in","ed"],["Ġsm","all"],["Ġus","ing"],["ut","e"],["sel","f"],["Ġa","round"],["o","h"],["in","ess"],["1","9"],["Ġdes","ign"],["Ġprov","id"],["ĠH","ow"],["ial","ly"],["Ġex","pl"],["a","ve"],["a","h"],["n","er"],["Ġo","p"],["Ġpo","int"],["Ġm","ain"],["Ġth","ree"],["Ġv","al"],["Ġp","oss"],["is","hed"],["e","red"],["Ġh","ist"],["âĢ","ľ"],["ivers","ity"],["ĠB","ut"],["v","es"],["ter","s"],["st","and"],["w","ay"],["Ġsupp","ort"],[",","âĢĿ"],["Ġs","ol"],["Ġcon","f"],["an","k"],["Ġd","on"],["Ġp","op"],["ou","se"],["Ġw","rit"],["Ġs","er"],["Ġbe","l"],["om","en"],["Ġstud","y"],["Ġ","8"],[".",","],["Ġf","un"],["l","ine"],["Ġc","le"],["Ġg","row"],["Ġl","ess"],["iv","id"],["Ġc","our"],["Ġth","ough"],["ect","ed"],["Ġnum","ber"],["Ġor","gan"],["i","en"],["Ġt","est"],["Ġs","om"],["ro","l"],["Th","is"],["Ġre","t"],["ĠI","nd"],["an","c"],["as","on"],["r","al"],["Ġ","&"],[".","\""],["Ġb","o"],["Ġc","are"],["ĠS","c"],["it","al"],["Ġcl","ass"],["Ġpro","t"],["Ġd","i"],["i","qu"],["Ġst","ill"],["e","g"],["Ġrep","ort"],["Ġm","ust"],["us","s"],["re","nt"],["Ġp","ower"],["ion","al"],["r","on"],["Ġb","est"],["Ġof","ten"],["c","l"],["al","k"],["ro","ss"],["Ġrese","arch"],["d","ition"],["Ġpl","an"],["v","iron"],["Ġpl","ace"],["Ġad","v"],["Ġ","9"],["Ġprodu","ct"],["Ġhe","re"],["Ġb","ook"],["g","er"],["ur","y"],["Ġm","ark"],["a","j"],["Ġb","us"],["Ġh","ome"],["Ġcont","in"],["Ġt","reat"],["m","an"],["Ġe","as"],["Ġo","cc"],["Ġf","ew"],["im","ate"],["i","red"],["iet","y"],["ĠThe","re"],["Ġqu","est"],["Ġb","ody"],["âĢ","Ķ"],["ĠW","h"],["Ġc","or"],["Ġgo","vern"],["a","red"],["Ġl","ast"],["Ġ","est"],["Ġall","ow"],["el","s"],["ou","n"],["ere","st"],["Ġinclud","ing"],["ain","s"],["i","um"],["b","s"],["Ġact","iv"],["ĠâĢ","ĺ"],["Ġp","ur"],["Ġm","at"],["et","s"],["Ġbu","ild"],["1","0"],["t","on"],["Ġpub","lic"],["ĠD","e"],["a","z"],["Ġ","ide"],["ĠS","h"],["b","ers"],["l","ess"],["am","p"],["Ġlar","ge"],["Ġa","ff"],["Ġm","ight"],["Ġr","is"],["viron","ment"],["ou","th"],["er","ing"],["Ġpro","f"],["d","e"],["it","ions"],["ul","ation"],["Ġs","l"],["Ġhum","an"],["en","se"],["Ġcon","st"],["iv","ing"],["Ġwith","out"],["u","g"],["Ġis","s"],["her","s"],["Ġl","and"],["Ġwe","e"],["Ġst","and"],["re","en"],["v","iew"],["Ġ","ess"],["es","p"],["ĠI","s"],["I","t"],["Ġe","qu"],["Ġe","lect"],["Ġf","in"],["Ġdis","e"],["l","ish"],["ri","es"],["w","ays"],["ert","ain"],["in","ing"],["Ġexam","ple"],["Ġcomm","on"],["Ġm","et"],["r","is"],["Ġapp","ro"],["m","e"],["pt","ion"],["Ġs","ay"],["os","ed"],["Ġre","du"],["Ġan","other"],["Ġp","ass"],["Ġunder","stand"],["Ġc","ell"],["act","er"],["Ġd","irect"],["r","ation"],["Ġ","$"],["con","om"],["Ġpro","ject"],["t","t"],["Ġbe","h"],["Ġto","p"],["ĠC","l"],["Ġcall","ed"],["ra","ph"],["in","c"],["Ġto","o"],["er","al"],["Ġp","ract"],["Ġwith","in"],["Ġpres","ent"],["ĠA","d"],["Ġl","ight"],["Ġs","it"],["ill","ion"],["Ġre","st"],["W","e"],["oc","i"],["i","k"],["Ġint","erest"],["Ġde","m"],["Ġre","al"],["i","or"],["Ġm","us"],["Ġch","ang"],["en","g"],["iz","ation"],["ord","ing"],["ater","ial"],["ivid","ual"],["Ġ19","9"],["Ġed","uc"],["id","s"],["Ġp","os"],["Ġb","ra"],["Ġha","pp"],["Ġke","ep"],["Ġbel","ie"],["Ġo","pp"],["er","gy"],["it","ive"],["Ġb","re"],["ĠE","ng"],["Ġbet","ter"],["Ġknow","n"],["Ġav","ail"],["Ġch","ange"],["Ġen","g"],["Ġc","ap"],["Ġsec","ond"],["Ġp","ri"],["in","ation"],["..","."],["ent","ial"],["r","ist"],["g","en"],["or","th"],["iel","d"],["cc","ess"],["Ġo","ld"],["âĢ","ĵ"],["Ġth","ings"],["i","od"],["ĠC","on"],["iv","en"],["Ġagain","st"],["Ġprog","ram"],["Ġare","a"],["Ġop","en"],["is","ion"],["Ġfe","el"],["t","o"],["are","nt"],["s","c"],["Ġfam","ily"],["Ġf","ree"],["Ġlear","n"],["ĠAmeric","an"],["am","es"],["Ġ","ve"],["ent","ly"],["ĠP","h"],["Ġle","g"],["ĠUn","iversity"],["om","s"],["Ġl","aw"],["Ġb","en"],["Ġim","m"],["our","ces"],["Ġa","ir"],["Ġb","ro"],["Ġor","der"],["if","ied"],["Ġb","as"],["Ġr","un"],["Ġimp","ro"],["w","are"],["Ġind","ividual"],["ĠâĢ","Ķ"],["Ġbus","iness"],["Ġl","ist"],["e","ad"],["ro","du"],["aj","or"],["he","re"],["h","or"],["Ġs","ince"],["Ġp","ost"],["Ġpro","per"],["Ġw","omen"],["ĠThe","se"],["g","in"],["at","s"],["Ġsoc","ial"],["Ġt","em"],["ab","ility"],["Ġper","form"],["Ġdevelop","ment"],["i","ans"],["ard","s"],["Ġd","el"],["if","y"],["e","ed"],["Ġm","aterial"],["Ġmon","th"],["ĠC","ol"],["ent","ion"],["er","n"],["Ġo","per"],["Ġhe","ad"],["ĠE","x"],["iss","ion"],["Ġlit","tle"],["Ġcom","e"],["Ġan","al"],["Ġp","ot"],["Ġu","nt"],["end","s"],["l","ing"],["Ġw","ar"],["ain","ed"],[",","\""],["Ġ","."],["Ġavail","able"],["eth","ing"],["Ġprov","ide"],["Ġp","ut"],["i","ver"],["Ġinclud","e"],["Ġab","le"],["ĠP","l"],["ill","s"],["or","ies"],["Ġ","ident"],["i","ents"],["ang","u"],["id","er"],["Ġbec","ome"],["Ġca","us"],["Ġse","ver"],["eth","od"],["er","c"],["Ġ","Q"],["Ġdes","c"],["Ġgovern","ment"],["Ġen","vironment"],["Ġe","conom"],["Ġp","ar"],["Ġp","ain"],["Ġcur","rent"],["Ġm","ove"],["ic","le"],["om","p"],["Ġe","l"],["ĠS","e"],["Ġgo","ing"],["Ġcont","rol"],["Ġ1","5"],["Ġen","ergy"],["iv","ed"],["Ġ","["],["_","_"],["Ġt","er"],["at","ic"],["Ġgrou","p"],["ot","e"],["s","p"],["Ġph","ys"],["Ġm","il"],["Ġc","ost"],["pl","oy"],["Ġan","t"],["Ġs","ays"],["Ġ","Z"],["Ġre","ce"],["Ġl","ot"],["Ġe","le"],["Ġte","am"],["Ġear","ly"],["Ġde","v"],["Ġl","ow"],["ag","n"],["Ġc","ult"],["Ġne","ar"],["u","se"],["ent","al"],["Ġre","ally"],["Ġm","ar"],["re","t"],["Ġpro","m"],["Ġadd","ition"],["Ġf","our"],["Ġle","t"],["ra","p"],["Ġf","ore"],["Ġh","ard"],["Ġf","il"],["at","or"],["our","ce"],["ab","ly"],["l","ished"],["Ġe","t"],["Ġme","ans"],["Ġsom","ething"],["o","int"],["Ġm","ajor"],["Ġt","re"],["Ġsy","m"],["b","e"],["ĠT","o"],["in","a"],["ĠR","es"],["a","per"],["ĠA","ll"],["at","ural"],["ĠW","hen"],["Ġhist","ory"],["Ġsever","al"],["Ġc","ase"],["Ġme","an"],["Ġf","ac"],["Ġal","ways"],["Ġposs","ible"],["Ġm","en"],["ĠC","an"],["Ġor","ig"],["ĠG","od"],["Ġt","ri"],["o","ad"],["Ġme","as"],["Ġsa","f"],["Ġpartic","ular"],["Ġn","ame"],["Ġ1","2"],["ĠC","omm"],["Ġv","iew"],["l","u"],["o","f"],["ĠUn","ited"],["Ġd","ri"],["ĠF","r"],["og","raph"],["an","e"],["le","ase"],["Ġper","cent"],["op","e"],["Ġcount","ry"],["Ġch","o"],["p","ort"],["I","f"],["Ġacc","ess"],["ĠB","e"],["ĠHow","ever"],["Ġne","xt"],["Ġn","on"],["Ġinv","est"],["Ġt","imes"],["Ġ","'"],["o","ot"],["ort","un"],["r","ing"],["olog","ical"],["th","ing"],["Ġinv","ol"],["end","ed"],["Ġm","ethod"],["Ġt","urn"],["ir","c"],["w","ork"],["Ġse","em"],["p","or"],["Ġl","im"],["Ġd","ays"],["ĠT","e"],["Ġfollow","ing"],["w","w"],["Ġdise","ase"],["Ġother","s"],["Ġt","ry"],["oc","us"],["is","ed"],["Ġh","ig"],["Ġ","0"],["pl","es"],["Ġin","tern"],["Ġper","iod"],["Ġben","ef"],["p","ar"],["W","hat"],["Ġsu","ccess"],["ĠA","t"],["Ġd","ra"],["at","ch"],["Ġris","k"],["u","rop"],["Ġan","im"],["ĠC","ent"],["Ġunt","il"],["Ġg","r"],["Ġne","ver"],["Ġc","ertain"],["Ġm","ov"],["ir","d"],["1","2"],["ar","c"],["Ġcon","n"],["Ġexper","ience"],["Ġb","ased"],["Ġg","ive"],["Ġas","k"],["ĠW","hat"],["id","ence"],["g","ed"],["Ġam","ong"],["Ġm","aking"],["Ġloc","al"],["ĠCh","rist"],["o","ver"],["el","f"],["Ġbl","ood"],["ĠU","S"],["ĠS","o"],["an","n"],["th","ough"],["an","ces"],["un","e"],["ĠB","l"],["og","n"],["il","ar"],["Ġsh","ort"],["Ġb","ig"],["Ġc","reat"],["Ġcour","se"],["Ġf","ar"],["one","y"],["Ġind","ust"],["Ġs","w"],["oh","n"],["p","r"],["em","ber"],["os","p"],["ĠO","n"],["Ġp","ast"],["ate","ly"],["ĠO","r"],["Ġf","re"],["s","y"],["Ġocc","ur"],["Ġproble","m"],["Ġto","day"],["Ġc","ause"],["c","ing"],["Ġwee","k"],["ur","ther"],["Ġl","angu"],["it","her"],["Ġc","al"],["our","s"],["Ġ1","7"],["ĠSt","ates"],["Ġal","ong"],["o","id"],["Ġf","ull"],["urop","e"],["Ġs","ing"],["end","ing"],["o","le"],["5","0"],["a","pp"],["b","r"],["on","es"],["Ġsign","ific"],["n","ot"],["Ġare","as"],["ure","d"],["Ġyou","ng"],["v","ious"],["1","8"],["Ġwh","y"],["Ġcommun","ity"],["Ġf","ri"],["id","es"],["res","ent"],["age","ment"],["as","ter"],["Ġspec","ial"],["Ġlear","ning"],["Ġl","ater"],["Ġc","r"],["Ġf","ocus"],["Ġl","ine"],["Ġp","ress"],["Ġm","ult"],["Ġmark","et"],["Ġsu","re"],["ut","ure"],["Ġwork","ing"],["Ġg","iven"],["Ġv","ol"],["A","s"],["gg","est"],["Ġa","way"],["ĠS","he"],["p","ect"],["ol","ution"],["ap","s"],["Ġpol","it"],["Ġl","ab"],["ort","s"],["Ġto","get"],["Ġprof","ess"],["Ġapp","lic"],["Ġtoget","her"],["m","it"],["1","1"],["Ġa","ge"],["Ġcon","d"],["b","y"],["h","y"],["id","d"],["le","y"],["Ġpl","ant"],["ak","en"],["Ġk","ind"],["Ġs","ide"],["Ġrec","ord"],["Ġam","ount"],["Ġcle","ar"],["Ġwor","ks"],["Ġproble","ms"],["m","in"],["Ġle","ft"],["Ġapp","ear"],["Ġp","ay"],["Ġt","ru"],["o","x"],["od","e"],["ic","ult"],["Ġcomple","te"],["le","t"],["S","t"],["Ġm","illion"],["Ġcol","le"],["Ġ1","6"],["E","R"],["Ġsu","ggest"],["ĠL","e"],["Ġspec","ies"],["Ġtyp","e"],["Ġtreat","ment"],["ĠE","urope"],["led","ge"],["Ġob","ject"],["Ġneed","s"],["it","ing"],["Ġle","ast"],["Ġcom","b"],["ĠW","or"],["Ġsc","ient"],["Ġst","ruct"],["ore","d"],["Ġd","ue"],["Ġhe","art"],["Ġaut","hor"],["ic","ation"],["Ġw","r"],["Ġen","ough"],["m","ost"],["ress","ion"],["ust","om"],["Ġn","atural"],["us","h"],["ĠTh","at"],["âĢ","¦"],["ĠA","f"],["Ġe","y"],["Ġs","al"],["Ġha","ving"],["er","y"],["i","pp"],["Ġpre","vent"],["Ġwor","ds"],["a","ut"],["ĠN","ational"],["re","am"],["Ġcomp","ut"],["Ġcre","ate"],["is","c"],["Ġl","ove"],["a","f"],["al","f"],["Ġf","low"],["Ġass","oci"],["a","im"],["The","re"],["Ġdisc","uss"],["ap","e"],["ot","t"],["3","0"],["Ġac","ross"],["ĠG","o"],["ut","es"],["Ġd","r"],["c","le"],["iqu","e"],["Ġen","c"],["Ġstr","ong"],["s","w"],["read","y"],["st","it"],["Ġc","irc"],["ec","ially"],["ra","y"],["Ġbe","gin"],["Ġacc","ount"],["s","ide"],["l","er"],["Ġresult","s"],["Ġo","pt"],["Ġimp","act"],["ĠM","ed"],["ot","s"],["Ġph","ot"],["Ġf","uture"],["Ġsignific","ant"],["ĠJ","ohn"],["Ġlike","ly"],["Ġde","ath"],["Ġcomp","any"],["Ġab","ove"],["ive","ly"],["Ġdet","erm"],["Ġc","rit"],["Ġb","i"],["Ġst","ory"],["f","or"],["Ġem","ploy"],["Ġme","et"],["Ġpro","p"],["Ġv","ir"],["Ġlevel","s"],["ate","g"],["Ġtechn","ology"],["Ġm","akes"],["au","gh"],["Ġf","r"],["Ġan","n"],["Ġoff","ic"],["Ġh","ouse"],["ĠC","ar"],["Ġev","ent"],["Ġte","ac"],["Ġchar","acter"],["Ġ3","0"],["Ġra","d"],["Ġeduc","ation"],["ru","g"],["at","ory"],["ar","ing"],["Ġp","arent"],["Ġsp","ace"],["esp","ecially"],["Ġex","ist"],["Ġdiff","icult"],["F","or"],["Ġf","und"],["ain","ing"],["Ġke","y"],["Ġcons","um"],["en","n"],["Ġal","ready"],["as","ing"],["Ġs","ite"],["Ġfun","ction"],["sel","ves"],["Ġrequ","ire"],["at","ors"],["Ġus","ually"],["Ġhow","ever"],["ĠW","ith"],["Ġsim","ilar"],["H","ow"],["m","ber"],["Ġm","ind"],["Ġspec","ific"],["Ġincre","ase"],["Ġproduct","s"],["g","es"],["Ġc","ity"],["Ġresp","ons"],["Ġsu","m"],["ĠS","y"],["ob","al"],["Ġd","one"],["ĠM","ar"],["Ġ","er"],["Ġre","ason"],["f","ace"],["Ġ","ill"],["at","ures"],["Ġn","et"],["Ġserv","ice"],["Ġrel","ations"],["v","al"],["Ġne","cess"],["ru","ction"],["Ġpl","ac"],["il","t"],["ic","ro"],["il","ities"],["Ġdesc","rib"],["Ġvar","ious"],["he","m"],["om","es"],["Ġqu","ality"],["u","ment"],["Ġsystem","s"],["Ġt","ell"],["Ġw","in"],["Ġserv","ices"],["Ġt","erm"],["Ġst","ep"],["Ġon","line"],["Ġto","ol"],["Ġd","am"],["u","ed"],["ur","ch"],["Ġm","ot"],["our","n"],["ff","ic"],["du","ct"],["Ġquest","ion"],["1","5"],["om","an"],["Ġl","ive"],["Ġlangu","age"],["ac","y"],["Ġ","i"],["Ġp","aper"],["ĠE","n"],["ere","nce"],["A","n"],["ĠA","m"],["Ġh","y"],["Ġpol","ic"],["i","am"],["Ġwhe","ther"],["ĠB","rit"],["Ġcl","ose"],["um","p"],["Ġ","especially"],["Ġcon","cer"],["it","or"],["Ġchang","es"],["Ġhig","her"],["Ġmem","bers"],["Ġj","ob"],["Ġob","s"],["Ġw","ays"],["ach","ing"],["Ġimpro","ve"],["ys","is"],["Ġc","ame"],["Ġ1","4"],["ĠM","an"],["Ġon","ce"],["Ġbe","low"],["Ġbeh","av"],["ĠT","r"],["Ġse","en"],["Ġgener","al"],["Ġsub","ject"],["Ġm","oney"],["Ġest","ab"],["Ġcons","ider"],["ut","ions"],["ward","s"],["ĠO","ne"],["Ġcond","itions"],["u","nt"],["ĠM","ay"],["ver","age"],["Ġcell","s"],["Ġ","z"],["Ġl","a"],["ne","y"],["s","h"],["Ġdet","ail"]]}}

tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "model_max_length": 2048,
+  "padding_side": "right",
+  "truncation_side": "right",
+  "clean_up_tokenization_spaces": false,
+  "add_bos_token": false,
+  "add_eos_token": false,
+  "bos_token": "<bos>",
+  "eos_token": "<eos>",
+  "pad_token": "<pad>",
+  "unk_token": "<unk>",
+  "additional_special_tokens": [
+    "<|system|>",
+    "<|user|>",
+    "<|assistant|>",
+    "<|eot|>"
+  ]
+}