Thai
tokenizers
thai
tokenizer
nlp
subword
Thaitokenizer / special_tokens_map.json

Commit History

  1. Add tokenizer configuration file with special tokens and settings

    Nattapong Tapachoom
    12f0d60