fastText skip-gram, Spanish Wikipedia, character n-grams 1-5 (dim 300)
Spanish fastText model used to embed syllables in
M. Zhang, J. Lévy, S. d'Ascoli, J. Rapin, F.-X. Alario, P. Bourdillon, S. Pinet, J.-R. King. A hierarchy of dynamic neural codes supports successive linguistic neural representations and coordinates language production in the human brain.
It differs from the official fastText Spanish vectors by using character n-grams down to
length 1 (-minn 1), so that short syllables get informative embeddings.
Files
| file | size | sha256 |
|---|---|---|
wiki_all_minn1_spanish.bin |
7,199,985,976 bytes | 79051c0a7be8262e26553632ab8d4b9e8707c0edb1eb03c0a9275ea13a9f0ac9 |
Usage
import fasttext
model = fasttext.load_model("wiki_all_minn1_spanish.bin")
vec = model.get_word_vector("pa") # 300-d
In the study's code, scripts/download_fasttext.sh places it at
scr/model_test/wiki_all_minn1_spanish.bin.
Training
Text: a Spanish Wikipedia dump (eswiki-latest-pages-articles.xml.bz2; the exact dump date
was not recorded), converted to plain text, one sentence per line, with
wp2txt:
wp2txt --no-convert -i ./eswiki-latest-pages-articles.xml.bz2 -o ./xml
wp2txt -s -i ./xml -o ./text
cat ./text/* > all.txt
./fasttext skipgram -input all.txt -output wiki_all_minn1 -minn 1 -maxn 5 -dim 300
All other parameters are fastText defaults.
License
CC BY-SA 4.0, like the Wikipedia text it was trained on.
- Downloads last month
- -
from huggingface_hub import hf_hub_download import fasttext model = fasttext.load_model(hf_hub_download("Lucyzmf/pinet-fasttext-es", "model.bin"))