Hugging Face
Models
Datasets
Spaces
Buckets
new
Docs
Enterprise
Pricing
Website
Tasks
HuggingChat
Collections
Languages
Organizations
Community
Blog
Posts
Daily Papers
Learn
Discord
Forum
GitHub
Solutions
Team & Enterprise
Hugging Face PRO
Enterprise Support
Inference Providers
Inference Endpoints
Storage Buckets
Log In
Sign Up
thunlp
/
SubCharTokenization
like
1
Model card
Files
Files and versions
xet
Community
Copy to bucket
new
main
SubCharTokenization
9.84 GB
Ctrl+K
Ctrl+K
1 contributor
History:
5 commits
This model has 1 file scanned as unsafe.
Show
files
chen-yingfa
Add Char model
4871469
over 4 years ago
.gitattributes
Safe
1.18 kB
initial commit
over 4 years ago
cangjie.pt
pickle
Detected Pickle imports (3)
"collections.OrderedDict"
,
"torch._utils._rebuild_tensor_v2"
,
"torch.FloatStorage"
What is a pickle import?
1.05 GB
xet
Add more tokenization methods
over 4 years ago
char.pt
pickle
Detected Pickle imports (3)
"torch.FloatStorage"
,
"torch._utils._rebuild_tensor_v2"
,
"collections.OrderedDict"
What is a pickle import?
1.05 GB
xet
Add Char model
over 4 years ago
pinyin.pt
pickle
Detected Pickle imports (3)
"collections.OrderedDict"
,
"torch._utils._rebuild_tensor_v2"
,
"torch.FloatStorage"
What is a pickle import?
1.05 GB
xet
Add pinyin
over 4 years ago
pinyin_no_index.pt
pickle
Detected Pickle imports (3)
"torch._utils._rebuild_tensor_v2"
,
"torch.FloatStorage"
,
"collections.OrderedDict"
What is a pickle import?
1.05 GB
xet
Add wubi and no index model
over 4 years ago
stroke.pt
pickle
Detected Pickle imports (3)
"torch._utils._rebuild_tensor_v2"
,
"collections.OrderedDict"
,
"torch.FloatStorage"
What is a pickle import?
1.05 GB
xet
Add more tokenization methods
over 4 years ago
wubi.pt
pickle
Detected Pickle imports (3)
"torch.FloatStorage"
,
"collections.OrderedDict"
,
"torch._utils._rebuild_tensor_v2"
What is a pickle import?
1.05 GB
xet
Add wubi and no index model
over 4 years ago
wubi_no_index.pt
pickle
Detected Pickle imports (3)
"torch._utils._rebuild_tensor_v2"
,
"torch.FloatStorage"
,
"collections.OrderedDict"
What is a pickle import?
1.05 GB
xet
Add wubi and no index model
over 4 years ago
zhengma.pt
pickle
Detected Pickle imports (3)
"torch._utils._rebuild_tensor_v2"
,
"torch.FloatStorage"
,
"collections.OrderedDict"
What is a pickle import?
1.05 GB
xet
Add more tokenization methods
over 4 years ago
zhuyin.pt
Unsafe
pickle
Detected Pickle imports (13)
"torch._utils._rebuild_tensor_v2"
,
"torch.utils.data.dataloader.DataLoader"
,
"numpy.core.multiarray._reconstruct"
,
"torch.FloatStorage"
,
"collections.OrderedDict"
,
"__main__.WorkerInitObj"
,
"numpy.dtype"
,
"__main__.pretraining_dataset"
,
"numpy.ndarray"
,
"torch.utils.data.sampler.BatchSampler"
,
"torch.utils.data.sampler.RandomSampler"
,
"torch.utils.data._utils.collate.default_collate"
,
"_codecs.encode"
How to fix it?
1.44 GB
xet
Add more tokenization methods
over 4 years ago
