语言: 越南语
标签:

• exbert
  许可证: cc-by-nc-4.0

越南语与英语RoBERTa模型(envibert)

该RoBERTa版本使用100GB文本数据训练（越南语和英语各50GB），故命名为envibert。模型架构为生产环境定制，仅包含7000万参数。

使用方法

from transformers import RobertaModel  
from transformers.file_utils import cached_path, hf_bucket_url  
from importlib.machinery import SourceFileLoader  
import os  

cache_dir='./cache'  
model_name='nguyenvulebinh/envibert'  

def download_tokenizer_files():  
  resources = ['envibert_tokenizer.py', 'dict.txt', 'sentencepiece.bpe.model']  
  for item in resources:  
    if not os.path.exists(os.path.join(cache_dir, item)):  
      tmp_file = hf_bucket_url(model_name, filename=item)  
      tmp_file = cached_path(tmp_file,cache_dir=cache_dir)  
      os.rename(tmp_file, os.path.join(cache_dir, item))  

download_tokenizer_files()  
tokenizer = SourceFileLoader("envibert.tokenizer", os.path.join(cache_dir,'envibert_tokenizer.py')).load_module().RobertaTokenizer(cache_dir)  
model = RobertaModel.from_pretrained(model_name,cache_dir=cache_dir)  

# 文本编码  
text_input = '河内理工大学。'  
text_ids = tokenizer(text_input, return_tensors='pt').input_ids  
# 输出示例: tensor([[   0,  705,  131, 8751, 2878,  347,  477,    5,    2]])  

# 特征提取  
text_features = model(text_ids)  
text_features['last_hidden_state'].shape  
# 输出: torch.Size([1, 9, 768])  
len(text_features['hidden_states'])  
# 输出: 7  

引用文献

@inproceedings{nguyen20d_interspeech,  
  author={Thai Binh Nguyen and Quang Minh Nguyen and Thi Thu Hien Nguyen and Quoc Truong Do and Chi Mai Luong},  
  title={{Improving Vietnamese Named Entity Recognition from Speech Using Word Capitalization and Punctuation Recovery Models}},  
  year=2020,  
  booktitle={Proc. Interspeech 2020},  
  pages={4263--4267},  
  doi={10.21437/Interspeech.2020-1896}  
}  

若本资源助力发表研究成果或被集成至其他软件，请务必引用

联系方式

nguyenvulebinh@gmail.com