许可证：apache-2.0
arXiv：2001.00059
流程标签：填充掩码
标签：

• 代码
• cubert

CuBERT：源代码上下文嵌入的学习与评估

概述

本模型是"CuBERT"的非官方HuggingFace版本。具体而言，此版本源自gs://cubert/20210711_Python/pre_trained_model_epochs_2__length_1024。该模型于2021年7月11日基于Python BigQuery数据集训练了2个周期，上下文窗口长度为1024个标记。我们手动将TensorFlow检查点转换为PyTorch格式并上传至此。分词器尚未完成转换。所有荣誉归属于Aditya Kanade、Petros Maniatis、Gogul Balakrishnan和Kensen Shi。

其他版本可在此获取：

cubert-20210711-Python-512

cubert-20210711-Python-1024

cubert-20210711-Python-2048

cubert-20210711-Java-512

cubert-20210711-Java-1024

cubert-20210711-Java-2048

引用：

@inproceedings{cubert,  
author    = {Aditya Kanade and  
             Petros Maniatis and  
             Gogul Balakrishnan and  
             Kensen Shi},  
title     = {Learning and evaluating contextual embedding of source code},  
booktitle = {Proceedings of the 37th International Conference on Machine Learning,  
               {ICML} 2020, 12-18 July 2020},  
series    = {Proceedings of Machine Learning Research},  
publisher = {{PMLR}},  
year      = {2020},  
}