license: apache-2.0
arxiv: 2001.00059
pipeline_tag: fill-mask
tags:

• code
• cubert

CuBERT：源代码上下文嵌入的学习与评估

概述

本模型是"CuBERT"的非官方HuggingFace版本。具体而言，该版本源自gs://cubert/20210711_Python/pre_trained_model_epochs_2__length_512。该模型于2021年7月11日在Python BigQuery数据集上训练了2个epoch，上下文窗口长度为512个token。我们手动将TensorFlow检查点转换为PyTorch格式并上传至此。分词器尚未完成转换。所有荣誉归属于Aditya Kanade、Petros Maniatis、Gogul Balakrishnan和Kensen Shi。

其他版本可在此获取：
cubert-20210711-Python-512
cubert-20210711-Python-1024
cubert-20210711-Python-2048
cubert-20210711-Java-512
cubert-20210711-Java-1024
cubert-20210711-Java-2048

引用文献：

@inproceedings{cubert,
author    = {Aditya Kanade and
             Petros Maniatis and
             Gogul Balakrishnan and
             Kensen Shi},
title     = {Learning and evaluating contextual embedding of source code},
booktitle = {Proceedings of the 37th International Conference on Machine Learning,
               {ICML} 2020, 12-18 July 2020},
series    = {Proceedings of Machine Learning Research},
publisher = {{PMLR}},
year      = {2020},
}