language: zh

prophetnet-large-uncased

ProphetNet的预训练权重。
ProphetNet是一种新型的序列到序列学习预训练语言模型，采用名为"未来n元预测"的创新自监督目标。
该模型通过n流解码器能够预测更多未来标记。原始实现为Fairseq版本，详见GitHub代码库。

使用方式

该预训练模型可针对序列到序列任务进行微调。例如可按以下方式训练标题生成任务：

from transformers import ProphetNetForConditionalGeneration, ProphetNetTokenizer

model = ProphetNetForConditionalGeneration.from_pretrained("microsoft/prophetnet-large-uncased")
tokenizer = ProphetNetTokenizer.from_pretrained("microsoft/prophetnet-large-uncased")

input_str = "美国国务院周三表示尚未收到玻利维亚关于驱逐美驻玻大使的正式通知，但指出对其指控纯属「无稽之谈」"
target_str = "美国驳回对其驻玻利维亚大使的指控"

input_ids = tokenizer(input_str, return_tensors="pt").input_ids
labels = tokenizer(target_str, return_tensors="pt").input_ids

loss = model(input_ids, labels=labels).loss

引用文献

@article{yan2020prophetnet,
  title={Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training},
  author={Yan, Yu and Qi, Weizhen and Gong, Yeyun and Liu, Dayiheng and Duan, Nan and Chen, Jiusheng and Zhang, Ruofei and Zhou, Ming},
  journal={arXiv preprint arXiv:2001.04063},
  year={2020}
}