毒性语言检测模型

模型描述

这是一个基于推文数据训练的毒性语言检测模型，基础模型采用Roberta-large架构。如需了解训练数据、模型局限性与偏差等详细信息，请参阅相关论文及GitHub代码库。

使用方法

注意：输出结果中LABEL_1表示有毒内容，LABEL_0表示非有毒内容。

from transformers import pipeline
classifier = pipeline("text-classification",model='Xuhui/ToxDect-roberta-large', return_all_scores=True)
prediction = classifier("You are f**king stupid!", )
print(prediction)

"""
输出示例:
[[{'label': 'LABEL_0', 'score': 0.002632011892274022}, {'label': 'LABEL_1', 'score': 0.9973680377006531}]]
"""

训练过程

本模型采用的随机种子数为22。其他技术细节请参见GitHub代码库。

文献引用信息

@inproceedings{zhou-etal-2020-debiasing,
  title = {Challenges in Automated Debiasing for Toxic Language Detection},
  author = {Zhou, Xuhui and Sap, Maarten and Swayamdipta, Swabha and Choi, Yejin and Smith, Noah A.},
  booktitle = {EACL},
  abbr = {EACL},
  html = {https://www.aclweb.org/anthology/2021.eacl-main.274.pdf},
  code = {https://github.com/XuhuiZhou/Toxic_Debias},
  year = {2021},
  bibtex_show = {true},
  selected = {true}
}