🚀 IndoBERTweet-HateSpeech

IndoBERTweet-HateSpeech是一个在IndoToxic2024数据集上微调的模型，可用于识别印尼语中的仇恨言论。该模型具有较高的准确性和F1分数，能有效助力相关领域的研究和应用。

🚀 快速开始

你可以按照以下步骤使用该模型：

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Specify the model and tokenizer name
model_name = "Exqrch/IndoBERTweet-HateSpeech"
tokenizer_name = "indolem/indobertweet-base-uncased"

# Load the pre-trained model
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(tokenizer_name)

text = "selamat pagi semua!"

output = model(**tokenizer(text, return_tensors="pt"))
logits = output.logits

# Get the predicted class label
predicted_class = torch.argmax(logits, dim=-1).item()

print(predicted_class)
--- Output ---
> 0
--- End of Output ---

✨ 主要特性

• 高性能：IndoBERTweet在IndoToxic2024数据集上进行了微调，准确率达到0.89，宏F1分数为0.78。这些性能指标是通过分层10折交叉验证获得的。
• 支持的分词器：支持使用indolem/indobertweet-base-uncased分词器。

💻 使用示例

基础用法

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Specify the model and tokenizer name
model_name = "Exqrch/IndoBERTweet-HateSpeech"
tokenizer_name = "indolem/indobertweet-base-uncased"

# Load the pre-trained model
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(tokenizer_name)

text = "selamat pagi semua!"

output = model(**tokenizer(text, return_tensors="pt"))
logits = output.logits

# Get the predicted class label
predicted_class = torch.argmax(logits, dim=-1).item()

print(predicted_class)

示例输出

Model name: Exqrch/IndoBERTweet-HateSpeech
Text 1: Kenapa sih mereka berantem terus?
Prediction: 0
Text 2: Orang gila emang elu!
Prediction: 1

📚 详细文档

局限性

• 该模型仅在印尼语文本上进行了训练，对于代码混合文本（code-switched text）的性能没有相关信息。

引用方式

如果使用该模型，请引用以下论文：

@article{susanto2024indotoxic2024,
      title={IndoToxic2024: A Demographically-Enriched Dataset of Hate Speech and Toxicity Types for Indonesian Language}, 
      author={Lucky Susanto and Musa Izzanardi Wijanarko and Prasetia Anugrah Pratama and Traci Hong and Ika Idris and Alham Fikri Aji and Derry Wijaya},
      year={2024},
      eprint={2406.19349},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2406.19349}, 
}

📄 许可证

本项目采用CC BY-SA 4.0许可证。