license: apache-2.0
datasets:
- MoritzLaurer/synthetic_zeroshot_mixtral_v0.1
language:
- en
metrics:
- f1
pipeline_tag: zero-shot-classification
tags:
- 文本分类
- 零样本学习
- 小型语言模型
- 检索增强生成
- 情感分析
⭐ GLiClass:通用轻量级序列分类模型
这是一款受GLiNER启发的高效零样本分类器。它在保持与交叉编码器同等性能的同时,通过单次前向传播完成分类,具有更高的计算效率。
该模型适用于主题分类
、情感分析
,并可作为RAG
流程中的重排序器使用。基于合成数据训练,可应用于商业场景。
使用方法:
首先安装GLiClass库:
pip install gliclass
初始化模型与流水线:
from gliclass import GLiClassModel, ZeroShotClassificationPipeline
from transformers import AutoTokenizer
model = GLiClassModel.from_pretrained("knowledgator/gliclass-small-v1.0")
tokenizer = AutoTokenizer.from_pretrained("knowledgator/gliclass-small-v1.0")
pipeline = ZeroShotClassificationPipeline(model, tokenizer, classification_type='multi-label', device='cuda:0')
text = "总有一天我要环游世界!"
labels = ["旅行", "梦想", "运动", "科学", "政治"]
results = pipeline(text, labels, threshold=0.5)[0]
for result in results:
print(result["label"], "=>", result["score"])
性能基准:
以下为多个文本分类数据集上的F1分数(零样本场景测试):
GLiClass系列模型对比:
数据集 |
轻量版-small |
轻量版-base |
轻量版-large |
标准版-small |
标准版-base |
标准版-large |
CR |
0.8886 |
0.9097 |
0.9226 |
0.8824 |
0.8942 |
0.9219 |
sst2 |
0.8392 |
0.8987 |
0.9247 |
0.8518 |
0.8979 |
0.9269 |
sst5 |
0.2865 |
0.3779 |
0.2891 |
0.2424 |
0.2789 |
0.3900 |
20新闻组 |
0.4572 |
0.3953 |
0.4083 |
0.3366 |
0.3576 |
0.3863 |
垃圾邮件 |
0.5118 |
0.5126 |
0.3642 |
0.4089 |
0.4938 |
0.3661 |
烂番茄影评 |
0.8015 |
0.8429 |
0.8807 |
0.7987 |
0.8508 |
0.8808 |
MASSIVE多语言数据集 |
0.3180 |
0.4635 |
0.5606 |
0.2546 |
0.1893 |
0.4376 |
银行客服 |
0.1768 |
0.4396 |
0.3317 |
0.1374 |
0.2077 |
0.2847 |
雅虎主题 |
0.4686 |
0.4784 |
0.4760 |
0.4477 |
0.4516 |
0.4921 |
金融短语库 |
0.8665 |
0.8880 |
0.9044 |
0.8901 |
0.8955 |
0.8735 |
IMDB影评 |
0.9048 |
0.9351 |
0.9429 |
0.8982 |
0.9238 |
0.9333 |
AG新闻 |
0.7252 |
0.6985 |
0.7559 |
0.7242 |
0.6848 |
0.7503 |
推特情绪 |
0.4012 |
0.3516 |
0.3951 |
0.3450 |
0.2357 |
0.4013 |
客服对话 |
0.3794 |
0.4643 |
0.4749 |
0.3432 |
0.4375 |
0.4644 |
平均分: |
0.5732 |
0.6183 |
0.6165 |
0.5401 |
0.5571 |
0.6078 |
示例数量对性能的影响:
模型 |
示例数量 |
sst5 |
垃圾邮件 |
MASSIVE |
银行客服 |
AG新闻 |
推特情绪 |
客服对话 |
平均分 |
轻量版-small |
0 |
0.2865 |
0.5118 |
0.318 |
0.1768 |
0.7252 |
0.4012 |
0.3794 |
0.3998 |
轻量版-base |
0 |
0.3779 |
0.5126 |
0.4635 |
0.4396 |
0.6985 |
0.3516 |
0.4643 |
0.4726 |
轻量版-large |
0 |
0.2891 |
0.3642 |
0.5606 |
0.3317 |
0.7559 |
0.3951 |
0.4749 |
0.4531 |
标准版-small |
0 |
0.2424 |
0.4089 |
0.2546 |
0.1374 |
0.7242 |
0.345 |
0.3432 |
0.3508 |
标准版-base |
0 |
0.2789 |
0.4938 |
0.1893 |
0.2077 |
0.6848 |
0.2357 |
0.4375 |
0.3611 |
标准版-large |
0 |
0.39 |
0.3661 |
0.4376 |
0.2847 |
0.7503 |
0.4013 |
0.4644 |
0.4421 |
轻量版-small |
8 |
0.2709 |
0.84026 |
0.62 |
0.6883 |
0.7786 |
0.449 |
0.4918 |
0.5913 |
轻量版-base |
8 |
0.4275 |
0.8836 |
0.729 |
0.7667 |
0.7968 |
0.3866 |
0.4858 |
0.6394 |
轻量版-large |
8 |
0.3345 |
0.8997 |
0.7658 |
0.848 |
0.84843 |
0.5219 |
0.508 |
0.6752 |
标准版-small |
8 |
0.3042 |
0.5683 |
0.6332 |
0.7072 |
0.759 |
0.4509 |
0.4434 |
0.5523 |
标准版-base |
8 |
0.3387 |
0.7361 |
0.7059 |
0.7456 |
0.7896 |
0.4323 |
0.4802 |
0.6041 |
标准版-large |
8 |
0.4365 |
0.9018 |
0.77 |
0.8533 |
0.8509 |
0.5061 |
0.4935 |
0.6874 |