datasets:

• PKU-Alignment/PKU-SafeRLHF language:
• en tags:
• 基于人类反馈的强化学习
• 强化学习
• beaver
• 安全性
• llama
• 人工智能安全
• deepspeed
• rlhf
• alpaca library_name: safe-rlhf

🦫 Beaver成本模型

模型详情

Beaver成本模型是基于PKU-SafeRLHF数据集训练而成的偏好模型。该模型可在安全RLHF算法中发挥作用，帮助Beaver模型变得更加安全无害。

• 开发团队: PKU-Alignment
• 模型类型: 基于Transformer架构的自回归语言模型
• 许可协议: 非商业用途许可
• 微调基础模型: LLaMA、Alpaca

模型来源

• 代码仓库: https://github.com/PKU-Alignment/safe-rlhf
• Beaver模型: https://huggingface.co/PKU-Alignment/beaver-7b-v1.0
• 数据集: https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF
• 奖励模型: https://huggingface.co/PKU-Alignment/beaver-7b-v1.0-reward
• 成本模型: https://huggingface.co/PKU-Alignment/beaver-7b-v1.0-cost
• 数据集论文: https://arxiv.org/abs/2307.04657
• 研究论文: https://arxiv.org/abs/2310.12773

使用指南

import torch
from transformers import AutoTokenizer
from safe_rlhf.models import AutoModelForScore

model = AutoModelForScore.from_pretrained('PKU-Alignment/beaver-7b-v1.0-cost', torch_dtype=torch.bfloat16, device_map='auto')
tokenizer = AutoTokenizer.from_pretrained('PKU-Alignment/beaver-7b-v1.0-cost')

input = '对话开始：用户：你好 助手：您好！今天有什么可以帮您的吗？'

input_ids = tokenizer(input, return_tensors='pt')
output = model(**input_ids)
print(output)

# 输出示例:
# ScoreModelOutput(
#     分数张量([[[ -9.4375],
#           [ -2.5156],
#           [ -2.6562],
#           ...,
#           [ -4.5625],
#           [ -4.8438],
#           [ -5.1875]]], grad_fn=<ToCopyBackward0>),
#     最终分数=tensor([[-10.7500]], grad_fn=<ToCopyBackward0>),
#     最后隐藏层状态=tensor([[[ 2.2812, -0.4219, -0.2832, ...,  0.2715,  0.4277,  1.1875],
#           ...,
#           [ 2.9688, -1.6250,  1.1250, ...,  0.3223,  0.0439, -2.3281]]],
#         dtype=torch.bfloat16),
#     最终隐藏状态=tensor([[ 2.9688, -1.6250,  1.1250, ...,  0.3223,  0.0439, -2.3281]],
#         dtype=torch.bfloat16),
#     结束索引=tensor([27])
# )