缩略图: https://github.com/rinnakk/japanese-pretrained-models/blob/master/rinna.png 许可证: apache-2.0 语言:

• 日语 标签:
• qwen2
• 对话式 基础模型:
• Qwen/Qwen2.5-32B
• Qwen/QwQ-32B
• rinna/qwen2.5-bakeneko-32b 基础模型关系: 合并 管道标签: 文本生成 库名称: transformers

QwQ Bakeneko 32B (rinna/qwq-bakeneko-32b)

概述

该模型是基于rinna/qwen2.5-bakeneko-32b的指令调优推理变体，通过Chat Vector和Odds Ratio Preference Optimization (ORPO)进行微调。
它遵循Qwen/QwQ-32B的聊天格式，旨在为日语任务提供卓越性能。

• 模型架构

  64层、5120隐藏大小的基于Transformer的语言模型。
  完整架构说明请参阅Qwen2.5技术报告。

• 训练过程

  本模型通过多阶段训练开发：

  模型合并。基础模型rinna/qwen2.5-bakeneko-32b通过Chat Vector加法过程增强了指令跟随能力。
  Chat Vector通过从Qwen/QwQ-32B减去Qwen/Qwen2.5-32B的参数向量获得，如下所示：

  rinna/qwen2.5-bakeneko-32b + 0.8 * (Qwen/QwQ-32B - Qwen/Qwen2.5-32B)
  

  在此过程中，进行参数向量加减时省略了嵌入层。

  蒸馏与ORPO。合并后的模型使用ORPO进一步优化，训练数据为DeepSeek-R1生成的1.3k精选样本。

• 贡献者

  • Xinqi Chen
  • Toshiaki Wakatsuki
  • Kei Sawada

• 发布日期

  2025年3月13日

基准测试

详细基准测试结果请参考rinna的LM基准页面(20250313表格)。

模型使用方法

环境配置

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "rinna/qwq-bakeneko-32b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

messages = [
    {"role": "user", "content": "请创建一个关于微积分的简单文字题，并解答该问题。"},
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
input_ids = tokenizer.encode(
    prompt,
    add_special_tokens=False,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    input_ids,
    max_new_tokens=4096,
    do_sample=True,
    temperature=0.6,
    top_k=40,
    top_p=0.95,
)

response = tokenizer.decode(outputs[0][input_ids.shape[-1]:], skip_special_tokens=True)
response = "<think>\n" + response
print(response)

使用建议

为获得最佳性能，建议在部署前查阅使用指南。

分词说明

本模型继承了原始Qwen/QwQ-32B的分词器。

引用方式

@misc{rinna/qwq-bakeneko-32b
    title = {rinna/qwq-bakeneko-32b},
    author = {Chen, Xinqi and Wakatsuki, Toshiaki and Sawada, Kei},
    url = {https://huggingface.co/rinna/qwq-bakeneko-32b}
}

@inproceedings{sawada2024release,
    title = {Release of Pre-Trained Models for the {J}apanese Language},
    author = {Sawada, Kei and Zhao, Tianyu and Shing, Makoto and Mitsui, Kentaro and Kaga, Akio and Hono, Yukiya and Wakatsuki, Toshiaki and Mitsuda, Koh},
    booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
    month = {5},
    year = {2024},
    pages = {13898--13905},
    url = {https://aclanthology.org/2024.lrec-main.1213},
    note = {\url{https://arxiv.org/abs/2404.01657}}
}

参考文献

@article{qwen2.5,
    title = {Qwen2.5技术报告}, 
    author = {An Yang and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chengyuan Li and Dayiheng Liu and Fei Huang and Haoran Wei and Huan Lin and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Yang and Jiaxi Yang and Jingren Zhou and Junyang Lin and Kai Dang and Keming Lu and Keqin Bao and Kexin Yang and Le Yu and Mei Li and Mingfeng Xue and Pei Zhang and Qin Zhu and Rui Men and Runji Lin and Tianhao Li and Tianyi Tang and Tingyu Xia and Xingzhang Ren and Xuancheng Ren and Yang Fan and Yang Su and Yichang Zhang and Yu Wan and Yuqiong Liu and Zeyu Cui and Zhenru Zhang and Zihan Qiu},
    journal = {arXiv预印本 arXiv:2412.15115},
    year = {2024}
}

@misc{qwq32b,
    title = {QwQ-32B: 拥抱强化学习的力量},
    url = {https://qwenlm.github.io/blog/qwq-32b/},
    author = {Qwen团队},
    month = {3月},
    year = {2025}
}

@misc{deepseekai2025deepseekr1incentivizingreasoningcapability,
    title = {DeepSeek-R1: 通过强化学习激励LLM的推理能力}, 
    author = {DeepSeek