license: apache-2.0 language:

• 英文
• 日文 base_model:
• Rakuten/RakutenAI-7B

RakutenAI-7B-chat

模型描述

RakutenAI-7B是一项系统性创新，旨在为日语大语言模型领域引入最新技术。该模型在日语理解基准测试中取得最佳成绩，同时在英语测试集上保持与OpenCalm、Elyza、Youri、Nekomata和Swallow等同类模型相当的竞争力。RakutenAI-7B采用Mistral模型架构，基于Mistral-7B-v0.1预训练检查点，成功实现了预训练模型权重的改造应用。此外，我们将Mistral的词汇表从32k扩展到48k，显著提升了日文的字符-标记比。

技术报告详见arXiv。

如需基础模型，请访问RakutenAI-7B。

如需指令调优模型，请访问RakutenAI-7B-instruct。

根据Kamata等人在Nejumi LLMリーダーボード Neo上的独立评估（采用llm-jp-eval与Japanese MT-bench的加权平均），截至2024年3月22日，RakutenAI-7B的chat/instruct版本在同类开源模型中表现最优，得分分别为0.393/0.331。

使用方式

# 使用RakutenAI-7B-Chat定制对话模板

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "Rakuten/RakutenAI-7B-chat"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype="auto", device_map="auto")
model.eval()

chat = [
	
	{"role": "system", "content": "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions."},
	{"role": "user", "content": "如何制作正宗的西班牙煎蛋饼？"},
]

input_ids = tokenizer.apply_chat_template(chat, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(device=model.device)
tokens = model.generate(
    input_ids,
    max_length=4096,
    do_sample=False,
    num_beams=1,
    pad_token_id=tokenizer.eos_token_id,
)
out = tokenizer.decode(tokens[0][len(input_ids[0]):], skip_special_tokens=True)
print("助手回答:\n" + out)
print()


# 不使用定制对话模板

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "Rakuten/RakutenAI-7B-chat"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype="auto", device_map="auto")
model.eval()

requests = [
    "「馬が合う」是什么意思？",
    "How to make an authentic Spanish Omelette?",
]

system_message = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. 用户提问: {user_input} 助手回答:"

for req in requests:
    input_req = system_message.format(user_input=req)
    input_ids = tokenizer.encode(input_req, return_tensors="pt").to(device=model.device)
    tokens = model.generate(
        input_ids,
        max_new_tokens=1024,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id,
    )
    out = tokenizer.decode(tokens[0][len(input_ids[0]):], skip_special_tokens=True)
    print("用户提问:\n" + req)
    print("助手回答:\n" + out)
    print()
    print()

模型详情

• 开发机构: 乐天集团
• 支持语言: 日语、英语
• 许可证: 本模型采用Apache 2.0许可证
• 指令调优数据集: 我们使用开源与内部手工制作的数据集混合对基础模型进行微调，创建了RakutenAI-7B-instruct和RakutenAI-7B-chat。指令调优与对话调优模型采用了以下数据集（CC by-SA许可）的train部分：
  • JSNLI
  • RTE
  • KUCI
  • BELEBELE
  • JCS
  • JNLI
  • Dolly-15K
  • OpenAssistant1

局限性及偏差

RakutenAI-7B系列模型虽然能生成涵盖广泛主题的类人文本，但与所有大语言模型一样存在局限性，可能产生带有偏见、不准确或不安全的输出。使用时请保持谨慎判断。

引用

如需引用RakutenAI-7B系列模型，请使用：

@misc{rakutengroup2024rakutenai7b,
      title={RakutenAI-7B: Extending Large Language Models for Japanese}, 
      author={{Rakuten Group, Inc.} and Aaron Levine and Connie Huang and Chenguang Wang and Eduardo Batista and Ewa Szymanska and Hongyi Ding and Hou Wei Chou and Jean-François Pessiot and Johanes Effendi and Justin Chiu and Kai Torben Ohlhus and Karan Chopra and Keiji Shinzato and Koji Murakami and Lee Xiong and Lei Chen and Maki Kubota and Maksim Tkachenko and Miroku Lee and Naoki Takahashi and Prathyusha Jwalapuram and Ryutaro Tatsushima and Saurabh Jain and Sunil Kumar Yadav and Ting Cai and Wei-Te Chen and Yandi Xia and Yuki Nakayama and Yutaka Higashiyama},
      year={2024},
      eprint={2403.15484},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}