🚀 基因柠檬水3.0版（GENETIC LEMONADE UNLEASHED v3）

基因柠檬水3.0版是一个实验性的模型版本，基于zerofata/L3.3-GeneticLemonade-Unleashed-70B进行SFT+DPO QLora微调。该模型旨在擅长基于角色的角色扮演（RP / ERP），能够提供较长且叙事丰富的回复，准确且主动地刻画角色。

🚀 快速开始

本模型是一个实验性版本，是对 zerofata/GeneticLemonade-Unleashed 进行SFT+DPO QLora微调后的成果。它主要用于基于角色的角色扮演（RP / ERP），尚未在冒险故事或大量创意写作上进行测试和训练。该模型旨在提供较长且叙事丰富的回复，能够准确且主动地刻画角色。

✨ 主要特性

• 角色扮演专长：专为基于角色的角色扮演（RP / ERP）设计，能出色地刻画角色。
• 长叙事回复：提供较长且叙事丰富的回复内容。

📚 详细文档

🍺 SillyTavern设置

这些设置并非“最佳”设置，只是一个稳定的基线。值得注意的是，该模型支持比其他L3模型通常建议的更高温度值。

推荐采样器设置

指令设置

使用Llama-3-Instruct-Names，但需要取消勾选“系统与用户相同”。

🔢 量化版本

GGUF

• iMatrix (Mradermacher)

EXL2

• 4bpw
• 4.5bpw
• 6bpw

🏋️‍ 训练过程

• SFT阶段：模型首先使用一个包含290万个标记（约750个对话）的小型合成数据集进行监督微调（SFT）。该数据集主要是角色扮演数据，包含少量随机指令/助手数据和创意写作内容。
• DPO阶段：然后，模型使用从SFT数据集中挑选出的约1100个高质量或能证明遵循指令的示例进行直接偏好优化（DPO）训练。被拒绝的样本是使用另一个以不擅长遵循指令而闻名的Llama 3.3微调模型生成的。

Axolotl配置

以下是SFT和DPO阶段的Axolotl配置文件，这些配置未针对成本/性能效率进行优化，实际效果可能因人而异。

SFT 1*H200

# ====================
# MODEL CONFIGURATION
# ====================
base_model: zerofata/L3.3-GeneticLemonade-Unleashed-70B
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
special_tokens:
  pad_token: "<|finetune_right_pad_id|>"
chat_template: llama3

# ====================
# DATASET CONFIGURATION
# ====================
datasets:
  - path: ./dataset.jsonl
    type: chat_template
    split: train
    chat_template_strategy: tokenizer
    field_messages: messages
    message_property_mappings:
      role: role
      content: content
    roles:
      user: ["user"]
      assistant: ["assistant"]
      system: ["system"]

test_datasets:
  - path: ./validate_dataset.jsonl
    type: chat_template
    split: train
    chat_template_strategy: tokenizer
    field_messages: messages
    message_property_mappings:
      role: role
      content: content
    roles:
      user: ["user"]
      assistant: ["assistant"]
      system: ["system"]

dataset_prepared_path:
train_on_inputs: false  # Only train on assistant responses

# ====================
# QLORA CONFIGURATION
# ====================
adapter: qlora
load_in_4bit: true
lora_r: 64
lora_alpha: 128
lora_dropout: 0.1
lora_target_linear: true
# lora_modules_to_save:  # Uncomment only if you added NEW tokens

# ====================
# TRAINING PARAMETERS
# ====================
num_epochs: 2
micro_batch_size: 4
gradient_accumulation_steps: 2
learning_rate: 1.5e-5
optimizer: paged_adamw_8bit
lr_scheduler: rex
warmup_ratio: 0.05
weight_decay: 0.01
max_grad_norm: 1.0

# ====================
# SEQUENCE & PACKING
# ====================
sequence_len: 8192
sample_packing: true
eval_sample_packing: false
pad_to_sequence_len: true

# ====================
# HARDWARE OPTIMIZATIONS
# ====================
bf16: auto
flash_attention: true
gradient_checkpointing: true

# ====================
# EVALUATION & CHECKPOINTING
# ====================
evaluation_strategy: steps
eval_steps: 5
save_strategy: steps
save_steps: 5
save_total_limit: 5  # Keep best + last few checkpoints
load_best_model_at_end: true
metric_for_best_model: eval_loss
greater_is_better: false
early_stopping_patience: 5

# ====================
# LOGGING & OUTPUT
# ====================
output_dir: ./output_model
logging_steps: 2
save_safetensors: true

# ====================
# WANDB TRACKING
# ====================
wandb_project: project_name
# wandb_entity: your_entity
# wandb_name: your_run_name

DPO 2*H200

# ====================
# MODEL CONFIGURATION
# ====================
base_model: ApocalypseParty/unleashed-fulldata30
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
special_tokens: {}
chat_template: tokenizer_default

# ====================
# RL/DPO CONFIGURATION
# ====================
rl: dpo
rl_beta: 0.07

# ====================
# DATASET CONFIGURATION
# ====================
datasets:
  - path: ./dpo_cleaned-v3_deduplicated.jsonl
    type: chat_template.default
    field_messages: conversation
    field_chosen: chosen
    field_rejected: rejected
    message_property_mappings:
      role: role
      content: content
    roles:
      system: ["system"]
      user: ["user"]
      assistant: ["assistant"]
dataset_prepared_path:
train_on_inputs: false  # Only train on assistant responses

# ====================
# QLORA CONFIGURATION
# ====================
adapter: qlora
load_in_4bit: true
lora_r: 32
lora_alpha: 64
lora_dropout: 0.05
lora_target_linear: true
# lora_modules_to_save:  # Uncomment only if you added NEW tokens

# ====================
# TRAINING PARAMETERS
# ====================
num_epochs: 1
micro_batch_size: 4
gradient_accumulation_steps: 2
learning_rate: 2e-6
optimizer: adamw_8bit
lr_scheduler: cosine
warmup_steps: 5
weight_decay: 0.01
max_grad_norm: 1.0

# ====================
# SEQUENCE CONFIGURATION
# ====================
sequence_len: 4096
pad_to_sequence_len: true

# ====================
# HARDWARE OPTIMIZATIONS
# ====================
bf16: auto
tf32: false
flash_attention: true
gradient_checkpointing: offload
deepspeed: deepspeed_configs/zero1.json

# ====================
# CHECKPOINTING
# ====================
save_steps: 10
save_total_limit: 10
load_best_model_at_end: true
metric_for_best_model: eval_loss
greater_is_better: false

# ====================
# LOGGING & OUTPUT
# ====================
output_dir: ./dpo_model
logging_steps: 2
save_safetensors: true

# ====================
# WANDB TRACKING
# ====================
wandb_project: project_name
# wandb_entity: your_entity
# wandb_name: your_run_name

📄 许可证

本模型使用llama3许可证。