许可证：apache-2.0
标签：

• 混合专家模型
• 模型融合
• mergekit工具
• lazymergekit工具
• Felladrin/TinyMistral-248M-SFT-v4
• Locutusque/LocutusqueXFelladrin-TinyMistral248M-Instruct
• Locutusque/TinyMistral-248M-v2-Instruct
• Locutusque/TinyMistral-248M-v2.5

🌟 请我喝咖啡是直接支持本项目的方式。

TinyMixtral-4x248M-MoE

TinyMixtral-4x248M-MoE 是一个采用混合专家（MoE）架构的模型，通过LazyMergekit融合以下模型而成：

• Felladrin/TinyMistral-248M-SFT-v4
• Locutusque/LocutusqueXFelladrin-TinyMistral248M-Instruct
• Locutusque/TinyMistral-248M-v2-Instruct
• Locutusque/TinyMistral-248M-v2.5

💻 使用方法

!pip install -qU transformers bitsandbytes accelerate  

from transformers import AutoTokenizer  
import transformers  
import torch  

model = "Isotonic/TinyMixtral-4x248M-MoE"  

tokenizer = AutoTokenizer.from_pretrained(model)  
pipeline = transformers.pipeline(  
    "text-generation",  
    model=model,  
    tokenizer=tokenizer  
)  

messages = [{"role": "user", "content": "用不到100字解释什么是混合专家模型。"}]  
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)  
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)  
print(outputs[0]["generated_text"])