library_name: transformers language:

• de license: mit base_model: openai/whisper-large-v3-turbo tags:
• generated_from_trainer pipeline_tag: automatic-speech-recognition

German-RAG-WHISPER-LARGE-v3-TURBO-HESSIAN-AI

该模型基于精心筛选的13小时数据集进行微调。

评估指标 - 词错误率

评估数据与代码详见此处

训练数据

本模型训练数据包含德语会话及英语商务短语混合内容，经过严格筛选和处理以优化识别性能。因数据可能涉及语音克隆风险，暂不公开数据集。所收集数据仅限用于训练语音转文本模型。

使用方式

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
from datasets import load_dataset
device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
model_id = "avemio/German-RAG-WHISPER-LARGE-v3-TURBO"
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id, torch_dtype=torch_dtype, low_cpu_mem_usage=True, use_safetensors=True
)
model.to(device)
processor = AutoProcessor.from_pretrained(model_id)
pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    max_new_tokens=128,
    chunk_length_s=30,
    batch_size=16,
    return_timestamps=True,
    torch_dtype=torch_dtype,
    device=device,
)
dataset = load_dataset("distil-whisper/librispeech_long", "clean", split="validation")
sample = dataset[0]["audio"]
result = pipe(sample)
print(result["text"])

框架版本

• Transformers 4.47.1
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0

German-RAG AI团队

Marcel Rosiak Soumya Paul Siavash Mollaebrahim Zain ul Haq