语言:

• 中文 许可证: apache-2.0 标签:
• whisper-event
• generated_from_trainer 基础模型: openai/whisper-small 数据集:
• mozilla-foundation/common_voice_11_0 模型索引:
• 名称: Distil-Whisper Small zh-HK - Alvin 结果:
  • 任务: 名称: 自动语音识别 类型: automatic-speech-recognition 数据集: 名称: mozilla-foundation/common_voice_16_0 yue 类型: mozilla-foundation/common_voice_16_0 配置: yue 分割: test 参数: yue 指标:
    • 名称: 标准化CER 类型: cer 值: 9.7

Distil-Whisper Small zh-HK - Alvin

• 此模型是alvanlii/whisper-small-cantonese在粤语上的蒸馏版本。
• 在Common Voice 16.0上实现了9.7的CER（无标点符号），11.59的CER（带标点符号）。
• 解码器层数为3层，而非Whisper small模型的常规12层。
• 使用约2GB的GPU VRAM。

训练和评估数据

训练数据包括：

• CantoMap: Winterstein, Grégoire, Tang, Carmen和Lai, Regine (2020) "CantoMap: a Hong Kong Cantonese MapTask Corpus"，发表于《第12届语言资源与评估会议论文集》，马赛：欧洲语言资源协会，第2899-2906页。
• Cantonse-ASR: Yu, Tiezheng, Frieske, Rita, Xu, Peng, Cahyawijaya, Samuel, Yiu, Cheuk Tung, Lovenia, Holy, Dai, Wenliang, Barezi, Elham, Chen, Qifeng, Ma, Xiaojuan, Shi, Bertram, Fung, Pascale (2022) "Automatic Speech Recognition Datasets in Cantonese: A Survey and New Dataset"，2022年。链接：https://arxiv.org/pdf/2201.02419.pdf
• Common Voice yue和zh-HK的训练集

评估使用Common Voice 16.0 yue测试集。

与Whisper Small的比较

注：推理时间是通过对CV16 yue测试集取平均推理时间计算得出。

使用模型

import librosa

import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor

y, sr = librosa.load('audio.mp3', sr=16000)

MODEL_NAME = "alvanlii/distil-whisper-small-cantonese"

processor = WhisperProcessor.from_pretrained(MODEL_NAME)
model = WhisperForConditionalGeneration.from_pretrained(MODEL_NAME)

model.config.forced_decoder_ids = None
model.config.suppress_tokens = []
model.config.use_cache = False

processed_in = processor(y, sampling_rate=sr, return_tensors="pt")
gout = model.generate(
  input_features=processed_in.input_features, 
  output_scores=True, return_dict_in_generate=True
)
transcription = processor.batch_decode(gout.sequences, skip_special_tokens=True)[0]
print(transcription)

• 或者，您可以使用huggingface pipelines

from transformers import pipeline
MODEL_NAME = "alvanlii/distil-whisper-small-cantonese" 
lang = "zh"
pipe = pipeline(
    task="automatic-speech-recognition",
    model=MODEL_NAME,
    chunk_length_s=30,
    device=device,
)
pipe.model.config.forced_decoder_ids = pipe.tokenizer.get_decoder_prompt_ids(language=lang, task="transcribe")
text = pipe(file)["text"]