库名称: transformers 标签:

• 自动语音识别 许可证: apache-2.0 数据集:
• reazon-research/reazonspeech 语言:
• 日语 评估指标:
• 字符错误率(CER) 基础模型:
• reazon-research/japanese-wav2vec2-large

japanese-wav2vec2-large-rs35kh

本模型是基于wav2vec 2.0 Large架构，在大型日语ASR语料库ReazonSpeech v2.0上微调而成。

使用方法

可通过transformers库调用本模型：

import librosa
import numpy as np
from transformers import AutoProcessor, Wav2Vec2ForCTC

model = Wav2Vec2ForCTC.from_pretrained(
    "reazon-research/japanese-wav2vec2-large-rs35kh",
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
).to("cuda")
processor = AutoProcessor.from_pretrained("reazon-research/japanese-wav2vec2-large-rs35kh")

audio, _ = librosa.load(audio_filepath, sr=16_000)
audio = np.pad(audio, pad_width=int(0.5 * 16_000))  # 推荐在推理前对音频进行填充
input_values = processor(
    audio,
    return_tensors="pt",
    sampling_rate=16_000
).input_values.to("cuda").to(torch.bfloat16)

with torch.inference_mode():
    logits = model(input_values).logits.cpu()
predicted_ids = torch.argmax(logits, dim=-1)[0]
transcription = processor.decode(predicted_ids, skip_special_tokens=True)

测试结果

我们报告了本模型与其他wav2vec2系列模型的字符错误率(CER)对比：

长语音识别性能对比：

引用

@misc{reazon-research-japanese-wav2vec2-large-rs35kh,
  title={japanese-wav2vec2-large-rs35kh},
  author={Sasaki, Yuta},
  url = {https://huggingface.co/reazon-research/japanese-wav2vec2-large-rs35kh},
  year = {2024}
}

许可证

Apache许可证2.0