语言: 韩语 标签:

• 文本转语音 许可证: 其他

Torchaudio_Tacotron2_kss

基于kss数据集训练的torchaudio Tacotron2模型。

许可证

• 代码: MIT许可证
• pytorch_model.bin权重文件: CC BY-NC-SA 4.0 (kss数据集的许可证)

环境要求

pip install torch torchaudio transformers phonemizer

同时需要安装 espeak-ng

Windows用户需设置额外环境变量，详见: https://github.com/bootphon/phonemizer/issues/44

使用方式

import torch
from transformers import AutoModel, AutoTokenizer

repo = "Bingsu/torchaudio_tacotron2_kss"
model = AutoModel.from_pretrained(
    repo,
    trust_remote_code=True,
    revision="589d6557e8b4bb347f49de74270541063ba9c2bc"
    )
tokenizer = AutoTokenizer.from_pretrained(repo)
model.eval()

vocoder = torch.hub.load("seungwonpark/melgan:aca59909f6dd028ec808f987b154535a7ca3400c", "melgan", trust_repo=True, pretrained=False)
url = "https://huggingface.co/Bingsu/torchaudio_tacotron2_kss/resolve/main/melgan.pt"
state_dict = torch.hub.load_state_dict_from_url(url)
vocoder.load_state_dict(state_dict)

声码器与原始seungwonpark/melgan相同，但权重文件已加载至cuda，因此单独提供。

text = "你好，我是Tacotron2。"
inp = tokenizer(text, return_tensors="pt", return_length=True, return_attention_mask=False)

with torch.inference_mode():
    out = model(**inp)
    audio = vocoder(out[0])

import IPython.display as ipd

ipd.Audio(audio[0].numpy(), rate=22050)