语言:

• 阿拉伯语
• 加泰罗尼亚语
• 德语
• 希腊语
• 英语
• 西班牙语
• 法语
• 印地语
• 意大利语
• 日语
• 韩语
• 荷兰语
• 波兰语
• 葡萄牙语
• 俄语
• 瑞典语
• 越南语
• 中文

小组件示例:

• 文本: "I Red Hot Chili Peppers sono stati formati a Los Angeles da Kiedis, Flea, il chitarrista Hillel Slovak e il batterista Jack Irons." 示例标题: "意大利语"

推理参数: 参数: 解码器起始标记ID: 250058 源语言: "it_IT" 目标语言: ""

标签:

• 序列到序列
• 关系抽取

许可证: 知识共享署名-非商业性使用-相同方式共享 4.0 管道标签: 翻译

RED^FM: 一个经过筛选的多语言关系抽取数据集

这是REBEL的多语言版本。它可以作为独立的多语言关系抽取系统使用，也可以作为预训练系统在多语言关系抽取数据集上进行微调。

mREBEL在ACL 2023论文RED^{FM}: a Filtered and Multilingual Relation Extraction Dataset中提出。我们提出了一个新的多语言关系抽取数据集，并训练了一个多语言版本的REBEL，将关系抽取重新定义为序列到序列的任务。论文可以在这里找到。如果您使用代码或模型，请在您的论文中引用这项工作：

@inproceedings{huguet-cabot-et-al-2023-redfm-dataset,
    title = "RED$^{\rm FM}$: a Filtered and Multilingual Relation Extraction Dataset",
    author = "Huguet Cabot, Pere-Llu{\'\i}s  and Tedeschi, Simone and Ngonga Ngomo, Axel-Cyrille and
      Navigli, Roberto",
    booktitle = "Proc. of the 61st Annual Meeting of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/2306.09802",
}

论文的原始仓库可以在这里找到。

请注意，右侧的推理小组件不会输出特殊标记，这些标记是区分主语、宾语和关系类型所必需的。有关mREBEL及其预训练数据集的演示，请查看Spaces演示。

管道使用

from transformers import pipeline

triplet_extractor = pipeline('translation_xx_to_yy', model='Babelscape/mrebel-large-32', tokenizer='Babelscape/mrebel-large-32')
# 我们需要手动使用分词器，因为需要特殊标记。
extracted_text = triplet_extractor.tokenizer.batch_decode([triplet_extractor("The Red Hot Chili Peppers were formed in Los Angeles by Kiedis, Flea, guitarist Hillel Slovak and drummer Jack Irons.", decoder_start_token_id=250058, src_lang="en_XX", tgt_lang="<triplet>", return_tensors=True, return_text=False)[0]["translation_token_ids"]]) # 将en_XX更改为源语言。
print(extracted_text[0])
# 解析生成文本并提取三元组的函数
def extract_triplets_typed(text):
    triplets = []
    relation = ''
    text = text.strip()
    current = 'x'
    subject, relation, object_, object_type, subject_type = '','','','',''

    for token in text.replace("<s>", "").replace("<pad>", "").replace("</s>", "").replace("tp_XX", "").replace("__en__", "").split():
        if token == "<triplet>" or token == "<relation>":
            current = 't'
            if relation != '':
                triplets.append({'head': subject.strip(), 'head_type': subject_type, 'type': relation.strip(),'tail': object_.strip(), 'tail_type': object_type})
                relation = ''
            subject = ''
        elif token.startswith("<") and token.endswith(">"):
            if current == 't' or current == 'o':
                current = 's'
                if relation != '':
                    triplets.append({'head': subject.strip(), 'head_type': subject_type, 'type': relation.strip(),'tail': object_.strip(), 'tail_type': object_type})
                object_ = ''
                subject_type = token[1:-1]
            else:
                current = 'o'
                object_type = token[1:-1]
                relation = ''
        else:
            if current == 't':
                subject += ' ' + token
            elif current == 's':
                object_ += ' ' + token
            elif current == 'o':
                relation += ' ' + token
    if subject != '' and relation != '' and object_ != '' and object_type != '' and subject_type != '':
        triplets.append({'head': subject.strip(), 'head_type': subject_type, 'type': relation.strip(),'tail': object_.strip(), 'tail_type': object_type})
    return triplets
extracted_triplets = extract_triplets_typed(extracted_text[0])
print(extracted_triplets)

使用transformers的模型和分词器

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

def extract_triplets_typed(text):
    triplets = []
    relation = ''
    text = text.strip()
    current = 'x'
    subject, relation, object_, object_type, subject_type = '','','','',''

    for token in text.replace("<s>", "").replace("<pad>", "").replace("</s>", "").replace("tp_XX", "").replace("__en__", "").split():
        if token == "<triplet>" or token == "<relation>":
            current = 't'
            if relation != '':
                triplets.append({'head': subject.strip(), 'head_type': subject_type, 'type': relation.strip(),'tail': object_.strip(), 'tail_type': object_type})
                relation = ''
            subject = ''
        elif token.startswith("<") and token.endswith(">"):
            if current == 't' or current == 'o':
                current = 's'
                if relation != '':
                    triplets.append({'head': subject.strip(), 'head_type': subject_type, 'type': relation.strip(),'tail': object_.strip(), 'tail_type': object_type})
                object_ = ''
                subject_type = token[1:-1]
            else:
                current = 'o'
                object_type = token[1:-1]
                relation = ''
        else:
            if current == 't':
                subject += ' ' + token
            elif current == 's':
                object_ += ' ' + token
            elif current == 'o':
                relation += ' ' + token
    if subject != '' and relation != '' and object_ != '' and object_type != '' and subject_type != '':
        triplets.append({'head': subject.strip(), 'head_type': subject_type, 'type': relation.strip(),'tail': object_.strip(), 'tail_type': object_type})
    return triplets

# 加载模型和分词器
tokenizer = AutoTokenizer.from_pretrained("Babelscape/mrebel-large-32", src_lang="en_XX", tgt_lang="tp_XX") 
# 这里我们将英语("en_XX")设置为源语言。要更改源语言，请将输入的第一个标记更改为您所需的语言或更改为支持的语言。对于加泰罗尼亚语("ca_XX")或希腊语("el_EL")（不包括在mBART预训练中），您需要一个解决方法：
# tokenizer._src_lang = "ca_XX"
# tokenizer.cur_lang_code_id = tokenizer.convert_tokens_to_ids("ca_XX")
# tokenizer.set_src_lang_special_tokens("ca_XX")
model = AutoModelForSeq2SeqLM.from_pretrained("Babelscape/mrebel-large-32")
gen_kwargs = {
    "max_length": 256,
    "length_penalty": 0,
    "num_beams": 3,
    "num_return_sequences": 3,
    "forced_bos_token_id": None,
}

# 要提取三元组的文本
text = 'The Red Hot Chili Peppers were formed in Los Angeles by Kiedis, Flea, guitarist Hillel Slovak and drummer Jack Irons.'

# 分词器文本
model_inputs = tokenizer(text, max_length=256, padding=True, truncation=True, return_tensors = 'pt')

# 生成
generated_tokens = model.generate(
    model_inputs["input_ids"].to(model.device),
    attention_mask=model_inputs["attention_mask"].to(model.device),
    decoder_start_token_id = tokenizer.convert_tokens_to_ids("tp_XX"),
    **gen_kwargs,
)

# 提取文本
decoded_preds = tokenizer.batch_decode(generated_tokens, skip_special_tokens=False)

# 提取三元组
for idx, sentence in enumerate(decoded_preds):
    print(f'预测三元组句子 {idx}')
    print(extract_triplets_typed(sentence))

许可证

本模型采用知识共享署名-非商业性使用-相同方式共享 4.0许可证。许可证文本可以在这里找到。