许可协议: MIT
语言:

• 英语

我们训练了一个语言模型，通过利用大量由人工评分员标注的训练数据集，自动对雅思（国际英语语言测试系统）作文进行评分。

该训练数据集包含18,000篇真实的雅思考试作文及其官方评分。
我们的模型评分结果根据雅思官方标准，从任务完成度、连贯与衔接、词汇、语法以及总分五个维度进行衡量。总分是雅思作文的综合评分。

在测试数据集上取得的显著成果如下：准确率 = 0.82，F1分数 = 0.81。就目前结果而言，我们的模型在一定程度上可以替代人工评分员对雅思作文进行评分，但我们将继续优化以提高其准确性和有效性。

如果您使用此模型，请引用以下论文：

@article{sun2024automatic,
  title={Automatic Essay Multi-dimensional Scoring with Fine-tuning and Multiple Regression},
  author={Kun Sun and Rong Wang},
  year={2024},
  journal={ArXiv},
  url={https://arxiv.org/abs/2406.01198}
}

以下是用于对新雅思作文进行评分的模型实现代码。在以下示例中，我们从测试数据集中选取了一篇总分为8.0的作文。
我们的模型对该作文的评分为8.5，与人工评分员给出的分数非常接近。

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch
import numpy as np

# 加载预训练模型和分词器
model_path = "KevSun/IELTS_essay_scoring"
model = AutoModelForSequenceClassification.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)

# 待评估的示例文本，测试数据集中人工评分为8.5的作文
new_text = (
    "It is important for all towns and cities to have large public spaces such as squares and parks. "
    "Do you agree or disagree with this statement? It is crucial for all metropolitan cities and towns to "
    "have some recreational facilities like parks and squares because of their numerous benefits. A number of "
    "arguments surround my opinion, and I will discuss it in upcoming paragraphs. To commence with, the first "
    "and the foremost merit is that it is beneficial for the health of people because in morning time they can "
    "go for walking as well as in the evenings, also older people can spend their free time with their loved ones, "
    "and they can discuss about their daily happenings. In addition, young people do lot of exercise in parks and "
    "gardens to keep their health fit and healthy, otherwise if there is no park they glue with electronic gadgets "
    "like mobile phones and computers and many more. Furthermore, little children get best place to play, they play "
    "with their friends in parks if any garden or square is not available for kids then they use roads and streets "
    "for playing it can lead to serious incidents. Moreover, parks have some educational value too, in schools, "
    "students learn about environment protection in their studies and teachers can take their pupils to parks because "
    "students can see those pictures so lively which they see in their school books and they know about importance "
    "and protection of trees and flowers. In recapitulate, parks holds immense importance regarding education, health "
    "for people of every society, so government should build parks in every city and town."
)

encoded_input = tokenizer(new_text, return_tensors='pt', padding=True, truncation=True, max_length=512)

model.eval()

# 执行预测
with torch.no_grad():
    outputs = model(**encoded_input)

predictions = outputs.logits.squeeze()

predicted_scores = predictions.numpy()  

# 标准化分数
normalized_scores = (predicted_scores / predicted_scores.max()) * 9  # 缩放到9分制

rounded_scores = np.round(normalized_scores * 2) / 2

item_names = ["任务完成度", "连贯与衔接", "词汇", "语法", "总分"]

for item, score in zip(item_names, rounded_scores):
    print(f"{item}: {score:.1f}")

## 输出:
# 任务完成度: 9.0
# 连贯与衔接: 7.5
# 词汇: 8.0
# 语法: 7.5
# 总分: 8.5