A fuzzy group decision making model for ordinal peer assessment

Massive Open Online Courses (MOOCs) are becoming an increasingly popular choice for education but, to reach their full extent, they require the resolution of new issues like assessing students at scale. A feasible approach to tackle this problem is peer assessment, in which students also play the role of assessor for assignments submitted by others. Unfortunately, students are unreliable graders so peer assessment often does not deliver accurate results. In order to mitigate this issue, we propose a new model for ordinal peer assessment based on the principles of fuzzy group decision making. In our approach, each student is asked to rank a few random submissions from the best to the worst and to specify, with a set of intuitive labels, at what extent each submission is better than the following one in the ranking. Students' provided rankings are then transformed in fuzzy preference relations, expanded to estimate missing values and aggregated through OWA operators. The aggregated relation is then used to generate a global ranking between the submissions and to estimate their absolute grades. Experimental results are presented and show better performances with respect to other existing ordinal and cardinal peer assessment techniques both in the reconstruction of the correct ranking and on the estimation of students' grades.