While running the MOOC at learn.moodle.net, couple of small usability issues with the current Workshop module raised. This META epic is here to summarize them all so they can be converted into separate issues. Ideas and notes are freely worded here with undefined order.

• When the final grades are re-calculated during the grading evaluation phase, we were missing settings like "Require at least X received assessments" and "Require at least X given reviews". If the participant did not pass these conditions, the grade (or grading grade respectively) would not be calculated to them. This is to prevent a situation when the final grade is based on a single received peer-assessment. The X number would actually be interpreted as a "weighted count" - if 3 received assessment were required, it would be enough to have a single assessment from a teacher with the weight set to 3.
• We were missing a possibility to hide enrolled students who did not actually participate in the Workshop. In our MOOC, roughly 10% of enrolled students actually participated in the Workshop (i.e. submitted their own work). Having all enrolled students listed in the grades overview made the grades report harder to navigate.
• When an additional assessment was done, or an assessment weight was changed etc. during the grading evaluation phase, there should be a warning/notice reminding to recalculate grades.
• Number of pending assessments to be displayed so the teacher knows "where are we", what the progress is.
• Ability to filter submissions that need an extra assessment (by the teacher typically). This mostly covers submissions that were allocated to reviewers who then did not review anything. It could also be more sophisticated to detect submissions where the teacher's assessment would be good to have as the existing assessments do not support (match) each other.
• When grades are re-calculated during the grading evaluation phase, some sort of report would be nice to have. It would show new/changed grades so the teacher can see and compare the effect of various evaluation parameters.
• Ability to flag submissions and assessments (like questions in the quiz attempt or e-mails in Gmail). The flag would be shared by all teachers so they could use it as a communication tool (e.g. to mark and remember submissions to look closely at)
• Ability to search for and filter submissions in the grades report (by the author's name etc) - including a way to display just one submission per page.
• Teachers can't see the calculated grading grade for an assessment when viewing the submission's assessments.
• Make the planner tool (the one with phases) responsive, using the 100% container width and collapsible.
• Ability to rename and delete submissions.
• New "overview" allocation method. It would not actually allocate anything, but it would provide some detailed stats about the situation (and it would become the default allocation screen).

Beside these, more conceptual issues (mostly the problem with isolated and distinct phases that are forced to everybody) were confirmed. But those will be addressed in a separate epic.

Edit:

Two great blog posts with very useful suggestions to be considered:

• http://blogs.ucl.ac.uk/ele/2015/02/13/peer-review-with-the-moodle-workshop-activity-a-close-look/
• http://blogs.ucl.ac.uk/ele/2015/03/18/a-good-peer-review-experience-with-moodle-workshop/