According to a recent New York Times article, computerized essay grading systems have the potential to replace human graders for standardized essays. In a 2012 study from the University of Akron, systems were able to grade hundreds of thousands of short essays written by middle- and high-school students in less than a minute, assigning grades that mirrored those assigned by human graders. Now, edX, the massive online, open-course enterprise, intends to make similar software available online for free. While the potential to quickly grade assignments may entice educators, caution should be exercised before schools replace current grading systems with software. Language and style are complex and difficult to measure, and universities should ensure that English instruction prepares students for the academic and professional writing required for success.
Anant Agarwal, edX’s president, claims that the edX essay grading software gives feedback similar to that given by class instructors. Other systems, such as e-rater, have claimed similar results when grading short essays on middle- and high-school level standardized tests. However, some of these systems grade only on “grammar, usage, mechanics, style and organization, and development,” and thus can be duped. Les Perelman, a researcher at the Massachusetts Institute of Technology, demonstrated this by submitting a grammatical, well-structured, but factually incoherent response for the short essay question to the e-rater system. e-rater assigned Perelman’s response the highest score possible. Current gaps in the software system suggest too many flaws for professors to move forward with this technology.
The edX software is based on machine learning, where it takes in about 100 scored exams from human graders and uses that as a basis on which to grade other exams. When evaluating essays, graders should not consider only grammar and organization, but also factual accuracy, argument and reasoning. Human Readers, a group opposed to the use of essay grading software in situations where students’ futures are at stake, has gained signatures from more than 3,000 college professors, instructors and teachers in support of its goals. According to the group’s website, multiple studies show the limitations inherent in computerized grading systems and how they incentivize superficial aspects of writing.
In college, students should have the opportunity to develop their writing skills so that they’re prepared for a whole range of writing tasks upon graduation. Writing and communication skills are increasingly listed among weaknesses of students applying for jobs and to graduate programs, and both graders and professors must take this into account. Currently, the University requires undergraduates to take two classes explicitly focused around writing during their tenure, a First-Year Writing Requirement class and one fulfilling the Upper-Level Writing Requirement. The University should continue to work toward securing high-quality writing instruction that acknowledges the complexities of the English language.