The hard truth about exam software

People say you don’t know what you’ve got until it’s gone. However, when it comes to technology, the opposite holds true: You don’t know what you’ve got until you’re stuck with it.

Over the course of the pandemic, technology has creeped further into our lives, education being no exception. All of my classes this semester have been conducted online, as have my club meetings and social events. Practically every interaction I have with the University of Michigan happens through a webcam.

However, there is one new technology that has seen particularly rapid adoption since the beginning of the pandemic: exam proctoring software.

***

There are a number of companies competing in the remote proctoring space, including ProctorU, Respondus, ExamSoft, HonorLock and Proctorio.

ProctorU, for example, offers four levels of online proctoring: record, record+, review+ and live+. The most basic service available is “record,” which offers identity verification and end-to-end recordings of the students’ camera and screen. During the exam, the artificial intelligence records instances of possible suspicious behavior — anything from eye movement to background noise. The service then produces a report including flagged events that may require further review by an instructor.

In comparison, “live+” offers more deterrence features designed for high-stakes exams. Along with monitoring exam-takers via artificial intelligence, this service includes a proctor who supervises students and intervenes when they suspect cheating.

With the breakneck transition to virtual learning, there has been a documented increase in cheating, and exam proctoring services have stepped in to address this issue. In an email to me, Scott McFarland, CEO of ProctorU, touted the advantages of software solutions to deter academic dishonesty.

“Changes in assessment design can mitigate some risks of misconduct,” wrote McFarland. “But so long as there are remote tests that require demonstrated knowledge in the form of a test, proctoring will be necessary. As of now, the only alternative to remote proctoring is not proctoring, which increases the risk of academic misconduct.”

***

However, all of this increased business for remote proctoring has coincided with a backlash over privacy and accessibility concerns.

Without the backing of their schools, some students have taken matters into their own hands. Daniel Farzannekou, a student at the University of California, Los Angeles, was featured in The New York Times for his act of resistance against the Respondus proctoring software. Before the exam could begin, Farzannekou replied to directions to scan his face, room and ID by holding a profane slip of paper up to his webcam and uninstalling the software. He called the software “dystopian,” comparing his peers’ numbness to its dangers to the placating tactics of authoritarian regimes.

When I reached out to Respondus via email to comment on criticisms of this kind, they actually addressed the issue of privacy head-on and sent a list of industry-standard protections.

“With online proctoring, students often have the misconception that Respondus views and uses the webcam videos for its own purposes,” wrote Jodi Feeney, Vice President of Operations at Respondus. “The data is owned and controlled by the university. Respondus doesn’t have human proctors reviewing the videos; the proctoring is done by an automated system.”

According to Sowers, Respondus also does not watch students while they take exams, sell or share data with third parties or access files on students’ computers. Sowers wrote that the software “exceeds the requirements” outlined in data protection legislation such as FERPA, GDPR, CCPA and frameworks like Privacy Shield and SOC 2 certification.

Nevertheless, the use of exam software has still resulted in legal controversies over privacy. When the Faculty Association of the University of Santa Barbara expressed its concerns and advocated a policy of refusal towards ProctorU in a letter to the Chancellor, ProctorU’s legal counsel levelled spurious allegations of trademark infringement, copyright infringement and defamation.

Finally, artificial intelligence implemented in these services tends to perpetuate discrimination. The behavior of students suffering from chronic illnesses, anxiety or other conditions are singled out by the algorithm. The facial recognition technology also discriminates against people of color. The fact that facial recognition systematically fails to recognize non-white faces is well-documented, so it should not come as a surprise that this phenomenon is being reproduced in the sphere of remote assessment.

When I asked if there has been criticism of their services, none of the software providers I reached out to for this story addressed discrimination or algorithmic bias in their email responses.

***

Though this national conversation has been brewing since the beginning of the pandemic, it was only after a few of my friends finished the Law School Admission Test in the first week of October that I began to take notice of the invasive nature of proctoring software. Hearing about their experiences — such as showing proctors their rooms or allowing proctors to control their laptop’s function — was pretty unsettling, and I was curious to know whether automated remote proctoring had penetrated into regular college courses as well.

The college of Literature, Science and the Arts has staked out a clear position on remote proctoring technologies. On the LSA Technology Services website, the college specifically recommends against the Respondus Lockdown Browser, which prevents students from navigating away from a webpage, and other third party services. They enjoin faculty to “Avoid the Temptation for Invasive Technology Solutions to Address Issues of Academic Integrity.”

And indeed, the LSA students that I spoke with had not experienced remote proctoring software in their college courses.

When I spoke over the phone with LSA senior Teresa Clark, she indicated her gratitude for the way professors were handling assessments.

“To accommodate people’s schedules, professors will give you the entire day to complete the exam,” Stark said. “They’ll release it at the beginning of the day and give you the entire day to work on it so they have been definitely less stressful and less high stakes than in-person exams.”

Stark also took the LSAT earlier this month, and the easygoing nature of her professors stood in sharp contrast to the ProctorU employee who kept an eye on her during the LSAT.

“You have to record your entire room,” Stark said with a laugh. “Which did cause some confusion because my room was pretty messy and they apparently prefer that they don’t see anything in the video.”

When I asked Stark how she would have reacted if she had been accused of cheating, she explained that the business about her cluttered room might have weighed against her.

“My room had like a bunch of random stuff in it,” Stark continued. “And I would have found it difficult to prove that I didn’t have some extra material around me.”

Outside of LSA, some colleges have employed different methods to deter cheating. During our phone conversation, Business and Engineering junior Vikas Chanduri explained how exams work in those two colleges. 

“In the entire engineering department, the engineering honor code works in a way where, even when we have in-person exams, there actually isn’t a proctor in the room,” Chanduri said. Instead, in pre-pandemic times, the honor code allowed engineering students to walk into an auditorium and take exams surrounded only by other students, with professors or Graduate Student Instructors waiting outside the exam room in case anyone had any questions.

The College of Engineering simply places trust in its students — what a concept! That attitude, however, differs quite a bit from the Machiavellian world of the Ross School of Business. 

“In all my (pre-COVID) Ross classes there were always (GSIs) walking around and my professors were at the front of the classroom for questions,” Chanduri explained. “So I guess in a way that has been transferred into a virtual environment, exactly the same way that it was before.”

Chanduri then talked about his experience using the Respondus Lockdown Browser during one of his Business School classes last semester.

“It was kinda weird at first having the camera on the whole time,” Chanduri explained. “But I guess it’s the same idea as if you had a proctor watching everyone in an exam room … so I didn’t think it was that big of a deal.”

***

In my view, however, the problem with exam proctoring technologies is that they are not really helping us to adapt. Rather, they are providing a vulnerable technological crutch we will be stuck with long after the pandemic ends.

Consider the fact that most of these services require students to have access to a computer and a good wireless connection. In Michigan, the rift between those who have internet connectivity and those who do not, known as the “digital divide,” poses a huge challenge for online education. Research conducted by The Education Trust shows that the percent of students without digital access in some counties in Michigan is around 30-40%, and a recent report from The Brookings Institution stated that 70 out of 83 Michigan counties have internet connectivity rates below the national average.

To the extent that adaptation is happening, it is actually students and faculty adapting to technology, not the other way around. For students to even be able to take exams, they have to download software or Chrome extensions, subject themselves to invasive security procedures and allow artificial intelligence to gather extensive data on their behavior. I suppose that for many of us who have become accustomed to pervasive online surveillance, this does not seem like too much to ask.

Here is the thing about erecting digital fences around university exams: Clever computer science majors will always find ways to hop over them. Some of them have even posted lists of possible circumvention methods online for everyone to see. The Virtual Machine, for instance, lets people run computer programs on software embedded within their physical computers. Even something as simple as Post-it Notes on one’s screen can work around the most basic software solutions.

The result will not be the ultimate victory of proctoring technologies, but an interminable arms race between software providers and cheaters. As the methods of security and prevention advance, so too will methods of circumvention. We might even see a revolving door between the makers of remote proctoring software and a growing suite of circumvention services.

Ultimately, educational institutions need to understand that any technological “solution” comes with its own host of problems. There are not side effects or unintended consequences; they are latent in the design of the technology itself. That is why many professors have turned to alternative methods of examination that require students to demonstrate their thought process and engagement with class material: some examples include ditching multiple choice exams for in-class essays, or having students present material in a presentation or project.

Remote proctoring technologies have some utility, especially when professors need to deliver high-stakes exams to hundreds of students. But where a relationship of trust already exists between student and teacher, this is what should be relied upon rather than technology.

Perhaps we should also turn our attention to the education system in which an overwhelming number of students believe they must cheat to get ahead. If only indirectly, proctoring software companies benefit from institutions that believe themselves to be full of tricksters and thieves. Therefore, they do not really want to solve the problem of academic dishonesty, even if such a thing could be done. What really matters is whether universities suspect their students of cheating and can think of no better way to stop them.

***

Despite his reputation for advising deceit, Machiavelli himself recognized that the ways of treachery could win a prince, “power, but not glory (imperio, ma non gloria).” Deep down, I think we all share this intuition that fraud impedes true greatness. And, therefore, that aspiring musicians, athletes and artists should avoid fraud as much as aspiring accountants, lawyers and engineers.

In the end, no matter how well exam proctoring technologies can deter cheating through surveillance and fear of punishment, they never will be able to convey its real cost; that in developing the skill of cheating, one neglects the virtue of learning.