DIG releases tools for course registration, including course evaluations

Tuesday, March 21, 2017 - 3:46pm

The University of Michigan Digital Innovation Greenhouse, housed within the Office of Academic Innovation, released a new version of Academic Reporting Tools earlier this week, which will make course evaluation data more readily available to students.

Mike Wojan, a DIG user-experience designer who worked with students and other team members to design ART 2.0, said the major difference between the newest version of ART and previous iterations is the inclusion of the course-evaluation data, a decision based largely on student feedback. He said the ultimate goal of ART is summed up in the slogan: “explore, discover and decide.”

“Those are the three things we're trying to help facilitate right now: students exploring their options when it's time to register courses, discovering things they might not have known about, courses or instructors or topics that the might not have known about, and then making more informed decisions when it comes time to register,” he said.

Wojan explained the data will be represented in terms of a bar graph showing the percentage of students who responded to course evaluations questions in a certain way, like the percentage that “strongly agree.” These feature will include icons representing the sentiment behind each question, like a crystal ball to indicate many students agreed that they knew what was expected of them in the course.

“We had to decide how are we going to give this information back to the user,” Wojan said. “What is going to be the easiest way for students to look at these evals, and right away understand what the data's actually saying about the course or about the instructor?”

Amy Homkes-Hayes, lead innovation advocate of DIG, said student feedback has been instrumental to the improvement of ART, and will be necessary for its continued success.

“We also have a lot of student support,” she said. “That's one of the major ways that we're spreading the word is by going through our student representatives to say this is a tool we want students to use while they're engaging the backpacking process and for course selection.”

She said the decisions of what information from course evaluations to include in ART were based on criteria established by the Office of the Provost, the Senate Advisory Committee on University Affairs and Central Student Government. The groups agreed on including standards set for data shown based on class size, number of evaluation responses and number of semesters taught. In addition to the use of aggregate data and displaying the number of student responses compared to the number enrolled in the class, these parameters, Homkes-Hayes explained, heighten the validity of the data shown.

“One of the benefits of using ART 2.0 is that this is official University data,” she said. “We have the blessing of all of the relevant parties in order to show it, and we also believe that we're showing it in a really sophisticated way … that makes it easy to understand.”

In an email interview, August Evrard, professor of physics and astronomy and the faculty lead of ART 2.0, acknowledged concerns about the validity of data, specifically noting the possibility of biased student evaluations. However, he feels University data will be more objective than other sources online.

“There are always concerns when dealing with quantitative data,” he wrote. “Some faculty are concerned about potential biases in the student evaluations of teaching that could put certain faculty members at risk.  Previous research has indicated that such biases are moderate at Michigan, and the common alternative, RateMyProfessor, is likely to be more biased than our own data, so the previous Provost decided that, on balance, the benefits of sharing (student evaluations) broadly outweigh this particular risk. But this is a concern that we need to study more closely and have more conversations around as we go forward.”

Evrard wrote he believes the expansion of ART will ultimately be beneficial for faculty members in that it may eventually allow professors the opportunity to include teaching statements and accomplishments in profiles on these pages.

“Instructors now know that there is a page for each of them that describes their teaching history at the university,” he wrote. “Making this information public is consistent with the open nature of our public institution, and I do hope that these pages will grow to contain more information about faculty’s teaching philosophies and accomplishments. … Also, some faculty are very innovative in their teaching - perhaps by inventing new courses to fill an emerging need - and these pages offer a potential means to highlight such innovations.”

Evrard hopes, in addition to other benefits for students registering for classes, ART will encourage students to fill out course evaluations more thoughtfully.

“An additional area of impact is the Student Evaluations of Teaching (SETs) themselves,” he wrote. “As students realize that their collective feedback on teaching will be shared with other students, we anticipate higher response rates and increasingly thoughtful responses to SET survey questions. This is where coordination with student government is essential.”

Wojan emphasized the importance of student feedback concerning ART to optimize its usefulness to students, not only in surveys, but also throughout the design process.

“It's important for me to remember that I have to design things not just in my own head but something that's representative of what the end users are asking for,” he said. “So (student input) is a good way of keeping us honest making sure that we have the right ideas about what people are asking for.”

LSA senior Noah Betman, CSG chief of staff, acted as a liaison between CSG and DIG throughout the development of ART, which he said was a productive experience.  

“I felt like my voice has been heard in the room, so it's not just a token student voice kind of situation,” he said. “I've been able to provide a lot of good input, and the product that is being released … is something that students will appreciate and that will find beneficial.”

Betman said he feels ART will be a useful tool for students in the future, in terms of giving them information about workload and past students’ experiences.

“Having that out there when creating your own schedule and making sure that the balance in the best way for you understanding that you know if a course has a heavier workload than another course matters a lot to students,” he said.

Wojan said DIG will continue improving and adding features, and mentioned the possibility of allowing students to craft personal profiles on ART or see grade distributions.

“Moving forward we're trying to do even more with ART, we're trying to get students even more personalized and engaging experience on the platform,” he said. “Our goal for ART is to have some new really useful functionality each term. And so you know we're excited about that and it keeps us on our toes because you know we're always designing something new and exciting that's going to be going out you can see right away.”