The following is an excerpt from John U. Bacon’s upcoming book “Endzone: The Rise, Fall, and Return of Michigan Football.”
John U. Bacon is a New York Times bestselling author, a lecturer at the University and a member of the Board of Student Publications, which works with The Michigan Daily.
On the morning of Tuesday, October 8, just three days before they were to make their presentation to the SRAC—and a room full of professors and peers—(then-Central Student Government President Michael Proppe) walked to the Ross Business School to meet with Royster Harper, the vice president for Student Life, who was attending President Coleman’s “State of the University” address. Like so many people in this story, Harper worked hard, smart, and unselfishly for people she would likely never meet—most of whom would never know her role in the story.
With the clock ticking, Proppe got to the point. “Royster, we have a survey designed, ready to go, and we have to get it out and back by Friday, October 11.” In hindsight, he says, “Now, that’s asking a hell of a lot. If we didn’t already have a good relationship with her, I don’t know if she would have considered it. But she didn’t blink.
“ ‘Okay, let me make a call. This is obviously important.’ ” Royster Harper got it out within two hours—about four days faster than usual.
Even with the survey completed and sent, they were at the mercy of their fellow students to respond. They needed enough of them to return the survey to get a statistically significant response, then analyze the results and publish a report before the meeting that Friday.
After Harper sent the survey out, they waited. She commended Proppe on his hard work. “But,” she said, “I’m concerned you’re not going to have enough responses in such a short period.”
Proppe was, too. So, he flipped open his MacBook Air and checked. Just 60 minutes after Harper had sent it out, they already had received 2,000 responses. When they closed the survey after 24 hours, 7,305 students had started the survey, and 5,892 had finished it. (They only counted finished surveys.) Proppe heard from many students unhappy that they closed it after 24 hours. He was sympathetic, but, he explained, “We had to begin putting together our report fast and we couldn’t do that if the data kept changing as people filled it out.”
Fortunately, they had plenty to work with already. When they had sent out their survey asking for student input for the presidential search that fall, only 2.6 percent of students had responded. For the general-admission survey, however, 14 percent responded, in only 24 hours. “That,” Proppe says, “is a ridiculously high response rate.”
You couldn’t say the students didn’t care.
Proppe didn’t waste any time gloating. He closed his laptop, loaded his backpack, and headed home to squeeze in some schoolwork. After they closed the survey, they started working on their report Wednesday night. What had seemed impossible two days ago now seemed likely: They were going to have solid data, thoroughly analyzed, to present to the committee Friday afternoon.
When he sifted through the survey data, Proppe boiled it down to two main points:
1. The students clearly did not like general admission. “This wasn’t my opinion. We had the data.” Of the survey respondents, 76 percent were opposed to it, and 17 percent supportive.
2. Whether students liked general admission or not, it had not achieved Dave Brandon’s stated goal: a full student section.
“Yeah, you could show photos of the empty rows in the student section, but I wanted real data,” Proppe says. “And for that I needed attendance figures, so I had to ask Hunter [Lochmann] earlier in the week, ‘Can you share this info?’ He provided it, no problem, and said, ‘We want to be very transparent about this.’ But he probably said that because he thought the figures made them look really good.”
Lochmann gave Proppe the figures Wednesday morning, two days in advance of the meeting, which showed the overall student attendance rate up slightly from 79 percent in 2011 and 2012, to 81 percent for the first four home games of 2013, and the on-time rate up from about 45 percent in 2011 and 2012 to roughly 65 percent in 2013.
Sure enough, as Proppe had predicted, Lochmann thought the data vindicated the department’s decision to switch to general admission. But, crucially, Hunter Lochmann was a former marketing man for the New York Knicks, not a statistics major at the University of Michigan. Proppe had been taught by some of the world’s best statistics professors that you can only make analytical conclusions when comparing apples to apples.
Instead of lumping all of 2011’s and 2012’s games into one pile and all of 2013’s into another, as Lochmann had done, Proppe analyzed comparable games from both seasons: noon games to noon games, afternoon games to afternoon games, and night games to night games. He noticed that, in 2011 and 2012, Michigan played a lot of noon games. In 2013, in contrast, Michigan had played one game at noon (Akron), two at 3:30 (Central Michigan and Minnesota), and one at 8 p.m., Notre Dame. Not incidentally, the quality of the opponent went up with the time of day. The distinction was crucial, Proppe knew, because students were far less likely to show up on time for noon games than 3:30 or night games, no matter what the seating policy.
“Look, I don’t know how to say this without sounding like a jerk,” Proppe told me, “but Hunter and his group were not as sophisticated as we were about analyzing data. When I looked at this data for ten minutes on an Excel spreadsheet, I could figure out what the data really meant. I realized once we segment it out by start time, the attendance for both being on time and showing up overall, was worse under the general-admission policy than it had been in 2011 and 2012. When Hunter gave me the data so quickly, and urged me to share it with the public, I don’t think he realized that.”
Looking at Lochmann’s numbers, Proppe quickly recognized something else: The department measured attendance one way in 2011 and 2012, and a different way in 2013. In 2011 and 2012, they used the number of tickets scanned, which is a flawed system because the scanners depend on cellular service, which any fan can tell you is notoriously spotty in Michigan Stadium. “Department employees have told me the electronic scanning system can miss 5,000 to 10,000 tickets a game—or about 5 to 10 percent, a remarkably high margin of error,” Proppe told me. “Yet these were the numbers Brandon used to justify switching to general admission.
“As shaky as that was, the department’s system for tracking attendance in 2013 was even worse: They measured student attendance by the number of section cards they handed out to students, which the ushers handed to them when they walked into the stadium. But the number of section cards handed out was higher—significantly higher—than the number of tickets scanned. When I saw this, I called Hunter and said, ‘We have to use the same measurement, which can only be tickets scanned, since they didn’t use section cards before.’ ”
Lochmann agreed, and gave them the data for the number of tickets electronically scanned in 2013. Looking at data gathered with the same methodology revealed student attendance was actually lower in 2013 than it had been in 2012 and 2011. Instead of the 81 percent attendance rate the department presented, based on section cards handed out, the ticket scanning data showed a 76 percent attendance—which was obviously lower than the 79 percent attendance rate from tickets scanned in 2012 and 2011.
Likewise, the department’s claim of an improved on-time rate from 45 percent in 2011 to 65 percent was at least overstated, as Lochmann’s ticket-scanning data showed 61 percent of students showed up on time for the 2013 games.
But even at that, Proppe knew, the data would tell an even more dramatic story once it was broken down by game times. And that’s a surprise he prepared for the meeting the next day.
To get all this done by the next day, Proppe stuck with his weekly routine that semester, working every night until two or three, then getting up at 6:30 a.m. But when Proppe walked to the Michigan Union that Friday, he had everything he needed in his bag. He was ready—and he knew it.
Proppe walked to the Michigan Union’s Welker Room, armed only with his laptop, where he met (then-Central Student Government Vice President Bobby Dishell) and four other members of the Central Student Government. They were surrounded by about 20 other people, mostly faculty members, plus Royster Harper and Hunter Lochmann.
Professor Potter asked Lochmann to start the meeting by explaining the general-admission seating policy.
“Hunter talked about why they started the policy,” Proppe recalls. “He explained the changes they’d made to respond to student complaints, and showed how well the policy had worked, based on their original data. He closed by saying, ‘We’re open to feedback.’ ”
Lochmann sat down, satisfied. Professor Potter thanked him, then asked Proppe to make his presentation. Before Proppe spoke, however, he noticed one of his former stats professors, Ed Rothman, in the first row. Proppe smiled. “I’m thinking, ‘Great! He’s on the committee. He’s going to love this.’ ”
When Proppe took to the podium, he left his prepared remarks briefly to tell the committee, “I had a great stats prof, Dr. Rothman, sitting here in the first row. This is the guy who taught me about stratifying variables. When you’re analyzing attendance, the stratifying variable is the start time of the game. If you miss that, you’ll read the numbers incorrectly.”
The trap, Dishell knew, sitting in the audience, had been set.
“You have to use a consistent basis of measurement,” Proppe continued, “but the department’s conclusions depend on two different bases for measuring attendance.”
Proppe glanced at Professor Rothman, who was already nodding his agreement, grinning. From Proppe’s introduction alone, Professor Rothman knew what was coming next.
Proppe then got to the two main tenets of his PowerPoint presentation.
“One: 76 percent of the students were opposed to the general-admission policy. The number one reason: it makes it harder to sit with your friends.”
That complaint had arisen as soon as the policy was announced, and confirmed with the first four home games, but since then CSG had discovered a serious safety issue before the Notre Dame game. Proppe read a few quotes from the 1,234 students who had responded to his survey that they “felt unsafe/in danger of being trampled while waiting to get to my seat”:
—“Going to football games is far less enjoyable now than it was during my 4 years of undergrad. It is much more stressful, been dangerous at times, and makes it hard to enjoy games with friends.”
—“I have gotten hurt with people pushing and have felt extremely claustrophobic while waiting in not only one, but TWO different lines to get into my section.”
—“I got my head slammed against a pole in a fight waiting in line.”
This explained another stat Proppe had uncovered: Of those students who were initially supportive of the general-admission policy when it was announced, 27 percent of them were now opposed to it, compared to just 4 percent who were initially against it and were now for it. Put it all together, Proppe said, and you can safely conclude, “General admission, in practice, was turning people against it who initially liked the idea.”
But that’s what a claustrophobia-inducing mob will do.
Proppe then pointed out that the football team, despite a few wobbly victories, had a perfect 5–0 mark that season, and was ranked 18th in the AP poll. It would be hard to claim the students’ dissatisfaction with the Michigan football experience was due to failure on the field.
“But we can agree,” Proppe said, building to his crescendo, “if the students didn’t like it, but it achieved the department’s goals of better attendance and punctuality, we’d have to debate the cost-benefit analysis of the policy. But once you segment out the attendance data by start times, you can see that it did not achieve the department’s stated goal, either.”
He then ran through the rest of his PowerPoint presentation, flashing data that compared similar games from 2011 and 2012 to 2013, matching the relative quality of the opponent, the time of year, and most important, the start time.
Proppe started by comparing the data from the games that started at noon, then 3:30, then 8:00 p.m. Once separated by kind, or “stratified variables,” as Professor Rothman had taught him, the data was damning. When Proppe clicked to a slide comparing the 2011 Western Michigan game and the 2012 Air Force game to the 2013 Central Michigan game, all of which started at 3:30, the numbers clearly showed that in 2011 and 2012, under the old system, more students showed up, and showed up on time, than in 2013, with fewer no-shows.
When he compared the two Notre Dame night games from 2011 and 2013, the slides showed the 2011 game attracted 727 more students at kickoff and 2,520 more total, with 1,413 fewer no-shows.
By Proppe’s last slide, Professor Rothman and Bobby Dishell looked a lot happier than Hunter Lochmann did.
“So, in conclusion,” Proppe said, “the students don’t like the general-admission policy, and it doesn’t work.”
In a nutshell, if you examined the data professionally, there was nothing to debate.
When Proppe finished, the faculty members gave him an ovation. Lochmann sat at the same table, head down, taking notes.
When Professor Potter returned to the front of the room, he opened the floor for questions—and he got them, directed from the faculty to Lochmann, most of them critical.
The faculty wanted to know more about student safety, for example, and how Lochmann believed the GA policy fit in with the mission of the university. Lochmann struggled to answer these questions, but took them with good humor.
When they finished the drubbing, Professor Potter asked Lochmann, “Are you going to reconsider this policy at the end of the year?”
“Yes,” he said, which surprised the crowd.
Proppe recalls, “Everyone kind of looked around at each other, kind of shocked, like, ‘Did he just say that? Did we just get a win here?’ ”
A full 10 seconds passed before Laura Blake Jones, dean of students, said, “Hunter, did I just hear you say that getting rid of general admission is on the table at the end of the season?”
“Yes,” he said. “It will have to be.”
When the meeting adjourned, Dishell recalls, “We were so excited! We weren’t sure if we should hug or high-five.”
Instead, to make sure they didn’t spoil the professional impression they had just made on the faculty members, they contained their excitement until they got out of the meeting room. Then their big smiles gave way to a quick high-five.