University researchers examine how data science can interpret music

Monday, April 1, 2019 - 7:10pm

Music, Theatre & Dance junior Joseph Mutone performs a Bach Trio Sonata at the Understanding How the Brain Processes Music presentation in Hill Auditorium Monday.

Music, Theatre & Dance junior Joseph Mutone performs a Bach Trio Sonata at the Understanding How the Brain Processes Music presentation in Hill Auditorium Monday. Buy this photo
Sarah Kunkel/Daily

On Monday afternoon, students and professors from the University of Michigan Data Science for Music Challenge Initiative conducted live research in Hill Auditorium during a musical performance and informational presentation before nearly 200 community members. James Kibbie, chair of the School of Music, Theatre & Dance Organ Department, and Daniel Forger, professor of mathematics, received a grant from the Michigan Institute for Data Science to collaborate on the analysis of organ playing by those with varying levels of music education.

“(Forger) and I have a big grant from MIDAS to study the big data science applications to specifically the Bach Trio Sonatas,” Kibbie said. “And we are looking at a number of things, but especially how data can reveal issues of performance and, as Danny (Forger) says, ‘What makes one sublime and another ordinary.’”

Kibbie said they chose to analyze the organ because it may be easily analyzed given only one note is played in each of the three keyboards at a time.

“The right hand is controlling one melody line, the left hand another and the pedal a third,” Kibbie said. “They are totally independent, but every detail matters so there’s no room for faking.”

Music, Theatre & Dance graduate student Sarah Simko, research assistant on the project, told the crowd about the rich history of the Frieze Memorial Organ housed in Hill Auditorium, which was used for the event’s performances. Dating back to 1894, parts of the organ are older than Hill Auditorium and were moved there upon the building’s construction.

“I always say that the organ is part history, part engineering and part actually being able to pick up your fingers and put them back down,” Simko said.

As it exists now, there are 7,599 pipes hidden behind the walls of Hill, which produce the music when someone plays the keyboards electronically attached to them. Simko said it is this capacity of the Frieze Memorial Organ that makes it an optimal instrument for their research.

“Professor Forger is the first person who figured out you can take this mechanism which already exists and actually use it for analytical purposes,” Simko said.

In addition to finding an optimal instrument, Kibbie and Forger also had to determine a good piece of music to analyze. Kibbie explained to the audience how they decided on Bach’s sonatas because they were originally used as a teaching tool for Bach’s eldest son and have continued to be used in music education programs for centuries since. Until Bach, trio sonatas were often written for three distinct instruments played by three individual musicians.

“Johann Sebastian Bach was the first composer to realize how beautifully this texture adapts to being played by one person on the organ,” Kibbie said.

Forger cited Bach as a fundamental baseline for an emerging study into the “grammar” of music. There are thousands of chord progressions possible on the organ of which Bach only uses about 11 percent, Forger’s research suggests. He said this indicates there are presumably rules as to which progressions are aesthetically pleasing to listeners and which our intuition tells us should be avoided in composition.

“There is some sort of an interesting structure to this,” Forger said. “Right now, we’re trying to figure out what is that structure.”

Forger explained the primary goal of their project is to determine the psychological or neurological basis for patterns in the practice and performance of the sonatas that are not prescribed by the composer. Kibbie said it is these nuances that make music interesting.

“The Trio Sonatas are notorious for this unpredictability of performance,” Kibbie said. “What Bach wrote on the music score is not everything that’s needed for a complete, successful artistic performance.”

Kibbie, Forger and their student assistants represent one of the four teams participating in MIDAS-funded Data Science of Music Challenge Initiative research this school year. Kibbie said it is wonderful to have received an opportunity from the University to engage in unique, interdisciplinary research between the fields of math and music.

“Musicians are sometimes skeptical of math analysis and data analysis because so much of what we do is intuitive, and I hope particularly the musicians will come away with an understanding that data science can support our creativity and our intuition and challenge it,” Kibbie said.

University alum Kevin Flannagan studied data analysis and music in conjunction as an undergraduate. He returned to campus to see the presentation. Flannagan told The Daily after the event that he sees many similarities between the studies of math and music.

“They’re both things that kind of reward focused effort and people who are sort of willing to spend a lot of time sitting alone working on something, so I think the skills that you need are the same for both,” Flannagan said.

Kibbie said the University provides the unique context most conducive to this type of interdisciplinary research.

“We have the best of both worlds,” Kibbie said. “We operate as a small, elite conservatory but embedded in this first-class research institution, so our students have the resources of the full University while we also have the focus study of music or theater and dance.”

Kibbie said the team intends to continue with and expand their research. The University is in the process of developing new technology to get more pure and detailed data.

“This is just the first phase,” Kibbie said. “The really exciting thing is that the School of Music, Theatre & Dance is raising funds now to commission a major new studio organ that is going to have groundbreaking data interface … It will be the first organ in the world that will allow data collection in a much more nuanced way that is possible here.”

Kibbie said he is extremely proud of the work this research is accomplishing at the forefront of its field.

“It is groundbreaking,” Kibbie said. “There are a couple of other places that are looking at similar things, but what people hear today are things that no one has done before.”