IU researcher develops virtual orchestra

floor office, a sophomore music student cradles her violin under her chin as the fingers of her left hand dance along the strings. Her right hand wields the bow alternately with a controlled grace and a dizzying speed. \nA nearby microphone records the notes of the violin and the simulated orchestra that plays through a speaker. This orchestra is Music Plus One, a computer program that Raphael has been developing for about 15 years. \nThe IU associate professor has dedicated a lifetime to music, first as a musician and now as a researcher. Raphael applies theories from artificial intelligence networks to the task of musical accompaniment, said Charles Fox, a researcher interested in applications of computers to music.\nMusic Plus One is a computerized program that provides musical accompaniment that responds to the performer's changes in expression and tempo.\n"Players can practice and perform with a computer backing track that follows their actions," Fox said of Raphael's system. "It's different from karaoke-style backings because it listens to how you change speed and expression and lets you take control." \nRaphael said his original goal in life was to be a professional oboist. He left the musical conservatory at Northwestern University before graduating to play music professionally, but while playing in an orchestra at University of California, Santa Cruz, he began to study computer science. \nWhat began as a practical decision soon grew into a genuine fascination, and as his interest in computer science grew, Raphael said he became increasingly aware of his tendencies for perfection. He decided this temperament made him a better scientist than musician. \nRaphael went on to study at Brown University, where he earned a Ph.D. in applied mathematics. After graduation, he learned about speech recognition technology. \nRaphael started wondering if it would be possible to follow a musical score in real time, as the music is being played.\n"For five or six months I thought about nothing else," he said. "I just went berserk." \nRaphael's previous work established a correspondence between a musical score and a recording, but the thought of doing matching in real time consumed him. He found the ideal environment to work on his project while serving as a post-doctoral student at Stanford's Center for Community Research in Music and Acoustics. \nRaphael worked and slept in a building that had technically been condemned after the Loma Prieta earthquake struck years earlier. It wasn't until the day he moved out that he experienced the sweet taste of success.\n"I got up at about 4:30 a.m. and I had my first successful experiment," he said, noting that he is never up that early. "I still have the recording of that." \nRaphael said he will never stop working on the project. It brings him joy, he said, and he sees the program as being a useful tool for serious musicians. And he's not the only one. \nMimi Zweig, a professor at the IU Jacobs School of Music, and her students are using the system to play orchestra accompaniments to violin concertos. She described the learning experience as invaluable for students. \n"My students have enthusiastically embraced the system," she said. "It gives them the opportunity to play with a 'real' orchestra, instead of piano accompaniment." \nThe opportunity for musicians to perform solo with an orchestra can be rare, which is too bad, Raphael said. \n"It's just a fundamentally informative experience," he said. "You're the leader, and you learn so much more as leader than follower in music and in everything else." \nYoo-jin Cho, a student in the Jacobs School of Music, has had the opportunity to use Raphael's system. \n"Whatever tempo you take, the orchestra will follow," she said. "This program seems to give performers more flexibility and accuracy." \nShe said she would definitely recommend the program to other musicians because it allows a thorough understanding of the music they're studying. \n"In the past, musicians have struggled learning their concerti because the piano part can't fully describe the orchestral sound as the composer intended," she said. "However, they had no choice unless you're fortunate enough to get to play with an orchestra." \nRaphael welcomes interested musicians to contact him about using his accompaniment system. For comprehensive information, visit his Web site, http://xavier.informatics.indiana.edu/~craphael/.