In March of this year, the Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT) received a prestigious grant from the Canada Foundation for Innovation totaling $9 million.
Housed at the Schulich School of Music and led by Professor Marcelo M. Wanderley (Music Technology, Department of Music Research), CIRMMT brings together researchers from across disciplines and institutions to create scholarship and artistic works spanning four key themes: instruments, devices and systems; music information research; cognition, perception and movement; and expanded musical practice.
We spoke with Marcelo and Prof. Fabrice Marandola (Percussion Area, Department of Performance; CIRMMT member and former director) about what led them to study music and technology, their current research projects, and the importance of the CFI grant for the future of CIRMMT.
The two of you followed quite different paths to your current positions, Marcelo through engineering and music technology studies, and Fabrice through percussion performance. What sparked your individual interests in music and technology, and how has this shaped your careers?
MARCELO: I come from a family of engineers (father, grandfather), so from a very early age, I was convinced that I would also be one. Then, around age 14, I developed an obsession with music (progressive rock & classical), mostly as a listener, though I studied classical guitar for a few years. The illumination came when realizing that most of the technology in recorded music is derived from electrical engineering (guitar pedals, amplifiers, synthesizers, mixing desks…) and, more recently, from computer science/engineering (audio software synthesis and processing, digital signal processing, etc.). Bingo! There I was – music and engineering, or, as we call it at McGill, Music Technology. This is in the early 80s in Curitiba, Brazil, where I was born. At 17, I applied to study electrical engineering at the Federal University of Paraná and was accepted. Five years later—the normal duration of an engineering degree in Brazil—I obtained a diploma in electrical (electronics) engineering but learned almost nothing about music technology. Nobody played even a single sound when explaining filters, amplifiers, signal modulation, etc. Lots of equations, though!
I then moved to another city, Florianópolis, an amazing place (I recommend a visit!), to do a master’s degree in electrical engineering. The course at the Universidade Federal de Santa Catarina covered digital signal processing, filters, analog and digital circuit design, etc., in other words, the basic blocks of a CD player—remember, we were in the late 80s. But again, those classes did not address audio signals per se. At most, spoken voice; forget about the singing voice!
After completing my master's degree, I promised I would never do a PhD in electrical engineering, then went to work for a commercial company in Lagos, Nigeria.
Things changed in 1994 when I had was offered a research position in Curitiba and could develop a first project using sensors in a musical application, creating dance-music interfaces. Then, in 1996, I had the opportunity to pursue a PhD at IRCAM in Paris, France. Finally, I could combine my music and engineering interests while working under the supervision of my current colleague, Philippe Depalle, and Xavier Rodet on IRCAM’s Analysis-Synthesis Team. I defended my thesis on June 07, 2001, and was offered a tenure-track assistant professor position at the then Faculty of Music at McGill after my job interview on June 18. I officially started at McGill in early October 2001.
FABRICE: I grew up with my father playing drum-set from time to time when reuniting with his high school friends, and I had been wanting to play percussion since my early childhood. I was very fortunate to attend the local music conservatory in Montluçon (France), whose young percussion professor turned out to be passionate and incredibly inspiring for a whole generation of wannabee drummers! I also loved mathematics and science, but at the end of my high-school years the music option was embedded in human sciences, so I had to let go the more science-based courses.
My first discovery of the link between music, technology and research occurred during my first year in musicology at the University of Paris IV-Sorbonne, where I had the chance to participate in an experiment on the influence of timbre on the perception of pitch in xylophone music: the experiment was led by Simha Arom, who later became my director of research for my Ph.D., and conducted by Frédéric Voisin. Arom’s team was back from the Central African Republic (RCA) where they had conducted field experiments with xylophone makers and players, bringing with them a Yamaha DX-7 synthesizer, which keys had been covered with light wooden bars to resemble a xylophone. Local tuners and players could modify the pitch and timbre of the synthetized sounds that had been created to imitate original sounds, and the research was aiming at studying the musical scales by allowing participants to check, in real conditions (i.e. by playing their musical pieces on this ‘xylophone’), if their modifications of the scale was correct. Discovering the video of this field-research, and the incredible ease with which the players were including this new instrument within their ensembles, pretty much decided my research trajectory afterwards: while I was pursuing my performance studies in percussion in various Paris conservatories, I began a parallel path in ethnomusicology at the Sorbonne, studying music first from the RCA, then from Cameroon. For my PhD, I implemented a new methodology involving digital multitrack recordings and the use of software developed by IRCAM researchers, to study the musical scale system of vocal polyphonies of the Bedzan people.
Fast-forwarding the following years, during which I taught in French conservatories and universities, and gigged in symphonic orchestras (Radio France), I started my career at McGill in September 2005. Soon after my arrival, I met Marcelo and we started working together, mainly in a spirit of exploration of new possibilities afforded by technology. One of the first things we did together was to test the responsiveness of various electronic drum systems, and to embark on trying to better understand percussionist gestures by using motion capture devices. In parallel, I started to do more and more music with live electronics, collaborating with our Dean, Sean Ferguson, at the Digital Composition Studios, and in 2007 I gave the first live@CIRMMT concert of what has been since then a long and successful concert series!
Tell us about your current research projects. What are you (and your students) working on right now?
MARCELO: My students and I work at the Input Devices and Music Interaction Laboratory (idmil.org), conducting research and development on digital musical instruments (DMIs). DMIs, in their basic form, are musical instruments in which an input device or musical interface controls sound synthesis algorithms on a computer by mapping sensor signals to sound controls. The main conference where we publish our research is the International Conference on New Interfaces for Musical Expression (NIME), which started as a workshop in 2001 and has been a full-fledge conference since 2002. Together, we have published 80 papers in NIME so far, including some of the most-cited papers in the conference’s history.
Projects span from the development of hardware interfaces for musical performance, for instance, the T-Stick, Probatio, the SlapBox and recently, the Agbau, and software tools for creating mappings (libmapper) or processing audio signals (the Digital Orchestra Toolbox), to the analysis of performer movements using tools such as motion capture systems. Strong interdisciplinarity is integral to almost all of our projects, with some of the main ones taking the form of multiple-year developments involving composers (Sean Ferguson and Sandeep Bhagwati) and multimedia artists (Chris Salter), as well as neuroscientists (Robert Zatorre and Virginia Penhune), psychologists/cognitive scientists (Caroline Palmer, Catherine Guastavino, Ilja Frissen) and other colleagues at the SSoM (Fabrice, Isabelle Cossette, Philippe Depalle).
Current projects aim to bring musical expertise to validate engineering developments, specifically for the T-Stick and the Agbau performance projects, involving student composers, performers, UX and music technology. Such projects bring another aspect of music research to traditional areas within music faculties and open up innumerable opportunities to propose new music and ways of performing, including the creation of musical instruments that address the needs of the d/Deaf community and/or cultural communities in Brazil (Agbau).
Fabrice and I collaborated on multiple projects related to the motion capture of music, including music performance analysis (sticking choices in timpani sight-reading), the synthesis of a virtual percussionist (Vannes), and the study of synchronization in small ensembles with humans and avatars (SynchMuse).
FABRICE: Besides the projects mentioned by Marcelo that are dedicated to scientific outcomes, but which often have potential implications for pedagogy or creative endeavours, my artistic research is mainly linked to my percussion ensemble Sixtrum. With my percussion colleagues, I work closely with composers to develop new works that often involve live electronics, and that you can discover on a regular basis in performances in the School’s Multimedia Room! Recent projects linked to CIRMMT researchers, collaborators and students (Myriam Boucher, Gabrielle Caux, Simon Chioini, Martin Daigle, Ana Dall’Ara-Majek, Pierre Michaud, Olivier Robin, Dominic Thibault, Jérémie Voix), include Les Espaces Physiques with Carmine Cella; the project Transformations that explored transformation of sound through water, live electronics and robotic drumsticks; a tribute to Xenakis with Polytope XXI and the construction of a new type of instrument; and the most recent Espaces d’interaction #1 that featured several works composed by CIRMMT students and collaborators.
Current and recent student projects range from expanded drum set practices with multiple devices (Martin Daigle, Tarcisio Braga) to the analysis of synthetic percussive sounds found in early video games and their reorchestration in percussion ensemble music (Joseph Chang), and from the creation of the first catalogue of timpani sounds counting over 2500 samples, captured in CIRMMT semi-anechoic chamber (Manuel López Tovar) to the exploration of percussion gestures with or without digital technologies (Antonin Granier).
The CFI Innovation Fund focuses on giving research institutions financial support to upgrade their infrastructure. What changes will this funding bring to CIRMMT in terms of resources and equipment?
MARCELO: There are three main aspects in this CFI Infrastructure grant:
- For the first time, it will partially cover operation salaries; up to 40% of CIRMMT’s operational costs. This will bring much-needed funds to cover CIRMMT’s daily operations, primarily for technical support for interdisciplinary research, allowing us to improve the sustainability of our operations.
- It will allow for the renovation of equipment in our six laboratories, which were completed in 2006. Some of this equipment is reaching the end of its useful life and will need replacement/improvement to catch up with current technology.
- It will allow for the acquisition of technology for new research activities, most notably a large Wave Field Synthesis system to be placed in the MMR (but moveable in the form of smaller, partial systems, if needed), newer acoustic and physiological measurement systems, and XR (Extended Reality) systems for immersive experiments and performances.
Furthermore, as with all CFI grants, 30% of the CFI portion, 40% of the grant value, or 12% of the total value, is made available as extra funds on top of the grant amount, for the operation of the infrastructure in the five years after its completion. This is called the Infrastructure Operation Fund (IOF).
Altogether, with a grant value of $9M and an IOF amount of almost $1.1M, we are talking about a $10M project. This is the fourth CFI IF grant CIRMMT obtained, the second one in the last 12 years. The total funds, not adjusting for inflation, exceed $35M for infrastructure and operations. Thanks to the CFI and the Government of Québec’s continuous support, CIRMMT has developed from a group of twelve world-leading researchers to an unparalleled infrastructure for interdisciplinary research in the science and technology of music and audio.
How will the CFI grant impact research at CIRMMT? What are the Centre’s future research priorities and what types of new projects can we look forward to hearing about?
FABRICE: As Marcelo mentioned, one of the most powerful impacts lies in the possibility for CIRMMT and its members to remain ahead of the curve in the exploration of music and sound, thanks to our access to cutting-edge equipment.
Regarding research priorities, CIRMMT is quite unique since its members are de facto driving research and research-creation, while the Centre provides a unique nest for these projects to develop, thanks to its laboratories, equipment, technical support and a remarkable network of researchers. Coming from very different disciplines, all researchers have in common a passion for sound and music, that is at the heart of what they come to explore at CIRMMT, and to do so in collaboration with each other. It is with their recommendations and priorities in mind that we targeted two main areas of research development: immersive environments, as well as interactive tools and techniques.
The new infrastructure will allow for research on multiscale immersive environments, with the development of systems that can accommodate everything from large orchestral and multimodal setups for research involving multiple participants, musicians, and audiences to individual in-ear and bone-conduction devices. It will favour hybrid technology integration by ensuring compatibility among various immersive audio and video technologies to facilitate the development and evaluation of innovative hybrid approaches and creations. It will also support investigating human auditory perception in immersive contexts, capturing, rendering and analyzing urban and personalized soundscapes, and innovations in music and sound art by developing new forms of sound installations and performances combining immersive audio and video, as well as wearable extended, virtual, and augmented reality (XR/AR/VR) studies simulating sound and visual environments.
With regard to interactive tools and techniques, we aim to develop new responsive, robust instruments and interactive installations for music interaction in the groundbreaking live@CIRMMT concert series and help democratize music access for disabled and/or underprivileged communities. Furthermore, novel motion and volumetric capture systems, combined with biosignal acquisition devices, eye trackers and in-ear devices (wearables and hearables, i.e., in-ear wearables equipped with various sensors), will allow for the study of realistic interactions in performance science research.
Our ultimate goal is to support ongoing interdisciplinary research and partnerships and have a transformative effect by systematically eliminating the barriers between research and creation with artistic, social, health, educational, and economic benefits for Québec and Canada.
What advice do you have for students interested in studying music and technology, and how can students get involved in CIRMMT?
MARCELO: I do encourage such students to break silos and experiment with taking classes and collaborating with colleagues and researchers from other areas. Students in my laboratory come from many different areas, including engineering, computing, mathematics, physics, and psychology, as well as music performance and composition. They work together on interdisciplinary projects involving multiple areas of expertise. At CIRMMT, the variety of backgrounds and the blend of disciplines is even greater. Students can propose their own projects, seek support from CIRMMT researchers, or collaborate directly with peers from other fields, often obtaining funding to do so. There are no limits on the types of projects that can be done at CIRMMT, except that they must include some aspect of sound and/or music.
Basically, any student in Québec can be a CIRMMT student member. They need to be sponsored by a CIRMMT regular member (researcher), who does not need to be their supervisor. Once sponsored, they will be asked to complete a one-page form indicating their main research interests. They will be required to attend an orientation workshop that provides basic information on the Centre’s functioning, safety, and laboratory/equipment booking rules, etc.
FABRICE: I “grew up” as a researcher in an interdisciplinary laboratory that was full of anthropologists, linguists, ethnobotanists, and of course ethnomusicologists, so I firmly believe in the necessity to break down silos and I enthusiastically second Marcelo’s words! I would add that we tend to create an imaginary barrier between science and technology, versus music and human sciences. Nowadays, everyone is constantly using advanced technological devices (computers and phones to start with), and it is not necessary to know how to make these tools to be able to use them creatively. The best asset for any student will always remain their curiosity: as much as I encourage them to go listen to all genres of music, explore museums, enjoy theatre and dance shows, I encourage them to attend as many CIRMMT events as they can, since inevitably, they will discover some new ways of thinking about music or science that will have an impact on their artistic or scientific path.
In short, do not let the system decide your next steps, but on the contrary exploit every opportunity that is provided to you to create your own adventure – and CIRMMT is definitely a place where such adventures may start!
To learn more about CIRMMT and its activities, see this video on their website.
