Sound and Image

AE interviews Composer and Video Artist
R. Luke Dubois
February 15, 2007

On Monday, February 26, EMF and NYU present Mark Hetzler & Luke Dubois in a multimedia concert combining instrumental performance with interactive electronics and video. We asked them both a few questions about working with sound and image together. Below is our interview with Dubois. Read our interview with Hetzler here.

AE You're a composer and a video artist. What are your ideas on combining music and image in live performance?

LD I originally started working with video as a way to defeat what I saw as the two main demons of a lot of electronic music performance, which paradoxically can be termed the problem of transparency and the problem of opacity.

Problems with transparency stem from the fact that the performer is often engaged in what appears to be utilitarian or pedestrian action (typing or moving a mouse) - the 'magic' is missing from the performance. The average audience member thinks to himself, 'I own a computer. How come I'm not up on stage?' This is a perfectly reasonable question, because the tools and performance practice of electronic music are hopelessly muddled with the tools and practice of our everyday engagement with technology. That is, they're transparent. At the same time, by disembodying music from the world of voices and acoustic instruments, computer musicians have made the experience of watching live performance hopelessly opaque. The average viewer has no way of understanding what's going on inside the computer, or what the performer is doing to create the music.

Video forces a middle ground, by (a) inserting a little bit of 'magic' back into the performance experience (no more simply watching a guy typing; now his motions trigger amazing visuals) and (b) explicating or, if you will, 're-embodying' what's going on under the hood, by creating a visual mapping of the acoustic experience.

Conceptually, I think working with projected imagery in the context of a musical performance is a natural outgrowth of how our imagination responds to sound in the first place. The tendency in terms of how we perceive our world is to unify rather than segregate our different senses. Film theorist Michel Chion speaks of an 'audiovisual contract' in cinema that allows the sound and picture to reinforce and inform one another on multiple levels. I'm interested in inserting that idea into the context of how we experience what is nominally a 'musical' performance, by making it not a 'musical' performance anymore but a new kind of performance where all sorts of interesting things can happen.

AE Are sound and image equal contributors to the aesthetic experience, or does one inform the other?

LD It's an interesting thing working in both sound and image. Conversationally, we privilege our sight above our hearing: this is why when asked to describe someone we usually say how tall they are rather than discussing their vocal timbre. Perceptually, however, our ears can process data faster and with a higher spatial accuracy than our eyesight, so working in sound gives you a lot more bandwidth, in a way, than working with video, which has no real depth and can be turned off entirely by closing your eyes.

A lot of my interest in working with sound and image at the same time is that there are always more than one way to experience a phenomenon. As a computational artist, I work a lot with algorithms that can be mapped visually or sonically with equal facility, and the choice of mapping and the choice of medium will reveal different things about the information being used as the basis of the work. For example, my pieces written using Lindenmayer systems (algorithms that can be used, among other things, to model biological forms) are visually mapped out as renderings of plant-life; sonically, however, they make for this interestingly recursive musical landscape that works along a fairly clear formal hierarchy and has a number of compositionally interesting features. These musical features would never be apparent in the picture, and vice versa.

AE Your electronic pieces have often involved live instrumental performance. Can you say a few words about your approach to electroacoustic composition, and how it differs from (or is similar to) that of earlier electroacoustic pioneers such as Babbitt, Davidovsky, Boulez, and others?

LD I think those specific composers were, in many ways, fenced in by the modernist project and the artistic framework it forced them to work within. Abstraction and formalism are interesting creative spaces, and the ability to keep a critical distance from the mainstream cultural discource is definitely useful at times. However, this creates certain blind-spots, not the least of which is a tendency to create work which is conceptually irrelevant to anyone not culturally invested in what's going on inside your fence.

I believe in the idea that every culture will leverage the highest level of technology available to it at the time in order to make art. People are scared of computers sitting onstage alongside traditional instruments, but the truth is that this kind of thing has been going on for centuries and it's a perfectly normal, healthy, and sane mode of cultural progress.

People criticized the use of industrial manufacturing for pianos; orchestras blocked the introduction of the saxophone; the synthesizer had musicians around the world up in arms. These moral panics are designed to delude you into thinking that 'technology and the arts' is a zero-sum game, i.e. when you add something to the mix (the computer) you lose something in return (musicianship, or whatever). However, instrumental performance is a priori a technological endeavor, and so integrating electronics, computing, video, et cetera into instrumental performance is merely using available technology to try and do something interesting.

Unlike the composers mentioned above, I do not have strong feelings about the tight integration of instrumentalists with digital machines. I'm much more interested in how people take in my work (or gaze back at it) than in the mechanics of how it was made or how it holds up to formal theoretical analysis. Of course, this is the advantage of time passing: composers my age have a technocultural literacy (and an apparatus for critical thinking about technology) that composers didn't have access to when first developing the electroacoustic approach to music. Our understanding of how electronic and digital technology integrates into our culture has changed as well (see the transparency / opacity problem mentioned above).

AE How does the algorithmic aspect of your music (and your video art) relate to the aesthetic aspect? To what extent is the algorithm the art?

LD I use algorithms to execute concepts... the algorithm is only as good as the concept, and the emanation or execution of the algorithm (i.e. the art) is only as good as the idea was in the first place. So the algorithm is an integral part of the artwork, but only insofar as it is a vital tool to get from point A to point B.

One of my more recent forays into algorithm design was a couple of years ago, when I was trying to figure out the sound you had ringing in your mind's ear when you left a concert. So I decided to write a computer program to radically speed up the timebase of sound. Lots of people have done this before (one pretty common group of techniques is called 'phase vocoding') but I wanted to do it in a way that simulates a long-exposure photograph, fusing everything into a gestalt impression of what transpired. The algorithmic technique I concocted (which I called 'timelapse phonography') is by no means a scientifically accurate, computationally effecient, or even very logical chunk of code. But it captures the concept, and the results are what I wanted.

© 2007