http://www.youtube.com/watch?v=1G0MzlfMPuM
My name is Golan Levin. I'm an artist and an engineer. Which is, increasingly, a more common kind of hybrid. But I still fall into this weird crack where people don't seem to understand me. And I was looking around and I found this wonderful picture. It's a letter from "Artforum" in 1967 saying "We can't imagine ever doing a special issue on electronics or computers in art." And they still haven't. And lest you think that you all, as the digerati, are more enlightened, I went to the Apple iPhone app store the other day. Where is art? I got productivity. I got sports. And somehow the idea that one would want to make art for the iPhone, which my friends and I are doing now, is still not reflected in our understanding of what computers are for. So, from both directions, there is kind of, I think, a lack of understanding about what it could mean to be an artist who uses the materials of his own day ( ), or her own day. Which I think artists are obliged to do, is to really explore the expressive potential of the new tools that we have ( ). In my own case, I'm an artist, and I'm really interested in expanding the vocabulary of human action ( ), and basically empowering people through interactivity. I want people to discover themselves as actors, as creative actors, by having interactive experiences ( ).
A lot of my work is about trying to get away from this. This a photograph of the desktop of a student of mine. And when I say desktop, I don't just mean the actual desk where his mouse has worn away the surface of the desk. If you look carefully, you can even see a hint of the Apple menu, up here in the upper left, where the virtual world has literally punched through to the physical. So this is, as Joy Mountford once said, "The mouse is probably the narrowest straw you could try to suck all of human expression through."
And the thing I'm really trying to do is enabling people to have more rich kinds of interactive experiences ( ). How can we get away from the mouse and use our full bodies as a way of exploring aesthetic experiences ( ), not necessarily utilitarian ones. So I write software. And that's how I do it. And a lot of my experiences resemble mirrors in some way. Because this is, in some sense, the first way, that people discover their own potential as actors, and discover their own agency. By saying "Who is that person in the mirror? Oh it's actually me."
And so, to give an example, this is a project from last year. Which is called the Interstitial Fragment Processor. And it allows people to explore the negative shapes ( ) that they create when they're just going about their everyday business. So as people make shapes with their hands or their heads and so forth, or with each other, these shapes literally produce sounds and drop out of thin air. Basically taking what's often this, kind of, unseen space, or this undetected space, and making it something real, that people then can appreciate and become creative with ( ). So again, people discover their creative agency in this way. And their own personalities come out in totally unique ways ( ).
So in addition to using full-body input, something that I've explored now, for a while, has been the use of the voice. Which is an immensely expressive system for us, vocalizing. Song is one of our oldest ways of making ourselves heard and understood. And I came across this fantastic research by Wolfgang Köhler, the so-called father of gestalt psychology, from 1927, who submitted to an audience like yourselves the following two shapes. And he said one of them is called Maluma. And one of them is called Taketa. Which is which? Anyone want to hazard a guess? Maluma is on top. Yeah. So. As he says here, most people answer without any hesitation.
So what we're really seeing here is a phenomenon called phonaesthesia, which is a kind of synesthesia that all of you have. And so, where as Dr. Oliver Sacks has talked about how perhaps one person in a million actually has true synethesia, where they hear colors or taste shapes, and things like this, phonaesthesia is something we can all experience to some extent. It's about mappings between different perceptual domains ( ), like hardness, sharpness, brightness and darkness, and the phonemes that we're able to speak with.
So 70 years on, there's been some research where cognitive psychologists have actually sussed out the extent to which, you know, L, M and B are more associated with shapes that look like this, and P, T and K are perhaps more associated with shapes like this. And here we suddenly begin to have a mapping between curvature that we can exploit numerically, a relative mapping between curvature and speech ( ).
So it occurred to me, what happens if we could run these backwards? And thus was born the project called Remark. Which is a collaboration with Zachary Lieberman and the Ars Electronica Futurelab. And this is an interactive installation which presents the fiction that speech casts visible shadows ( ). So the idea is you step into a kind of a magic light. And as you do, you see the shadows of your own speech. And they sort of fly away, out of your head. If a computer speech recognition system is able to recognize what you're saying, then it spells it out. And if it isn't then it produces a shape which is very phonaesthetically tightly coupled to the sounds you made. So let's bring up a video of that.
