22: Isaac Cohen—An XR Trailblazer on How to Build Humane Technology
Listen to the episode on iTunes
AMLG: I’m excited to have Isaac Cohen here today — he goes by Cabbibo on the internet. I’ve been a longtime follower of his work and he’s become a friend through the Wave VR and the content that he puts on their platform. He’s an XR artist technologist and has published a number of otherworldly audio-visual objects and experiences, everything from rainbow membranes and jelly crystals to reverberating ethereal beings and swimming fish eyeballs — we’ll get into it and I’ll share visuals. Isaac has a fascinating background in physics and interface design. He’s worked at Leap Motion, been an artist in residence at places like Unity and Adobe, and has released a bunch of great open source tools. Isaac you’re such an interesting thinker and an optimistic human — thank you for coming on the podcast.
IC: Hello and thank you for the descriptions. Those are some good ones.
AMLG: To start — tell me a bit about your background, where you’re from and your journey so far.
IC: I grew up outside Portland, Oregon. It was pretty rural. We didn’t have a TV so I spent a lot of my time in the woods making things. One of our favorite activities, because our house was on stilts above the ground — was when it rained we would go underneath the house and craft these waterways and get covered in mud. It was me and my older brother, and my mom would be livid because we’d get mud everywhere. So we had a lot of outdoor time. My mom’s rule was “you can do anything you want as long as you’re wearing a helmet.” One time I was building a tree fort and asked, “Mom can I borrow the saw I want to saw off the branch I’m standing on to see if it breaks differently?” and she said, “OK, just put on the helmet!” Obviously it was a mistake. Also my dad was a really good teacher and I always wanted to keep up with my brother, so even though he was three years ahead of me in school my dad would sit down and teach me the stuff that he was learning. I remember in preschool him trying to explain negative numbers and me being like “what in the hell…”
AMLG: That’s an advanced age for negative numbers.
IC: I don’t think I learned it, I just remember him trying to explain it.
AMLG: Is he also a programmer, your dad?
IC: Kind of. He’s a patent attorney. He worked for a bit at the MIT AI lab and helped them way back in the day program Logo. He taught me logo. I literally learned Logo before I learned to read. I learned programming very young. But then in second grade I stopped doing it because nobody told me that you could make the universe, that you could manifest whatever you wanted. I thought it was just drawing spirograms. That it was cute and fun and you could make pretty patterns. I ended my programming career at second grade and didn’t do much with computers after that or even much art.
My grandma lived out at Cannon Beach which is a beach town. Honestly going out and sitting in the waves on the Oregon coast is probably the thing that has helped me the most in terms of patience, in terms of focus, in terms of being OK with the unknown and knowing how to not panic. Because some days you’re out there chilling for eight hours and you catch three waves — it’s relaxing and meditative. You’re doing digital signal processing in your mind because you’re watching all these waves come in and trying to filter out the signal from the noise and trying to say OK if this wave comes this way and that wave comes that way, those two will meet each other three points to the left so can I get to that when they double up and peak. So you’re doing all of this processing —
AMLG: That’s your physics brain at work.
IC: Yeah haha. But then other times it’s just lizard brain survival. Like oh my god I’m stuck and there’s a massive wave. The craziest thing about it is you have to stay calm when it is the scariest. You get thrown over and say I know I’m going to be held under and it’s going to be too long and I’m going to be uncomfortable. But if I freak out I might die. Stay chill or you will die—
AMLG: Good way to approach life in general. Here comes the tidal wave but I’m chill about it and I’ll be fine.
IC: Totally. The best you can do is all you can do.
AMLG: It sounds like that’s also an approach you’ve brought to your work. Lets explain to listeners what you’re doing. I first discovered your stuff through the Wave VR where we’re invested — you’ve got some amazing experiences in there that people love. But your first VR pieces that I saw were things like L U N E, which is one of the experiences that I always show people, it’s a go-to.
IC: That’s definitely my favorite one that I’ve done in VR.
AMLG: Really? I’ll pull it up here and you can describe it — what is L U N E and why did you do it?
IC: Before I started doing VR I did a lot of WebGL pieces. I made this one WebGL piece called “Enough” and it’s sort of it’s like a 45 minute experience — to use a music parallel you could say it’s the only album that I’ve ever released. I’ve done a lot of singles. Then I came to VR. I started by making BLARP! which is a fun cute arcade game.
AMLG: That’s the one where the eyeballs fly at you and you bat them away?
IC: Yeah. There’s some parts of it that are important to me. It’s interesting to think about control and movement and that one forces you to make long, slow movements which I like. But in general it isn’t meaningful, it isn’t me trying to say something that has helped me deal with being alive and the pain of it or the good parts of it.
But after I published “Enough” I thought, I’m done with the Cabbibo name. That was the thing that I wanted to say. I want to do something else. And what I ended up doing was making another website called L U N E. L U N E is a mathematical symbol that’s the intersection of two circles. One of the things that’s difficult for me to do as an artist, and I think it’s true for a lot of artists, is to say the quiet things. It’s easy and survivally fit to say the loud things. If it’s loud and it’s bright people will notice it. I wanted to challenge myself to say the quiet things because to me those are the moments that really matter.
AMLG: I definitely felt that while doing L U N E and when I put friends in it. I do want to talk about how your work feels less goal-oriented. Generally when people go into VR they’re like, what am I supposed to do next? A B C, how do I finish, when does it end? But when you go through the L U N E experience you’ve got this blanket and these sticks and you kind of have to figure it out and be patient. It feels meditative — to your point about being calm and not so loud — it does feel like that.
IC: Yes and it’s hard to do in a way that isn’t frustrating, because it’s easy to say, “OK you have to calm down because you have no idea what you’re doing and you’re getting frustrated.” But it’s more difficult to make something that is learnable by subtle cues and juicy feedback. This piece you can finish in a minute or in an hour and you can keep on going through it again and again and again.
For me this one is about how do you do that in a way where you’re calm but there still is specific narrative progression. Your first emotion is that of being alone under the night sky and that being a bit scary but full of awe. Then feeling “ohhh this is beautiful.” Then the cloth starts falling. At first you think it’s the ceiling because there’s no structure to it, so you think oh my god I’m going to be crushed. That existential terror turns into claustrophobia and being scared that everything’s closing in. Then the moment it hits the rods there’s this golden divine cloaking you in comfort. So there’s a specific progression. Finally the cloth falls through you and through the ground and there is a feeling of, OK you became intimate with this cloth and it sheltered you from the darkness. It kept you from the abyss. Then you watch it slowly fall away. For me it’s this moment of, to have loved and lost. Oh that thing that was so beautiful I’m watching it go. There’s this sadness when you can no longer pull it up and people are like no no and they try to pull it back up. But at the same time there’s a feeling of gratitude — thank you for cloaking me in comfort.
AMLG: Right — thank you for letting me just live the experience. That’s one of the amazing things about your work. People always ask me, “well what do you actually do when you go in VR?” I do a lot of things but your work is some of the only times that I actually have feelings in VR. I want to read one of your quotes and ask you about it. You’ve said:
“Let’s get weird with this new medium, spreading out our tendrils into the majestic new possibility space until we find the places that are *truly* compelling, that let us as humans understand more about the universe around us and help us discover more of our magical reality.”
How do you think about VR as a medium? Do you still feel that’s true?
IC: Totally. The thing that I’ve always felt is true about technology is, if you are trying to make it take the place of something in our world it is going to fail. Whether it’s realistic rendering or compelling mechanics — like Rocket League has compelling mechanics — but it is nothing like BMX or actual soccer. If you’re trying to replicate this reality you’re going to fall short. Our minds, the human scale of things is this tiny little part, like the visible spectrum of light. There’s so many other wavelengths but we only experience a tiny little part. There’s only a minuscule part of our reality that we can see and feel and therefore understand. What else is there? That is the question to me. Like when have you played with gooey pillows that hold up a golden cloth that shimmers?
AMLG: Yeah the closest is your example of pillow forts —the L U N E experience recreates some of those wonderful childhood memories for me, of when I would cover our room in tunnels for the pet mouse and then go in there with him running around. It felt like a different universe. Things that we don’t do in the real world anymore — that seems to me the most interesting in VR.
I’m going to share a document you wrote called, “10 quick things I’ve learned in VR” because these things rang so true to me. Not to hate on the people that are doing Solitaire in VR… but yes it should be transformative and not derivative. Think about how you can transform engineering into something creative that was never before possible, that pushes the limits of human understanding. To play more and experiment more. I’m curious — you are experimenting more with augmented reality as well. Are you doing both, have you fully transitioned, what’s been the journey from one to the other?
IC: I’m doing both. I definitely enjoy doing both. There are certain aspects of each one that are compelling. Think about what’s the thing that you want to do? OK then what medium should you do it for? Going from WebGL to VR was difficult. For “Enough” I had three million hits or so whereas Blarp! — which did very well in terms of VR — at most 20,000 people have bought it.
AMLG: So the audience is an order of magnitude bigger.
IC: Yes and especially with ARKit there is something exciting about a lot of people having iPhones. The problem is that it is still different than WebGL, because with WebGL you publish and everybody can touch and play with it immediately. With ARKit to publish you have to go through an entire process. I’ve actually only published one ARKit piece ARQUA!, which is done now I don’t know how to fix it —
AMLG: I loved ARQUA! I have a video of me ARQUA’ing — if that’s the verb — I’ll share that. I really enjoyed it while it was going. It was like making magical underwater glitter fish crystal kelp scenes around you — some combination of those things?
IC: Yeah! Making your room a bit more underwater-y and rainbow-y. One thing that’s exciting to me about all these mediums is that they’re more participatory and they are more about creative agency. It may be idealistic but I think that the more agency we’re taught in our day to day life and the more agency that we experience when we’re creating or playing or being with technology, the more that we expect out of those things and the more we feel we have agency in our own lives. My hope is that if somebody makes something with ARQUA! they might be like, oh I’m allowed to make art. I can try drawing if I want. Or even better — if I can digitally change this world, maybe I can actually change this world, what can I do to participate? Giving people the understanding that they are important and that they have the power to create things.
AMLG: When you think about the Internet and the way that it evolved — it was always meant to be more of a random walk, more of an experimental playful journey with user agency, or at least some people had that vision. Then it turned into this functional utilitarian Internet. One thing you’ve talked about in interface design which I think is true of computing in general is that it is very concerned with efficiency. So like, how do you achieve this goal, we click this button to get there, but there are other dimensions we’re not tapping into that are more about the wonder and the emotional side of computing and how technology makes us feel. I’d love to hear what you think about VR and AR — we’re at the beginning of these and how they might be different?
IC: Totally. We want the most out of things. That’s human nature. And efficiency is predicated on quantifiable data. Take something like Twitter we can say it’s efficient because you’ve scrolled X amount of tweets you’ve read this many things, so it’s efficient in a way that’s quantifiable. That’s the hardest part about so many of our rigorous, data driven techniques — they only work for things that are quantifiable. There’s some saying; we couldn’t measure what we thought was important, so we decided that what we could measure was important. We see that all the time.
AMLG: Right, eyeballs and clicks don’t measure how someone felt when they did that thing.
IC: Totally. I’m down to reframe it. Efficiency should not be oriented to just how many ads somebody saw — that’s a capitalistic, monetary efficiency. What about emotional efficiency? What about spiritual efficiency? We can also be thinking about emotional efficiency, not just how much did somebody scroll but how much of their energy did they use up while they were scrolling.
AMLG: And how much joy did they get out of doing it.
IC: Exactly. I remember one study where someone had to do a super banal task putting numbers into a spreadsheet. Then they gamified it so you’re flying around and putting numbers into a spreadsheet. There are some parts of this that are still wack because obviously a game where you’re flying around putting numbers into a spreadsheet sucks. But for the first part of the day the people in the Excel spreadsheet were beating the crap out of everybody who was playing the game because it was more efficient to do it that way. But then they burned out, because a human being is not meant for that. By the end of the day the people who were playing the game would get more done because they were having fun doing it. So one of those was more efficient, just in the longer term. That’s the crux to me — we aren’t allowed to think long term when all we’re worried about is returns. Timoni West who’s one of my favorite creators —
AMLG: Shoutout to Timoni! Love her.
IC: I wrote her down when you asked “who are your favorite creators?” — one of them is definitely Timoni. I love what they’re doing at Unity. They’re doing all these interface designs. But they also have to think about, how do we release something that people can use now? Which is a different question than, how do we research what design should be in 50 years? I’m sure they’re doing that and we just don’t get to see it yet. But those long-term research questions focused on how do we actually take a step forward that is not just optimizing and polishing a turd in the sense of UX, are much more interesting questions.
AMLG: One example that I love around efficiency and doing things differently — in Job Simulator instead of just exiting the game you finish by picking up the burrito and… taking a bite.
IC: Yes! You take a bite and then it says in beans EXIT and then you take another bite to exit. That to me is just like ugh, even leaving this thing is awesome. That’s one of the cool things about VR and XR right now. As one of my high school English teachers said, “if you can’t find something beautiful in a poem that is your fault.” That’s super true in VR. In every single piece you try there’s something massive to learn. That’s exciting. Even if something is garbage there is one little shimmer of aha! That’s the reaching the tendrils out into possibility space. Maybe you hit a tiny little moon instead of an Earth like planet. But building those up and figuring out that network of things that work is what we need to do for this medium to be a platform.
AMLG: And as you push the edges, as you experiment and discover these islands of possibility and learn about how you feel and how you interact with it, what are the boundaries or frustrations you come up against? In terms of tools or what’s missing? What do you wish existed?
IC: I mean I wish there was more money for people to experiment with. That’s the truth and it sucks but so much of it is just, how do you get money to have the mental space and the physical space to be able to experiment. A banal answer. As far as tools go one of my favorite parts about developing for AR on the Magic Leap is the fact that I can wear my headset and type. That is massive because iteration is always the thing that’s most difficult. For ARKit for my phone — luckily there’s this ARKit remote where you can use your phone as a remote but even that is still temperamental. And you aren’t using the actual thing. If you want to test it on your iPhone you have to build it out from Unity into Xcode and build it from Xcode into your phone. On the web you change a line of code and it’s already there. In Unity when you’re doing it locally you press play and it restarts.
AMLG: One of the other things people have said — you probably know Danny Bittman right? He announced that he’s taking a hiatus from creating in VR. He’s one of the people I’ve been following for a while. He said he felt isolated and lonely making stuff, that was one of the reasons. I presume in AR there’s other people around you can be iterating with while you build?
IC: Yeah but the isolating and lonely is what I like about VR. Those are the experiences that I want to exist in VR. I don’t think being isolated and lonely is a bad thing. It’s a bad thing if it’s the only thing you do. For me it’s important to work really hard but when you’re not working be with other humans —
AMLG: And run around the natural world to balance it out.
IC: Exactly. I do think being alone is an OK thing, though it can get to be too much. Tools wise yes, anything where you can create with other people is magical. If I could program in VR with other people there and we could have this golden floating jellyfish and I was like, OK let me grab this and put this here and actually start typing into it and do it that way, that would be it. If you have a slow iteration cycle the types of things you make are going to be different. So right now in VR we see a lot of box collider games, there’s a box collider you can touch it, you can move it, something like Job Simulator. They’re more about the memetics, about what everything is than the physicality or what the space is made of. The only way that you get to build mechanics that are compelling in VR is to be building those mechanics in VR. Same thing with AR.
AMLG: What about the limitations of physics on the CPU and how many vertices you can have in there, how many atoms you can describe within each physical thing that you make?
IC: Well you can’t complain. On one of these phones we’ve got 10x what we needed to send people to the moon!
AMLG: True. And it’s getting better every day.
IC: Yes there’s a lot of development in terms of hardware but there’s also a lot of development in terms of the structure of software. We are moving towards a compute shader framework that… so compute shaders are general purpose programming objects for the GPU. Everything up till now is the CPU much of which is single threaded. You can think about the CPU as a super sick artist who can draw a cat pretty well and fast. A GPU is a thousand not as good artists who draw much slower but there’s a thousand of them. So if I’m like yo draw me a thousand cats, they might take a bit longer but it’s different than one person drawing a cat and then drawing a cat and then drawing a cat —
AMLG: Simultaneous processing.
IC: Exactly. So it’s SIMD which is single instruction multiple directions — I don’t know what the D is but it means that for something that is atoms, each atom is calculating information about itself. That leads to a lot more interesting structures. Because at first the only things you were talking about to the GPU were vertex shaders and fragment shaders. Vertex shaders take vertices and triangles and tell them where to go on the screen. Fragment shaders tell you what color those sections are. That was the traditional pipeline for so long. But now that we have these more general purpose things, it’s opening the doors to different representations of space and color. It allows us to do things differently which is exciting.
AMLG: I feel if we were living in the Matrix you’d know how to manipulate the universe around us, you’d be like “that vertex is just wrong so I’m going to rip a hole in space time and redo this — ”
IC: Haha that human being you’re talking about is not me. It’s a dude called Timothy Lottes. There’s a few people who when you make eye contact with them you’re like you live in mathematics. Inigo Quilez who was at Story Studio and Pixar and also made the website Shadertoy is another one of those people. Timothy Lottes helped design Mantle, the precursor to things like Metal, and Vulcan is kind of based on Mantle. He is so low level. His entire operating system is like a Linux and he’s like that’s too high level. I’m lower level than that. He has two GPU buffers and a tiny bit of CPU memory but basically everything lives in these CPU buffers. He doesn’t want any latency. So when he does gamepad input he writes it directly to GPU memory — like, how do you do that? You need to be so intimately involved and knowledgeable in such a massive pyramid of what computation is currently. There are so many levels of who makes the chips and who does this and how you get in there and how do you do that and who made this — and he somehow understands all of that or at least has torn it down enough that it’s something that he can hold.
AMLG: So that’s someone you think is amazing on a low, systems level. What about on a high level, people who are creating hacking experimenting. Who do you admire, who inspires you, who do you look to in your community that you talk to about stuff?
IC: For me it’s important to actually not be inspired by our community. There’s a lot of super sick people I could name who are doing great things. But as with any nascent medium, people haven’t slated at that top level yet. If you want to look at the people who inspire me the most — it’s Miyazaki, Otomo who did Akira, Encyclopedia Pictura a studio that does animations, David O’Reilly.
AMLG: And what is it about them?
IC: They get it. They want to try to say things that… Their wave sets… When you’re sitting in the ocean, a single sine wave gets boring. There’s the building of tension and the releasing of tension. In our current paradigm so much of what’s being created are one hits. Like a Banksy piece. “Now you get it!” It’s one single thing. It’s surface level. There are people who are doing surface level in amazing ways, like Beeple to me is a person who slays so hard —
AMLG: I project his new pictures on my wall every day!
IC: I definitely have a lot of his art in my house. But somebody like Miyazaki, when I think about his stories — one story feels too complex for a human to hold in their mind. And the way they tie together and work together. Take Akira — how do you understand and comprehend and put in words something that is too big to understand? That’s exciting. These people who are dealing with things that are mysterious, that are too big for a human. Twin Peaks does a similar thing. And a lot of magical realism does it. I like when somebody tries to say everything and they know they’re going to fail at saying everything. Akira tries to say too much.
AMLG: But the viewer takes away some piece that reflects back on them that they relate to.
IC: Right. They’re looking at that same thing whatever it is, that weird morphing bigger-than-all-of-us thing, and also pointing to it. The role of the artist is not to say look at this thing I made, it’s to say — holy crap look at that out there! Oh my God isn’t that crazy! There’s one author, and oh man there’s no way I’m going to pronounce his name right— it’s C I X N…
AMLG: Cixin Liu — I love. Those. Books.
IC: I loved them! Oh my God I just finished Deaths End and told my dad about it. He was like “I loved Three Body Problem but Dark Forest and Deaths End were meh.” For me Three Body Problem was bonkers good but Death’s End — he goes there. It makes 2001 Space Odyssey look like Frasier. It’s sick for somebody to try to say the thing that is too much to say.
AMLG: And you get the sense that in Chinese it’s even a whole other level. After reading it in English, you come out of that and feel, wow my brain changed a little bit. My brain was speaking another language.
IC: I bet it is way tighter in Chinese. Also I think that’s just his language. He manifested that world in his mind and he’s the only human who can actually speak like that.
AMLG: Right. A lot of sci-fi feels iterative to me, it builds on prior stuff. But his just feels new and fresh.
IC: It’s interesting because there’s other people who do something that feels new. Another one of my favorite creators is Ursula le Guin.
AMLG: Me too. We are just jamming right now! I love her. And all the dragons.
IC: So after I finished “Enough” I was depressed because I thought how am I going to be able to make something like this again. “Enough” is still by far the best thing I’ve done and it scares me that I’m not going to be able to make a second thing as big as that. Then I read “The Dispossessed.” And I was like, oh I get it. Everything is alright now. So I wrote her a letter and said, “hey no need to respond to this I know you have a lot of younger readers and they need your support more. But I just want to say that your light shines so deeply through your work that I’ve felt it help reignite my light. And now your light is shooting through me and refracting into others.” Then two months later I get a card back from her. She’s got to be getting thousands of letters a day. So there’s a poem and some preprinted stuff and I thought OK it’s just a postcard — but then at the bottom she writes “trust in your light. It will shine through you and others.” I was like — you are my absolute hero.
AMLG: Wow. Did you read the Wizards of Earthsea? Those were my favorite growing up.
IC: Yeah definitely. She’s awesome. So much of sci-fi is about efficiency and she’s down to slow down and ask what are the emotional or spiritual or ramifications of technology. Have you read her rant on technology? You would love it. So everybody’s like oh Ursula leGuin isn’t sci-fi because she’s not “hard sci-fi.” And she’s like yo what? I’m not hard sci-fi because I don’t talk about technology? Technology is the way that we as humans interface with the world around us. That is all technology is. If you think that technology is something that is so complex that somebody can’t understand it, that means you haven’t tried building a fire in a while. Go build a fire and remember how hard it is to do that and then respect that because that’s a sick technology. She thinks about technology as a way that we interface with the world — it doesn’t necessarily have to be a complex progressive march towards a Matrix future. Technology can be something that is humane, technology can be something that is oriented towards us being humans with each other. It can be something that gets out of the way. Like that pen sitting there isn’t in your way right now but you’re able to use it even while still having a conversation.
AMLG: And you’re able to use it for evil or good (like a lightsaber...) Let’s talk about Dynamicland for a second because you’ve been spending time there. I’ve pulled up some of the stuff you’ve been prototyping and experimenting with. So Dynamicland is a center in Oakland, a communal, computational media authoring environment with paper and objects, no screens, a projector. Can you describe what’s going on in this scene here?
IC: So in that scene there’s a paper called a “Jellyfish” on everything. For every page it sees on the table it makes a Jellyfish and the Jellyfish connects every page to every other page. Then I pull out another piece of paper that’s called “this is everywhere” and whatever you point it at runs on every single computer in the entire space. So all of a sudden those Jellyfish were just limited to that table—
AMLG: They extend all the way over here to this wall as well. So everything is connected and interactive?
IC: Right. Dynamicland is a long-term research facility that’s thinking about what is computation in X amount of years. It’s inspired by Xerox PARC. Xerox PARC is the reason why computers suck a bit less than they would. Suck a lot less than they would. There’s the basic understanding that computation is awesome. If you’d like to grok why that is important, watch the Bret Victor talk “Stop drawing dead fish.” It’s a good representation of what makes computation cool. What sucks about computation right now is it isn’t human scale. It isn’t communal. And it isn’t about agency. Those are the big focuses of Dynamicland. The tagline — and I’m just a volunteer so nothing I’m saying is official I just love the place and spend my free time there —
AMLG: It looks amazing. I want to come hang out.
IC: It’s special. The tagline is “incubating a humane dynamic medium that’s full power is accessible to all humans.” That last part is important because that piece of paper, somebody else made that and then I changed it. Then somebody else made the “this is everything” paper. In order to combine code we didn’t have to learn how to use Github or use Terminal, learn how to merge them, we just pointed these papers —
AMLG: So you’re democratizing computation, interactive computation?
IC: Yes — but even more than democratize it, we are also making it something that other people can build off of. Let’s say you’re using Gmail and it doesn’t work for you because you’re colorblind. You would have to build that app from scratch. That would be difficult to do maybe even impossible. Even more impossible is to actually go and change those colors to what you want them to be. The Internet had this period when you could view source but that’s pretty much gone now because everything’s minified and out of the way. But you used to be able to go and tinker with stuff. Imagine being able to go to your Gmail app and you want to change the colors, so you point at it and you go baa baa baa baa baa then it’s something that becomes yours. It’s a computation that can be yours. It isn’t this siloed thing that is exclusively and only something that you’re viewing — it’s something that all humans get to be present with together.
AMLG: It is interesting to think what the world would look like if we were all able to express ourselves directly through our technology. There’s a few more things I want to jump to. You touched on Xerox PARC — one of my favorite things to study is centers of more open-ended research and exploration and whether we have that now or whether we are missing that. You’ve done a bunch of freelancing in VR AR and you’ve also done residencies at Adobe and Unity. A lot of the most interesting creative folks have done work for big companies or become artists in residence. Tiltbrush of course went to Google and other artists have joined Apple. Are there things you wish you could do if you were working at a company like Google or Apple that you can’t do freelancing? What are the pros and cons?
IC: The pros are a type of stability. If it were possible to be part of those big companies in healthy ways… For example “Delilah’s Gift” was funded by Google. It was through the Tiltbrush artist in residency program. So I’m working with those big companies in some ways. ARQUA! is because of Viacom Next. There are things I would love to do with that division. One of the things that is quintessential to being human is sharing. Working together to build a community. There is some communal open source stuff. But my friends who work at Apple or Google — I have a friend who works at Apple and he’s working on something crazy but I have no idea what it is. He comes over and says yo I have a question about this thing in Unity what do you think about this interface. And I give him information. And then I get nothing back. I’m not blaming him for that. I mean I’m kind of blaming him for that, break your NDA if you’re with a big company... But at the same time it sucks how much those entities take in versus how much they give out.
AMLG: Well hopefully they will give back at a certain point when they’re ready — when we see what Apple is going to do and they finally push it out into the world.
IC: Yes and there are some parts of Apple that are tight. The fact that they are a luxury bourgeois brand allows them to not take defense contracts. They’re like the only tech company that does not take defense contracts. If you are powerful enough you can say no to the government — that’s pretty baller.
AMLG: To be honest a lot of emerging technologies start there because there are budgets.
IC: Right that’s the only place where long term research is actually allowed to happen. Which is frightening.
AMLG: DARPA has been experimenting with this stuff forever.
IC: When I was in college doing physics I was studying temperatures on superconducting phase transitions of Lutetium Nicobar Carbide. We got all of our liquid helium from Los Alamos lab, which is a byproduct of building nuclear weapons. So yes it’s where there is money that there are things happening. I wish that there was more oriented towards things like play — play is how we grow. How do we play at a bigger scale? Xerox PARC is a good demonstration of it. Xerox committed to having people play and they played and they made the tightest thing ever. And Xerox was a big enough corporation that they kind of missed it. I feel like this is the same thing with Viacom Next. They shut down. I don’t think Viacom understood how special that tiny entity was —
AMLG: Right. And what would you say people like you are doing for those entities in terms of steering their thinking or showing them possibilities? I’m sure they’re getting a huge benefit.
IC: Part of it is the research. If you are oriented towards where you’re going—so when we’re talking about putting our tendrils out into space, when you get into space people are like, well we’re going to Mars or we’re going to this specific planet. But there is infinity out there. What if there’s a better direction? So much of what these entities are about is crystalizing, they’re about choosing a direction and going in that direction. The role of an artist is to be a flow like object, to point and say look over here over here over here over here, and then those entities can crystallize in those directions. The only way we get to a better future is by demonstrating that better future. It’s easy for people to call you an idealist unless you have the actual thing. But when you take a thing and you’re like look at this, experience this, use this, it is so obviously better — whether it be an economic structure or a voting structure or a set of morals or a UX — if you can show what the thing is that is better it is much easier for people to let that future unfold.
AMLG: That’s one of the things that’s special about you — you do have this optimism, you are critical and there are things that bother you but you also have an optimistic spirit that runs through a lot of what you’re creating.
IC: Cynicism is cowardly to me.
AMLG: And it’s also lazy.
IC: It’s so lazy! It’s so easy to do the bad thing. Take Black Mirror — it’s such garbage, it’s so easy.
AMLG: I’ve never heard anyone say that. I don’t watch it because I don’t like dark things and it upsets me.
IC: It’s disgusting, it’s goddamn lazy. Do your masterbatory “technology is bad” thing in the first five minutes and then spend the next 40 minutes saying how do we live healthy lives? Because that’s a problem. It is difficult for us to live healthy lives. It is not difficult for us to be like “yeah that’s bad. I hate that” it is so much more difficult to say, I’m going to delete Twitter.
AMLG: We all know social media is terrifying but OK then what? What’s the alternative? One other thing I want to ask you is about how VR and AR might help us push our understanding of reality. You’ve said: “What we comprehend about reality is so tiny and if we can expand that just a little bit we would feel more gracious about reality.” Tell me what you mean by that.
IC: Each of us understands reality differently. Even if our ability to communicate was just a tiny bit better that would be something that already let us examine reality in a different way. Way back in the day we’d have an idea in our mind and then we’d have to put it through our brain and then turn it into the things that make our mouth go blah blah. Then the sound waves come over to you and hit your ear and then somehow from the little hairs in your ears moving it would have to turn into the object that I was trying to say. When Einstein was like oh my god! theory of relativity! he had to take that thing and then like shebeebboppp and then through his arms and then into his hand and he had to write it out…
AMLG: Which is why riding on a light beam in VR would be so much better than reading about it in a book.
IC: Yeah! Because then he’d have to take that object and then he send that on a car I guess, or a horse and buggy, or bicycle like 200 miles and four months later it gets to another person and that person then somehow has to take those symbols and turn them into the same object. And finally eight months later they’re like oh my god I get it. And their reality all of a sudden expanded a little bit. Like how can we do that in other ways, how can we make that communication better? Once you have these ideas bopping around you start getting emerging complexity. Then you get happy accidents of combination. That’s where the special stuff comes from. That’s one of the reasons why Dynamicland is so crazy — when you’re all sharing that space when you’re all working together when you’re all sitting in the same reality, you start making things that interact with other things in unknown ways and that is where life comes from.
AMLG: I’m excited. So what are we going to see next from Isaac?
IC: I have one long project that I’ve been working on, and I need to do something big again. It’s been four years since “Enough” and it’s weighed me down that I haven’t done something big yet. So my primary concern is how do I find space, mental monetary physical space, to get to focus on that. I have a concept. It’s about a scientist who gets trapped inside of a computer. She is exploring the computer and discovering different flora and fauna. The user is following her journal so they see her cataloging things. So if she like rigs up a piece of electricity to this skin, when I move this slider it will change the looseness of the skin. So all of a sudden somebody is engaging with those creatures and has agency.
I also want to do stuff with face. It’s frustrating to me that with Snapchat or Facebook filters you download the thing and you use it and you’re done. What if you had a tool so you could express what you wanted to express. Like oh I’m angry, I’m going to paint a bunch of red dots on my head, oh I’m feeling silly I’m going to paint these weird kelp things on my face. Making it so that we get to have a bit more agency in that dimension is exciting too.
AMLG: So an AR app of some sort?
IC: Yeah. That definitely is a silly fun thing and maybe would help with self-exploration and understanding reality more. But the one about the scientists is what it’s really about.
AMLG: That’s awesome. Will it be a game?
IC: Kid’s book is the best the best thing I’ve got to describe it. And an app.
AMLG: Very cool. There’s so much more that I’d love to get into with you but we should probably wrap up. Thank you so much for coming. I love talking to you and the way your brain works and all the optimistic things you have to say. I can’t wait to play with your future experiences when you release them.
IC: Awesome. Thank you!
***
Resources:
