0:06

Welcome to Practical AI.

0:09

If you work in artificial intelligence, aspire

0:12

to, or are curious

0:14

how AI-related tech is changing the

0:17

world, this is the show for

0:19

you. Thank you

0:21

to our partners at fly.io, the

0:23

home of changelog.com. Fly

0:26

transforms containers into micro VMs that

0:28

run on their hardware in 30

0:30

plus regions on six continents. So

0:32

you can launch your app near

0:34

your users. Learn more at

0:36

fly.io. Welcome

0:43

to another episode of the Practical

0:45

AI Podcast. I am your co-host,

0:48

Chris Benson. Usually

0:50

I have our other co-host, Daniel Whitenack

0:52

with us. He is not able to

0:54

join today, but we have a great

0:56

show in store. We have with us

0:58

a super interesting guest. You may

1:00

very well, if you follow AI, have heard about

1:02

this guest and this company doing

1:04

some super cool stuff. So I'd

1:08

like to introduce

1:10

Anastasios Dramenidis, who

1:13

is the co-founder and CTO at

1:15

Runway. Sorry, I screwed up your

1:17

name there. Did I get it anywhere close to

1:19

right there? Yeah, all good. Thanks

1:22

so much for having me. No, sorry

1:24

for the stutter there. Thanks for joining us on

1:26

the show. You guys are doing

1:29

some really cool stuff at Runway. Wanted

1:31

you to actually, before we dive fully in, kind

1:33

of tell us a little bit about your own

1:35

background, and then we'll kind

1:38

of dive into kind of the environment

1:40

that you find yourself in and the

1:42

industry and what kinds of problems out

1:45

there are interesting as we dive

1:47

in. First of all,

1:49

CTO of a hot AI company. How did you

1:51

get there? How did you get to where you're at right now?

1:54

Well, the first thing I would say is that I

1:56

did not get here by planning for it, I

1:59

think. in some ways planning

2:01

against being where I am today. So just

2:03

to give a little background. So

2:05

my background is kind of a

2:07

hybrid of engineering and art. So I

2:10

was for the past decade

2:12

or so, I've been kind of in different

2:14

startups working as an engineer at the same

2:16

time having my own art practice. And so

2:18

doing kind of a variety of work in

2:21

media arts and interactive arts.

2:23

Runway was the first time where those two

2:25

kind of different worlds have converged for

2:28

me, but Runway started in art

2:30

school. So this is not really where

2:32

AI companies get

2:34

started usually. So

2:37

my motivation for going

2:39

to art school was actually to take

2:42

a break from technology to really explore

2:44

the more creative and in

2:47

some ways open-ended exploration of

2:49

those technologies without any concern about

2:51

making something that would make a

2:53

commercial sense at some point. But

2:56

it just so happened that I met my

2:59

co-founders there and we started kind of making

3:01

those small tools. And one

3:03

thing led to another. And we

3:05

realized that this was kind of a

3:07

really useful thing to build out and

3:09

kind of spend our focus time on.

3:12

It sounds like it was a bit

3:14

of a passion project without that commercial

3:16

intent up front. In the beginning, you

3:18

kind of fell into it because it was

3:20

what you love. Yeah, and I think that's

3:22

how the best thing gets started very usually.

3:26

And that's been a general pattern,

3:28

I would say, not just at the start,

3:30

but just throughout the way we rebuild the

3:32

company. We're very bespoke that

3:34

we really give every employee that's

3:36

called why greatness cannot be planned.

3:38

And it just talks about this

3:40

idea that when you have very,

3:42

very concrete goals in mind, it's

3:44

actually very often you end up

3:46

not meeting them. And sometimes going

3:49

for the next stepping stone is

3:51

the right approach to actually get to very

3:53

interesting findings or novel insights. That's

3:56

been part of how Runway started and that's been part

3:58

of how Runway has changed. continue to grow.

4:01

But yeah, initially, I would say our

4:03

main goal was like, these

4:05

machine learning models are super difficult

4:08

to understand, super difficult to use,

4:11

especially when we started like around five years

4:13

ago, but they're super interesting for artists and

4:15

they can make really compelling things with it.

4:17

Once they get to the point where they

4:20

can actually use them. At that point, you

4:22

know, generative models, AI was a

4:24

bit at an earlier stage in terms

4:27

of how many people cared about it

4:29

and also the result of those models.

4:31

But it was still even at that point,

4:33

really useful for artists the moment we gave

4:36

the right tools for them to use it.

4:38

And so that was kind of the inception

4:40

of runway. I'm curious, recognizing that there

4:42

wasn't, you know, the master plan that you were

4:44

implementing, you know, there was a bit of serendipity

4:47

to how you arrived there. I am kind

4:49

of curious, you mentioned that you would kind

4:51

of set aside technology before you were going

4:54

back into art right there. And I'm kind

4:56

of curious, did the technologies you're

4:58

in prior to art school play into

5:00

where you've come out here with, you

5:02

know, in terms of runway being that

5:04

end result? Or did you, you know,

5:06

is there any connection there? Or were

5:08

they just you happen to be in

5:10

a different area? And we're

5:12

finding AI? Were you active in AI

5:14

prior to going back into art school?

5:16

My interest in AI kind of goes

5:18

back into like at least high school

5:20

and before. So I've been before runway,

5:23

I was working as a machine learning

5:25

engineer, as a kind of distribution engineer

5:27

at different companies. So definitely had a

5:29

background in this area was very interested

5:31

in AI. My interest was specifically

5:34

in neural networks, which, you know,

5:36

when I was kind of decades ago,

5:38

they had become kind of like

5:40

ignored area of machine learning, like they were

5:42

kind of seen as a dead end, like

5:44

they wouldn't be able to, like, at that

5:47

point, like support vector machines throughout their kinds

5:49

of models were more popular. But

5:51

there was still something very compelling about

5:53

neural networks that made me

5:56

actually get kind of start working

5:58

with them from any type of machine. initial

6:00

projects. So in very interesting

6:02

AI kind of throughout the motivation for

6:05

going to art school was

6:07

just to kind of keep more context on the

6:09

kinds of art school. It was a program

6:11

at NYU that was exploring

6:13

the intersection of art technology. Technology

6:16

was still part of it, but it

6:18

was less kind of technology for the

6:20

sake of technology or for just like

6:22

novelty for the sake of novelty, more

6:25

understanding like how the technology could be

6:27

used in creative ways or in way

6:30

better, maybe unconventional. As you

6:32

were coming into art school and you

6:34

have this background as a machine learning

6:36

engineer and the passion for art,

6:39

what has been your initial vision for that

6:42

industry? Like within entertainment, human creativity,

6:44

which are things that you currently

6:47

are targeting, how did you see

6:49

them? How did you expect to be able

6:51

to impact the industries with AI

6:53

going into the process? So like things are moving

6:56

so fast and we're seeing these amazing technologies which

6:58

we're going to be talking about in the minutes

7:00

to come, but I'm really

7:02

curious what your perspective was about where

7:04

this was going for art

7:07

and entertainment prior to actually arriving

7:09

there. The perspective for us

7:11

has always been that those models, those

7:13

techniques are never going to be a

7:15

source of ideas. They're going to be

7:18

an acceleration and expression of like

7:21

creator's ideas. This is kind of

7:23

a mindset that we started doing those tools

7:25

around and that's why from the beginning we

7:28

started working very closely with filmmakers or designers

7:30

or with artists in making

7:32

those tools and getting their feedback on

7:34

how to make them. The other aspect

7:37

in terms of how we were kind of seeing the

7:39

trajectory of those models was when we

7:41

looked back at like 2017 or

7:43

2018 when we just started kind of working

7:45

on this, the results of

7:48

those models were you know fix-related,

7:50

low resolution, very experimental, you

7:52

know the composition was off, but

7:55

you could see the trend very clearly that you

7:57

know every year the resolution without playing the fidelity

7:59

was in improving at the first physics world way.

8:01

And so it was not a matter of if,

8:03

it was a matter of when this would arise.

8:06

And those things are always really

8:08

difficult, so we didn't really know

8:11

exactly when we're going to get

8:13

to this breakthrough where those models

8:15

really started becoming actually useful, but

8:18

we knew that it was going to happen at some point in the

8:20

next few years. Most people who

8:22

were machine learning engineers, and

8:25

I work with university students a lot and

8:27

people at the company I'm at now and

8:29

previous companies, and that's kind of

8:31

their dream job. And I find it's

8:33

really interesting to me that you said, I'm going

8:35

to set that aside for a little bit and

8:38

go and do art school. What

8:40

was the driving factor for you? Because obviously

8:42

that turned out for your story, that turned

8:44

out to be crucial, that

8:46

juxtaposition, if you will,

8:48

of those different factors. I'm just curious, what

8:50

made you say, I think I'm going to

8:53

put down machine learning engineering for a while

8:55

and go back to art school. I was

8:57

just curious what that was because obviously that

8:59

seemed to create a perfect environment for you

9:01

to spring from. I would say mainly

9:04

just the motivation and the need

9:06

to explore the possibilities

9:08

of something without a very

9:11

clear expectation that

9:13

it needed to result in

9:16

a tool that was necessarily

9:18

useful. Or just being in

9:20

an environment where it can

9:23

have this open and exploration of the

9:25

possibilities of the technology. It

9:28

was less that I wasn't interested in

9:30

machine learning, I wanted to get away

9:32

from it. It was more I wanted

9:34

to explore it in a context where

9:36

there was no expectation that I needed

9:38

to feel something that was commercially valuable

9:40

or super useful. Of

9:42

course, that took a turn and

9:44

I ended up with that was the way

9:46

to get to something that ended up being

9:48

a very good fit for a company. But

9:51

I would say initially I

9:53

was very interested in, at some point,

9:55

I think in 2015, 2016, we

9:58

were just starting to emerge. is going

10:00

to new movement around making art with

10:02

AI. And there were so

10:04

many new explorations, a lot of them in

10:06

front of the open source world. And I

10:08

just started contributing to making kind of small

10:11

projects around making kind

10:14

of tools to make art with AI. And

10:16

so really just want to spend more

10:18

time doing those things and less kind

10:21

of in the, kind of purely in

10:23

the industry working with machine learning because

10:25

I think those two things, you're working

10:27

with the same underlying model from the

10:29

same technology, but the actual results

10:32

are very different that you're creating with them.

10:34

And just one more kind of story from our

10:37

school, like to illustrate, we, like

10:39

one of the first projects that we built

10:41

with my co-founder Chris, was

10:44

this drawing tool essentially where

10:47

there was this model that Nvidia released that was

10:50

meant for kind of self-driving car research.

10:53

And the main idea of this model was

10:55

you could give a kind of a layout

10:57

of essentially a street view. So

11:00

like kind of indications of where pedestrians

11:02

are or like the road user or

11:04

other cars are, and then generate an

11:06

image using that layout. It

11:08

doesn't sound like the most kind of creative

11:10

model or like creative use case for

11:12

a tool. The context of

11:14

that model is very much for like, as

11:16

part of like self-driving kind of car research

11:18

and just kind of creating synthetic data for

11:20

that and so on. But we decided to

11:23

build this drawing tool around it where you

11:25

could define kind of the layout of a

11:27

scene and then generate kind

11:29

of street views based on that layout. We

11:31

saw that the moment we gave it to artists, the

11:34

kinds of scenes that we were creating were super

11:36

different than like what the regular office

11:38

of the model was. So they would

11:40

create like giant pedestrians or like street

11:42

signs flying from the sky. So

11:46

there's the same insight there that, you know,

11:48

you're working with the same types of models,

11:51

the same types of technologies, but seeing

11:53

them with a fresh set of eyes

11:55

and a different perspective makes all the

11:57

different. And so this is what

11:59

I came to. are cool to do

12:01

is to see the same underlying ML

12:03

AI technologies with a new set of

12:06

eyes, exploring new possibilities. And this

12:08

is why we hope to be awesome with

12:10

the tool itself. What's

12:24

up, friends? Is your code getting dragged

12:26

down by joins and long query times?

12:29

The problem might be your

12:31

database. Try simplifying the complex

12:33

with graphs. A graph

12:35

database lets you model data the way

12:38

it looks in the real world instead

12:40

of forcing it into rows and columns.

12:42

Stop asking relational databases to do more

12:44

than what they were made for. Graphs

12:47

work well for use cases with lots

12:49

of data connections like supply chain, fraud

12:51

detection, real-time analytics, and generative AI. With

12:54

Neo4j, you can code in your favorite programming

12:56

language and against any driver. Plus, it's easy

12:58

to integrate into your tech stack. People are

13:01

solving some of the world's biggest problems with

13:03

graphs. And now it's your turn. Visit neo4j.com

13:06

slash developer

13:09

to get

13:11

started. Again,

13:13

neo4j.com/developer. That's

13:15

neo4j.com/developer. So,

13:36

you arrived at art school for that purpose of

13:38

seeing all this through a new set of eyes.

13:41

And you met your co-founder, Chris, and

13:43

you guys had that spark of an

13:45

idea which would become Runway. Can you

13:47

talk a little bit about the insight

13:49

that you had there that created Runways

13:51

before we dive fully into what Runway

13:53

has done since? I'm really curious

13:55

what the moment we're where you and Chris, you

13:57

know, kind of said we have something here. The

14:00

something we're gonna go do. Oh, they're distinct

14:02

bomb I did. You just kind of gradually

14:04

arrived there. What was that moment? Like.

14:06

For you decided it's time to go be an entrepreneur.

14:09

In this context, So. I will

14:11

say was that one moment that come

14:13

with with a turning point though we

14:15

were working in a little different projects

14:17

with a crease in them hundred or

14:19

the other. the other cop on there

14:21

are and the he so this for

14:23

this is kind of a standalone tool

14:25

around on the helping for a say

14:27

are causing for an artist or for

14:29

a specific kind of medium of the

14:31

city of contact. Our time we

14:33

realize that there is another the same

14:36

thing for we have he do for

14:38

it he project and had a fine

14:40

spending as like being able to run

14:42

models would be even more like difficult

14:44

than an a day even like can

14:46

and running and a google and call

14:48

of not to with like to my

14:50

slot from time for like i like

14:52

an artist without any technical on a

14:54

bike around or know how about are

14:57

those models work so then saw idea

14:59

was. Let's. Start from was already

15:01

out during the officers world like there

15:03

is already and her wealth of different

15:05

models that perform different top. Five.

15:07

List: Snake economic really fool around them

15:09

so let's bring the kind of interface

15:11

in the kind of I see you

15:13

in that are these are familiar with

15:15

some other races. But used

15:17

to newman with a refunding out. There

15:20

are differences and possibility. And. On

15:22

the back in foes make the

15:24

main idea of runway initially on

15:26

and also. As. A message before there

15:28

was kind of that. That reason with their

15:30

from the third them. As. Is most

15:32

was becoming better and better. more

15:35

on the up, the ability of

15:37

those models would go increasingly more

15:39

from them. You know the more

15:41

experimental use cases do something that

15:43

actually driving production and it's like

15:45

really really useful for of right

15:47

of credit workflows and we saw

15:49

that happened very quickly as were

15:51

certainly. You. mentioned along the way

15:54

they're that the difficulty of implementing some

15:56

of the models and and even today

15:58

with a number of to choices

16:00

out there, it's still something that

16:02

many companies are contending with is

16:04

how to address models, how to

16:06

train them, where they're going to

16:08

train them, what the deployment, how

16:10

it fits into products. There's a

16:12

gazillion questions out there. You

16:14

were doing this at a moment where

16:16

that wasn't even as sorted as it

16:19

is now, and it's still in development

16:21

at this point. How did

16:23

you manage that? Because when

16:25

I've talked to other people, that's often been one

16:27

of the biggest challenges is just getting the resources

16:29

in place, especially at that time

16:31

when it was still in early development. What

16:34

was that like to try to bring that,

16:36

bring your vision out when obviously the environment

16:38

that we were doing AI in was still

16:41

fairly exclusive in a lot of ways in

16:43

the sense of access to expertise, resources.

16:46

You're in an art school that's designed

16:48

to help you do that, but that

16:50

couldn't have been easy. Yeah, so we

16:53

essentially had to figure out a lot

16:55

of things from scratch as we were

16:57

building this. As I mentioned,

16:59

initially Runway was based around providing

17:02

access to existing open source models.

17:05

We quickly actually realized that we

17:07

needed to build

17:09

a new house, like research team in

17:11

order to really get those models from

17:13

something that makes it good

17:15

demo, or good prototype, or something that's really

17:18

useful. That was actually from

17:21

the first month of Runway became very clear that

17:23

we needed to do this. Of course,

17:25

none of our three had built a

17:28

research team before. I had engineering

17:30

and research in a

17:33

research background, but the experience of how to

17:35

build the team, like what skills to bring

17:37

in was like nobody on the team had

17:39

it. A lot of the things we just

17:42

didn't figure out from scratch. One nice thing

17:44

I would say is that because we started

17:46

so early, we had years to figure this

17:48

out. If you're just

17:50

coming into AI as part

17:53

of building a new company today, the

17:56

time horizon, you need to figure those

17:58

things out in much more in a

18:00

more entrepreneurial fashion. So for

18:02

us, we spent the first years figuring out what

18:05

does it mean to actually build a research organization?

18:08

We think that started, and what does it

18:10

mean to build a robust deployment

18:12

pipeline so that you

18:14

can not only kind of serve those models, but also serve

18:17

them interactively, because a big part of the way

18:19

we build tools that run with

18:21

the interaction, it's a

18:23

very key aspect of really making those models

18:25

useful. When I've talked

18:27

to other entrepreneurs about this, they have

18:30

a tough time, as you're kind

18:32

of getting to the place where you're at now in

18:35

terms of being able to, you may not have the research,

18:38

you're doing amazing research, but you had to

18:40

kind of get from A to B in

18:42

the meantime and kind of keep the

18:44

company alive. How did you

18:46

approach from a funding, customers, things like

18:48

that while you were kind of figuring all these

18:50

things out? Because that strikes me

18:52

as a pretty hard problem to tackle

18:54

as you're moving along,

18:57

but you still have to pay the bills, if you

18:59

will. How did you tackle those kind of issues in

19:01

terms of creating an AI startup that

19:03

couldn't instantly be everything

19:05

that it is today from day one?

19:08

I would say the main insight is

19:10

to, we wanted to make

19:12

sure that runway was useful at each stage

19:14

of evolution. So even

19:17

though the generative models were

19:19

not quite as powerful, when

19:21

we started as they are today, they weren't

19:23

as big a part of the initial kind

19:26

of tool offering and we wanted to make

19:28

the tool as useful from the

19:30

very beginning as possible. So

19:32

the product of runway went through many

19:34

evolutions that really tracked how the kind

19:36

of AI models evolved and at which

19:39

stage they were useful for which things.

19:42

A big part of early runway was building

19:44

out a video editor that was really combined

19:46

some of the more traditional video editing techniques

19:49

with AI-based techniques to speed up the

19:51

process of a lot of video editing

19:53

workflows. And that wasn't necessarily something that

19:55

had generative models powering it, but

19:58

it was a really useful tool that... that really gave

20:00

us a little bit inside about how to

20:02

build tools that are really useful for creative

20:05

workflows and how to really solve real pain

20:07

process. But at the same

20:09

time, while we're building those tools, we're also at this

20:12

research that was ongoing that was

20:14

still remaining and more academic level

20:16

of just really demonstrating how we

20:18

can improve the results of generative

20:21

models. And at some point,

20:23

there was that intersection point where we

20:25

started bringing those generative models to production.

20:28

So the overall strategy was we

20:30

knew that generative models would be really

20:32

powerful given that time and if we invest the

20:35

resources on the research side. At

20:37

the same time, we knew that at the

20:39

beginning, not everything is to be powered by

20:41

generative models. So we're building

20:43

a lot of AI-based tools that incorporated

20:46

that were really useful from the beginning

20:48

and that they were used by VFX

20:50

artists, by video editors to speed up

20:53

a lot of their workflow. Even

20:55

far before, we released things

20:58

like Gen 1 or Gen 2 for

21:00

text-to-video functionality. You're

21:02

saying generative, but it was definitely the

21:04

early days of generative. And you certainly,

21:06

right now, it's all the rage. Everyone's

21:09

talking generative in every context. But

21:12

you had some insights into that. You talked

21:14

about the fact that you guys knew that

21:16

that was going to be the case going

21:18

forward. But to your

21:21

credit, not everybody did. A

21:24

lot of people went, aha, much later than you

21:26

went, aha. And I'm curious,

21:30

is there anything that stands out as

21:32

what drove the insights that you guys had

21:34

and why? Because you were really

21:37

one of the very first to get these

21:39

kinds of functionalities to product. That's

21:42

very notable. And you might

21:44

say the rest of the world didn't,

21:46

not that many. And

21:48

so what were some of the

21:50

things that gave you that confidence to say,

21:53

this is clearly going to be critical

21:55

to our future. This is going to drive

21:57

the industry at an early stage. pioneering

22:00

that thought process, how did you get there?

22:02

From the very beginning, a big part of

22:04

the rendering was working directly with artists and

22:07

building the tools. And so when we gave

22:09

them even early versions of alternative models, we

22:12

could already see that they, like,

22:14

there was really compelling aspects

22:16

of work to me then, even if there

22:18

is also a low resolution and like not

22:21

as high fidelity. So like early

22:23

forms of things like prompt engineering, like figuring

22:25

out how to traverse the latent

22:27

space of those models were still there at

22:29

the beginning of the runway. And

22:31

we saw how artists were engaging with

22:34

them, like how they were kind of,

22:36

they were finding them to be really

22:38

compelling and really useful. And

22:40

so really part of it has been

22:43

this early view into how artists

22:45

with some of more early adopters,

22:48

I would say, were engaging with

22:50

those models and just extrapolating that

22:52

once those models improve, other people

22:54

will equally find them as compelling.

22:57

Working with artists, I think, has been a really

22:59

important part of just really understanding

23:01

some of the future of those models and

23:03

extrapolating of how they would be used. And

23:06

also just looking at the kind of

23:08

history of art and how tool

23:10

making was always part of, like how

23:13

new tools always allow kind of new,

23:15

create a new kind of movements or

23:17

allows new kinds of kind of genres

23:19

to emerge and just assuming and kind

23:21

of predicting that the same would happen

23:24

with those sharing models. Along the way,

23:26

as you were going down this path, what

23:29

stumbles did you have as part of putting,

23:32

because it's quite remarkable because you clearly

23:34

could see the future before you got

23:37

there and with more clarity

23:40

than others that might be in a

23:42

similar position. As you did

23:44

that, what kinds of things did

23:46

you, were either unexpected or

23:49

challenges that were bigger than you thought? You

23:51

know, the things were maybe at a moment

23:53

in time, you were grinding your teeth and

23:55

going, or this is not exactly how I had

23:57

it planned. Do you have any stories to that?

24:00

that affect during this process? Many

24:02

stories and many learnings along the way, for

24:05

sure. I think the biggest

24:07

requiring insight that we've had around how to

24:09

deal for those tools and the things that

24:11

I think is still not fully appreciated today

24:13

is how important control is

24:16

in terms of interacting with those models. And

24:19

so every time we invested into adding

24:22

more ways in which you can really

24:24

control the outputs of the models

24:26

that people were using inside Runway,

24:29

we saw a whole new set of possibilities, a

24:31

whole new kind of usage. So

24:33

that has been a really consistent thing.

24:36

And even at the beginning, we just

24:38

saw that those models had a lot of

24:40

flaws that they might not always, like if

24:42

you have a very simple way of controlling

24:44

them, it might not really give you what

24:46

you want. They might have to do a

24:48

lot of tries with the same old kinds

24:50

of other outputs to get to where

24:53

you want your desired

24:55

result. And so that's really

24:57

what we saw with the early, when

25:00

we first released Gen 2, you could

25:02

only control things with a front. And

25:05

we saw very quickly that that side of

25:07

people just generating like tens

25:09

or hundreds of outputs in order to

25:11

get to the result that they wanted.

25:13

And so we invested, kind of continues

25:15

more and more, adding more and more

25:17

ways in which you can manipulate things

25:19

essentially as the film director would

25:22

think about creating a scene. So

25:24

a film director would have a vision, notice

25:26

of like a description, high level description of

25:28

what the scene is, but how the camera

25:30

moves in the scene, or like how do

25:32

the characters interact with each other. So

25:35

having ways in which you can control really the

25:38

kind of camera motion or like the

25:40

motion, the object motion, like the motion

25:42

of the characters in the scene, like

25:44

all those things that make total sense

25:47

from a career's point of view. But

25:49

they're not necessarily how like maybe ML

25:51

researchers would necessarily think about those models.

25:53

I think that has been always the insight

25:55

that, you know, we never saw

25:58

some negative effects from adding. more this

26:20

is a change log news break pewter

26:23

is the Internet OS pewter

26:26

is an advanced open source desktop

26:28

environment in the browser designed to

26:30

be feature rich exceptionally

26:33

fast and highly extensible it can

26:35

be used to build remote desktop

26:37

environments or serve as an interface

26:39

for cloud storage services remote servers

26:41

web hosting platforms and more I've

26:44

been around long enough to see

26:46

a bunch of these desktop OS

26:48

in a browser window demos and

26:50

toys but this is the first time

26:52

I've been impressed by one enough to keep

26:54

the tab open longer than 30 seconds from

26:57

the URL structure to the cloud

26:59

storage integration to the developer portal

27:01

pewter strikes me as an actually

27:03

viable internet-based operating system with potentially

27:05

real-world use cases and that's saying

27:08

a lot oh and it's also

27:10

entirely built with vanilla JavaScript and

27:12

jQuery so you know the devs

27:14

haven't cargo-cultured together something they can't

27:16

grow and maintain on that note

27:18

they say for performance reasons pewter

27:20

is built with vanilla JavaScript and

27:22

jQuery additionally we'd like to avoid

27:25

complex abstractions and to remain in

27:27

control of the entire stack as

27:29

much as possible also partly inspired

27:31

by some of our favorite projects

27:33

that are not built with frameworks.

27:35

BS code, photo pee, and only

27:37

office. You just heard

27:39

one of our five top stories

27:42

from Monday's changelog news. Subscribe to

27:44

the podcast to get all the

27:46

week's top stories and pop your

27:48

email address in at changelog.com/news and

27:50

to also receive our free

27:52

companion email with even more developer

27:54

news worth your attention once

27:57

again that's changelog.com/news

28:06

So before the break, you brought up Gen

28:08

2. And I'd like, we've had a little

28:10

bit of a history on the development, which

28:12

is fascinating. It's an incredible story you have.

28:14

Tell us all about Runway today.

28:17

You've arrived here, you have

28:19

Gen 2. Just talk a

28:21

little bit about how you're impacting industry

28:23

today. And for listeners

28:25

who haven't been to your website, you

28:27

talk about advancing creativity with artificial intelligence.

28:30

And you specifically note that you're an

28:32

applied AI research company shaping the next

28:34

era of art, entertainment, and human creativity.

28:36

What does that mean in 2024?

28:39

As you're out there in the space, can you talk a

28:41

little bit about the company as it is now? Yes,

28:43

to give some context, Gen 2 is

28:46

a text-to-video and image-to-video generation model. Essentially,

28:48

it takes a description of a scene

28:50

and generates a video output from that

28:52

scene. And it's one of the many

28:55

models that we have at Runway, the

28:57

most well-known one. The broad vision of

28:59

the company has remained the same over

29:01

the last five years. And it's understanding

29:04

and creating the new generation of creative

29:06

tools, and then working with artists directly

29:08

to figure out to help them shape

29:10

those tools as much as possible. And

29:13

so I think where we are today

29:15

is, I would say we're

29:17

still at the very early stages of

29:19

where those models can go. I think

29:21

video generation, this is really the year

29:23

where video generation gets really

29:25

good. And so we're really excited

29:28

to be part of building

29:31

out those technologies and figuring out how

29:33

to work with artists to make

29:35

them as useful as possible. We've seen over

29:37

the past year, I think we're at least

29:39

Gen 2, film studios, streaming

29:42

companies, ad agencies adopting Runway. And

29:44

that adoption is not just from

29:46

kind of individual creators, but it's

29:48

really we see companies starting to

29:50

use those models and incorporate them

29:52

in the workflows. And I think

29:54

it's not going to be a

29:56

binary shift where you go

29:58

from not using generally one. at all

30:00

as part of making video

30:02

or making art, to using it everywhere.

30:05

It's a more gradual transition. And for

30:07

us, the big goal is teaching folks

30:09

how to use those models, supporting all

30:12

the creators that are making interesting things

30:14

with those models. So we have an

30:16

AI film festival that we showcase films

30:19

that use AI in

30:21

different ways. So I would say for us, the

30:23

goal is very much holistic.

30:27

We do the research, we do research

30:29

and development in building up the next

30:31

generation of those models. We build useful

30:33

tools around those models. And we also

30:35

work with artists and with companies that

30:37

want to adopt those models in their

30:39

creative workflows. As you have

30:41

been working into this for years, for most

30:44

of the rest of the world, the past

30:46

few months have been a big eye-opener, particularly

30:48

with big cloud companies producing

30:50

their models and stuff and competing in

30:52

that. There's the obvious aspect of

30:54

you have the industries that you're playing in and that

30:57

you're strong in. But what concerns

30:59

do you have from a competitive standpoint

31:01

against other companies, especially these big, all-encompassing

31:03

cloud companies that are in the AI

31:05

arms race to

31:08

produce the ever larger, more

31:10

capable model? At no point in

31:12

this conversation, have you expressed any concern? Have

31:15

you raised that or anything? Which is quite notable. Usually

31:17

people are a little bit worried about that.

31:20

And you seem very strong in your space.

31:22

How do you see those other big players

31:25

that are out there? Do you see them

31:27

as competitors even? Or are they far enough

31:29

from you that that's not a big deal?

31:31

Or are you so tightly into the industries

31:33

that you're serving specifically that you have a

31:35

huge competitive advantage? How do you see all

31:38

that? Well, we've

31:40

always had the perspective and mindset

31:42

of running around race. And so we

31:44

try not to kind of be too

31:46

distracted, but especially at this

31:48

stage, like there's so much kind of

31:50

noise and discourse around AI that it's

31:52

easy to kind of get back and

31:54

like following the latest developments. So I

31:56

think that's kind of the number one

31:58

aspect. first release Gen

32:00

2 last year, one of our positions that

32:03

was not as popular, I would say last

32:05

year was that video generation

32:07

models were going to be the

32:09

kind of like video with the modality that's

32:12

kind of encapsulated as much

32:14

world knowledge and just for

32:16

myself possible. And last year, the focus

32:18

was on language. And for us, it

32:20

was a big kind of unorthodox to

32:23

kind of maybe pay so much attention

32:25

to video statistically and and claim that

32:27

video generation models were like really the

32:29

way to build really broadly useful AI

32:31

system. And over the past

32:33

month, we've seen more companies kind of

32:35

entering this space of video generation models. And

32:37

so it was not nothing unexpected, like we

32:40

know that those models are going to be

32:42

really useful for a wide variety of use

32:44

cases, they're going to be useful

32:46

beyond reading creative tools, which is really our

32:48

focus. And so for us,

32:50

it's really important to maintain that focus

32:52

of really, like not just building those

32:55

models and like making kind of cool

32:57

demos around that, but really figuring out

32:59

like bridging that gap between, you know,

33:01

those demos and really deploying them to

33:03

product and really getting kind of people

33:05

to use them and getting kind of

33:08

making them controllable. So there

33:10

is still that gap, I would say

33:12

from doing just the research and developing

33:14

the model to actually making those models

33:16

controllable and deploying useful tools. And

33:18

for us always, it has been the focus to

33:20

bridge that gap. And that kind of continues to

33:23

be our focus. So again,

33:25

like video generation models are still very early.

33:27

And like, we haven't seen anything yet

33:30

about what the ultimate be

33:33

capable of, you can imagine, you

33:35

know, a year from now to years from

33:37

now, every company can have like a photo

33:39

realistic video generation model. And that's an assumption

33:41

that we're making that the competitive advantage is

33:44

shift over time. And at that

33:46

point, like, what's the differentiation of

33:48

runway for us, it's always been

33:50

working very closely with artists, building

33:53

really useful tools and bridging and bringing

33:56

making those models really controllable and useful.

33:59

It's fascinating me because I talk

34:01

to so many people in different

34:04

companies and they're busy trying

34:06

to just AI everything and they're kind

34:08

of all about the AI. You guys

34:10

are doing the AI but it sounds

34:12

like competitively having

34:14

been so embedded into

34:16

the artistic ecosystem with

34:18

your tooling is really

34:21

kind of something that keeps you right there

34:24

while everybody goes through the kind of the

34:26

AI model wars in terms

34:28

of trying to produce so much. Do

34:30

you think that long heritage of the tool making is probably

34:33

key to your future in that sense? Is that kind of

34:35

how you're thinking about it? I think it's

34:37

the most important aspect of how we're operating. Otherwise,

34:40

again, it's too easy

34:42

to get lost in the short-term race

34:44

of just having kind of a marginally

34:46

better model for a few weeks versus

34:48

kind of really having the mindset of

34:51

building the most useful tool long-term and

34:53

then obviously updating the model, making sure

34:55

you get state-of-the-art results with it. But

34:57

it's not the goal, it's not the

34:59

focus to have the best model. The

35:01

focus is to get artists to make

35:03

the coolest things or the most compelling

35:06

things with those models. And if that

35:08

remains the goal, then that also informs

35:10

how we build those models. And

35:12

so another aspect of

35:14

Runway is that we have a research

35:17

team and then we also have a creative

35:19

team in-house that works with the research team

35:21

on a daily basis and tries out the

35:23

latest model, informs how do the

35:25

research, like what kind of controls needs out

35:27

the models. And having that

35:29

perspective is really like when I talk

35:31

to researchers that work in academic labs

35:34

or kind of large industry labs, they

35:37

might notice papers about the potential creative

35:39

applications of those models, but they don't

35:41

interact with artists daily. They don't often

35:43

know like is it actually useful or

35:46

is it just a hypothesis that I'm making? And

35:48

I run away, as a researcher, you

35:50

get that feedback on a daily basis.

35:53

And I think that really changes how you

35:55

approach building those models. For listeners,

35:57

you and I can see each other. an

36:00

audio only podcast, but you had this

36:02

glint in your eye a moment

36:04

ago when you were talking about kind of where

36:06

you expected these video models to be going. For

36:09

just a minute there, you reminded me of the

36:11

kind of the kid in the candy store. You

36:13

could see your passion really flying out of your

36:15

eyes there and obviously I'm the only one that

36:17

could see that. Talk a little bit

36:19

about where you think this is going. That's what

36:21

everybody is wondering. There's so many questions, you know,

36:24

that people have in terms of, you know, how

36:26

video fits in their life, what life becomes like

36:28

when you have generative capabilities

36:30

that essentially, you know, simulate life

36:32

in so many ways. What are

36:35

you expecting over the next year

36:37

or so? And like

36:39

I'm not holding you to it obviously, but

36:41

just what do you anticipate might

36:43

happen in the video space generatively? And then

36:46

how would you see it several years out,

36:48

you know, when it's kind of exponentially been

36:50

had time to grow a bit? What does

36:52

that look like to you? The

36:54

way we would like to think about

36:57

those generative video models is we have this

36:59

term of their general world

37:02

model. Essentially, they simulate different aspects of

37:04

the world because in order to kind

37:06

of similar to how, you know, you

37:08

have large language models that have been

37:10

trained with a very simple task, just

37:12

predict the next token in a sentence.

37:15

In order to predict that the next

37:17

token and perform the task really well,

37:19

they have to gain all this understanding

37:21

about different aspects of human

37:23

knowledge, different aspects of the world

37:25

just to solve this task well, because

37:28

they need to complete sentences that might

37:31

come from an encyclopedia or like a

37:33

forum post or it's like a wide

37:35

variety of cases. So we think very

37:38

similarly of how the video distribution models

37:40

operate. In order to predict the next

37:42

frame, you need to gain kind of

37:45

not understanding of basic kind of rules

37:47

of motion or like physics. You

37:50

really need to gain a kind of more

37:52

comprehensive like broader understanding of the world. And

37:55

so like if I think, you know, a

37:57

year from now, where do those models go?

37:59

Essentially because more and more higher

38:01

fidelity simulations of the world, giving

38:04

you the ability to really imagine all

38:06

sorts of different kind of scenarios, like

38:08

build out, tell all kinds of different

38:10

kind of scenarios and stories. And

38:12

I think that the applications of that are

38:14

kind of really, there is kind of a

38:17

wide ranging kind of application that goes beyond

38:20

the kind of creation use cases, which

38:23

I think for us are kind of

38:25

still in mind the focus, but just

38:27

building models that can perceive the visual

38:29

world, like of

38:31

course, like can be used in all kinds of

38:33

other ways as well. Thank you for

38:36

sharing your story. As we finish up here,

38:38

we have a lot of young listeners on the

38:40

show and there is, I guarantee that there are

38:43

quite a few young artists that are

38:46

technically inclined out there, you know,

38:48

high school, maybe early college age, and

38:51

they're listening to this and they're going, that guy just lived

38:53

the life that I'm wishing I could live. You know, that's

38:55

the kind of thing that I wanna do. What

38:58

would you, whether they identify themselves kind of

39:00

as a young artist who's technically inclined or

39:02

technologists who loves art, however they see themselves,

39:05

do you have any guidance on how they might

39:08

step into the future and kind of

39:10

get to that sweet spot for them, given

39:12

the fact that clearly the technology, specifically

39:15

with AI and the artistic role will

39:17

continue to merge and develop together for

39:19

years to come, where should

39:21

they go, what should they do, any thoughts? I

39:23

would say the number one thing is following your

39:26

curiosity and tinkering as much as possible, so there

39:28

is a lot of ways in which you can

39:30

start kind of building those

39:32

models yourself, you can start kind of running them, you

39:35

can start to get kind of an understanding

39:37

of what you can do with them, and

39:39

that's available to really kind of anyone, and

39:42

so really, you can start getting

39:44

involved today in building projects, kind

39:47

of exploring AI or making creative projects with AI.

39:50

That would be the number one thing. It's also,

39:52

I would say for me, planning, trying

39:55

to plan ahead too much has

39:57

never quite worked, really focusing on

39:59

my question. that I can build today, like

40:01

where kind of curiosity and interestingness

40:04

will drive me next, has always been

40:06

kind of the guiding principle.

40:08

And so that would generally be my,

40:11

my recommendation is not trying to think

40:13

of, you know, what, where technology will

40:15

be five years from now, because really,

40:17

nobody can fully plan ahead. But

40:20

rather trying to really build interesting things

40:22

today. It's actually surprisingly, I would say,

40:24

easy to like, if you started making,

40:26

you know, projects open source and just

40:28

showing them to others, it can be

40:30

quite fast, but you can get noticed

40:32

for those projects. And you can like

40:34

start to, you know, build a community

40:36

around and work with other people and

40:38

collaborate on your project. And kind

40:40

of with those collaborations, kind of one by one,

40:42

you can kind of get to a point where

40:45

you can kind of start kind of doing this

40:47

work full time. So, like really focusing

40:49

on the next project, I think, for

40:51

me has been really the way to go. Well,

40:54

Anastas, thank you so much. That

40:56

was fantastic guidance. Appreciate your, your

40:58

perspective, fascinating story leading into this,

41:01

and especially in all the early

41:03

insight that you guys had. Thanks for coming

41:05

on and talking about runway and the world

41:07

in which you guys are trying to make

41:09

a bit better. Appreciate it. Thank you, Chris.

41:19

All right, that is Practical AI

41:21

for this week. Subscribe

41:24

now. If you haven't already,

41:26

head to practicalai.fm for all

41:28

the ways and join

41:31

our free Slack team where

41:33

you can hang out with

41:35

Daniel, Chris and the entire

41:37

changelog community. Sign up today

41:39

at practicalai.fm slash community. Thanks

41:42

again to our partners at fly.io

41:44

to our beat freaking residents, Breakmaster

41:46

Cylinder and to you for listening.

41:48

We appreciate you spending time with

41:50

us. That's all for now.

41:52

We'll talk to you again next time. MMMMMMM

42:04

!