0:19

So we have a guests with us today. Go

0:21

ahead and introduce yourselves

0:23

I'm Ben Glickstein. I am

0:25

a filmmaker, immersive

0:27

artist, immersive and interactive artist.

0:29

And Doug and I run

0:31

the Art of AI podcast and

0:34

soon to be newsletter. And we

0:37

are both filmmakers and we both

0:39

got into AI from the art

0:41

perspective through stable diffusion and

0:43

chat, G p t things like that. And,

0:45

Just are looking at AI

0:49

as what it means in

0:51

for artists and what that means

0:53

in terms of a both a workflow for

0:56

them and what actually is

0:58

art at this point since now we can just

1:00

put in a word and it spits out 300

1:02

images that look great.

1:05

Nice. I am definitely curious to hear

1:07

your thoughts on that, Doug.

1:10

Thanks so much for having us. I'm Doug Carr and

1:12

I'm a filmmaker and entrepreneur

1:15

and do a lot of different stuff in tech.

1:18

And most recently

1:20

getting deeper down the rabbit hole of

1:22

ai. And co-hosting the

1:24

Art of AI podcast with Ben. So

1:26

yeah, just really excited to chat with you about

1:29

prompts and how AI

1:31

is beginning to reshape everything we do.

1:33

Yeah, definitely. So before

1:35

we get into that, and I'm very curious

1:37

cuz you're the first artists that I've had

1:39

on. How did you both start learning

1:42

about prompt engineering?

1:45

So I would say for me it goes

1:47

back many years. I

1:49

want to, if I'm gonna put a number on it, I'd say

1:51

maybe seven or eight. When AI

1:53

systems were just starting

1:56

to be, have chat

1:58

bots that you could interact with I got

2:00

really excited about the idea of storytelling.

2:03

With ai and,

2:05

at that time these

2:07

LLMs were so limited in

2:10

terms of what you could do with

2:12

them. And it was a frustrating

2:14

exercise. But fun, like interesting.

2:16

And it just felt like basically I

2:19

would. Try to interact with the

2:21

AI and it would give me some vague,

2:24

barely cohesive

2:27

language. But even like even

2:29

at that point I was like, oh, but there's two

2:31

sentences here that I can pick out. And that's

2:34

actually really interesting. And it's, just

2:36

as a kind of brainstorming exercise,

2:39

it would bat. It would just trigger

2:41

things in my head that

2:43

would help me with writing and Ben

2:46

and I have worked on a number of different

2:48

collaborative writing projects and for

2:50

film and video games, et cetera. And I

2:52

was always let's just dip back into the

2:54

the ai, let's see what it can do to help

2:57

prompt and do interesting things. And I think

2:59

at that point it was more like the kind of reverse of what we

3:01

have now, where it was prompting

3:03

us cuz it was so vague. And

3:06

it's only of course, Yeah.

3:09

Yeah. And now it's really flipped where, now

3:11

when we interact with the ai, it's so

3:13

much more organized together and

3:15

able to complete language so well

3:18

that, now you have to get smart at prompting

3:21

the ai and then it becomes this kind of wonderful

3:23

dynamic interchange back and forth.

3:26

In terms of text and imagery But

3:28

yeah so that's where it started for

3:30

me. And obviously in the last

3:33

few months, it's been really

3:35

exciting to now, be able to.

3:39

Feed the AI a,

3:42

story elements and then have it

3:44

begin to organize, an entire arc

3:47

for how the story can be told. And

3:49

there's a lot of really crazy advancement

3:51

going on there. And it's really dynamic,

3:54

amazing process. Ben, you gonna,

3:56

you

3:56

Yeah. I actually think I started

3:58

with prompting from the visual side of it.

4:01

So I, like we, we

4:03

were, I remember Doug was playing around more with that,

4:05

and I like, touched on early G P T

4:07

and I was eh, I'm not like super into

4:09

this right now. But then I got into

4:11

Starry AI and Wombo

4:13

I think were like these early apps,

4:16

like before Dolly was released to the public,

4:18

right? There were these early apps

4:21

that were coming out and they were doing like text to

4:23

image generation. And then three months

4:25

later, Dolly two got released and

4:27

I got into the open beta for that and

4:29

that was off to the races for me. Like

4:32

I was literally, like literally the day

4:34

I left, I just moved to California a few

4:36

months ago and literally the day we

4:38

were driving across country, I got access

4:40

to Dolly and I was just literally in

4:42

the car like half the time just prompting Dolly

4:45

and having it spit out all this weird stuff.

4:47

And then I think by the time like we got settled

4:50

in la like stable diffusion started

4:52

to be a thing. I got really into that and That sort

4:54

of took it out and then chat, G B T

4:57

3.5 came out and I started playing

4:59

around with it and I was like, oh, okay, it's actually good. And

5:01

four came out and then I was like, oh, great. And

5:03

I literally signed, paid for, plus

5:05

the day before four came out. So

5:08

it was. It's been working pretty good.

5:10

Yeah, and that's where I've been coming

5:12

from it and I've been using t p t four

5:14

a lot to help me with code since

5:16

I'm not a native coder and I understand

5:19

a little python, but like it's really useful

5:21

to like getting in structures and stuff

5:23

that I just don't understand.

5:26

So how do you approach the very

5:28

iterative process of refining

5:30

and testing prompts to make them give you

5:32

what you want?

5:35

So for me it's not so much

5:37

like I focus left less on

5:39

the prompts and more on the results. So

5:41

for me it's especially with that cod dev

5:43

prompt that I shared with you, that

5:46

has been really great because you can break

5:48

everything down into a structure. You

5:50

can then put that in like in Unity,

5:52

which is what I'm using for the current thing I'm working

5:54

on. You upload that into Unity

5:56

and you hit run and if it doesn't

5:59

work, it spits out an error and then you just

6:01

take that error and I put it in chat, G B T,

6:03

I'm like, yo, what's up with this? And it's ah, sorry.

6:05

It actually should be this. And I put that in. I'm

6:07

like, no, that's not right. And it's oh, sorry, it should be this.

6:10

And then, it like, it eventually works and between

6:12

that and like feeding documentation

6:14

to chat. G P T. Like

6:16

also looking up the errors and then being, oh,

6:18

this is what it's saying, since you're not connected

6:21

to the internet, like this is I

6:23

found that is the best way to work with it.

6:25

That's for chat G P t.

6:28

If I'm using stable diffusion honestly,

6:31

I mostly go and, if

6:33

I have an idea, like I will

6:36

start with a prompt. And then

6:38

And then basically start doing image

6:40

to image stuff. A lot is, I

6:43

find prompting is a great start off point,

6:45

but I find most of the control comes

6:47

from the other tools they have, like image

6:49

to image control net. I

6:52

am, I'm of the mind that prompt is

6:55

a great start, but I don't.

6:58

I don't buy into prompt

7:01

refining as much since

7:03

I find that the jumping

7:05

off point is more what you need and then the other

7:07

tools come in and refine it as much

7:09

We haven't actually covered image to image or

7:12

control net yet, so if you could explain

7:14

both of those for the audience, that would be great.

7:16

Absolutely. In stable diffusion

7:18

image to image is when you take

7:21

a photo you have already created, or a

7:23

photo from a prompt bring it

7:25

up into stable diffusion.

7:27

And it's using that as a guide

7:30

to create the new image. So you're going, Hey,

7:32

this is the basic layout of the image, but I want the

7:34

person there to have red hair or

7:37

something like that. And then it,

7:39

gives you something that's completely different. But

7:41

that's in theory how it works. And what

7:43

Control Net does is it

7:45

basically creates a depth model of the image

7:48

so that you can then like isolate a person

7:51

to to get them out of there

7:53

and Use that to

7:55

then create a much more controlled image.

7:58

And what people are eventually gonna be

8:01

doing is using this to create videos

8:03

and they already are. So you can

8:05

basically use that, take 24 frames of

8:07

a video, run it through that, and then pretty

8:10

cleanly have that do in there.

8:11

That's one area that we're super excited

8:14

about is as this moves into the moving

8:16

image and as AI

8:18

gets better at dealing with elements

8:20

on a timeline, that's the next

8:23

big evolution from the art scene.

8:25

That's awesome I haven't used it, but

8:27

I believe Mid Journey has the same ability to

8:29

do image to image. Is that

8:31

correct?

8:33

You can feed mid journey images through

8:35

the web, basically. No,

8:36

Gotcha.

8:37

But is it image to image or are you using

8:39

that to

8:39

not,

8:40

a person? Like

8:41

it's more, yeah.

8:42

right?

8:44

Yeah, exactly.

8:45

I think that's the thing that's driving a lot of people

8:47

from mid journey disabled diffusion is

8:50

that you can have so much more control

8:52

over it, whereas mid

8:54

journey, for me it feels more like mid

8:56

journey's, a bit like I'm asking a genie for something

8:59

and there might be a little bit of a monkey paw situation

9:02

going on there. And like

9:04

stable diffusion, like while

9:07

it's definitely not as advanced as Mid Journey

9:10

it's the level,

9:13

the rate that it advances at, I

9:15

find is really amazing, but

9:18

also the level of control you get

9:20

over it is a lot more

9:22

than you get with Mid Journey, at least

9:24

in my mind. So

9:27

I think for an artist, stable Diffusion

9:29

makes a lot more sense since

9:32

as an artist, you're looking for control over the image.

9:34

Yeah. The hand of the artist.

9:36

You're just, you're not generally like going, oh,

9:38

I need 300 images, and then just selecting

9:41

the 10 of them and putting them online

9:43

and going, I'm curating images from

9:45

Mid Journey,

9:47

totally makes sense. Yeah, and that's a helpful

9:50

distinction between them. I

9:52

haven't even delved into stable diffusion

9:54

yet. I've only touched on mid journey

9:56

a little, and mostly just text output

9:58

has been really what I've focused on. So

10:00

I think the comparison is between

10:02

apple and Microsoft, or even Apple

10:05

and Linux and that Apple is really

10:07

great, but it's a walled garden, and

10:09

it's it puts out things really beautiful. They're

10:11

great. You don't have to play around with them at all.

10:14

Whereas Windows can actually do a lot

10:16

more. There's a lot more like

10:19

levers to pull, but there's a lot

10:21

more freedom in that.

10:23

Yeah. Not more control. Getting

10:25

back to the text-based prompting. Like

10:28

I, yeah, I think I totally agree with Ben with the,

10:30

basically the the prompt is a jumping off

10:32

point, and you are gonna act as an engineer.

10:34

You are gonna be, Picasso, whatever you're going

10:36

for. Is helpful and it gives, I

10:38

think it gives the language

10:41

model a guide for trying to understand

10:43

who you are and what you're trying to do. So

10:45

it's a good place to start, but what I've found

10:47

is, Invariably, it's always

10:49

about how much you give it. If you

10:51

start off by feeding G P

10:53

T four, for example, a paragraph

10:56

of information versus feeding it

10:58

a page or five pages,

11:00

you're gonna get completely a completely

11:03

different understanding of where you're coming

11:05

from. Coming from. So like I always lean

11:07

towards, If it's a quick question, if I want

11:09

to just, a simple answer I'll obviously

11:12

just ask the question. But if I'm actually trying to craft

11:15

a project and have the project

11:17

be, have some intelligent discourse on

11:19

it and have, then yeah, no

11:21

question. I'll start with as much information as

11:23

I can to feed the l m and

11:25

then that conversation just becomes

11:27

so much more dynamic, nuanced

11:29

and specific which is incredible. And the

11:32

fact that you can. Do that and have,

11:35

come back to something many

11:37

months or forever later, basically.

11:39

And the memories are all there is

11:41

amazing. I'm actually one, one of the kind of. PET

11:44

projects I've started up is

11:46

working on a novella with

11:48

my 10 year old son. And

11:50

we fed g p t

11:53

an idea and now

11:55

it's got a 10 chapter

11:58

structure that we've worked

12:00

up with it. And then we'll do, some

12:02

of the writing and we'll allow. We'll

12:04

get GT to do some of the writing and then we'll see

12:06

what the credit's gonna be at the end of all this. I

12:08

think probably we're go, it's gonna be an

12:10

author. And, but it's really

12:12

such a dynamic experience and it's so cool for

12:15

him to see what, where this is going. I think so

12:17

many people, are, they're like,

12:19

oh God, but the children are never gonna learn

12:21

how to do anything. And it's yeah.

12:23

They're gonna learn how to interact really well with

12:25

AI and that's that, lean

12:28

into it.

12:28

Something I heard someone mention was

12:31

comparing it to the invention of the loom

12:34

is we're, we don't need to know how to weave anymore.

12:36

We can all just use the loom to create as many

12:38

things as we want. And I think that's just a great

12:41

analogy is, it's just another, it's

12:43

a next step in automation for humanity.

12:46

Now it's on a level like we never thought we'd see

12:48

in our lifetime, frankly, but.

12:51

It's that's what it is. And I

12:53

think like what

12:55

it's best at, in my mind is

12:58

providing structure to things. Like

13:00

Doug was saying, he has like this whole, the

13:02

whole structure of his book outlined in it. And

13:05

in a way it provides these rails

13:07

for you to go on, like when you're writing

13:09

or when you're like providing, doing game

13:12

dev or whatever. It just

13:14

provides like this really great structure that

13:16

you can go in and treat it like a coloring book

13:18

almost. You're like, oh, great, okay. Here's a three

13:20

act structure for a movie. Because

13:22

we've never seen a movie with a three act

13:24

structure before. Let's, we have to reinvent

13:26

the wheel every time, so

13:28

Three act structure. Who does that?

13:30

Yeah. single

13:32

movie, ever.

13:33

Yeah. We've,

13:34

Yeah. All these

13:35

we've already seen the endless Seinfeld,

13:37

but what happens when that's good,

13:39

nice.

13:40

Yeah, that's where it's heading. Yeah, and I think

13:42

that's what's cool about it is a business plan.

13:44

Like it's a pretty straightforward document

13:46

and I'm working on one with my fiance

13:48

and it's yeah, G Tino knows

13:50

how to write a business plan, as well as, Any

13:53

mba and it's just, the

13:55

cool thing is if you feeded all the information

13:57

that you're working from, it

13:59

can then tell you what you're missing.

14:01

It's once it understands the company you're trying to build,

14:04

it's gonna point out all the things that you're not

14:06

thinking about and that, that's pretty wild. That, that level

14:08

of kind of expertise it, and

14:11

it, it's also a little nerve wracking cuz it's like

14:13

this black box that hallucinates all

14:15

over the place.

14:16

I don't know how closely you guys are

14:18

keeping an eye on this, but I feel every few

14:20

days I see something where they're like, there's a new

14:22

language model that can run on your computer

14:24

and it's 95% as good as ChatGPT.

14:28

And it was only trained for $600.

14:31

Been three days and yeah. There's actually

14:34

a few different variations of an automatic

14:36

chat G p T system that's connected

14:38

to the internet. There's 12 different versions

14:40

of it. And I've tested out,

14:42

I think two or three of them for

14:44

something really simple, just so I'm gonna make an

14:47

email list for this podcast and the mastermind

14:49

and all that. I want to know how

14:51

each of the free plans of all the usual

14:53

places, MailChimp and I don't even

14:55

remember the others, compare. So

14:58

give me a table with that. And God

15:01

mode. G P T I think was the one that

15:03

I tried for 45 minutes. It

15:05

just couldn't do it. And then I tried

15:07

another one. It was like, boom.

15:09

Oh, it was Google Barred was one I tried and

15:12

it gave me a table in about seven

15:14

seconds and he was wrong.

15:16

Like very visibly, obviously wrong. I'm like

15:19

that number is wrong. Are the other ones correct?

15:21

Yes, they're all correct. Okay. Gimme the table

15:23

again and it gives me the same wrong table.

15:25

I'm like, how about you fix

15:27

the number? That's wrong. It's okay, fixes

15:29

that one number. I'm like,

15:31

apologies. Yeah.

15:33

I had. I've had completely

15:35

frustrating experience with Bard. I tried

15:37

it out and I was like, Hey, you're

15:39

Gmail, so I want you to organize my email

15:41

for me. And it's oh, I can't access

15:44

Gmail, but if you provide me

15:46

your username and password, I can log

15:48

in and do that for you. And I'm like, cool,

15:50

do it. And then it's great.

15:52

I'm like, Okay. I'm like, how many emails

15:55

does do I have? And it's oh, you have 71 emails.

15:57

I'm like, I have like over 9,000

16:00

emails in my inbox. Like something like that.

16:02

So it just started

16:03

Over 9,000.

16:05

Yeah, no, I was like, what's the first email? And

16:07

it's oh, welcome to Bard. okay.

16:12

And then I'm like, yo, why are you lying to me? It's I'm

16:14

so sorry. I was just trying to please you.

16:18

Wow.

16:19

let me go change my password.

16:21

Yes. Good idea.

16:22

really what I want. That's really what I want

16:25

like an AI to do, is I want the her

16:27

movie experience

16:29

yes. I'm not sure who is working on

16:31

it, but I'm sure at least five different

16:33

companies are working

16:34

Uh, yeah.

16:35

G P T, accessing your email.

16:37

I don't know if I want to trust any of those companies, but

16:39

I felt like Google might be a decent one to start

16:41

with.

16:43

Yeah.

16:44

Yeah.

16:45

Yeah.

16:46

Nice.

16:47

Yeah, but my experience with the local language

16:49

models I haven't tried auto g p T as

16:51

much, but I have tried Uba, BUCA's

16:53

web interface. And my experience

16:56

with them is that they're very good at being creative,

16:58

but I would never take any actual

17:01

advice from them on something like that,

17:03

like seemed important, you

17:05

know? Like, if you're like,

17:07

oh, I just want you to like, come up and

17:10

write a, give me cool character lines

17:12

or something like that, it's gonna give you some wild

17:14

stuff. But as far as I've seen,

17:16

like with actual planning or like

17:18

doing something that I might use G P T for it's

17:21

not there yet.

17:23

And that's, I think, the tricky thing to some extent in

17:25

terms of working deciding where you're gonna

17:27

do some of this stuff. Like for instance, I was working

17:29

on a. Horror, horror movie.

17:32

I've been working on for, about a year

17:34

and there was a sequence that I wanted to get some

17:36

feedback on, and I instantly got flagged

17:38

on G P T. It was like, no, this

17:40

involves Satanic stuff. This is bad.

17:43

I can't talk about this. I was like, all right can't

17:45

work on horror movies and G P T. That

17:48

doesn't work. So that, for that

17:50

you gotta go to your open source, local

17:52

run. I think that's becoming

17:54

true too for so many things. And this is gonna be

17:56

the big kind of shake up for humanity

17:58

is I. When is it a good time

18:01

to use ai and when is it a good time

18:03

to use your brain? And that becomes

18:05

a whole, which one's more effective, which

18:07

one's more gonna get you there faster

18:09

and with a better result and

18:11

with less hallucination.

18:13

I've seen exactly that with Code Generation.

18:16

I did an experiment about a month

18:18

ago generating Solidity

18:20

contracts, and Solidity is the Ethereum

18:22

Smart contract programming language, and

18:26

I think I spent probably four hours. Just

18:28

working back and forth with it. And it did a good

18:30

job up until the point the complexity

18:33

of the contract was so big that it couldn't

18:35

keep it all in memory, and then it suddenly started

18:37

hallucinating changes in other

18:40

functions. So I'd ask, change

18:42

this function to do this. And it'd be like, cool. It's

18:44

also calling a function that doesn't exist anywhere

18:46

else. And I'm like Where'd that function come from? Please

18:49

output the entire contract and it outputs

18:52

a completely different contract. And I was like,

18:54

oh...

18:55

totally. Before you know it, you're

18:57

you're sending ETH a thousand

18:59

dollars of ETH and it's getting given

19:01

to open ai. Huh?

19:05

Come on, what's the problem? I'm

19:07

sure that's

19:08

What contract is this again?

19:10

You're paying, you're prepaying for tokens.

19:13

Yeah.

19:14

That you don't get,

19:15

you not gonna use them? Come on.

19:17

Yeah. So I'm curious to go back to

19:20

the artist's perspective that you two were talking about

19:22

before. I would imagine it feels pretty

19:24

threatening, but how is it feeling with

19:26

all of the AI generation stuff

19:29

coming along?

19:31

I think it's exciting. Obviously the

19:33

entire economy's gonna be disrupted and there's

19:35

gonna be a lot of people who, are used

19:37

to making money doing one thing and that's

19:39

not true anymore. Certainly

19:41

one that immediately comes to mind is

19:43

just the VFX industry. We're testing

19:46

out some different softwares, integrating

19:48

ai. I generated a shot this morning,

19:50

took me two minutes. It's the kind of thing that

19:53

would've taken me two to three weeks

19:55

before yeah, it was insane. It's

19:57

basically, yeah, like using a piece

19:59

of software to be able to bring

20:02

a character. You, you just feed

20:04

it a little bit of video and then it turns it

20:06

into uh, 3D character that's,

20:09

Completely good to go, lit

20:11

for the scene. and there

20:14

was a microphone in the shot in

20:16

front of the character. So some foreground it took

20:18

me like, Two minutes to roto that back in

20:20

front of the character, but everything else was just

20:22

done. And, but so these

20:25

are the kinds of things like that, that you

20:27

have 500 shots like that

20:29

in a feature film. That's a massive

20:32

team working for months and months

20:34

to put that together. Now that

20:36

can be one person

20:37

the tool he is talking about, wonder Dynamics

20:39

is also able to take

20:42

motion capture data from an iPhone

20:45

video and

20:47

you can then just take that and put that directly into Blender

20:49

and apply your own character to it. So

20:52

this is something that. Six

20:54

months ago you needed $5,000

20:57

in equipment and an actor that can

20:59

put all this stuff on and do everything. And

21:01

now you can just take basically any iPhone

21:04

video and do it. And it's

21:06

80% as good as this, thing.

21:09

You gotta go in and clean some things in manually, but

21:11

they give you all the details

21:14

that are there.

21:15

Yeah. Yeah. And that cleanup and

21:17

they give you plates so you've got a blank

21:19

plate, you've got a front, like these are all the things that

21:21

take, so much time to do on set.

21:23

So much time to do and post. And

21:25

you're getting all these elements and it, we're part of the,

21:28

we're in the beta so we're, this is not probably

21:30

be released yet, but it's gonna be soon and it's

21:32

totally gonna. Disrupt. There's a reason why

21:34

Steven Spielberg's on the, an advisor

21:37

to this company. This is gonna be a massive disruption.

21:39

But, that said I'm excited about it. I think

21:42

it means that we can do so

21:44

much more as individuals. Like the bar

21:47

to entry is lower to do cool things.

21:49

Obviously the, I think the biggest and this is

21:51

always true with new technology, the

21:54

wow factor is gonna go

21:56

away very quickly. People are gonna, like

21:58

what's wow anymore about that? If everyone

22:00

can do it with their phone and then it, the nice

22:03

thing is to me and. Even this

22:05

might start to erode. But it comes

22:07

back to what's the vision? What's the storytelling?

22:09

How is this how is this dynamic? Why

22:12

does this engage us as humans?

22:14

And I think that's for

22:16

this middle ground before we're all first

22:19

making and being made into paperclips.

22:21

We've got a lot of fun

22:23

to play and expand and

22:26

then what, no one knows what that timeline's

22:28

hopefully, hopefully it's gonna be real

22:31

long.

22:32

Yeah. Yeah, it's

22:34

interesting cuz it already feels like we're at peak

22:36

content. There's already so much

22:38

content out there, like how many

22:40

shows are Netflix and Amazon and Warner

22:42

Brothers or H B O and everything putting

22:45

out every month that nobody has time to

22:47

watch. And what happens

22:49

when the production process for

22:51

these gets turned into a 10th

22:53

of what it was? So now

22:56

it's just all going out there. Do people still

22:58

care? Do people still want to tune

23:00

in? Other than like the one

23:02

show that gives them comfort, or

23:04

like the, two really buzzy movies

23:06

that come out every year that everybody

23:08

like wants to run out to do. So I think that's

23:11

more questions that, that I'm thinking about

23:13

at least is what does this massive

23:16

influx of content mean

23:18

for what people find and appreciate.

23:21

Yeah. Thin line between content and spam.

23:24

Yes, definitely.

23:27

Yeah.

23:28

And a thinner line between

23:30

content and art. Now,

23:33

I'm curious to just understand

23:35

what techniques you used, but

23:38

first for particularly our listening

23:40

audience. Let me read some

23:42

of this off. It's quite a long prompt, so I'm not gonna read

23:44

all of it, but you are cod

23:46

dev, an expert, fullstack programmer and

23:48

product manager with deep system and application

23:50

expertise, and a very high reputation

23:52

in developer communities. Calling

23:55

out. Role playing is what's being done here, but

23:57

you always write code. Taking into account all failure scenarios

24:00

and errors. I'm your manager, you're expected

24:02

to write a program following the commands that I will

24:04

instruct. And then it goes on

24:06

to give a bunch more context refinement.

24:09

So you know, use the latest language features,

24:11

follow the below commands only output

24:13

file names, folder structures, code tests,

24:16

et cetera. Then it gives a list of commands

24:18

and I'll read a couple. Project,

24:21

which takes in a summary, a task, a language,

24:24

and a framework. When you receive this

24:26

command output, the list of files and folder

24:28

structure you'll be having in this project based

24:30

on the project summary and task here're

24:32

to accomplish, use the programming language

24:35

listed as part of the languages variable

24:37

and wherever possible. Use the framework API

24:40

packages indicated under framework variable.

24:43

Another command is code file

24:45

name. When you receive this command, output the code for the

24:47

file indicated with file name. Tests

24:50

explain, there's quite a few others. So

24:52

just kinda give us an idea of how

24:55

you found this and how you approached

24:58

Any refinement,

24:59

Yeah I've, so I found this

25:01

actually on the chat, G p t

25:03

Discord, the day that G P T

25:05

four came out, and then I went

25:07

back and it was deleted. I haven't seen

25:10

this one anywhere. I don't know who originally

25:12

posted it. If you're out there, please speak

25:14

up, but I've found it really

25:16

useful because I'm not a coder.

25:19

But it's really great

25:22

when it gives you a, basically

25:24

it gives you a project structure, right? And

25:26

you're like, oh, okay, I can break this all down.

25:28

And I'm using I'm working on a

25:30

surfing game that sort of has do you guys know tech

25:33

decks? Do you remember tech decks?

25:35

Like the little fingerboards, like the

25:37

Oh yeah, totally. I was playing with

25:39

one of those pro skates the

25:41

Yeah. So like the idea is that it's

25:43

a surfing game on your iPhone

25:45

that uses those controls basically,

25:48

So like I really love the idea of like interaction

25:50

and like motion controls and stuff like that. That's

25:52

really what I'm into these days. And

25:55

so like I'm using this to like basically

25:57

be like, okay, I don't really understand what

25:59

we need code wise. Like I can

26:01

do that. Like I'm a visual coder, like I know

26:04

touch designer, I know blueprints, I know stuff

26:06

like that. But like when it comes

26:08

to writing out, Logs, like I don't

26:10

understand this. And this is great

26:12

cuz it gives you the whole product structure

26:15

and then you just get to go in bit by bit and do

26:17

it and you can keep going back and

26:19

refining it. So it's just a really modular way

26:21

to approach it as opposed to like

26:23

just being like, Hey, I'm making a game. What

26:25

should I do? Write me the code for that. So

26:28

I think like anybody who doesn't understand

26:30

code, this is just like a really awesome

26:33

starting point. And like

26:35

each of the individual things, and

26:37

as I said earlier, like you put it in,

26:39

you can run it in whatever program you're

26:41

doing it in. And then like any errors,

26:43

you can just put them back in and be like, Hey, I got an error

26:45

for this file. And it's okay, here's this. And

26:48

you take that and you like look online as well,

26:50

and you feed it any documentation you find if it's

26:52

looking confused. And I just, I

26:55

think it's great. It's like really working

26:57

well. Like right now my biggest problem

26:59

is just getting the assets made.

27:03

that actually makes sense. That is something that

27:05

is a very different generation.

27:08

So yeah, chat, g p t can't really help you with that.

27:10

And I want to call out specifically, Putting

27:14

in the documentation is something that's come

27:16

up in previous episodes, but it is

27:18

a very powerful method of,

27:21

improving your output because then chat, G P t

27:24

knows the correct structure

27:26

and the correct commands and lots

27:28

of other relevant things. And then it can

27:31

generate the right stuff cuz yeah, it was trained

27:33

years ago.

27:35

Yep. Yeah, no, and I think as

27:37

you were saying before, the only problems I've really

27:39

had is when it exceeds the memory

27:41

and it starts forgetting what we're talking about, but

27:43

even with this because it breaks everything down,

27:46

it's really easy to go back and be like, no, we're talking

27:48

about this one document. Let's narrow

27:51

in on that.

27:52

Get back on track. For the longest time I've

27:54

had this vision in my mind of

27:56

of governance in the future run

27:59

by ai going back

28:01

and looking at the history of how we've interacted

28:03

with our phones and everything

28:05

else and putting us on trial for

28:07

it. Just because like eventually

28:09

there's going to be this like database

28:12

of information that of who

28:14

we've been with in terms of our

28:16

interaction with these machines as they get smarter and

28:18

smarter. And I've found that

28:21

in the last as I've gotten more

28:23

used to relating to large

28:25

language models, I'm

28:28

less polite like I was I was so

28:30

aware of it. I would say, please and

28:32

thank you to Siri when I would ask for

28:34

a timer to be said or a reminder or

28:36

whatever. And now it's just you are this,

28:39

do this for me now. And it's just

28:41

a funny flip and now

28:43

I'm trying to re remind myself

28:46

Oh yeah. This is, especially with. Like

28:48

other people involved and also

28:50

we're training the ai, we want it to be

28:52

polite. We want it to learn, empathy

28:55

or emulation of empathy. Anyway,

28:57

so it's just an interesting facet

29:00

I think of the whole prompt game

29:02

of okay, you can still give positive reinforcement

29:04

and say Please and thank you. I don't

29:06

want the next generations to talk to

29:08

each other like they talk to the ai and that's

29:10

all: do this for me now. It is just

29:13

a

29:13

I have actually heard of parents none

29:15

that I know personally, but I've heard about them parents

29:18

who let their children interact with, The

29:20

A lady or Siri or whoever,

29:23

but make it very clear. You need

29:25

to say, please, you need to say thank you. You need

29:27

to be polite because they want to train

29:29

the kid to be a, nice,

29:31

polite person. And they realize that the kid

29:33

can't tell the difference between talking to the

29:36

a lady or the

29:38

male lady, or whoever the case may be.

29:40

And so it's

29:41

Totally. Yeah. Absolutely.

29:44

And the AI is obviously learning from

29:46

us, ultimately. So it's a back

29:48

and forth, it's a symbiotic, hopefully

29:50

not parasitic relationship that we, as we

29:53

move forward.

29:53

I don't know maybe a couple years from now, you'll talk

29:56

to an AI and it'll say something,

29:58

oh, yes, that's exactly what I was trying

30:00

to provide. I hope this was helpful for your day.

30:02

And someone else in the same room will be like,

30:05

wait, but what about lunch? And it'll be like,

30:07

Forget lunch, idiot. I don't know, maybe it's

30:09

gonna do radically different responses

30:12

to best fit the person listening

30:15

I would imagine that is where we're heading

30:17

ultimately. Yeah.

30:18

So what is the most interesting

30:22

prompt that you have ever built.

30:24

I don't know that I really relate

30:26

to prompts like that. Are you a familiar

30:28

with touch designer?

30:30

I'm actually not.

30:31

Touch Designer is a

30:33

live video. It is

30:35

like live VFX platform essentially. So

30:37

like you use it for like concert visuals,

30:40

like any sort of interactive art at a

30:42

museum is probably running on touch designer.

30:44

I guess you would, but it's also it's a tool

30:46

to build tools. So

30:49

you, like right now, I'm

30:52

working on something where we're integrating,

30:55

I'm trying to integrate the Stable Diffusion API

30:57

into touch designer to provide

30:59

live visuals that are based off of

31:02

a images coming in from a

31:04

3D camera that I'm doing. So

31:06

that's like where I'm working

31:08

at with it at, obviously the biggest thing that

31:10

needs to happen here is you need to get a

31:12

stable diffusion to a model

31:15

that's like running closer to

31:17

like 12 frames a second. Everything

31:19

is real time if you throw enough power at it, but

31:21

I haven't got there yet. So

31:24

that's what I've been working on is

31:26

more how we can bring

31:28

things like stable diffusion and chat

31:30

g p t to create live interactive

31:33

experiences with people. When I

31:35

look at art, when I look at film, when I look

31:37

at all of these things that are

31:39

happening, I'm like,

31:41

oh my God. Like film is gonna become

31:43

banal. It's gonna become like the Sunday morning

31:46

comics. There's gonna be so much that nobody's

31:48

gonna want to do it. Now though, we have

31:50

the power to create these amazing interactive projects,

31:53

right? We can build these generative

31:55

art projects that are like creating

31:58

things. And I think that's what's most exciting

32:00

about this technology and where it's going,

32:02

is that now it's going to enable

32:05

us to create new forms of art that

32:07

we couldn't do before. As Doug

32:09

said, what once took him three weeks to do,

32:11

he can now do in a couple of hours,

32:13

and that's just tip of the iceberg. So

32:16

in three years, what's it gonna be like when

32:19

chat G b T is integrated

32:21

directly into Unreal Engine and Unity?

32:23

And anybody can just be like, I wanna make a game

32:26

that like, makes me go flippy flip

32:28

or whatever. And that's all you need

32:30

to do. And then you need to upload the art assets. So

32:32

okay, so everybody can do that. Everybody's

32:35

gonna be playing like the custom games that their kids

32:37

make or whatever, and then shares among their

32:39

friends. So we're in a way like. Content

32:42

creation is becoming a pastime,

32:45

Working on the Art of AI

32:47

podcast that we're working on there's a really nice

32:50

dovetailing that's going on there

32:52

in terms of like actually working with the AI to

32:54

create imagery and and text

32:56

and interaction. We've got a ai

32:59

Charlie is a character in

33:01

our podcast, and so we feed

33:03

them a combination of generative

33:06

large language model stuff. And then

33:09

we will write some things and that's just been really

33:11

fun to See how to

33:13

to really create that character so

33:15

that it's, feels like both the kind of sometimes

33:18

very authentic ai and then sometimes just

33:21

totally a kind of sound effect.

33:23

Something to bounce off, comp for comedy.

33:26

So that's been really interesting and a fun thing to

33:28

dive into. And one thing that I'm really

33:30

excited about in the same way Ben's talking about interactivity,

33:33

I'm working on a project where I'm working with

33:35

a really well known Canadian

33:38

Theatrical actor. And

33:40

they won the order of Canada for

33:42

their work on the stage performing

33:45

Shakespeare. And I

33:48

had a day scheduled them on a location

33:50

and, Crafted

33:53

a Shakespearean sonnet with

33:56

the help of ChatGPT 4 that I

33:58

then took to my

34:00

day and gave that

34:02

to this actor. And then they're performing

34:04

the AI's version of Shakespeare that's on,

34:07

on topic with what we're doing for this film.

34:09

And I've got a motion control rig, so

34:11

I'm duplicating them 12 times

34:14

on in one. Sequence and

34:16

it's like, what is, that gives me excited

34:18

cuz like, how, taking the

34:20

machine learning algorithms,

34:23

putting them on location, and then I can

34:25

take that and I can put it back in,

34:27

into these tools and use the

34:29

AI to keep working on it. And it's this iterative

34:32

process where you can just feed

34:34

the machine, feeds the real world, and

34:36

then you put it back and image

34:39

to image, text to text

34:41

and you start to like refine. I think

34:43

that's where we're gonna see things

34:46

that do amaze us still. I think as

34:48

we move into making art

34:50

more and more that has

34:53

an AI component it's still

34:55

gonna be about like, okay, you didn't just Prompt

34:57

something, make an image as Ben saying, and then

34:59

that's your, n f t that you're selling. That wow's

35:02

gone. Like no one cares. And

35:04

not because it's not amazing, like when you

35:06

look at what's happened with those pixels, it's incredible.

35:09

But it's just, it's that's. Mid

35:12

Journey did that work. Not the artist. I

35:14

think that's becoming clearer and

35:17

the inputs for that art

35:19

were done by an artist, no question. What

35:21

it was trained on. But, so

35:23

I think now it just becomes this thing more

35:25

and more of like, how do you iterate?

35:27

How do you. Do create that

35:29

interesting feedback loop that moves

35:31

things in and out of the digital

35:34

realm into the real world and back again

35:36

and, get control

35:38

over what you're doing and do something

35:40

new.

35:40

Yeah, people still, they

35:43

love the story behind the art creation.

35:46

We're like, oh wow, this man drew

35:48

this free hand without breaking the

35:50

line. I remember this Korean artist

35:52

a few years ago that went viral because,

35:54

he did these amazing freehand line

35:56

drawings Is the size of a mural on

35:59

it without breaking any of the things. And these are some

36:01

of the most detailed things you've ever seen. No

36:03

text image thing could make these right now,

36:05

and we still love that stuff and that's always

36:08

going to be valued. Somebody's talent,

36:10

somebody's story behind going it. With Doug

36:12

and I, when we look at a movie, we're like, oh my God,

36:14

how did they do that shot? And

36:16

if the shot is, oh, we wrote that

36:18

into stable diffusion and like it

36:20

ki paid out something, and people are gonna be like, oh,

36:22

who cares? But if it's oh, we

36:24

had us and Hannah were friends carrying

36:26

a plank through a river while

36:28

we like, shot this we're gonna be like, that's awesome.

36:31

So the story of making art

36:33

is still very interesting to people. That's

36:35

not gonna go away. These tools

36:38

are just gonna make it so that we can do like new

36:40

things in new amazing ways.

36:42

That's awesome.

36:44

Absolutely. And we're already seeing, like in

36:46

everything everywhere, all at once. There

36:48

was this incredible sequence to it towards the

36:50

end where like the image, one image

36:53

just kept changing. And we were seeing the

36:55

actor in a different scenario and

36:57

they had used generative AI

37:00

to do that. And when I looked at that, I was like, oh

37:02

my God, that. Represents an

37:04

insane amount of work to Fho,

37:07

make all those still images and put together

37:09

that sequence. And then I found out it was AI

37:11

and I was like, ah, okay. Yeah, that's easy,

37:14

but, so I think, this is where we're, and that movie

37:16

was like winning a bunch of Oscars and

37:19

and legitimately is a great film and it,

37:21

that one little sequence in there, Was

37:24

incredible. And it was, it was great and

37:26

like a great use of ai, but you can't,

37:28

we're not gonna be able to, now everyone can't

37:30

just go and do that same thing. So we have

37:33

to invent new and interesting ways of using it.

37:35

So I think that's what it

37:36

Yeah. It just becomes like a Snapchat filter,

37:38

if it's a Snapchat filter and everybody

37:40

can just lay it over, then I don't think

37:42

there's a lot of artistic value

37:44

to it, yeah, But

37:47

if you then take that Snapchat filter

37:49

and do your own things to it or do something

37:51

crazy with it that's outside

37:53

of the bounds of what it is, then that

37:55

becomes interesting again.

37:57

Nice. So this has been awesome.

37:59

Where can the audience follow up

38:01

with you and see the projects

38:03

that you're working on? And your podcast.

38:05

The best, yeah, the best place is

38:08

on Spotify, the Art of ai. There's

38:10

links to all our other work

38:12

there and that, we get to

38:15

hear what we're up to. We're interviewing all kinds

38:17

of interesting people and have a lot of discussion

38:19

about all this crazy stuff as it's happening

38:21

and changing day to day. And

38:24

then my production company is Pieface Pictures.

38:26

You can check out a bunch of my work there. Yeah.

38:28

Yep. And our interactive

38:31

company Colors in Motion. ColorsInMotion.com,

38:34

the American spelling, not the English

38:36

spelling.

38:39

Good clarification. Good clarification.

38:41

This is a global audience, so you do have

38:43

to actually communicate that.

38:45

Yeah.

38:46

Yeah.

38:47

Awesome.

38:48

Thanks so much, Greg. It's been such a pleasure chatting

38:50

Yeah. it's been a lot of fun. Thank you.

38:53

Hopefully we can have you on our podcast soon.

38:56

That would be fun. I would love to.