0:01

Welcome to another episode of the Mapscaping

0:04

Podcast. My name is Daniel and this

0:06

is a podcast for the geospatial community.

0:08

Today on the show we're talking about synthetic data.

0:11

And so this is not the synthetic you

0:14

think about when you think about synthetic aperture radar.

0:17

This is the synthetic we think about when we think about

0:19

fake, not real, generated, that

0:21

kind of synthetic data. Here's a little

0:23

bit of background before we jump into the conversation. Data

0:26

vision is everywhere, but teaching an algorithm to

0:28

identify objects requires a lot of data.

0:31

And this is definitely the case when we think about

0:33

geo AI. But it's not enough

0:35

that we need a lot of data. We also need

0:37

that data to be labeled. So

0:39

for example, if we're looking for cars and images, we

0:42

need a lot of images of cars and we need

0:44

to know which pixels are the car. Of

0:46

course, I'm oversimplifying things

0:48

here, but I hope you get the idea. Now imagine

0:51

that you can automatically generate a large label

0:53

data set of realistic images

0:55

of cars based on the specifications

0:58

of a specific sensor. These

1:00

data sets are often referred to as being synthetic or

1:02

fake data. And to help us understand more

1:05

about this, I've invited Chris Andrews from

1:07

Rendered AI on the podcast. So this

1:10

episode is building off a bunch of different episodes

1:12

that we've published before around computer

1:15

vision. And there'll be a few interesting links

1:17

for you in the show notes today that are well worth checking

1:19

out. Before we get started, a big thank

1:21

you to the Open Geospatial Consortium,

1:24

the OGC, for inviting me to their

1:26

members meeting in Alabama and for helping

1:28

make this episode possible. Thanks

1:30

very much.

1:31

I really appreciate your support.

1:36

Hey, Chris, welcome to the podcast. We

1:38

met a while ago now a couple of months ago at

1:40

an OGC meeting.

1:43

And you're doing something amazing, at least in my mind, you're

1:45

creating synthetic data. You

1:48

work for a company called Rendered AI. And

1:50

I've been wanting to have this conversation for quite some time.

1:54

Before we get into the synthetic data stuff, maybe

1:56

you could just tell us a little bit more about yourself, perhaps

1:58

a brief introduction is what I'm looking for.

1:59

for who are you, how did you

2:02

get involved in this, and what

2:04

is your title at Rendered AI?

2:07

Yeah, thanks Dan. I appreciate the chance

2:09

to chat with you. It was a lot of fun to take

2:11

a walk with you and

2:13

chat a couple of different times in

2:15

the sun in Alabama there at the OGC meeting.

2:18

I'm definitely happy to share a bit about what

2:21

we do and who

2:23

I am. So my background

2:25

is I started off in the sciences,

2:28

in geology and ecology.

2:31

Along the way, I was always a programmer,

2:33

and then somewhere in my

2:36

graduate work, I was directed

2:39

toward GIS because I had built

2:41

my own mini GIS to

2:43

map out ants and populations.

2:46

And yeah, and that

2:49

actually started me working

2:51

with, I got interested

2:53

in ArcView, so

2:56

old, the original ArcView, I

2:58

think it was actually 2.0 or something like that. And

3:00

when I actually sat down at it for the

3:02

first time, I actually started programming.

3:05

I think there had just been an update to

3:07

ArcView, the avenue programming language.

3:10

And so from the very beginning, I

3:12

was always interested in customizing,

3:15

extending

3:18

GIS for analytical

3:20

and other purposes, not

3:23

so much cartography

3:25

or the kind of map production

3:28

side of GIS. So

3:30

starting to work in GIS

3:32

brought together some of my interests around

3:35

data investigation and

3:37

analysis and then visual

3:40

stuff. I've always liked to work in visual

3:42

domains. That's why I've been in

3:45

GIS or GIS-centric

3:47

things for so much of my career.

3:49

And from there,

3:52

having an ecology degree in the mid

3:54

90s was not such a great thing. And so

3:57

in terms of for jobs,

3:59

but having...

3:59

some programming and GIS skills was

4:02

a fantastic thing. It got

4:04

me

4:05

job offers at startups and at bigger

4:07

companies like Bentley Systems

4:09

within a couple weeks of

4:11

graduation.

4:13

I'm pretty sure I was one of the first

4:15

people to put a map on the web. I think

4:17

there might've been people who did it earlier, but

4:20

this was an interactive map that you could zoom

4:22

in on and pan around on it. From

4:24

there, that got me into

4:27

more web mapping stuff and then

4:29

from there, a synthetic

4:32

Kennedy Space Center and

4:34

work in

4:35

enterprise integration, which then took

4:37

over the next few years of my career.

4:40

I eventually ended up picking up a variety of

4:42

specialty around utilities and AEC.

4:46

And then I ended up at Autodesk. After

4:49

seven years at Autodesk, leaving

4:51

and then less than a year later

4:54

going to Esri, where I started

4:56

out as the product manager

4:58

for 3D across the ArcGIS platform,

5:01

that was an interesting opportunity

5:03

because it allowed me to touch just about anything.

5:06

My joke used to be, people would ask me

5:09

about different things and I'd say, well, gosh,

5:11

I don't know, who's the 2D product manager

5:13

here? And the reason

5:16

why that's kind of an inside joke is because

5:18

really it's Jack Dangerman, right? He's

5:20

the

5:21

ultimate arbiter of all that.

5:23

And so it was an interesting opportunity

5:26

to kind of carve out

5:28

a new role that touched just

5:31

about everything. So

5:33

it was a pretty

5:34

interesting role. I

5:37

almost immediately helped get the

5:39

ArcGIS Earth's effort started.

5:41

Later, I theorized

5:43

a lot was that Autodesk and I got together and

5:46

we

5:46

helped get the Autodesk-Esri

5:49

partnership started. That was a huge effort

5:52

to convince Esri that

5:53

BIM was really relevant to GIS. And

5:55

now it's

5:57

a deeply established part of their market.

5:59

rapidly. So

6:01

that was all fantastic. I

6:03

ended up, after seven years, being one of

6:06

the PM leads, I had a team of 20 and

6:09

there were about 45-ish products

6:11

under me. But I kind of

6:14

wanted to get back to my smaller company

6:16

roots

6:17

and have a bit more direct influence

6:19

on the development of a small company in

6:21

an

6:22

emerging technology domain.

6:24

And

6:25

not that everything

6:27

was completed in 3D and Mgis

6:30

integration, but I've always

6:33

leaned into these new areas.

6:35

And

6:36

so an opportunity came up to

6:38

join a small company who was working on

6:41

synthetic computer vision imagery

6:44

for AI training. And

6:46

this particular company happened to do

6:48

a lot of their initial work in the

6:51

remote sensing space, simulating satellite

6:53

imagery.

6:54

So it kind of brought together my

6:56

interest in

6:58

contributing an exec level at a small

7:00

company to help build and grow a business and market.

7:03

And then also

7:04

background in everything from 3D

7:06

to digital twins to reality

7:08

capture simulation. And

7:11

then still is quite

7:13

open beyond that because computer vision

7:16

is everywhere across just about every

7:18

human industry these days. And

7:20

it's just starting. So there's tremendous opportunity.

7:23

I ended up joining Rendered AI as the

7:25

COO and had a product.

7:28

And that was about two years ago. And we've been continuously

7:31

and steadily

7:33

growing and expanding our business since

7:35

then. And it's been a quite

7:37

exciting adventure so far.

7:39

Well, you have come a long way.

7:42

So a couple of key points I

7:44

just want to sort of highlight here. You started

7:47

off writing your own software, building

7:49

your own GIS to map

7:51

ads. You're one of the first people

7:54

to create

7:56

a web map, to put a map on the web, which

7:58

is amazing in itself. We

8:00

worked at some massive companies.

8:02

It sounds like you've been on the forefront of

8:04

a lot of technologies as

8:07

they were emerging. So 3D, like I remember

8:09

back in the day when I started at GIS,

8:11

the joke was it's all flat. If

8:13

you want to do 3D, if you want to

8:16

actually model the world, then you need to

8:18

go to BIM because over there,

8:20

there at a 3D, you guys are still flatlanders.

8:23

It sounds like you're on the forefront of that, at

8:26

least at Esri. And now you're working

8:29

with synthetic data. You're getting back to this, I think

8:31

you call it your small company startup

8:33

roots again. So let's

8:35

dive into this synthetic data thing because that's the

8:37

promise of this episode is to help people understand more about

8:39

it. A while ago, I

8:41

published an episode called Fake Satellite

8:43

Data and it was all about using GANTS, so

8:46

Generative

8:48

Adversarial Networks to produce

8:50

fake imagery. Is that what we're talking about when we

8:52

talk about synthetic

8:55

images or synthetic data?

8:57

No.

9:00

Before I address that directly, I just want to

9:02

tell you an anecdote about myself

9:05

in 3D. So the actual first 3D map that

9:07

I ever built was I hand surveyed

9:09

an antifilled in Tallahassee,

9:12

Florida in 1996.

9:14

So

9:17

I used a front and compass, a geology tool and

9:20

a one and a half meter stick to basically

9:22

build a

9:23

3D map of the entire

9:26

meter by meter. I had a one meter resolution

9:28

for this entire 50 meter

9:31

radius ant field

9:33

and study field. And

9:36

then I actually used Excel

9:39

to plot it as a 3D, as

9:41

a bunch of 3D points. So that

9:44

was my first brush with 3D. And

9:46

then later I saw I actually

9:48

did touch 3D analyst

9:50

about two years later

9:53

when I was with a different startup company

9:55

from the first one. I don't know. I always

9:57

had a nose for leaning.

10:00

Into technologies that

10:02

were likely to take off and and avoiding

10:05

some that were not for example

10:07

I avoided cold fusion and PHP

10:09

Like they were the plague because I

10:11

just looked at them and said these are not

10:13

Going to stick around and you could say

10:16

that they kind of have because there's a lot of like zombie

10:18

Apps out there that are still based upon

10:21

both of them actually frighteningly enough But

10:23

uh, but I've just had a nose for it more

10:25

on the tech side than the business side I guess if I had

10:27

more of a nose for it on the business side that I'd be

10:30

I'd have a few extra zeros At the

10:32

end of my bank statements, but but

10:34

I don't

10:35

back to your question

10:37

As I said, we focus on synthetic

10:40

computer vision data Actually, what what

10:42

we build is a platform as a service

10:44

for customers to generate

10:47

Tailored

10:49

Synthetic computer vision data for

10:51

training algorithms We don't do

10:54

algorithm training and in some

10:56

cases

10:57

We don't even need to help customers with

11:00

the process of setting up their synthetic

11:02

data simulations and we can talk about that

11:05

Today all of the synthetic

11:07

data Channels, which we

11:10

call synthetic data applications a

11:12

synthetic data channel all of the synthetic

11:14

data channels that run on rendered today

11:16

are not Gan

11:19

based there is a step where we can

11:22

use Gans and I can come to that But

11:24

so today what all of the simulations

11:27

that we run do is essentially

11:30

use kind of classical

11:32

simulation techniques to

11:34

emulate physical processes in digital

11:37

form and then simulate

11:39

the collection of real sensor

11:42

imagery

11:43

and this is true for we have

11:46

Customers who have channels for RGB

11:48

and panchromatic satellite imagery.

11:51

We work with dear SIG from

11:53

RIT that is a Long-term

11:57

government funded simulator for multi-spy

11:59

spectral and hyperspectral imagery.

12:02

We have our own SAR, synthetic

12:04

aperture radar simulator.

12:06

And all of those things are physics-based

12:10

simulation systems. And

12:12

so we don't actually use

12:16

GANs to generate the primary

12:18

imagery that a customer

12:20

might be interested in. It's

12:23

highly conceivable that in the future or

12:25

even next

12:28

week that we could build or a customer

12:30

could actually build in again

12:33

into a synthetic data channel in

12:36

rendered AI and start generating

12:39

synthetic computer vision imagery using a GAN

12:42

or something in one of these diffusion models.

12:44

However, there are today

12:47

some major differences

12:49

between physics-derived synthetic data

12:51

versus generative AI-derived

12:53

synthetic data that would

12:56

make that generative AI data less

12:58

usable than physics-based

13:01

synthetic data.

13:03

Okay, so let's talk about the physics-based

13:05

synthetic data that you are creating or your

13:08

customers are creating on your service.

13:11

What do I start with?

13:12

What do I need to get going with this?

13:14

There are two parts to our platform. One

13:17

part, every platform as a service out there

13:19

is all about

13:21

hiding the complexity of cloud capability

13:23

to accomplish some kind of job. And

13:26

the job for

13:27

us, the job that our customers need accomplished

13:30

is to

13:31

get

13:32

lots of fully labeled

13:35

computer vision imagery

13:37

out of this process that

13:40

is effectively magic to most data

13:42

scientists who today rely

13:45

on real sensor data collection.

13:48

On top of the platform is

13:50

this notion of these containerized

13:53

synthetic data channels. And so by

13:55

containerized, what I mean is

13:57

we literally use Docker. Docker

14:01

is a technology product

14:03

set for building

14:05

these packages, wrappers around

14:08

a self-contained programming

14:11

programs of some kind. A Docker is

14:13

used for many, many things not just synthetic

14:16

data generation. In our platform,

14:18

we provide a framework to basically

14:21

construct one of these Docker containers

14:23

to have

14:27

some simulation capabilities

14:29

or a renderer of some kind. This could be Nvidia

14:32

Omniverse or Dearsig

14:33

or consumably

14:36

Blender or we've had

14:38

experiments with Unreal Engine, for example, as well. Then

14:42

access to content to

14:44

be able to assemble a digital

14:47

twin of a scene, a

14:51

real scene somewhere

14:54

that you might point to sensor at.

14:57

Then some sensor parameterization and

14:59

then other

15:01

diversity that can be introduced

15:04

to that simulation. For example, do you need fog

15:06

effects or variable lens distortion

15:08

or variable daylighting

15:11

or anything like that. Basically,

15:12

this containerized simulation

15:14

packages up access

15:17

to 3D and 2D content, the simulation

15:20

capability itself, parameterization

15:23

around the simulation. Then that

15:25

container can be deployed to our platform

15:28

and then a data scientist or

15:30

computer vision engineer

15:31

can use a very simple web interface

15:34

to configure the

15:36

actual jobs that will be

15:39

executed using that simulation.

15:41

Then the platform is

15:44

there so that

15:45

when the computer vision engineer pushes

15:47

a button and says, go, they

15:49

can get 10 or 100 or 10,000 runs

15:51

of that simulation.

15:56

As I hinted at a second ago, in

15:58

that simulation is built-in

16:01

variability

16:01

that can be stochastically

16:03

varied,

16:04

such that if you're looking for

16:07

zebras, you're

16:09

trying to build an algorithm that counts zebras

16:11

on

16:12

a grassy plane,

16:14

well, part of the

16:16

structure of that simulation

16:19

includes information

16:21

about how to place zebras, how many zebras

16:24

could be placed, and

16:26

then even where against

16:28

the background that the zebras would occur.

16:32

So that when you run that simulation 10,000

16:34

times, you'll get 10,000 image chips with

16:38

the zebras in

16:40

different patterns according to your parameters

16:43

on those image chips. And then

16:45

you'll also get annotation identifying

16:48

things like bounding boxes around the zebras.

16:51

And then

16:52

for RGB and a few other

16:54

data sets, you'll also get pixel image

16:57

masks because when you're rendering

16:59

that image, you actually know what

17:01

is at each pixel. So

17:03

I know where the zebras are, I know

17:06

where the grass is, I might

17:08

know where the river is if

17:10

I had a river in there and I care about it.

17:12

That's something that you can't get through

17:15

generative AI today,

17:16

not directly.

17:19

In the generative AI world today, we're

17:21

typically talking about a

17:23

text instruction that

17:26

is interpreted through

17:28

some kind of

17:30

vector database. And then that

17:33

information that is then extracted from

17:35

the text instruction is used to drive

17:38

the

17:38

generation of the image in

17:41

whatever generative model is used to

17:43

spit out a set of pixels

17:45

at the end.

17:46

But those pixels are not

17:48

deterministically

17:50

prescribed in that model such that

17:53

you don't know

17:54

where the zebras are in the image, you just

17:57

happen to know that there are zebras against

18:00

grass in an image,

18:02

spit out by that generative model. We

18:06

cover a lot of ground there. Let me try and summarize this a little bit,

18:08

and you can tell me where I'm going wrong. So

18:10

my understanding is this, you keep

18:13

calling it containerization. We have this container.

18:15

Inside the container, we have a model. I think you

18:17

referred to this as being the content.

18:19

What is it that we want to

18:21

simulate here? So in your example,

18:23

again, it was a zebra. So I have a physical

18:25

model of a zebra, at least

18:28

this is my understanding. In this

18:30

container, I have parameters.

18:32

So I can say, I'm using this kind

18:34

of sensor. The

18:36

look angle is from

18:38

this side, from this height, from this angle,

18:40

or whatever. Maybe I need some

18:42

fog or something like this. And I can say,

18:45

OK, as the data scientists generate,

18:47

run.

18:48

And this container, this

18:50

program, will go off and create maybe 10,000 images of zebras,

18:53

given the parameters,

18:56

given the scene that I have described that I wanted

18:58

to take images of.

19:00

And in those images, I not only do

19:03

I know that there is a zebra somewhere in this image,

19:05

but I know which pixels are the zebras. So

19:07

not just the bounding box of the zebras, but which

19:09

pixels are the zebras. And my guess is this

19:12

would be an amazing training set.

19:14

If, for example, maybe I don't

19:16

have time to wait for all

19:19

of those to build up a massive training

19:21

set of satellite imagery,

19:24

where I know that I have zebras in it because of clouds,

19:26

because of the passover rates, whatever,

19:28

because of the orbits, because maybe it's really

19:30

difficult to find zebras. This sounds like a

19:33

way of generating such

19:34

a training set. That's

19:36

exactly right. And then

19:39

there are a few other questions that

19:41

come out of that. For example,

19:44

are my pixels, is it

19:46

really like real imagery

19:49

collection? And there are,

19:51

we provide a couple different techniques

19:53

to help both post-process

19:56

the simulated imagery to be

19:58

more like a real data. data set, and

20:01

then also to compare data sets to make

20:03

sure that the synthetic data is

20:05

going to behave as if it is real data

20:09

when training an algorithm. So there's

20:11

more to it than just this

20:13

containerized digital twin of the sensor collection

20:16

scenario. That's part of it. And

20:19

after that, then you may do

20:21

things like actually use a cycleGAN

20:23

that's been trained on real sensor

20:26

data to post-process the synthetic

20:28

data to make it kind of look

20:31

to an AI more like the real

20:33

data.

20:34

And then beyond that, like I said, there

20:36

is this opportunity to then start comparing

20:38

real and synthetic data sets and actually

20:41

exploring if

20:42

you make changes to your synthetic data,

20:45

does it actually make it

20:47

appear more like real data to

20:49

an algorithm?

20:50

And that's another nuance is suppose you generate

20:52

one data set with zebras

20:55

in one particular pattern,

20:56

but then your research comes back and

20:58

you decide that you know what, zebras actually,

21:01

the way they follow each other or the way they stand

21:03

around looking at each other actually

21:06

is in some other patterns than what

21:09

was built into the original simulation.

21:11

Or suppose you want to mix zebras

21:14

and buffaloes and make sure that you're not accidentally

21:16

counting water buffaloes. Well,

21:19

you just go back to your synthetic data

21:21

channel. You maybe add in some

21:23

more 3D models. You

21:26

can figure an additional run

21:28

and you kick off 10,000 more images. And

21:30

it's really that

21:31

easy once that channel is set

21:34

up

21:35

to experiment with a wide variety

21:37

of data variation. And suppose

21:39

you want to make sure that the algorithm

21:42

is going to fail predictably. And

21:44

one thing you can do, for example, is

21:47

you could go in and you could say, I

21:49

want all my zebras to be blue, knowing

21:52

full well that zebras are never blue in the wild.

21:55

And in that case, you just basically add

21:58

in a modifier node is what we can do. call it

22:00

in the graph interface that you use

22:02

to configure synthetic data runs.

22:04

And you'd say, I want

22:06

my zebras, but I want them all to be blue.

22:09

And you kick off a job, you

22:11

generate 10,000 more images of blue

22:14

zebras standing around the grassy field.

22:16

And then you go and see if your detection

22:18

algorithm works or fails, as

22:21

you would expect it to,

22:22

against those images.

22:24

There's a ton of opportunity

22:26

to explore with

22:28

algorithm training and validation verification

22:32

once you have that initial

22:34

synthetic data channel set up

22:37

and you have the right pipeline in

22:39

place to assure

22:41

yourself that the synthetic data is

22:43

performing as if it was real data

22:46

during algorithm training. This

22:49

is a slightly off topic a little bit here,

22:51

but I get this idea of creating

22:54

these training sets and how we can

22:56

change and adapt and test and retest

22:59

and redo things. This makes a lot

23:01

of sense to me.

23:02

Is anybody showing up and saying, hey, I'd like to do

23:04

this to see what kind of sensor we should build? Here

23:06

is a scene that we're interested in capturing.

23:09

What is the best possible sensor? And then just adjusting

23:11

the

23:12

parameters of the sensor

23:14

inside the model and rerunning it and figuring

23:17

out how is our sensor going to perform over

23:19

the scene?

23:20

How can we tweak the sensor before we even

23:22

build it?

23:22

To collect the best possible data?

23:25

That's a fantastic question.

23:29

It's very close to one of the common use

23:31

cases for rendered AI today.

23:34

You might say that

23:36

people who build hardware today

23:38

do use a lot of simulation tools like

23:41

ANSYS tools, things like that, to try to

23:43

do engineering design and refinement.

23:46

Our

23:49

platform typically

23:51

comes into play

23:54

after that engineering design

23:56

step when somebody has settled

23:58

on a

23:59

sensor that they want to build

24:02

and then they want to generate a bunch

24:04

of simulated imagery from it

24:06

to prove that they can do AI training.

24:09

So you're hitting on

24:11

an

24:12

important distinction between

24:15

simulation for engineering design

24:18

versus simulation for AI training.

24:22

It's entirely possible down

24:24

the road that and those things are already

24:26

merging. There's been,

24:28

Autodesk did a lot of work with,

24:31

early work with generative AI. I'm

24:33

talking 10 years ago.

24:35

They actually built

24:37

cloud-based

24:39

products that would do things like optimize

24:42

the

24:43

shape and form and

24:46

volume of structural parts

24:48

in

24:50

chairs and race cars,

24:52

motorcycles,

24:53

so that you could

24:55

get out of the this

24:56

algorithmic pipeline

24:58

an optimized part that

25:01

is as light as possible, uses as little material

25:03

as possible, but is still as strong

25:06

as possible given your parameters.

25:09

That's a type of synthetic data.

25:12

We have not been used for that,

25:14

but we absolutely, I'd say

25:16

half of our customers are

25:18

using us

25:20

to prove out

25:23

algorithm training on future

25:26

systems that may not actually yet

25:28

be deployed in the field. That's

25:31

true for satellite systems, drone

25:33

systems, vehicle-mounted

25:35

systems. That idea

25:37

that I need data to innovate

25:39

is

25:40

definitely one of the key use cases

25:43

that we are used for today.

25:44

When we talk about physical

25:46

models, could you just explain this idea to me

25:49

please? Is it enough just to have the

25:51

extremely accurate dimensions of

25:53

the object that we're trying to model or

25:55

do we also need to know about the materials

25:58

being used? My thought process is here.

25:59

here is it's one thing, at least

26:02

in my mind, to identify an object. It's another

26:04

thing to identify to simulate

26:06

the reflectant level at different wavelengths.

26:09

So when we talk about these models, are we talking about

26:11

are we looking for an accurate shape or

26:13

are we also looking to understand the materials

26:15

used within the models? Great

26:18

question. So when a customer

26:21

is building a synthetic data channel in

26:23

rendered AI, they are trying to emulate

26:25

some kind of real sensor data, for one

26:27

thing, and then also real

26:30

world conditions for collection of that

26:32

sensor data. The sensor itself

26:34

may actually determine how much information

26:37

you really need on that 3D model.

26:39

So

26:40

if you are doing a nadir

26:43

satellite imagery of elephants

26:46

and zebras on a plane with

26:48

one meter resolution,

26:51

you probably don't need

26:53

the same kind of materials,

26:55

reflectance information

26:57

that you want if you're simulating

26:59

a SAR collection of that same scene.

27:04

It really ends up depending upon

27:06

the sensor domain and kind of the collection

27:08

regime that you're working on. There

27:11

are some really hard cases like

27:14

infrared. The way that infrared

27:17

bounces off of water depends greatly

27:19

on the wavelength of infrared

27:21

that you're trying to collect.

27:24

And if you are trying

27:26

to accurately simulate a scene

27:28

in rendered using water, using a

27:30

sensor simulating infrared,

27:35

then you're going to need to have scene

27:38

elements that have the right properties so that

27:40

you can simulate the right

27:42

physical effects. Obviously,

27:46

the RGB simple

27:48

lighting cases, of which there

27:50

are lots out there, they are the most

27:53

easy for us to address. Nathan

27:56

Kunst, the founder

27:59

of primary. founder of Rendered, he

28:01

is a physicist with a background in

28:03

general relativity and electric materials.

28:06

So he founded the company

28:08

under the premise that the

28:11

data that we simulate needs to be physically accurate.

28:14

Right. Thank you very much. Appreciate that.

28:17

Do you feel like the trend around reality

28:20

capture, digital twins, we

28:22

seem to use these terms everywhere now, not

28:24

just in terms of buildings, but in terms of scenes,

28:27

in terms of

28:28

much smaller objects as well,

28:30

or in entire cities.

28:32

Is this in any way accelerated, like

28:35

the ability to create these

28:37

synthetic datasets? My thinking

28:39

is here, the more accurate models we have in the world,

28:42

the more we can do this, this thing that

28:44

you're talking about, the more perhaps we can take these

28:46

similar models,

28:48

use similar techniques to capture reality

28:50

and put them into a system like yours.

28:53

There's opportunity there. I

28:55

say in many cases that that is

28:58

ahead of the market

29:00

today. And it's not that there aren't people

29:03

doing what you're saying. I think the

29:06

synthetic data market is kind of bimodal,

29:09

as most technology markets are when they're

29:11

starting out.

29:12

You have big, deep-pocketed

29:13

organizations

29:18

who have a lot of money

29:21

to invest in synthetic data development.

29:24

And there are those today like Waymo,

29:27

Tesla,

29:28

Apple, Amazon,

29:31

how's the synthetic data pipeline for training

29:33

drone data delivery, for example.

29:35

In that case, those

29:37

groups are using

29:40

sophisticated content

29:43

pipelines to kind of infuse content

29:45

into the simulation. And then they're also

29:48

using highly specific, highly tailored

29:51

simulation models that say

29:53

emulate the exact

29:56

parameters of the sensors that they're going to

29:58

use on their cars or trucks.

29:59

or drones or whatever it is.

30:01

And then they're also feeding that

30:03

data into a highly bespoke

30:06

AI training pipeline

30:09

because they are trying to accomplish some

30:11

really hard but

30:14

huge potential market opportunity problem

30:17

like self-driving cars or trucks

30:19

or

30:20

automated drone delivery packages, that

30:22

kind of thing.

30:23

And

30:25

those folks, I think, have

30:27

benefited tremendously from

30:30

at least concepts but also

30:33

implementation around reality capture

30:35

and kind of the not

30:39

just static but mostly static digital

30:41

twinning of the real world. In

30:44

the medium and longer tail of the

30:46

market,

30:47

one of the reasons for the founding

30:49

of Rendered because most companies

30:52

out there don't have the deep pockets of OAMO

30:54

or

30:55

Tesla or somebody like that. So

30:58

if I'm a 200

31:00

person

31:02

agricultural startup

31:04

that is well-funded,

31:07

has profitable business but needs

31:09

to get ahead by training

31:12

computer vision algorithms to

31:14

do things like detect

31:17

disease or damage to

31:20

crops or something like that or maybe optimize

31:23

pesticide spraying,

31:25

then I probably can't afford

31:27

the full stack of stuff that

31:29

OAMO or Tesla can afford.

31:32

And there's no real option out there for

31:34

me today. And that's where Rendered

31:36

comes into play because we hide a lot of that complexity.

31:39

In those cases, in that we

31:42

not only hide the complexity, we also help mitigate

31:44

some of the cost of setup and maintenance

31:47

of that cloud infrastructure to support data

31:49

set generation. What I'd

31:51

also say about that middle and longer

31:53

tail of the market is that

31:56

their needs are often less sophisticated

31:58

than the needs of

31:59

somebody trying to train self-driving

32:02

cars

32:03

to be much, much, much better safety-wise

32:05

than a human driver.

32:07

So you're in

32:09

orders of magnitude different,

32:12

higher complexity problem set in those

32:14

big

32:15

companies that have built their

32:17

own synthetic data capability.

32:20

Amazon, they don't want to deliver

32:22

packages to the wrong addresses. They

32:24

also

32:25

don't want to deliver those packages to the wrong place

32:27

at those addresses, and they don't want to drop packages on

32:29

people's heads or on people's cars. So the

32:32

problems that the big folks are solving

32:34

require much more sophistication around

32:36

digital twinning the real world and

32:39

making it a dynamic

32:41

simulation playground for training

32:43

AI than the middle and

32:45

longer tail of the market that

32:48

we are trying to serve. We're happy to

32:50

work with those folks, too, but

32:52

they've got tons of deep investment

32:54

in their specific problem set. What

32:57

we tend to see today as the state of the market is

33:00

a lot more people are just discovering computer

33:02

vision, trying to tackle basic computer

33:04

vision problems, and then they

33:07

run into issues where they either can't get data,

33:09

it's too expensive for them, it doesn't

33:11

have the right diversity of problems

33:14

in it. You can get lots of pictures of strawberries,

33:16

but can you get enough pictures of

33:19

strawberries with a particular kind

33:21

of blight or damage or insect or

33:23

something like that?

33:24

So we tend to be

33:27

in a space that is

33:29

somewhat less sophisticated digital

33:31

twin-wise. But

33:32

yeah, all the developments

33:35

from everybody, from Epic Games

33:37

to Esri, Bentley Systems, Hexagon,

33:40

Autodesk, all of those companies

33:42

are fueling

33:45

commoditization of capability

33:47

to capture the real world at many

33:49

different resolutions,

33:51

both in time and space.

33:53

All of that

33:55

interest is driving increased focus

33:57

on standards to be able

33:59

to... to make

34:01

all that data more usable across many

34:04

more use cases. And

34:06

so we're seeing the evolution of things like, there

34:08

was just announced this OpenUSD consortium

34:12

around

34:13

universal scene description that was originally started

34:16

by Pixar and Autodesk and

34:18

a few others.

34:19

And now they're, Nvidia and

34:21

others are really investing in

34:24

it as an open format for sharing 3D

34:26

content

34:27

around all kinds of applications from

34:29

games to simulations to, you know,

34:32

to you name it. So yeah, the digital

34:34

twin world has definitely influenced the ability

34:36

to generate

34:37

synthetic data kind of from a fundamental technology

34:39

perspective. I'd say if you were

34:42

actually to look at a

34:43

pipeline

34:44

developing or creating synthetic data today,

34:48

if you go into one of those big

34:50

shops that is kind of

34:52

DIYed their synthetic data pipeline,

34:55

you'd probably look at that and see that as much more

34:57

of a digital twin of their problem set

34:59

than you would many of our customers.

35:01

Right, so if

35:03

we get back to sort of your sweet

35:05

spot, your customers, we talked about,

35:08

it sounded like we're talking about almost discrete objects. We

35:10

talked about, as opposed to a scene

35:13

in urban landscape, we talked about strawberries,

35:16

we talked about zebras, that kind of thing.

35:19

Does this replace or does it augment

35:21

things like manual labelling and

35:24

ground truthing?

35:25

Before I go on to that, so

35:27

yes, many of our customers are focused

35:30

on discrete objects. At

35:32

that same time, we are helping them simulate

35:35

real sensor data capture. So that may mean

35:37

in some cases that we are building

35:40

out digital versions of real environments.

35:44

It is usually more sophisticated than just an

35:46

isolated object in space. In

35:49

terms of replacing manual

35:52

labelling or ground truthing, it

35:55

is not always a replacement

35:57

for it. broad

36:00

categories that I run into when

36:02

I listen to customers' needs. One

36:05

is customers who are looking for

36:08

warm hits. They have an idea

36:10

on training an algorithm.

36:14

They can describe kind of what they

36:16

want to look for. They may even have some 3D models

36:18

of what they're trying to look for or some other information.

36:21

But they just can't get enough data today,

36:24

manually

36:25

labeled or not.

36:27

They need more data

36:29

to supplement that process

36:31

of the algorithm training.

36:33

Many of our customers have been quite successful at jump-starting

36:36

in that way. Actually,

36:40

a couple of different partners and customers have shown

36:42

that

36:42

the optimum results can really

36:44

be achieved through a combination

36:47

of using some real,

36:50

manually-leveled imagery and then also

36:52

synthetic data

36:53

at the same time. I wouldn't

36:55

say it's a complete replacement. On

36:58

the other side, the other kind of group of

37:00

customers are customers that are looking to fine-tune

37:03

the precision of

37:06

some kind of established model. This happens in manufacturing

37:09

and a few other areas where they're constantly

37:12

looking at the same

37:13

objects, but they're trying to tune

37:16

models to

37:17

recognize really rare defects

37:19

or

37:21

defects is a common one or counting

37:24

things and need high precision.

37:26

For that, synthetic data alone is definitely

37:28

not going to get you

37:30

to that tuned performance.

37:33

Typically, what we see is that in

37:36

those cases, customers have to have

37:38

some

37:39

real, manually-labeled data. They

37:41

can then supplement it

37:43

with synthetic data.

37:46

We have one partner, for example, that

37:48

has demonstrated that they create

37:50

a type of synthetic data from just

37:52

simple reality capture technology

37:55

where they take images of real parts and then

37:58

image or generate a type of data. 3D model

38:00

of those, they were able to prove that

38:03

by simulating imagery

38:05

off of those kind of static reality

38:08

capture examples, they can actually better identify

38:11

defects. And then they worked with us to

38:13

actually use our platform to introduce more

38:16

variety into defects and they were able

38:18

to show

38:19

yet another performance jump

38:22

once they introduced synthetic data

38:24

with

38:25

stochastic variability.

38:27

So in reality, to

38:29

get to high precision use cases, improve

38:32

accuracy incrementally,

38:35

what you'll see is some combination

38:38

of real and different types of synthetic

38:40

data come into play. What

38:43

are some of the stranger use cases you've seen

38:45

for this? So

38:48

actually, most of the use cases that

38:50

we service have a lot of real

38:52

world impact. For example,

38:55

helping detect,

38:58

generating greater diversity in

39:01

damage to train parts

39:03

to improve algorithms

39:06

looking for, inspection algorithms looking

39:08

for defects in damage on

39:10

in-service trains. That has real impact

39:13

to their thousand derailments

39:15

a year in the US.

39:17

Quite a few military

39:19

and defense use cases where you're

39:21

looking for rare and unusual objects in the field

39:24

in different locations.

39:25

We've got a medical customer looking

39:28

at trying to increase diversity

39:30

around imagery to do

39:33

detection of a certain skin

39:36

condition through a mobile

39:39

app. So a lot of our

39:41

customers actually have real important,

39:43

real world

39:45

applications of computer vision.

39:47

We've talked to some amusing ones. One

39:50

of the more amusing ones was one of the robot

39:52

vacuum

39:53

makers. They

39:56

actually

39:56

have difficulty

39:58

getting enough imagery.

39:59

of dog poop

40:02

on living room floors, that kind of

40:04

thing. We did

40:06

a project for a defense

40:10

contractor and they were

40:13

using us to simulate a marine

40:15

imagery and apparently

40:17

in some parts of the world, dead

40:20

animals are commonly found

40:23

floating in the water, in the ocean,

40:26

and they wanted to be sure

40:28

to, you know, one of their concerns

40:30

was that they didn't want to be accidentally

40:32

detecting the dead animals as

40:35

they wanted to detect something else.

40:37

So

40:38

there's quite a lot of oddball

40:41

computer vision use cases.

40:43

We've all seen like images of say Tesla's

40:45

pipeline or something like that where

40:46

they're simulating ostriches running across

40:49

the street in

40:50

San Francisco or something like that.

40:53

You could actually point to the recent

40:56

failures around some of the

40:58

automated taxi in San

41:01

Francisco as somebody's failure

41:03

of imagination around their synthetic data pipeline

41:06

because

41:07

what was happening, I'm not sure if you saw the articles,

41:09

is that local locals

41:11

are getting frustrated over these taxis

41:14

and they've been able to disable them by

41:16

simply placing a traffic cone on the hood of

41:18

the car, this automated

41:20

car.

41:21

It's something that, yeah, the car

41:23

should raise a flag, should send a warning

41:26

to the system operators

41:28

if there's a traffic cone been placed on my

41:30

hood, but should

41:32

it really be impeding the full operation

41:34

of that car? Probably not

41:37

and there are some frightening implications of like

41:40

the car not working if other

41:42

things like that happen that might be more dangerous.

41:44

The options for if

41:47

you're just using real sensor

41:49

data,

41:50

then your imagination

41:53

around what you might need

41:55

to train for is going to be bound

41:57

by

41:58

what you think you can get in real life.

41:59

sensor data.

42:01

One of the challenges that we have

42:03

is sometimes

42:05

backing our customer up from

42:07

the point at which

42:09

they usually start, which is getting that real

42:11

sensor data, and instead getting them

42:13

to talk through

42:14

the full range of, and think

42:16

through the full range of

42:19

possibility around

42:21

what they really should be training

42:23

for to support whatever business outcome

42:26

they're looking for.

42:27

And so you can think of it as

42:30

almost a scientific method process where

42:34

what you really want your customer to do is describe

42:36

the ideal

42:38

data set or data sets that

42:40

they would need to train their algorithms.

42:43

And that's their starting hypothesis.

42:45

And then we can help them, in

42:48

RenderDI can help them generate

42:50

data to support that hypothesis and test

42:53

it.

42:53

What we almost always find is that

42:55

from there, the customer needs to back up

42:58

and say, well, you know what, our assumptions weren't right. We need

43:00

to change the data that we're generating

43:03

and generate more data. So it's

43:06

been really fascinating to see

43:08

how customers start to

43:10

change their mindset once they have

43:13

this ability to just generate an unlimited

43:15

amount of synthetic

43:16

computer vision content.

43:19

Yeah, like when you talked about that,

43:21

being able to disable an autonomous

43:24

vehicle by putting a traffic cone on the hood of it,

43:26

it

43:27

made me think, well, where

43:29

do you stop, for example. So in

43:31

that scenario, if we start heading

43:33

down that path, then anything's possible.

43:35

And it seems like an impossible task to train

43:38

for every

43:39

conceivable and inconceivable

43:41

situation. And this must be

43:44

a similar problem that your customers

43:46

are facing. Where do we stop?

43:48

If you can train for everything, should

43:50

you do it? Because, I mean, it's impossible.

43:52

But where do you draw the line? Yeah,

43:55

that's a deeper AI ethics type question.

43:58

And it's gonna, it's gonna depend depend

44:01

on the domain as well, right? If

44:03

you're building an AI to enable

44:05

some kind of game, you know, tabletop game

44:07

or computer game, that's different from building

44:10

a healthcare detection algorithm, which is different

44:13

from building, you know, a flight

44:16

safety mechanism and algorithm.

44:19

So it really is gonna depend

44:21

somewhat on the

44:23

use case. One, it might've

44:25

been at the OGC talk. I

44:28

think I described that really

44:30

what we're talking about doing today

44:32

is something that we've been doing for a long

44:34

time. Since the

44:36

dawn of the ledger, we've used

44:38

math to

44:40

support

44:41

decision-making. So, you know, even going

44:43

back hundreds of years,

44:45

a written ledger and some

44:47

basic math tools

44:50

were, you could think of those as decision

44:52

support tools. What we've

44:54

done in the last few years

44:57

is exponentially level up decision

45:00

support and move decision

45:02

support

45:03

closer to the end

45:05

decision-making,

45:08

like inflection point.

45:10

So we're now relying on, you know,

45:12

instead of humans looking at pixels

45:14

and counting zebras, we're now

45:16

able to increasingly rely on an algorithm

45:19

to count zebras and then

45:21

maybe even make some inference about

45:23

the

45:23

specific activity the zebras are engaged

45:26

in across a

45:28

whole bunch of images.

45:30

At the end of the day though, there's a human who

45:33

is probably making some

45:36

decision based upon that information that's

45:38

being extracted at that point.

45:40

A couple of years ago, or actually

45:43

still in practice today,

45:45

you know, earlier in that decision support tooling,

45:48

there would have also been human decisions

45:51

to say, yeah, I think these are zebras

45:53

standing around, these are zebras drinking, these

45:55

are zebras, you know, on the

45:57

hoof in a migration.

45:59

What we're talking about is gradually reducing

46:02

the

46:03

amount of human effort in all of that

46:06

inference so that we can

46:08

get to more decision making.

46:11

The hope is that

46:13

you're not just turning a bunch of

46:15

naive

46:16

decision making over to an

46:19

automated system.

46:22

And that's where the diversity of

46:24

training information is

46:27

really going to depend

46:29

upon the criticality of the domain. For

46:31

self-driving cars and medical applications,

46:34

in one sense there isn't an end,

46:37

but if you listen

46:42

to an interview from Elon Musk

46:44

a few years ago, he was talking

46:46

about trying to improve

46:49

on

46:50

vehicle safety

46:52

by I think it was like an order

46:54

of magnitude better than humans in

46:57

terms of avoiding accidents and

46:59

death and injury.

47:00

That

47:03

maybe is the, therein lies

47:05

the answer to your question really, which is if

47:07

you're going to really

47:11

move beyond decision support

47:13

and into decision making,

47:15

what is the threshold that you need to

47:17

feel that the algorithm

47:20

performs at that's better than

47:22

humans

47:23

to be able to trust it? From

47:26

there, you get into the question of, okay, well, what

47:28

is the data diversity, data set

47:30

diversity that's going to be required in order

47:32

to train the algorithm to that level? And

47:36

so that's a

47:38

long-winded way of saying it, but it's

47:40

a much more nuanced

47:42

answer than would

47:46

seem

47:47

because it really just depends

47:49

on the nature of the computer vision

47:51

application.

47:53

You tie it together so beautifully at the

47:56

end there that what are we trying to do? Are we

47:58

trying to move beyond decision?

47:59

decision support to decision making.

48:02

Okay, so what needs to happen?

48:03

At what level are we comfortable handing

48:06

over the reins kind of thing? I think that was a really

48:09

brilliant way of describing it.

48:11

And yeah, of course we ended up with, yeah, it depends

48:13

on the situation, which also makes a lot of sense. Sure.

48:17

So I'm a firm believer that nothing is for

48:19

everyone.

48:20

In what situations would people, would

48:22

you turn down customers, turn down work

48:24

and say, hey, I understand what you're trying to do, but

48:27

this synthetic data is not for

48:29

you. Who shouldn't be thinking about synthetic

48:31

data to

48:33

create data for their computer vision applications?

48:37

I guess one area that we definitely

48:40

would avoid is there have been

48:42

a few kind of human

48:45

monitoring detection

48:47

use cases brought to us that are

48:51

ethically questionable around things like

48:54

generating diversity of people

48:56

looking at different ethnicities for certain

48:59

types of decision making

49:00

that we just would stay away from because

49:03

we don't want to be involved with

49:05

the end outcome of that algorithm

49:08

training. There's probably

49:11

a few other cases like that, but the

49:13

other place that I'd say we would

49:16

probably avoid is

49:18

on that engineering optimization

49:21

kind of novel simulation side of

49:23

things. It's just a different problem set

49:25

from synthetic

49:26

data emulating real

49:28

world scenarios for training detection.

49:31

And there's

49:33

a lot of prior art that has already

49:36

been invested by Autodesk,

49:38

Dassault System, ANSYS and

49:40

others into that simulation

49:43

optimization for engineering and product

49:45

design kind of things. So we would stay

49:48

away from

49:49

that. It's just

49:51

so much computer vision that's being

49:54

applied everywhere. Everything

49:56

is being sensorized. Microelectronics

49:58

is a thing where you've got pins.

49:59

and head cameras being installed in light bulbs

50:02

above your head in retail stores

50:04

all around the world, things like that.

50:07

There's so much computer vision that

50:09

is going to be done in the future.

50:12

The opportunity for us to work with customers

50:15

is really endless. An

50:17

example is I was

50:19

at a conference last fall where

50:22

there

50:22

was a group talking about this kind of point of

50:24

sale,

50:26

human observation system to

50:29

make sure that people were being charged

50:31

for things they were carrying out of a store.

50:33

And they said they were using off-the-shelf trained

50:36

backbones

50:37

that often missed things like when a human

50:39

had a scarf around their neck or was wearing a ski

50:42

jacket of a particular type

50:44

or other things. And so you can

50:46

see that

50:47

in many cases, back to your other question, there

50:50

really isn't an end to the amount

50:52

of diversity that you need in

50:55

simulated or real sensor data to train

50:57

algorithms.

50:58

You have to call

51:01

the end to generating and using data

51:03

when you've satisfied the

51:05

business model objective or the

51:08

other detection

51:09

objectives that you need for that particular

51:12

training.

51:13

But in the case of the ski jackets and scarves,

51:15

you really wanna be able to detect that people

51:18

are

51:18

carrying chocolate bars out

51:21

of a store in the winter just like you do in the summer.

51:23

So that's a legit use case where rather

51:25

than standing up a bunch of people and taking pictures

51:27

of them in scarves and ski

51:30

jackets, you could actually simulate a lot of that

51:32

and train algorithms.

51:34

So it's

51:36

quite diverse. There are definitely cases

51:38

we would not touch, but

51:41

there is 99.9999% of stuff that we see is

51:47

well within the scope of what

51:50

synthetic data could be used for.

51:52

So given this sensorization

51:54

of the entire world and

51:57

cameras are a big part of this. talking

52:00

about your senses that are taking air

52:02

temperature or something like that, we're talking about your visual

52:05

senses, senses that are taking pictures, collecting

52:08

film, that kind of thing.

52:09

Given that, and my guess

52:12

is like you, that this is only going to increase dramatically

52:14

over time, what do you think the landscape

52:17

of synthetic data is going to look like in

52:19

the future? Let's say the next five years. How

52:22

are things going to change from the way they are today?

52:26

One quick distinction to call out is that there

52:29

are really two branches of synthetic data,

52:31

and we've focused on computer vision and imagery,

52:34

which also can touch video and lidar

52:36

and things like that.

52:37

That is where rendered operates.

52:40

There's a whole other branch of synthetic data

52:42

that is focused on

52:44

NLP text form

52:46

data, that kind of stuff. And

52:49

we don't really

52:50

play there today. There are other companies

52:53

that are more established in that space.

52:55

And it's actually, I'd

52:57

say that the barriers to entry

53:00

are lower than the computer vision side,

53:02

so it's been an area with more

53:04

entrance. The market

53:06

will grow.

53:08

There's a great analyst

53:11

in, I think, Germany named Elise

53:13

Deveaux, who actually tracks the synthetic data market.

53:16

And in her reports,

53:19

what you can see is that

53:21

the number of

53:22

companies out there offering synthetic data has

53:24

tripled in the last two years,

53:26

I think it's two years.

53:28

And what we're seeing is really

53:30

what I've seen multiple times in my career, which is the

53:32

start on

53:34

a market explosion

53:36

into what

53:38

will become an essential technology

53:40

component in this AI world

53:43

that we're all confronted with.

53:46

Over the next five years,

53:48

one of the things that is absolutely

53:50

going to change the way

53:53

that we

53:54

provide synthetic data

53:56

will be generative AI.

53:58

And back to... what I said earlier,

54:01

today generative

54:03

AI is not

54:05

ideal for generating whole image

54:07

chips.

54:08

But we can already see advancements

54:11

like text to 3D model and

54:13

kind of infinite texture development

54:16

out of generative AI and other things

54:18

that

54:18

can be used as part

54:20

of a

54:21

synthetic data, like the

54:24

simulation and scene assembly

54:26

part of synthetic data generation.

54:30

So it's entirely conceivable

54:32

that in, I don't know, five, ten

54:34

years, however long, that you

54:37

might be able to actually speak

54:39

the scenarios that you're trying to generate

54:42

and actually get

54:44

out of a rendered,

54:46

could get to the point where you could

54:48

get an auto-constructed synthetic

54:51

data channel and then start

54:53

generating data from some kind of text

54:56

to synthetic data experience.

54:59

We're not there yet today and there's

55:02

a lot more infrastructure

55:04

and scaffolding

55:06

to be built before we get there.

55:08

But that's one place

55:10

where the synthetic data,

55:12

computer vision synthetic data market is definitely going

55:14

to change.

55:15

There are already some synthetic

55:17

computer vision

55:19

data providers using generative

55:21

AI.

55:22

They tend to be in more fixed domains,

55:25

so say vehicle autonomy or

55:27

medical.

55:29

And that's because you need to have

55:31

a lot of data to inform the

55:33

generative algorithms to be

55:36

specific enough around the type of diversity

55:38

and the nature of imagery that you want to get out. And

55:41

we are a horizontal platform

55:43

and intentionally don't have that deep

55:46

specialty in just one domain.

55:48

That's one thing that's going to change.

55:50

It may seem like a tired old

55:53

theme, but it is amazing actually

55:56

to see what some of the open standards

55:58

activity of the data is.

55:59

it's converging these

56:01

days, I think

56:03

that Nadine and NOGC

56:06

have done a tremendous job over

56:09

the last few years at kind of changing the

56:11

persona of open standards to

56:13

be more like a shepherd than

56:16

a dictator. And that's

56:18

really opened up a tremendous amount of collaboration

56:21

around major companies

56:23

like Epic Games

56:25

and Nvidia

56:27

and others, and then smaller companies who are

56:29

also major in their own way like Cesium

56:31

and others. And so what

56:33

we're seeing is this really interesting convergence

56:36

of reality capture,

56:38

game technology,

56:39

rendering, and

56:42

even human modeling and all

56:44

kinds of other things that are

56:47

really creating the opportunity for

56:49

kind of an open playground of

56:52

capability on top of which

56:54

you could do all kinds of things, including build

56:56

synthetic data pipelines

56:58

and all kinds of things. So I think that open

57:00

standards move is very interesting.

57:03

I personally will be

57:05

really interested to see how USD

57:08

ends up somehow

57:11

blending with the web. Right now there's a huge

57:13

disconnect between streamable

57:16

3D on the web and what USD is,

57:18

in my understanding. I

57:20

think USD is also just at the

57:22

really early stages of being able to serve

57:24

geospatial content. For

57:27

any of your listeners who don't know what USD

57:29

is, it's kind of a dynamic cash

57:32

format that is now

57:34

kind of an open interchange

57:36

for 3D information

57:38

for, in some cases, extremely large scenes.

57:41

It was originally built for films which

57:43

can have

57:44

massive, massive gigabytes

57:46

to petabytes of 3D content and

57:49

2D content

57:50

used to render highly detailed

57:52

film scenes.

57:54

And over time, that has been

57:56

adapted to additional workflows

57:58

around gaming.

57:59

and simulation and other things.

58:02

Like I said, it's just started. I saw

58:05

the preliminary attempt at geospatial

58:08

in USD. I think that came out last year. It

58:10

was really early. That's

58:12

going to keep going. And then I'm

58:14

really interested to see how that influences

58:17

or merges with the web. You know, you've got a whole

58:19

set of different technologies with i3s,

58:22

which I was involved with, and then 3D tiles

58:25

through OGC for streaming

58:27

geospatial information on the web. So

58:30

that's a really deeply techy, nerdy thing,

58:32

but it's an interesting area that

58:34

will kind of bring things together down the road.

58:36

Wow, that

58:38

is a lot to think about. I'm going to have to go back and listen to this

58:41

last section of the podcast quite a few times, I think,

58:44

to really get a handle on everything that you just mentioned

58:46

there.

58:47

But I really appreciate

58:49

it. I think probably here is a great place to sort

58:51

of wrap up the conversation as well.

58:53

And I want to say that I really appreciate

58:55

your time. This is a complicated subject, and I

58:57

think you've done a great job of explaining it to me. I'm

59:00

hopeful and confident that the listeners also

59:02

have got a lot out of this, so I appreciate it. We've

59:05

mentioned the name of your company a few times,

59:07

but let's clarify it for the listeners

59:10

one more time. Where can people go if

59:12

they want to reach out to you, if they want to find out more

59:14

about the work that you're doing? Sure,

59:17

and Dan, thank you very much for the

59:19

opportunity to chat

59:21

with you. It's been fantastic.

59:23

I'm Chris Andrews. I

59:25

work with rendered.ai.

59:28

You can find us on the web at rendered.ai.

59:31

And we have a

59:34

trial that people can actually sign up

59:36

for right on our website. We have

59:38

quite a few videos and kind

59:40

of explanatory webinar,

59:43

recorded webinars about synthetic data. And

59:46

what users can really kind of

59:49

think about is essentially,

59:51

if you're relying on real

59:53

sensor data to train your algorithms, it's

59:55

not going to be affordable or possible

59:58

in many cases to to acquire

1:00:00

all the real sensor data that you would want to

1:00:03

build out your business case or business

1:00:05

model with AI.

1:00:06

And rendered AI

1:00:09

can be used to generate essentially fake,

1:00:11

fully labeled computer vision content

1:00:14

for training a variety of

1:00:16

computer vision algorithms. And in that

1:00:18

fake data, you can actually design in all

1:00:20

the diversity that

1:00:22

you might want to try to address the

1:00:24

business problems you're tackling.

1:00:26

And it's a fun area. It touches

1:00:28

everything from

1:00:30

messes on carpet to

1:00:33

satellite data collection to real

1:00:35

hardcore use cases of emergency

1:00:37

medical,

1:00:39

vehicle damage, all

1:00:41

kinds of stuff. So it's a great area

1:00:43

to be in and it really does apply broadly across

1:00:45

computer vision.

1:00:47

Thanks again, Chris. Really appreciate it. I

1:00:49

can see a lot of people trying this out.

1:00:52

It seems to me that it's almost impossible to imagine

1:00:55

a future without this when

1:00:57

we think about computer vision. So again,

1:00:59

really appreciate your time. Thanks very much.

1:01:01

All

1:01:02

right. Thank you, Dan.

1:01:06

I really hope you enjoyed that episode with Chris, COO

1:01:09

and head of product at Rendered AI. And

1:01:11

I hope this has helped you understand a little bit more about this idea

1:01:14

of synthetic data. I mentioned at the

1:01:16

start of the episode that

1:01:18

this is building on top of other podcasts

1:01:21

that we've published previously. And

1:01:23

there'll be links to those in the show notes today. A

1:01:26

couple that you might find particularly interesting

1:01:28

are the following. So

1:01:30

computer vision and geo AI. So

1:01:33

this is a comparison between the two. They're not exactly

1:01:35

the same thing. And it's probably

1:01:37

worth understanding this. There's an episode called

1:01:39

Labels Matter. So this dives into

1:01:42

the idea of annotating

1:01:44

images and how that happens today and the process

1:01:46

that's involved if we're doing it manually. And

1:01:49

some time ago I published an episode around fake

1:01:52

satellite imagery. And I think I mentioned

1:01:54

it in this episode that you've just listened to. So

1:01:57

this was the idea of using GANs.

1:02:00

So generative adversarial

1:02:02

networks to create fake

1:02:04

satellite imagery. So I think those two would

1:02:07

be a really good start, but if I find other episodes I

1:02:09

think that you might find interesting I'll include those in the

1:02:11

show notes, so please check them out. Again,

1:02:13

thank you to the Open Geospatial Consortium for helping

1:02:15

make this episode possible. Really

1:02:17

appreciate your support. And that's it for me. I'll

1:02:20

be back again next week. I hope that you'll take the time

1:02:22

to join me then.

1:02:30

Thanks for watching.