0:07

Welcome everyone to the AI

0:09

in Business podcast. I'm Matthew

0:12

D'Amelio, Senior Editor here at

0:14

Emerge Technology Research. Today's

0:16

guest is Steve Astorino, Director

0:18

of the Canada Lab and

0:20

VP of Development and Data

0:22

in AI at IBM. Steve

0:25

joins us on today's program to

0:27

talk about the biggest challenges for

0:29

enterprise leaders when it comes to

0:32

driving the infrastructure innovations necessary to

0:34

leverage new emerging AI use cases,

0:36

especially in new data-hungry, generative AI

0:39

tools. Without further ado, here's our

0:41

conversation. Steve,

0:49

thanks so much for being with us on the show today.

0:52

Thank you for having me. Especially, you

0:54

know, talking to someone who has

0:56

like a divisional control and we're

0:58

talking to someone from IBM, Canada.

1:01

I think this is a

1:03

particularly great perspective to have

1:05

on infrastructure challenges, especially

1:07

to know where they are in

1:10

the pipeline and what those specific

1:12

challenges are. So thank you so

1:14

much for being on the show.

1:16

We wanted to know what you

1:19

see as the biggest infrastructure challenges

1:21

when it comes to driving AI

1:23

capabilities, especially from a divisional

1:25

enterprise standpoint. Yeah,

1:27

sounds good. So we have, when

1:29

we look at where AI is

1:31

today, especially with, you know, foundation models,

1:34

large language models, the biggest

1:36

challenge that I see, and it's

1:38

kind of across the world, is

1:40

the availability of the GPUs that

1:42

are required, both for,

1:44

you know, faster training of these

1:46

models, as well as the

1:49

ability to actually execute and infer these

1:51

models when we're asking a question. I

1:53

think for most organizations, they want to

1:55

get their hands on it. That

1:57

is one of the biggest challenges that I see.

2:00

in the market in terms of

2:02

how do these clients get started and get

2:04

their hands on this infrastructure. There's

2:06

lots of different things that are trying to be done to

2:09

accelerate that, but the reality is we're probably going to be

2:11

in this situation for a little while. But

2:13

it's probably a good thing, I would say, as well,

2:16

because this technology has been moving

2:18

way too fast. I think everyone

2:21

can acknowledge that. And there are risks associated with

2:23

it, so we want to be careful. So

2:26

I would say GPUs, the availability of

2:28

that infrastructure needed for large language models,

2:30

is the number one, I would say,

2:32

challenge that we have. But

2:34

there are other things that are super critical,

2:37

in my opinion, to make sure that companies

2:40

and users of this technology are getting

2:43

into the right use of it. Having

2:45

the right tools is super important. What

2:48

I've seen is a lot of

2:50

businesses are overwhelmed, they're underprepared, they

2:52

want to get into this

2:55

AI space, but they don't know

2:57

how to do it exactly, they don't know how to

2:59

profit, but they're all having board-level

3:01

meetings that need to happen. The

3:03

business leaders are expecting this to happen.

3:05

So choosing the right

3:08

tools, I think, is also critical

3:10

in the success. And we've seen

3:12

a lot of mishaps in the market already, just

3:14

in the last six or 12 months,

3:16

I would say. Absolutely. It's

3:18

all moving so fast. It really feels like in

3:20

the last year, life has changed entirely in a

3:22

very much a paradigm shift. And

3:25

AI was here and in people's faces,

3:27

and a topic of conversation well before

3:29

that. But I think generative

3:32

AI on a cultural level is just way

3:34

bigger than even original AI for reasons we

3:36

can get to in another podcast. I really

3:38

want to pull apart what you were saying in

3:40

GPUs there for a moment. We did a big series

3:42

not too long ago on GPUs.

3:45

The audience is free to go back to

3:47

that episode, check it out for at least

3:49

the dynamic so we can get a little

3:51

deeper baseball here. Basically, as you describe it,

3:54

it's a good thing that on an entire

3:56

infrastructure level, there haven't been and there don't

3:58

seem to be what are called... step-level

4:00

improvements to the entire system. The

4:03

step-level improvement is basically like the

4:05

wheel, inventing the wheel. It makes a

4:08

lot of things a lot easier, you

4:10

know, or the cotton gin, if you

4:12

want a slightly better, slightly more recent

4:14

example. But it doesn't seem like there's

4:16

gonna be step improvements

4:18

for the recent future. Audience members who

4:20

tune into that series will remember our

4:22

guests talking about, well, when we do

4:24

have step-level improvements, we'll be able to

4:26

take data from every single cell phone

4:28

and do incredible things and yeah, taking

4:31

data from every single cell phone. Now,

4:33

I know we're already doing that on

4:35

some level, but even taking visual data,

4:37

that is a giant leap, not just

4:39

in the infrastructure and in the step-level

4:41

improvements, but also in the customers being

4:43

comfortable with their privacy. That, to your

4:45

credit and to your last answer, brings

4:47

up some ethical questions. So I'm

4:50

just wondering there, you know, also I'm curious

4:52

what makes you say, not that our big,

4:54

you know, message from that last series was

4:56

that there's all these step improvements coming. What

4:58

makes you say it's slowing down right

5:00

now? I get that sense too, but I'm

5:02

curious as to what makes you say. Do

5:05

you think it's just the cultural backlash of

5:07

AI? We all, everybody kind of agreeing, we

5:09

all need to slow down right now? Or

5:11

is there something a bit more in the

5:13

systems behind the scenes? Yeah, I think

5:15

there's a couple of things. I mean, so the

5:17

availability of the infrastructure is helping us slow it

5:20

down a bit or the inavailability of it. Then

5:22

I think, you know, it's almost

5:25

like in January everybody woke up and said,

5:27

oh AI is here. Well, to your

5:29

point, like you said, AI was here before. It's

5:31

just now we really went to

5:33

the next level to show its capability

5:36

to everyone rather than

5:38

just maybe data scientists, right? So

5:40

that has kind of, I

5:43

think everybody going through this year has

5:45

learned a lot more about it. That

5:47

has raised a lot of questions about

5:49

the reliability of it, about the security

5:51

of it, about all the wrong things

5:53

that can happen. There's a lot of

5:55

lawsuits happening right now and for

5:58

probably for the right reasons, right? We've kind

6:00

of dove in really, really fast. And

6:02

I think everyone is now to the point, okay, I

6:04

understand it a bit better. I know I need to

6:06

continue to get into it, but now I'm going to

6:09

step it back a little because I want

6:11

to make sure I do the right thing for my

6:13

company. Right. So I think that's the biggest thing. So

6:16

everyone is taking a little bit of a step back and

6:18

be more mature about the technology, if I can use those

6:20

words. Yes. Yes. Yeah. You can, you

6:22

can, you can bring maturity into the mix. No

6:24

one's no one's going to know. And from our

6:26

legal department is going to write in to me

6:28

with, with some angry messages. And I'll take that

6:30

hate mail. I'll take that hate mail. You also

6:33

mentioned the right tools. I love this conversation and

6:35

I want to put a finer point on this.

6:37

I'm seeing, and our audience

6:39

can go back in the last few episodes

6:41

that it's been with the, especially with the

6:44

explosion of generative AI beforehand,

6:47

the C-suite management, not everybody, of course,

6:49

but let's, let's call worst case offenders

6:51

or worst case scenarios, it was like,

6:53

you want to do what with AI?

6:56

You know, they hadn't heard of it yet. You know, what

6:58

is this machine learning? And then they have to

7:00

go find it with the explosion of generative

7:02

AI. It seems like, especially with, with the

7:04

media coverage and the notice, now it's like,

7:07

oh, you have C-suites, their kids are using

7:09

it, and now they're coming back down to

7:11

the data science and saying, why can't we

7:13

use this gen AI stuff to do whatever

7:15

we want? And what's being

7:17

lost in that mix is that the

7:19

first generation capabilities of AI, just the

7:21

plain old machine learning, the predictive analytics,

7:24

they're not even being leveraged well enough

7:26

for specific workflows where they can really

7:28

help in enterprises. I'm wondering

7:30

if that's what you mean there, but

7:32

by the right tools, maybe something a bit

7:35

deeper, but that's, that's what, what I'm hearing

7:37

for the conversations we have on the show

7:39

in terms of, you know, enterprises barking up

7:41

the wrong tree, not really thinking about what's

7:43

the right hammer to nail this nail. Yeah,

7:46

I think so. Yes. And what I

7:48

would say is what we know about

7:50

this technology, there's still a lot of

7:52

the answers are inaccurate. The, you know,

7:54

it's not scalable, it's not adaptable. There's

7:56

a lot of risk with it. Right.

8:00

There's a hallucination where you get the wrong

8:02

answer altogether. So I think

8:04

all of that has been helping

8:06

slow us down. But yeah, so I

8:08

think line of business, you

8:10

know, C-suite, they've been looking at this,

8:13

give me this fast, but I

8:15

think there's a learning curve that's been happening. And

8:18

I think it's the right thing for all of

8:20

us to understand what's there, what it

8:22

can do, what are the risks associated with

8:24

it. And I think we're

8:26

in the, probably I would say in the

8:28

right place, we should leverage this technology because

8:31

I believe it's going to transform kind

8:33

of everything that we do on

8:35

our daily lives in the good. And

8:38

we have to be careful for the bad because there

8:40

are a lot of risks with it. The

8:42

other thing I would say is the skills are

8:44

not there. I, you know, I applaud the market.

8:46

Everyone is becoming more and more knowledgeable

8:49

of the technology, trying to understand it, what it can

8:51

do, how it works. I think that's all goodness. And

8:53

by right now, there is a lack of skill and

8:56

we need to continue to build it. And

8:59

the other thing I would say, look,

9:01

at the end of the day, this is

9:03

all about data. And every conversation I have

9:05

with clients, no matter where we

9:07

start, we always end up with data. And

9:09

there's a lot of things associated with that

9:11

around data security, privacy, what's got done with

9:13

that data, the type of answer, who owns

9:15

it and all of that stuff. It's, there's

9:17

a lot of complexity around this,

9:20

where we need very strong regulations. But, you

9:22

know, there are a lot of things that

9:24

kind of get into this mix on how

9:26

we do this successfully. Very, very

9:28

interesting stuff. And I think you're bringing up a

9:30

lot of, a lot of really incredible points for

9:32

the solutions portion of our show. I'd like to

9:35

break these down because I think you're bringing up

9:37

very distinct problems. And even a few

9:39

of them, I just don't think that you can,

9:41

you can talk about them. Hey, how are, how

9:43

is everybody using solutions? Let me go to GPUs

9:45

for a second. It almost sounds like you actually

9:47

kind of think this, you know, this problem is

9:50

a good thing that it is slowing down. It

9:52

might not need a solution. Maybe the question then

9:54

for solutions is how should we go about talking

9:57

about being more

9:59

proactive? about when

10:01

that next step level improvement

10:03

comes, implementing it safely and

10:05

ethically? Yeah, I'm going to

10:08

tie it back to the comments I made before about

10:10

the right tools. So right

10:13

now, there's a lot of every

10:15

company probably in the world is coming up, oh, I

10:17

have an AI tool, I

10:19

have a Gen AI tool. But

10:21

the reality is that at

10:24

the enterprise level, our clients

10:26

expect the ability to, they either

10:28

want to build their own models, or they want to

10:31

fine tune models that exist out there, or they want

10:33

to just do prompt engineering or prompt tuning. But

10:35

that's just a kind of a small aspect of

10:37

it, right? Once you get past that, then

10:40

you really need to be able to want

10:42

foster innovation, but also in a secure way.

10:44

I think that's the biggest challenge right now. And

10:48

the problem is the guardrails don't exist,

10:50

both from legislation and regulations, as well

10:53

as kind of the tools themselves. So

10:55

it's very easy to go off a cliff

10:57

and get in trouble. And we're seeing that

11:00

we're seeing that in the market. So

11:02

to me, tying it

11:04

back to having the right tool is the number

11:06

one thing. I can share

11:09

what we're doing. And we're very proud of

11:11

what we're doing at IBM in terms of

11:13

how we have been taking our technology to market.

11:16

But it's really the ability to manage and

11:18

to end the entire lifecycle in a secure

11:21

risk-free and compliant way. I

11:24

said a lot in that one sentence, but at

11:26

the end of the day, we want to make sure our clients are

11:29

able to get the

11:31

innovation out, disrupt their own business in

11:33

a positive way, and then do it

11:35

in a safe way. Yeah, that

11:37

makes a lot of sense. And I think that also

11:40

narrowed down my question on the right

11:42

tools, like how do we think of

11:44

those solutions? I think we got a

11:46

good there. I want to talk about

11:48

what you just brought up in the

11:50

skills gap, especially what you find are

11:52

good approaches, at least maybe for hiring

11:54

or at least training in-house to develop

11:56

this know-how for enterprises. Yeah, we've

11:59

been working through this. challenge also with

12:01

data science and machine learning, right?

12:03

You go back five, six years,

12:05

most companies did not even have

12:07

data scientists. So there's

12:09

multiple approaches to this. We with an IBM

12:11

have done a ton of

12:13

an enablement and education and courses and

12:16

leverage our research team. Our research team has been

12:18

at the forefront of this technology

12:20

for many years. We didn't just wake up in

12:23

April or May and we said, oh, we

12:25

have Gen AI too. We've been working at

12:27

this for five, six years. And we

12:30

knew exactly what the capability was, also

12:32

what the dangers were. And that's important

12:34

to us. But on the skill side,

12:37

we've been leveraging that team to enable

12:39

and to train and we have dedicated

12:41

courses specifically for it. We're

12:43

doing a lot of hands-on challenges within

12:45

the company. Our brand is What's the

12:48

Next, as you can see on my

12:50

shirt here. But we are What's

12:53

the Next challenges where we're getting the

12:55

entire company to use the technology, to learn

12:57

about the technology, to see how we can innovate

12:59

with the technology. And this is a safe zone

13:01

where we can do that. And we can see

13:03

what the art of the possible that then we

13:05

can pass on to our clients. We've

13:07

been working with academia to be able to

13:10

strengthen these programs. And we've gone from machine

13:12

learning now to Gen AI as well. The

13:15

reality with academia is that they're a

13:17

little slower than probably everyone else in

13:19

the market. And it takes time to

13:22

kind of modify the courses that they have

13:24

and the programs. But we're working with them

13:26

to accelerate as much as we can. And

13:28

there are also a lot of other external

13:31

companies that are doing this education. So I

13:33

think it's an industry-wide task

13:36

that we all have to improve the skills,

13:38

increase the skills in the market. And I

13:40

think we all got to collaborate on that.

13:43

It doesn't matter which company is doing what,

13:45

but I think it's important for everybody. Absolutely.

13:47

Absolutely. I want to give some time to

13:49

the problem of data that you had brought

13:51

up before as well. But we could do

13:53

a whole episode on data, data

13:55

problems in the enterprise. We could do probably three

13:57

or four episodes on data problems in the enterprise.

14:00

enterprise. Let me know. Do you want to

14:02

know what is data mesh versus data fabric?

14:04

Hit me with that answer, but where I

14:06

was going to go with this is what

14:08

do you find is the biggest problem, biggest

14:10

conversation with data that you're having with enterprise

14:13

leaders? If you could narrow it down to

14:15

one. Data mesh in that difference, totally

14:17

on the table if you feel that's the case. I'll

14:20

talk about the biggest challenges really. There's

14:22

actually two. There are two that I

14:24

think are super critical. One is the

14:26

data needs to be clean, especially for

14:28

foundation models just because whatever answer you're

14:31

going to give is going to be

14:33

based on that data. Maybe it's

14:35

more than two, maybe it's three, but the ability

14:37

to access the data is still

14:39

a challenge in probably I

14:42

would say most of

14:44

not all organizations. The larger the

14:46

organization, the bigger the challenge is.

14:49

The way AI works is the more

14:51

data that is clean and accurate you

14:53

have, the better your model and

14:56

your answers that you get from those models

14:58

will be. Then the

15:00

other one is privacy around our data. It's

15:02

twofold. It's who can access the data, but

15:05

also if you look at the models, we

15:08

can put governance and controls around the data

15:10

on who can access it, but once you train

15:13

a model, then that model is a generic model

15:15

that users can access.

15:17

Let's say that you're a more senior person

15:20

in an organization and I'm a less senior

15:22

person and I want to be able to

15:24

ask a question, we're going to

15:26

get the same answer. The model doesn't actually

15:28

know that yet. When you think about where

15:30

we need to go in the future around

15:33

this type of capability, we should be able

15:35

to provide access control or role control granularity

15:37

so that I can get potentially a different

15:39

answer that you can based on what privileges

15:42

I should have. When we talk

15:44

about the data itself and privacy and

15:47

all of that, it's super critical that

15:49

we have full governance around that, not only

15:51

on the data itself, but also on the

15:53

models. When the models

15:56

get called, are we able to provide

15:58

a safe and right answer? are based on

16:00

not only the data itself, but also the

16:02

privileges of that data and that model. You're

16:05

making a lot of sense to me, and I think you're

16:07

making a lot of sense to the audience. I think

16:10

these are new ways of looking at the problem.

16:12

And very excited in our next episode to really

16:14

dive into AI ethics with you. So we'll have

16:16

to have the audience tune back in for that.

16:18

Steve, thank you so much for joining us today.

16:21

It was my pleasure being here, thank you. Before

16:34

we wrap up today's show, and

16:37

I'll give a full disclosure

16:39

here, while we often record

16:42

two-part episodes separately and

16:44

over a fair amount of week's time,

16:46

I'm actually recording this outroduction after we've

16:48

recorded our second episode with Steve. And

16:51

I would like to plug that episode, even though I'm

16:53

not quite sure when it's going to publish, and

16:56

just encourage our audience to stay tuned,

16:58

especially given Steve's last answer in this

17:00

first episode, because we're going to

17:03

focus a lot more on AI ethics

17:05

in our second episode. And

17:08

in many ways, especially for those of

17:10

you who remember my conversation with Scott

17:12

Zoldy, Chief Analytics Officer at FICO

17:14

from earlier this year, you know, and by

17:16

this year I mean 2023, you

17:19

know that many AI

17:21

ethics problems as they occur

17:23

at the end of the

17:26

pipeline are PR disasters. That

17:28

impugn the morality and

17:31

the characters of the leadership at

17:33

the brand in question. And

17:36

often these products begin as

17:38

simple, non-biased, no

17:41

social issues involved, but simple

17:44

data governance problems, just at

17:46

least where the problems

17:48

are originating. And

17:51

Steve goes into a lot of really great

17:53

depth on how that works, how the mistakes

17:55

are made, how these mistakes

17:57

happen, often with all the good intent.

18:00

the world or at least intend to cut costs and

18:02

make processes faster. But

18:05

as Steve emphasizes both in this

18:07

episode and in the coming episode,

18:10

there's many ways that just cutting

18:12

corners is not worth it. And

18:15

that is sort of the underlying

18:17

conversation underneath both conversations about data

18:19

governance and conversations about AI ethics,

18:22

which very often for that Venn

18:24

diagram, they're almost perfect circles. I

18:26

find the more we talk about

18:29

these topics throughout the show. On

18:32

behalf of Daniel and the entire team here in

18:34

Emerge, thanks so much for joining us today and

18:36

we'll catch you next time on the AI in

18:39

business podcast.