0:01

It's almost compulsory, not even a necessity,

0:04

because when you start working,

0:07

that's my work experience, when you start

0:09

just trying to solve a problem, you realize

0:11

that some things tend to happen not

0:14

once, but twice, thrice, and then

0:17

it becomes not a

0:19

daily occurrence, but it's something that happens almost

0:21

all the time. It keeps happening on the intervals

0:23

and stuff like that. So then you realize that you

0:25

cannot keep doing things by hand because then

0:27

you have the problem that is error prone,

0:31

and

0:31

second, it becomes boring. And

0:33

one thing that happens to people when they get bored is that they

0:36

make mistakes. Me in particular,

0:38

I'm really good at making mistakes when something

0:40

is actually boring me to death. So

0:43

one of the things that I try to do is just to

0:46

automate it if it's possible, and

0:49

also to document or make it easy

0:51

to use. That's Jose

0:53

Vicente Nunez, and if

0:55

he sounds familiar, it's because he was

0:57

a guest on our episode about sysadmins.

1:00

He shared something with us

1:01

in that episode that stuck with me. Sometimes

1:05

when trying to automate a repetitive task in

1:07

order to save time, you end up spending

1:09

way more time than you expected to. I

1:11

thought that was pretty interesting, and I wanted to dig into

1:13

that concept a little bit more. When

1:16

does it make sense to automate? When

1:18

should you avoid it? And most importantly,

1:20

what can you gain from automation?

1:27

This is Compiler, an original

1:29

podcast from Red Hat. We're

1:31

your hosts. I'm Brent Simino.

1:33

And I'm Angela Andrews. We

1:35

go beyond the buzzwords and the jargon

1:37

to simplify tech topics. Today,

1:40

we hear about adventures in

1:42

automation.

1:47

In case you missed our episode on sysadmins,

1:50

Jose Vicente Nunez is a DevOps

1:53

engineer for a fintech company.

1:55

He has years of experience in the field, and automation

1:57

is a big part of his job.

1:59

As I mentioned earlier, automation is something to

2:02

be embraced, especially with repetitive

2:04

tasks.

2:05

It saves time, provides

2:06

consistency, and it

2:08

preserves your sanity. But even though

2:11

it's something he does on a regular basis for

2:13

systems he works with every day,

2:15

he can still find himself lost in the puzzle

2:18

of automation. In my head,

2:20

I thought that I understood how it worked, then

2:22

you start testing and playing with that, and then you start

2:24

finding a small box or

2:26

unexpected behavior, and then you have to

2:29

spend time trying to figure out why it's not working

2:31

the way you expected. By the time you realize

2:33

that, or you figured out what was wrong,

2:36

you spend X amount of time just

2:38

testing or debugging something that was supposed to

2:40

be straightforward.

2:42

That is so true. It

2:45

is a rabbit hole, okay? When

2:48

you start trying to automate something, again,

2:51

that you think it'll be, oh, this is simple

2:53

and we'll figure this out, the time

2:55

you spent

2:56

rerunning and reapplying

2:58

the automation and then

3:00

rolling it back and seeing if that worked and

3:02

watching the error message, it's a whole

3:05

thing.

3:05

It

3:08

was supposed to be straightforward, but-

3:10

It's like opening up a time warp, you

3:13

know, where you're just like- Where'd the day go?

3:16

I feel them on that one. Yeah, and I assume

3:19

it can be frustrating too, because if it's something

3:21

that you do over and over again, you

3:23

know the process, right? And so you think, oh,

3:25

I can just write a script or

3:28

write a module or whatever, I'll

3:30

just

3:31

make sure all the commands are the same, and then

3:33

you run it and it's not what

3:35

you were expecting. Something goes wrong and you're like, what

3:38

is going on? I think this is something

3:40

we can all relate to, right, Johan? Yeah,

3:44

I don't do all that much automation, but

3:47

in the past I've focused on an interesting

3:49

problem and lost track of time.

3:52

But if time saving

3:54

were the only benefit for automation, then,

3:56

you know, can sometimes be

3:58

a dubious proposition.

3:59

What time? What

4:02

time are you saving exactly? Who

4:04

needs to keep track of time, right? But

4:06

when you have a process that you need to roll

4:09

out for multiple machines, the

4:11

idea of writing a script becomes

4:14

much more attractive. Jose told me

4:16

why it's important to take your time, even

4:18

when automating something like patch

4:20

updates for the switches on his company's

4:23

network, because a mistake

4:25

could mean costly downtime. So

4:28

it's a combination. Like say that I'm just fixing

4:30

one switch with a handful of servers

4:32

connected and they're used for test or

4:34

QA. It's an acceptable

4:37

risk. You are going

4:39

to modify or update 500 switches

4:42

where you have a bunch of servers connected to them. Then

4:45

actually we're talking about they cost the same, but

4:47

you know, the

4:50

consequences are dire. If actually you have

4:52

half of those 500 switches failing just because

4:54

the patch wasn't good. Same

4:57

for hardware components. You're

4:59

fixing probably one GPU card on

5:01

a machine. Okay, yes,

5:03

they're expensive, but maybe you can afford

5:05

to be down while you unroll back the change

5:08

or figure out what went wrong with the update. If

5:11

you're talking about maybe an FPGA card that

5:13

costs like a $20,000 and the server is

5:15

used for low latency trading, then

5:17

that's when you start thinking, you know what? Maybe it's not

5:19

such a good idea.

5:23

Oh, I'm sorry.

5:25

It's, this is funny to hear because

5:28

we've all been there. We're doing

5:30

our best. We're testing it on the smaller,

5:33

you know, QA test systems.

5:35

And we're thinking, oh, okay, this

5:38

works. This looks good. Everything passed

5:40

all the tests, but then you

5:43

went and deployed it to a thousand

5:45

systems. And you know, you just

5:47

watch those lights light up

5:49

on the monitoring board of all these systems

5:52

going down off of the network. And you're just sitting

5:54

here going like,

5:54

oh. Oh

5:57

my God. Oh no. Where's

6:00

the undo button? Right. Right. Right.

6:03

Right. So what does it mean to

6:05

take your time, right? You talked about

6:07

testing your scripts, adjusting

6:09

them, and then testing them again

6:12

and again until you're confident beyond

6:14

a reasonable doubt that the update's

6:16

going to roll out the way that you want it to, right? That it's

6:18

going to be smooth. That

6:20

way, you're not going to break a large

6:22

chunk of your hardware, and you're going to keep your

6:24

infrastructure running. And for that really expensive

6:27

piece of hardware, that FPGA card

6:29

that's like $20,000, sometimes it's OK

6:31

to do things by hand. Now, another

6:34

way to minimize things going wrong is to

6:36

keep the scripts simple and

6:38

have them do one single

6:40

job. And that's not always as easy

6:42

as it seems, because it can be tempting to add

6:44

functionality to a script. And there can

6:47

also be pressure from other teams to add functions

6:49

as well.

6:50

Now, over the years and over his career, Jose has

6:52

had to learn an important skill. I

6:55

learned how to say no. So

6:57

it's like, can you add this feature to put

7:00

this data into the database? No. Can

7:02

you add this feature to launch a Docker

7:04

container? And no. What

7:08

we can do, we can do this. But you don't

7:10

need to change the tool. Keep the tool simple. And

7:12

anything that you want to put on top of that, we can

7:14

do it. All right, let's all say it together. No.

7:17

No. No.

7:18

No, it's a complete sentence.

7:21

It gets your point across. And

7:24

it is OK to lead with no.

7:26

Because not every request

7:29

that you get requires

7:31

your attention, or not your immediate

7:34

attention. So it is OK to say

7:36

no. Learn how

7:38

to say no and be comfortable in it. Because

7:41

it may save you in the long run.

7:43

Because you can always come back and say yes. It's

7:45

hard to come back from saying yes

7:47

to a no.

7:48

So think

7:50

about that. Yeah, if you start with that simple

7:52

script and you start adding things to it, it's

7:55

harder to remove those features than it is

7:57

to add them later on. That's

7:59

a great point.

7:59

So we're talking about automation

8:02

and there's this famous automation

8:04

instructor. And one of the words, this

8:06

phrase he always says is, keep

8:09

it simple, complexity kills

8:11

productivity. And I

8:13

believe that

8:14

wholeheartedly. You want

8:16

to do things in modular,

8:19

small, little piecemeal things.

8:22

You don't want to go messing up

8:24

a whole lot at once, because it's really

8:26

harder to undo something

8:28

that is complex. And it's really harder

8:30

for someone else to come behind you and

8:32

understand what you did when it's so complex.

8:35

So, yeah, I'm

8:37

all for putting the brakes on.

8:40

Well,

8:40

it sounds like

8:43

there's a lot of benefit to simplicity

8:45

here, right? I think that there, you know,

8:47

it's not

8:48

because you don't want to sort

8:50

of, it's not that you're lazy and don't want

8:53

to do more work, right? It's

8:55

because it has all these like really great

8:57

effects. Yeah, and keeping it simple

9:00

makes it easier for the thing to just work and just to keep

9:03

working. And, you know, you're minimizing

9:06

the surface area for problems to occur at

9:08

an hour later on. Saying no is

9:10

really great for keeping it simple. And those

9:13

simple scripts, well, they can

9:15

last a lot longer than you might

9:17

imagine. You will surprise

9:18

me. I've seen pieces of code that

9:20

are like 20 years old and they're

9:22

still kicking pearls, especially. Oh my

9:25

God, these guys, I'm telling you, if

9:27

there is a catastrophe like an asteroid

9:29

hits the earth or something like that, you will see the roaches

9:31

and the pearl scripts taking over. Probably

9:35

those will be the only two things that will survive because

9:38

they're so

9:38

resilient. That was a perfect visual.

9:42

Pearl will never die.

9:44

Neither will cockroaches. Like that has

9:47

already been established. How funny

9:49

is that? That's great. I love

9:51

it. I love talking to

9:53

Jose. But what is it about these scripts that like keeps

9:56

them around for so long?

9:58

is because not

10:00

a lot of people know Pearl anymore. And

10:03

if they work, you don't touch them.

10:06

They're doing whatever task they were

10:08

designed to do, and they do them

10:10

very well. That's the real reason.

10:13

No one knows Pearl anymore. It is

10:15

such an old scripting language, but it's very

10:17

solid. Whatever it's been tasked

10:19

to do, oh, it does it and does it well,

10:22

seriously. So that's my one

10:24

big

10:24

reason. Yeah, yeah, that's essentially

10:27

what he told me, is that he said he was

10:29

looking back over some old code, and he found

10:31

some few scripts that were still

10:33

kicking around in production that he wrote years ago,

10:35

and they still worked because they do

10:38

the one thing that he mentioned that they should

10:40

be doing, right? They're just focused on a singular

10:42

thing. They do one thing. They do one thing.

10:45

They're very simple, right? They don't do that

10:47

one thing in a complex way, and

10:50

no one is allowed to touch them or add anything

10:52

to them or change them in any way. And

10:54

so because software doesn't really decay,

10:58

if a thing works and it's

11:00

a thing that you need to have running for years and years,

11:03

just don't touch it unless you

11:05

absolutely have to, and it might

11:07

outlive you. Might

11:10

outlive us all. Except

11:12

for the cockroaches. Eventually

11:14

they'll become sentient and take

11:16

over those Pearl scripts and change them for their own needs.

11:19

And we won't be

11:21

here to see it, so that sucks. We won't, yeah.

11:24

I don't think I want to be here to see that.

11:26

Yeah.

11:28

So next we spoke to Patrick

11:30

Arminio. He's a developer

11:33

advocate at PolygraphQL, and

11:35

for him, automation is about

11:37

leaning in to a self-applied stereotype

11:40

about developers. Yeah,

11:42

I guess I'm a bit lazy in terms

11:44

of, I guess most programmers are lazy. So

11:48

I really, well,

11:49

other than being lazy, I'm also,

11:51

I don't intend to procrastinate a lot.

11:54

And especially like tiny tasks, because I say I can

11:56

do those anytime. And

11:58

I realized that after a while,

11:59

especially for some of the automation, I guess we

12:02

talk about it in server. There were a few

12:04

things that always like ended up putting off

12:06

like weeks or even months

12:08

at the time sometimes. And I noticed

12:11

that I was gonna automate this, so I don't have to think

12:13

about them too much. Now, Jose

12:15

said something similar

12:17

about programmers being lazy, but

12:19

he kind of caught himself and rephrased it in a way that

12:22

I think is much more accurate.

12:24

It's not necessarily about laziness, it's

12:26

about being efficient with your time, right?

12:29

If you can automate a thing, something that you do

12:32

on a repetitive basis, you can avoid

12:34

spending all that time doing the thing over and over

12:36

again, and spend that time focusing on

12:38

something else instead. And writing

12:41

that script can be an interesting challenge in

12:43

itself. It's definitely more

12:45

fun. I mean, seeing something that

12:47

works without having you to do anything,

12:50

it's very nice. I mean, even if it takes more

12:53

time to actually automate it than actually do

12:55

the task, I think it's very fun. And

12:57

sometimes it's actually a

12:59

good way to learn new skills,

13:01

because you can say, oh, I'm gonna try this new library or

13:03

gonna try this new language. He's right

13:05

about having that

13:08

moment where you've automated something

13:10

that is so cool, and it just makes

13:12

things so much easier for you. It

13:15

is an amazing feeling. And when

13:17

it works,

13:19

it's an amazing feeling.

13:21

And that's just one less thing

13:23

you have to do.

13:25

So you can go off and learn new

13:27

things and try new things, but this

13:29

one thing that may have plagued you or been

13:32

in your back pocket and you kind of put it off

13:34

and put it off, once you've finally done

13:36

it and got all the bugs out and it works like

13:39

it was supposed to work, that's pretty

13:41

clutch. I've had that

13:43

feeling and it is a really good feeling.

13:45

On top of that, you get to learn something new as

13:47

well. And how often do we make

13:49

that point on this show that it's super important to

13:52

just keep learning things in this industry,

13:54

right?

13:56

This is one way to do that while also being

13:58

very productive.

13:59

even if the time spent doesn't initially

14:02

look like a good trade off, right? You spend a lot of time

14:04

automating and you're not sure

14:06

if you're saving all that much time on the

14:08

backend, but you're learning something

14:10

new, you're getting something that's consistent and

14:13

you're

14:14

putting in that investment for time saved later

14:17

on.

14:18

So let's hear a little bit about what Patrick

14:20

was being allegedly lazy about.

14:23

He's a maintainer for an open source project that's

14:25

pretty popular. And that means

14:27

people are submitting pull requests.

14:29

That also means that there's some regular work for

14:31

the release process once those pull requests

14:34

get merged. Yeah, I think the

14:36

fact that it wasn't automated, I would tend to kind

14:38

of postpone it a lot of times, which also

14:41

means I would

14:42

forget how it works. So I would need

14:44

to read the docs. Because it's something, I mean,

14:47

releasing a package is not

14:49

something I used to do often in

14:51

the past. So I would need to check what are the

14:53

commands I need to run? What do I need to check to

14:55

make sure that everything works?

14:58

So it's a process that

14:59

wouldn't take a huge amount of time,

15:01

a little more than 10 minutes each time

15:03

and a few times a week.

15:06

But he had to often go

15:08

back and

15:09

relearn how to do it, right? Which

15:12

would probably increase the amount of time every

15:14

time. And not getting to

15:16

it would cause delays, not just for him,

15:18

but for his whole team.

15:20

He was a bottleneck and that didn't feel right. So

15:24

he decided he'd had enough and he started

15:26

to automate the process. And now with

15:28

the automation, it's much smoother. And

15:31

if he's unable to do it himself, or if

15:33

he wants to take a break, someone else

15:36

can step in and do it. That seems pretty

15:38

great, right? Yeah, it does. That's

15:40

how it started anyways. Okay.

15:43

But maybe Patrick and Jose need to have a

15:45

little bit of a chat about scope creep. The

15:48

initial kind of scope was to automate

15:50

the release of

15:53

this library. And then we added the invitation, we

15:57

added the tweets, release cards, and adding.

16:00

message to basically compile

16:02

the form to get a sticker. Yeah,

16:04

so we definitely went, well I definitely

16:07

went overboard and added new features because

16:09

I don't know it's fun to automate things

16:11

and once you start then

16:14

it's hard to stop by. I think

16:16

the only thing that's that kept me from adding

16:18

more features is just the fact that at the moment

16:21

it's like a lot of scripts, GitHub

16:23

actions and it's a bit hard to maintain.

16:25

It's also not easy to test because

16:28

you have to push

16:29

on GitHub and wait for everything

16:32

to run to test it. So

16:35

he's saying that he started wanting

16:38

to automate the release of a

16:40

library, right? And

16:42

then he was just like, oh if I can automate

16:44

that I can also add a tweet

16:47

about it, right? I can also add in a release

16:49

card, I can add a message

16:52

to so-and-so, right? Yeah,

16:55

he wanted to add basically

16:58

little some of the little

16:59

ancillary stuff that they do to build

17:02

the community, right? They wanted to announce

17:04

that there's a new release that's available,

17:07

right? So that people, users can know

17:09

that they can download something new. They

17:12

wanted to add a little message for

17:14

the the person who wrote the pull requests as

17:16

like a little thank you to say, hey you know your code

17:18

has made it into the main branch

17:21

and we've released it so thank you very much. There's a little

17:23

invitation to a Discord server to say

17:25

like, hey if you want to chat with us you

17:27

can become part of the community that way and

17:29

that

17:29

just you just kept adding another

17:32

thing one after the other and it seems

17:34

like it's gotten a little bit too too big

17:36

to handle. Yeah,

17:38

it definitely sounds like Patrick and Jose need

17:40

to have a talk. It

17:44

started out simple and then it's not

17:46

so simple anymore. Patrick

17:48

wrote up a pretty successful script even though

17:51

it's got all those fancy features. It's

17:53

been fairly successful and fairly smooth and

17:56

it's so successful actually that it's making

17:58

its way into other

17:59

other open source projects whose maintainers

18:02

were impressed by the script

18:04

that Patrick wrote. Now Patrick

18:06

is keen on automating a lot of things. It's

18:08

fun, it keeps him from being a bottleneck,

18:11

but he does think that automation should have

18:14

its limits. Definitely the human

18:16

interaction is something I'm

18:18

afraid of losing. I really like, you know,

18:21

being able to, I don't know, I

18:23

feel like I've said that this a lot of times.

18:25

I've loved people to be welcome.

18:27

Like when I do a pro quest, like

18:30

I don't want to just have proven, I want to prove with a message,

18:32

this is nice.

18:33

There's been a push on his team to

18:36

automate a lot more than what

18:38

he thinks is necessary. And he's

18:40

worried about having scripts take over what

18:43

really helps build open source communities, even

18:46

if it does mean doing some of that work himself.

18:48

There's something to be said for being

18:51

able to interact with end users

18:53

and having those conversations

18:56

and being able to help them while

18:58

you're helping yourself in your own education.

19:01

So I feel him on that. There's sometimes

19:04

where some things don't need

19:06

to be automated, especially when there's a little

19:08

bit of human interaction that can help. As

19:11

you know, the more remote we are,

19:14

the more we're losing that,

19:16

those types of interaction. So I don't blame

19:19

him for wanting to, you know, keep

19:21

a little bit of that. It's probably

19:23

what makes his job very

19:24

rewarding. Yeah. There's something

19:26

to a very personalized,

19:30

unique message to someone, you

19:32

know, not just an automated response that

19:34

you

19:35

thank you first name and last name. Yeah,

19:37

exactly. Yeah. The form fill.

19:41

Well, there's something to receiving that too,

19:43

right? Like getting that little message to know that

19:45

you're appreciated. And you can you

19:47

can tell when it's an automated message. And when it's

19:49

been something that's actually

19:52

been handwritten for you, maybe not handwritten,

19:54

but custom personalized

19:57

to go on is sending it through like a carrier pigeon.

19:59

A hand written note. So

20:04

those are Patrick's projects. And again,

20:06

he's really just been getting started with automation.

20:15

Our next story is from someone who's working on

20:17

a massive enterprise scale

20:19

automation project.

20:21

This one has the same benefits as the others, but

20:24

instead of saving a little bit of time

20:26

and providing a little bit more consistency, it's

20:29

designed to smooth out a complicated process

20:32

that's a fundamental part of a financial

20:34

institution's business. One that

20:36

was extremely prone to error. Satyajit

20:39

Lele is a senior consultant here at Red

20:41

Hat, and he's been working on a massive

20:44

automation project. They've had

20:46

a process cobbled together for

20:48

many different tools over several years.

20:51

And the documentation isn't great. So

20:54

a lot of this organization's fundamental systems

20:56

rely on the know-how of a few people.

20:59

The application was often

21:02

installed or updated during

21:05

pre-coordinated upgrade window in

21:07

collaboration with networks as

21:09

admins. And this was usually done on

21:11

the weekend. So this

21:14

work could range from a few hours

21:16

to a half dozen of hours. But

21:20

there have been a lot of instances where

21:22

the team has had to work throughout the weekend

21:25

in order to debug or fix

21:27

a problem because they were doing

21:29

everything manually and they were

21:32

installing so many of these

21:34

components manually. And

21:37

if they miss

21:39

even one config file or

21:42

anything, they would have to manually debug

21:44

it

21:45

and make sure that the

21:47

application comes up. So

21:50

that is what we were trying to automate

21:53

and increase the reliability

21:57

that

21:58

it will not fail.

21:59

on manual errors at least. So

22:03

there was no testing systems. That's

22:05

for sure. And have we

22:08

all worked at the same place? Like, sounds

22:10

so familiar. We've all

22:12

been here. This must, he must

22:14

be a former coworker

22:15

because this sounds familiar. Well,

22:18

this is a project that they're just wrapping

22:20

up now, I think. Oh. Like

22:22

it's, it's in progress and they're either

22:25

just finishing it or they just recently finished.

22:27

I'm not entirely sure on the details, but

22:30

it's, it's one that's been in a long time

22:32

coming. This process was

22:34

not a recipe for a good time for those teams

22:36

working for that client, right? There had

22:38

been attempts to automate this process in the

22:40

past, but they didn't get very

22:43

far. They had a lot of components

22:45

that needed to start and stop

22:47

in a particular sequence. This

22:50

included a lot of checks along the way to ensure

22:52

that these components started correctly in

22:54

the right configuration.

22:56

Satyajit's team had its work

22:58

cut out for him.

22:59

Sounds like a case for automation. This

23:02

is textbook right here. This is textbook.

23:05

For this project, we estimated

23:08

around six months of work, which

23:10

is quite a long time, but

23:12

it turned out to be quite an accurate estimates

23:15

because there were many roadblocks

23:18

along the way.

23:21

Now, one of the biggest things

23:23

that they had to do initially that took a lot of time

23:25

was to recreate the client's developer

23:28

environment to as close as they could because

23:31

it was such a complex system.

23:33

If they didn't have it close enough and

23:36

they then set up the automation,

23:38

when they tried to port that automation

23:40

over back to the client system, it just wouldn't

23:43

work, right? Because the configuration would be different.

23:45

The machinery would be different, right? That you're

23:48

essentially building something that's just not

23:50

going to be applicable to that situation.

23:52

Let

23:54

me see if I understand this, Johan. So, and Angela,

23:57

maybe you can help me out too. So this is sort

23:59

of. sort of like a prime use

24:02

case for automation.

24:06

This one is difficult though, because

24:09

the processes are

24:11

so complex and

24:13

a lot of the knowledge

24:16

is in people's heads. Is that

24:18

kind of what's going on here? Like

24:20

what helped me understand like the root of the

24:22

difficulty here? Well, it sounds like

24:25

they never had a proper testing environment

24:27

that mirrored their production environment. It

24:30

could be that it's running on disparate

24:32

hardware. It could be

24:35

different operating system, different packages,

24:38

everything is different. And when you're

24:40

trying to automate something, you want

24:42

it to be as close to

24:44

the production system as possible.

24:48

Any small deviation

24:50

could be problematic. We can see

24:52

that if these aren't the same systems

24:55

and you're trying to build a workflow that

24:57

works for this, you

24:59

could possibly have problems

25:01

when you're trying to test it and say,

25:04

oh, it works in development. And then

25:06

you try to get it to prod and all heck breaks

25:08

loose because there was something really

25:10

very small, but it did deviate.

25:13

So it's hard when you don't have

25:15

those mirrored environments to do this

25:18

type of testing and it can

25:20

bite you. I mean, as we can see, if it's

25:22

a really complex system, it's

25:25

hard to line

25:25

up all the stars to get

25:27

it right. And the complexity

25:30

came from, it's a system that

25:32

they built up over a really, really long

25:35

time. A long time. Yeah. And

25:37

when we say system, but it's not like it's one computer,

25:39

right? It's cobbled together

25:42

from different pieces of hardware, different

25:44

pieces of software that they've added to over the

25:46

years. And whenever

25:49

they had to run an update, all

25:51

of these things had to be updated

25:53

in concert with each other. And

25:56

it's like conducting a symphony, right?

25:58

You got all these different. pieces

26:00

moving together and if if one piece

26:02

is off then yeah it doesn't sound

26:05

right and it's not gonna come out right so

26:08

they were having to update these and because it's

26:10

a financial institution they

26:12

didn't have a lot of downtime where they'd be able to

26:15

figure something out if something basically have

26:17

like a weekend right yeah they'd have they'd have

26:19

the weekend to update something so they'd start

26:21

it and hopefully everything went right

26:23

they'd be done in a couple hours which again

26:26

is a pretty long time already but if

26:28

something went wrong they'd

26:31

have to get those systems back online by Monday

26:33

morning that's it no pressure

26:35

yeah no pressure at all right and

26:38

actually this customer they got in a little bit

26:40

of trouble because even though

26:42

they tried to automate things in the past they

26:44

found it too difficult and they kept making mistakes

26:47

they ended up being told to automate the

26:49

system because of the frequency of the

26:52

mistakes that they were making oh wow and

26:54

so that's that's when they brought the team

26:56

that Satyajit is on to kind of help them

26:58

do that

26:59

so this is twofold to me they

27:01

were lacking automation that they so desperately

27:04

needed but this is also a great

27:07

use case for app modernization

27:09

hmm think about what you just said

27:11

they're talking about disparate hardware

27:13

and old systems and cobbled together

27:16

and those words you hear sounds

27:18

to me like wow they would really

27:21

benefit from something like containerization

27:24

on top of automation yeah

27:26

to get ding like that though is

27:29

pretty scary stuff you know

27:31

yeah like how does that the

27:33

project go like but how's it going

27:35

now

27:36

well what I spoke to Satyajit he told

27:38

me that it had been going really well it still

27:41

took a while still took months of work but

27:43

they're getting to that that finish line now

27:46

thankfully the client is getting a lot

27:48

of benefits from this project this automation

27:51

that we did was more

27:53

to increase the reliability

27:56

and security more than speed speed

27:58

is byproduct

27:59

but it was more to increase

28:02

the reliability and

28:04

security. And my architect,

28:06

Jeff, he always gives an example

28:09

of a robot arm. So when

28:11

you install a robot arm, it's

28:13

not only to save time, but

28:16

it is also to have the

28:18

exact weld at the exact spot.

28:21

Right, so that robot arm is

28:23

providing computer exact

28:26

efficiency and consistency, right?

28:28

You're getting that exact same result

28:31

every single time because

28:34

this is what they needed, right? They needed to

28:36

have a system where they could

28:39

run it without worrying about one

28:41

little thing, derailing a multi-hour

28:44

process and then having to go back and

28:47

figure out what went wrong and rebuilding

28:49

everything. They needed reliability and

28:51

security, like he said, but

28:54

speed

28:55

is a byproduct of that. Yeah, yeah. For

28:57

sure,

28:57

once you've developed a

29:00

workflow that can move through

29:02

this system accurately and efficiently

29:04

to do everything that you needed, that's where

29:06

the speed comes in. You're

29:09

not doing these things manually anymore. So

29:11

this is a good story. I'm

29:14

glad there was a happy ending here,

29:15

for sure.

29:19

Because yikes. So

29:21

I'm thinking now about both

29:24

Patrick and Satyajit, right?

29:27

And I'm thinking about the difference in scale

29:30

between the two types of automation

29:33

that they were working on. And

29:35

I'm curious what

29:38

they have in common.

29:40

Well, automation allows you

29:42

to remove the manual processes. And

29:45

I think that's what they were both dealing with before

29:47

they brought automation into the fold. And

29:50

we know manual processes lead

29:52

to errors and mistakes

29:55

and issues. So

29:57

what they do share is that they

29:59

said,

29:59

going to automate this issue. Yes,

30:02

it could be the small issue. Yes, it could be this

30:05

big issue. But what we're going to do is

30:07

we're going to take away that

30:09

error-prone-ness that we were plagued

30:12

with all along. And what we're going to do

30:14

is we're going to create this workflow where

30:16

we can get things done consistently,

30:19

automatically, time and time

30:22

again. And they wouldn't have to think about it again.

30:24

So that's what they share in common.

30:27

Automation provides you with

30:29

that safety

30:29

where it's like, oh, okay, I

30:32

did this thing. I'm not

30:34

making the mistakes that I've made in the past. I've

30:36

taken my hands off of the wheels and

30:39

I'm letting the automation take care of itself.

30:42

Johan, how are you thinking about automation

30:44

now? Well,

30:46

when I first started researching this episode,

30:48

it was very much in that frame of what

30:51

Jose had said, which is

30:53

you try to automate something sometimes and you

30:55

get down that rabbit hole and you're

30:57

not actually saving all that much time because

30:59

it's something that would take you a few seconds and you

31:01

spend hours automating it. And

31:04

if you were actually to account for

31:06

the time writing

31:07

the automation and actual

31:10

time doing the thing by hand,

31:12

maybe that balance doesn't look so good in the end.

31:14

But

31:15

speaking to Jose again, speaking

31:17

to Patrick, speaking to Satyajit, it

31:20

really opened my eyes to some of the other things

31:22

that can be gained from automation. Just like

31:25

Angela was just saying, which is, you know,

31:27

you get that consistibility.

31:30

I like that word. Reliability and consistency

31:32

all at the same time. Just put it all together.

31:35

Boom.

31:36

You get that consistency, you get that

31:38

reliability, and

31:40

from those two things, you get things

31:42

like speed, increased security,

31:45

time to focus on something else.

31:48

And really, whether it's just

31:50

automating little things

31:53

that make your life a little bit easier to

31:56

these huge enterprise

31:58

systems, you know, that, that.

31:59

that's just a matter of scale at that point.

32:02

Indeed. Indeed.

32:05

There's one more thing I think we should address

32:07

here, which is we heard from Jose

32:09

about keeping things simple.

32:11

Yeah, okay. Right, having things do one thing.

32:14

And one thing

32:16

that came up in the conversation with

32:18

Satyajit is obviously this six

32:20

month work is for a huge system

32:23

and the automation that they came up with. It's

32:26

just, it's complex, right? It's doing

32:28

a lot of different things. Yeah. But

32:31

apparently the way in which they set it up is

32:33

that

32:35

even though the whole of

32:37

the automation is very

32:39

complex, it's split up into little

32:41

different parts that each do one

32:43

thing. Oh. And so what that then allows

32:46

them to do is that if they

32:48

discover a bug in part of the process,

32:51

it's not something that's gonna cascade down

32:53

and affect all the rest of the other automations

32:56

and the other modules and the other scripts

32:58

that they're setting up. They can go in and

33:00

fix things here and there and

33:03

allows them to kind of debug

33:05

a little bit more in a more compartmental

33:08

fashion. Yeah. Right. So

33:11

it makes it a lot easier for them. And then later

33:13

on for the customer, if they need to change something

33:15

down the line, they can then go fix

33:18

those individual parts more than having

33:20

to look through the

33:21

entire thing and fix

33:23

every single bit of it when they only need one

33:26

little change. Mm-hmm, mm-hmm. Making

33:28

it modular. Mm-hmm, exactly.

33:31

Keeping it simple.

33:34

Well, we wanna know what you think about

33:36

this episode. I mean, automation is

33:39

all the rage. That's all we're talking about.

33:41

That's all you hear. Automate all

33:43

the things. We wanna hear what you

33:45

have to say about it. So tweet us at

33:47

redhat using the hashtag compilerpodcast

33:50

and we would love to know about your

33:52

automation journey and what do you think about

33:54

it? Let us know.

33:59

And that does... for this episode

34:01

of Compiler. Today's episode

34:03

was produced by Johan Philippine

34:05

and Caroline Craighead.

34:07

A big thank you to our guests, Jose

34:09

Vincente Nunez, Patrick Arminio,

34:12

and Sachijit Lele.

34:13

Victoria Lawton is the irreplaceable

34:16

component we will never automate.

34:19

Never. Our audio engineer

34:21

is Robin Edgar. Special

34:24

thanks to Sean Cole. Our theme

34:26

song was composed by Mary Anchetta.

34:28

Our audio team includes Lee

34:30

Day, Stephanie Wonderlick, Mike Esser,

34:33

Nick Burns, Aaron Williamson, Karen

34:35

King, Jared Oates, Rachel Ortell,

34:37

Devin Pope, Matthias Fonda's, Mike

34:40

Compton, Osia Matthews, Paige Stroud,

34:42

and Alex Trebulci.

34:44

If you'd like today's episode, please

34:46

follow the show, rate the show,

34:48

leave a review, share it with someone you

34:50

know, it really does help us out.

34:53

All right, we will see you next time. See

34:55

y'all, bye. All right,

34:57

bye. Bye.