0:00

Welcome. I'm playing with

0:02

some different kinds of episodes,

0:04

and this episode is going

0:06

to be a little different. Instead of doing an

0:08

interview or a mastermind, what

0:11

I'm gonna be doing is actually telling you about something

0:14

I've been using ChatGPT, and

0:16

Bard, and a bunch of other tools for,

0:19

Welcome to the Prompt Engineering Podcast,

0:21

where we teach you the art of writing effective

0:24

prompts for AI systems like ChatGPT,

0:27

Midjourney, DALL-E, and more. Here's

0:29

your host, Greg Schwartz.

0:33

First place to start is a little bit of backstory.

0:36

So back in 2019, my

0:38

mom fell down a flight of stairs and broke

0:40

her neck. And, uh, fortunately

0:42

she was never paralyzed, but I

0:45

ended up moving from San Francisco,

0:47

where I previously lived, to

0:49

San Diego to be able to help her. So

0:53

the reason I bring this up is she

0:56

is looking for work and that

0:58

can be a little challenging for a couple of reasons. One

1:00

is the neck injury. She has, mostly

1:02

recovered from that. So she can drive, but

1:04

she can only lift five or 10 pounds. And

1:08

she's a very outgoing

1:10

person and so a lot of the

1:12

stuff that would make a lot of sense typically

1:14

for someone with her personality, say, sales

1:18

is a little more challenging for her because she can't

1:20

work most normal retail jobs. Because

1:22

most normal retail jobs require you to

1:24

lift 20, 30,

1:26

40, 50 pounds. Second

1:29

challenge is unfortunately she never finished

1:31

her college degree. For the past few years,

1:33

what she's done is she's worked

1:36

as basically a home health aid,

1:38

helping people with their medication and

1:41

getting them to eat if they have a hard time eating or

1:43

even cooking for them if need be. The

1:47

challenge though is as her clients

1:49

get older or

1:51

their health declines, she

1:53

loses a client. She only works with one at a time,

1:56

so it's not the most stable.

1:58

Not surprisingly, I'd like to help my mom

2:00

have a more stable career in life.

2:03

So I've been thinking, all right, maybe there's

2:05

some certifications that I could find that

2:08

she could go get and then be

2:10

able to get a job that's more stable.

2:14

So naturally I

2:16

turned to ChatGPT, and

2:18

I said, act as a career counselor,

2:20

an executive coach with 20 years of experience.

2:23

Your client is a 67 year old woman

2:25

without a college degree living in San Diego. She

2:28

is outgoing, sociable, but unable

2:30

to work eight hours a day, do a due

2:32

to a neck injury. She's

2:34

previously been a salesperson at Chicos,

2:37

but struggled with the political battles in the store.

2:39

Suggest 15 certifications or

2:41

trainings that could lead to a pleasant career

2:43

for the remainder of her life, including links

2:45

to organizations offering certification, job

2:48

posting, and seller estimation. I

2:51

tried using it with browsing, and

2:54

maybe it was just that day, but

2:56

for whatever reason, I ran

2:58

it three times and each time it would

3:00

just crash. So

3:03

I would hit regenerate and

3:05

it would then sometimes give me answers,

3:07

but often it would just get stuck trying to load links.

3:10

So browse has usually worked

3:12

fairly well for me, but on this

3:15

prompt, it really did not. So

3:17

I turned off browse. And just

3:20

went with this. It gave some

3:22

interesting suggestions. It said, life

3:24

coach, real estate agent, which is something

3:26

we've actually talked about before. A bunch

3:28

of other stuff. Virtual assistant,

3:31

mediator, genealogist (which

3:33

was actually an interesting one cuz she's very into genealogy).

3:37

And customer service and that kind of thing. And

3:39

I said, okay. Yoga

3:41

and some of these others that are fitness related aren't

3:44

really a good fit for her with the neck

3:46

injury. So I said, another 15,

3:48

please, nothing fitness related.

3:51

Gave me more. And

3:53

most of these were not great, I'm

3:55

scrolling through them if you're watching the video.

3:58

And so then I said, okay, gimme more ideas.

4:02

And then, It did,

4:05

but it started drifting. So for example,

4:07

it said how about a certified spiritual

4:09

counselor or a certified etiquette

4:12

consultant, and a bunch of other things that

4:14

I went, I'm not sure that's a real

4:16

certification. Nothing against those jobs

4:18

just didn't sound like a, an actual certification

4:21

from the way it was describing.

4:24

Particularly certified travel

4:26

writer and photographer. Which

4:29

is from some company called

4:31

Great Escape Publishing.

4:34

I'm not entirely sure I believe

4:37

that's a certification. So anyway, I

4:39

kept going and what I

4:41

noticed is it just

4:44

drifted. I should mention

4:46

also she really likes cooking and baking

4:48

and that kind of thing. So I said, Hey, what

4:50

about stuff that involves food? And

4:53

it just went off the wall. It started

4:55

giving all these certifications that, yeah I

4:57

don't think these exist. Certified

5:00

barbecue judge really, I don't know

5:02

about that. But anyway tried

5:04

to narrow it down and what I started noticing is

5:06

each time I would say, give me more ideas. It

5:09

started recycling 'em. For example, in

5:11

the first response it suggested mediator,

5:14

and again, here is a suggestion

5:16

for mediators and

5:18

some other ones repeated as well. So

5:21

eventually it was really starting to repeat itself.

5:24

So then what I tried was more ideas

5:26

that you haven't suggested yet, which if

5:28

you understand how context windows worked, unsurprisingly

5:31

didn't work. So just to be clear,

5:34

the context window is the memory

5:37

of the conversation that ChatGPT

5:40

has, and part of the challenge

5:42

is even though it's still on

5:44

screen for you in the ChatGPT

5:46

interface, it

5:48

won't remember after a certain number of messages,

5:51

especially when it's outputting a lot of text or

5:53

you're inputting a lot of text. And so

5:55

obviously the context

5:58

window had gotten so far

6:01

that even the things that it had previously

6:03

suggested had fallen out.

6:05

At this point, I said more ideas

6:08

that you haven't suggested yet, and it said, here

6:10

are additional certifications she might consider

6:12

that were completely just off the wall.

6:15

And strange and repetitive

6:17

travel agent was something that had already come up.

6:20

Personal stylist had already come up. Interior decorator

6:22

had already come up. So many of these things, it

6:25

just clearly wasn't keeping track. So

6:28

I finally said, all this isn't really working

6:30

quite as well as I hoped, so let

6:32

me give it some more constraints. How about more ideas

6:34

that don't involve fitness programming,

6:36

ux, graphic design, manual labor or

6:39

pets. And it still

6:41

wasn't doing great, at least

6:43

did that, but now it's giving totally off the wall

6:45

stuff with a certified mindfulness instructor,

6:48

which I like mindfulness, but I'm not

6:50

sure there's an actual certification for that.

6:52

And a bunch of others that, again, just weren't really

6:55

fitting. So what I finally started doing

6:57

was I just completely

6:59

copy and pasted everything out

7:01

of here and pulled out all

7:03

of the job titles that it had listed so far.

7:06

And so then I gave it the exact same prompt.

7:09

Act as a career counselor, et cetera, and

7:11

then do not include any of the following jobs.

7:13

Life coach, real estate agent, personal stylist,

7:15

blah, blah, blah, blah, blah. You can see it's quite

7:18

a long list and

7:20

that... partly worked.

7:22

It did a decent job of giving

7:24

a few more suggestions. Master

7:26

Beekeeper was actually one that would be fun,

7:29

but isn't really very relevant to

7:31

San Diego, but it was definitely one that made

7:33

her go, Ooh, that could be fun. Kept

7:36

going through, more ideas. Most of

7:38

these were not great. So then I finally just said, all right,

7:41

grab all of that again. Run it again.

7:43

So that helped. It definitely continued

7:46

to be a little on the repetitive side.

7:48

So again, as you can see, as I just

7:50

keep scrolling I kept giving

7:53

it that prompt again with an even longer

7:55

list of things not to include. So

7:58

finally I said, okay, I think I've mined

8:01

ChatGPT of as much as possible,

8:04

and then I went, There is another spot,

8:06

let's go check one of the auto

8:08

GPTs and see if they can

8:10

come up with anything. So I tried God mode,

8:12

which by the way is http://GodMode.Space.

8:15

Not that I would recommend it. I have

8:17

not had it work any

8:19

of the times I've tried it, and unfortunately, this

8:21

is another example of when it

8:23

just totally failed. So

8:25

I gave it the original prompt. It

8:29

literally ran as its first action

8:31

to not take an action. I

8:34

don't know what to make of that. That's

8:37

silly, but that's what it did. And

8:40

then it said, oh, okay, great. Let's go search for

8:42

sales and customer service careers. And

8:44

then it spent so many

8:46

cycles trying to retrieve

8:49

job posts from indeed. And

8:51

failing and retrying

8:54

and, okay, let's try it again. And, okay,

8:56

let's just go search again. And ultimately

8:59

it was basically useless. So

9:02

here's the results that it came up with.

9:05

And needless to say, this is not

9:07

a particularly helpful list. It's customer

9:09

service representative, customer sales and service

9:12

representative, customer service representative,

9:14

entry level sales, and B2B sales.

9:16

I was not looking for five, three of which are the

9:18

exact same thing. So yeah,

9:22

God mode a fail, but

9:24

then I thought, oh, bard has access to the

9:26

internet, so let's go try Bard.

9:29

And it, it wasn't

9:31

great. Same suggestion of

9:34

customer service, administrative

9:36

assistant bookkeeper web

9:39

developer, graphic designer. And

9:41

then I said, okay. More ideas, it

9:44

said, great. Here are some more ideas for

9:46

certifications or trainings that blah, blah, blah, blah,

9:48

blah. Customer service representative, administrative

9:51

assistant bookkeeper. It's all the same list.

9:54

Now, since it's a much more concise list,

9:56

it gave me more, but. It

9:59

says more ideas for certifications

10:01

or trainings, and it's coming

10:04

up with tour guide, dog

10:06

walker, babysitter, house cleaner, personal

10:08

shopper, and volunteer.

10:11

Okay. Obviously, volunteer

10:14

is not something you get certified for

10:16

as a career. So just

10:18

completely a fail, but the other ones aren't things you get

10:21

certified for either. So it was just

10:23

kinda okay. Barb, did you not understand

10:25

what I was saying? But sure. Let's give it one

10:27

more shot, gimme some more ideas. And

10:30

again, it just wanders off:

10:32

volunteer comes up as a suggestion the third

10:34

time, tutor, house cleaner,

10:36

dog walker. Bard is not paying attention

10:39

to the prompts, which again, let

10:41

me scroll back up. Mentioned

10:44

that she has a neck injury

10:47

and it just tossed that

10:49

out. I

10:51

did bring up. You know what

10:53

about food related stuff? Cuz she's a talented

10:56

cook and a baker and it just

10:58

gave a bunch of jobs. There wasn't anything

11:00

about, these are certifications,

11:02

these are jobs that require certifications. There was

11:04

nothing about that at all. So

11:09

finally I tried reprompt it

11:11

with the same prompt again, but just to try

11:13

and get it back to are there certifications

11:16

that would be relevant here? And it just

11:18

completely lost the thread at that point. It

11:20

starts suggesting personal shoppers, which

11:23

there's no certification for that. Life coaching,

11:25

there are certifications for that, but they aren't particularly

11:27

helpful. Grief counselor and

11:29

just so many things. It.

11:32

It had so clearly lost the thread

11:34

that I finally just gave up. Oh, yes. It also

11:36

suggested being a podcaster or a

11:39

YouTuber, and there are absolutely certifications

11:41

for that. If you can't tell. I'm being sarcastic.

11:43

I am. So

11:46

then I tried Bing and

11:48

was hoping it would be a little better cuz it,

11:51

it does definitely do

11:53

a good job of searching for things. It

11:55

didn't do a great job, but let's walk through

11:57

it. So I gave it the same prompt and

12:00

it gave an interesting, I'm sorry to hear about her neck

12:02

injury. There are many sales opportunities

12:05

available. Alright, great.

12:07

Thanks. I should note this is, Basing

12:10

the search off of certifications for sales jobs

12:12

without college degree, which was an interesting

12:14

leap for it to take. So then

12:16

it says, advertising sales, real estate

12:18

broker insurance sales. Retail sales.

12:21

Yeah. Okay. Some of those

12:23

have certifications, most of them do not.

12:26

Some links around organizations

12:29

that provide the certifications and that kind of thing.

12:32

I said, all right, that's not super helpful. More

12:34

ideas. And

12:37

it then said, all right, I'll search for jobs

12:39

for outgoing, sociable people with neck

12:41

injury. Totally

12:44

lost the certification concept and

12:48

the results: exactly the

12:50

same as everything else has suggested

12:53

[except for GodMode], customer service,

12:55

social media, event planner, travel agent,

12:57

writer, virtual assistant, graphic

12:59

designer, web developer, blah, blah, blah.

13:03

Interestingly, then it

13:06

said, here's a list of jobs

13:08

that are available for people with chronic

13:10

pain that offer schedule flexibility

13:12

or minimal physical demands, and listed

13:14

them off, most of which were the

13:16

same jobs of, writer, editor,

13:19

graphic designer, et cetera. But

13:21

it was an interesting idea. I

13:24

then said, all right, let me try and guide it a

13:26

little more. More ideas given

13:28

that she doesn't have a college degree. So

13:30

now it says, okay, I'm searching for jobs for

13:32

outgoing, sociable people without a college degree.

13:35

And again, it's giving me almost all

13:37

the same results. So

13:41

once again, There's a

13:44

big difference in quality that I notice

13:46

where ChatGPT, at least for

13:48

a certain amount of time, keeps

13:51

the context window pretty large.

13:54

So this is another example of how I

13:56

have seen that ChatGPT

13:58

so far is getting by far the best

14:00

results. Bing

14:02

and Bard both give

14:05

a lot of duplicate results,

14:07

even just one reply

14:10

later. In this response,

14:12

it's suggesting customer service

14:14

social media travel writer virtual

14:16

assistant and graphic designer. And then

14:18

when I say more, it just tells me a

14:21

bunch of the same things. Really surprisingly

14:25

poor results from both Bing

14:27

and Bard. I was genuinely expecting

14:30

at least one of them would do better. So

14:32

all in all, ChatGPT is

14:34

by far the winner, by far

14:37

the best results. Honestly, shockingly,

14:40

how much better, especially

14:43

shockingly, because I wasn't that happy with the results

14:45

from ChatGPT until I went to

14:47

the other ones and thought, this is

14:50

really bad. This is amazingly

14:52

bad. I can't even believe how bad this is. All

14:55

in all, I took everything

14:57

from ChatGPT threw

14:59

away most of the rest of it because honestly, all

15:02

of the others gave me the same information. Put

15:06

all of that into a nice Google

15:08

Doc and showed it to my mom,

15:10

walked her through it. She thought it was some interesting

15:12

ideas. So she's exploring some of these

15:14

concepts now. So all in

15:16

all, if you're doing this kind of brainstorming

15:19

[and there's a little bit of research involved, but it's

15:21

mostly brainstorming] chatGPT,

15:24

particularly G P T four is still

15:26

your best bet. Bing and Bard,

15:28

as nice as it is that they

15:31

have access to the internet, their

15:33

ability to remember the conversation

15:36

is much worse. And so for this kind

15:38

of brainstorming and research, it just

15:40

doesn't work very well. Your

15:43

mileage may vary. Every time

15:45

I've tried Bard and Bing, the results

15:47

have been pretty mediocre It just doesn't remember

15:49

the rest of the conversation to give you new information.

15:52

It just keeps giving you the same information

15:54

in Bing each time you ask,

15:56

and the same information in Bard, each time

15:58

you ask. I'm

16:00

told Bard is better for code generation.

16:03

Haven't tested it yet. Maybe that'll be next

16:05

week. In the meantime, I'm gonna be

16:07

launching my course next week, so if you'd

16:09

like to give me some beta feedback on an early

16:12

version of it, for a nice discount.

16:14

I would love the feedback cause

16:16

I wanna make it as good as possible for all of

16:18

you. So if that would be of interest,

16:21

please either comment below or send me

16:23

an email. Talk to you soon.

16:26

Thanks for coming to the Prompt Engineering Podcast. Podcast

16:29

dedicated helping you be a better

16:31

prompt engineer. I also host

16:34

weekly masterminds where you can collaborate

16:36

with me and 50 other people live

16:38

on Zoom to improve your prompts.

16:41

Join us at http://promptengineeringmastermind.com

16:44

for the schedule of the upcoming masterminds. Finally,

16:48

please remember to like and subscribe.

16:51

If you're listening to the audio podcast, rate

16:53

us five stars. That helps us teach more

16:55

people, and if you're listening to

16:57

the podcast, you might want to join us on YouTube

17:00

so you can actually see the prompts. You

17:02

can do that by going to youtube.com/@promptEngineeringPodcast.