Narrator: You're listening to the humans of DevOps podcast, a

podcast focused on advancing the humans of DevOps through skills,

knowledge, ideas and learning, or the SK il framework.

Jason Baum: Hey everyone, it's Jason Baum, Director of Member

experience at DevOps Institute. And this is the humans of DevOps

podcast. Welcome back. Hope you had another great week this

week. I always hope you have a great week. So I hope this one

was even greater than the last one. Today we're gonna be

talking about incidents, mistakes, do overs, we talked

about blameless culture. It's one of the core principles of

DevOps. But in reality, is it as easy as just saying, we have a

blameless culture. In preparing for today's episode, I'm

reminded of a quote by Phoebe Waller bridge, the creator of

fleabag, and show runner of killing Eve. That's the very

reason they put rubbers on the ends of pencils because people

make mistakes. I love that quote. If you don't know who she

is, you know, Waller bridge has made a career of bringing to

life unconventional women who make a lot of mistakes. But what

her series have in common is that her flawed characters get a

chance at redemption, moving past mistakes, and offering them

an opportunity to prove something to themselves and come

out stronger and more confident on the other side. I feel like

this quote, the metaphor she made is a perfect setup for our

conversation today. Incidents are a great opportunity to

gather both context and skill. They take people out of their

day to day roles and force teams to solve unexpected and

challenging problems together. Joining me today to discuss this

topic is Lisa Carlin Curtis, Lisa is a product engineer

incident.io. In fact, Lisa was employee number two@incident.io.

She started out as a consulting working at Accenture before

accidentally becoming a developer. I'd love to hear how

that happened. Lisa loves building stuff. But it's also

interested in how people interact with each other in a

work environment, particularly in software engineering. Outside

of work, Lisa loves cooking, and pretty much any competitive

sports, we definitely have that in common. But I guess she likes

the British one. She says, I really don't know that much

about them. So Lisa, welcome to the podcast. Thank you for

joining me.

Lisa Karlin Curtis: Hey, lovely to be here. I'm really looking

forward to awesome.

Jason Baum: Are you ready to get human? Alright, let's do it. All

right. So we're talking about incidents? How can incidents be

considered an opportunity to gather context and skill?

Lisa Karlin Curtis: So I kind of started thinking about this.

When I was reflecting on Yeah, I kind of became a software

engineer accidentally, and I've accelerated quite quickly, I've

been very fortunate. And part of that is because a lot of the

stuff I did before I was an engineer was actually quite

useful. But also part of it was, I realized, I started doing this

thing where I was basically running towards the fire. So

stuff would go wrong. And I'd be like, Oh, that looks kind of

interesting. And all of the times where I learned most the

sort of step changes in my understanding or my context,

were around incidents. So they were like, something would go

wrong. And either I would learn, like, while we were fixing that

problem, I would learn about your stuff. Or straight

afterwards, when I was like reflecting on it and talking to

people about it, I'd learned a bunch of stuff. And so I kind of

started thinking about this and talking to people about it. And

it turns out other people had had the same experience, what a

surprise. And so I think that there is there is something very

unusual about incident, which is why it is an incident Right?

Like something, something happens that is unexpected that

you didn't know was going to happen. And then you have to

react to it. And that pushes people outside their comfort

zone. And it pushes you to do things and see things that you

wouldn't otherwise see. So I guess like, I can think of like

three, three key areas where it's really useful. So one is

about like broadening your horizons because you see the

stuff that you wouldn't see in your day to day. One is about

like teaching you how to build stuff that fails gracefully, and

then an observable way. So I think that one of the one of the

key differentiators between good software engineering and great

software engineering is about what happens when the thing that

you didn't think could happen happens. So like step one, make

it work. Step two, make it work really fast. Step three, make it

work really fast. And when you get a negative number that you

weren't expecting you explode really, really loudly as opposed

to just take the negative number and let's just Pay somebody a

negative amount of money or you know, whatever it might be. And

then sorry, just to finish off, the third is about building your

network. So you have a whole bunch of connections with

different people in your organization, you work with,

like your team. But in an incident, often you have to, you

have to work with lots and lots of people from across the

organization. And that builds bonds that are really important.

And I think really valuable both to you as an individual and to

the company.

Jason Baum: Yeah, absolutely. If you listen to this podcast,

you've heard me use parenting often as examples. Because I

think parenting is so applicable to what goes on in day to day

life outside of your home. And one of the things that I was

told when I was a new parent was plan for the unexpected or plan

for the implantable. And I think that's applicable here. I think

with incidents and mistakes, it's almost like, you need to

expect it, it's going to happen, if you're building a program,

it's going to have a bug. It's what happens after it happens.

That really matters, right? That's where all the everything

happens.

Lisa Karlin Curtis: Yeah, I think that's the differentiator

in terms of, if you're, if you're building a system, you,

you will predict a certain number of the possible things

that are going to happen, and you will make your system behave

well in them. And that's all great. And then you read a book

that's like, oh, you should make your system observable. And you

go, Okay, I will add some loglines. And I will add some

metrics. And like, it's really easy to do that in a way that

doesn't really add any value. And we've all we've all seen

examples of that. We've all seen dashboards that really mean

anything. And the way to get from like, having read in a book

to being actually able to do it, valuably, I think is just to see

it. And it's very difficult to do that into learn that in the

abstract, but as soon as you see someone trying to debug a

problem, and you see like, you know, what, what is the

breadcrumb? What are the breadcrumbs that they are

following, in order to get from our API is slow to this is the

root cause I can now fix this problem. And if you see somebody

do that enough times, you start to be able to lay your own

breadcrumbs. Because you can kind of imagine you can, you can

empathize, you can put yourself into that person's shoes who's

trying to debug it and be like, Oh, maybe it'll be useful for me

to like, have a metric here. Because if this specific bit

starts to go weird, we want to know about it. And I think that

that's something as you say, where it's like, it's all about

preparing for the unexpected. And a lot of that is actually

counter intuitively, perhaps it's not being able to handle

every case, it's being able to either be sure that what you're

doing is right, or get a human to help you out, right, and that

ENCODE is like throwing an exception or panicking or

whatever you want to call it. And that's the most important

thing, particularly if you're building your billing software,

like cars and planes and you know, software that we trust

with our lives, then you need to be sure that if the software

sees anything that it doesn't expect it the first the human.

And that's the same in like FinTech, which is my background.

And it's the same in lots of bits of software. And that's

something that like, you're not really taught that and if you

read it in a book, it doesn't really land. But once you see

it, you can really start to engage with it and sort of do it

yourself.

Jason Baum: So I mentioned blameless culture. And that's

really important in DevOps, with incidents of on blameless

culture anywhere, really. But I feel like it is something that

is said, I've heard it is said so much that it almost becomes a

buzzword. And you really question the authenticity of

when someone says, oh, we have a blameless culture? How do you

actually have a blameless culture? How do how do incidents

lead to learning? Without feeling like, the person who

made the mistake is getting in the way or, you know, is? Well,

yeah, feeling like you made a mistake, let people down. I

think that's, that's inherent nature for all of us, right?

Lisa Karlin Curtis: Yeah, absolutely. I think that there's

a lot of people have a lot of shame around, like making

mistakes at work. And that is a very like human thing, humans

are incredibly susceptible to shame. And what that means is

that there is so much psychological pressure when you

make a mistake to try and cover it up. And that is like the

worst possible thing that you can do in a software engineering

environment. And we know that, and yet all of us still have

that, right. All of us still have that moment when we find a

mistake was made. And we're like, maybe I'll just fix it,

and it will be fine. And no one will ever know. And I think that

is such a, it's so hardwired into our brains, that you have

to work very, very hard to combat it. And so some of the

obvious things that you can do as an individual, try and be

very open about your mistakes, particularly if you're in a

leadership role. Or some or if you have quite a lot of social

capital, because you've been in that organization for a long

time. That means that people will kind of monkey see monkey

do right. If you do it, other people will, will copy you and

we'll follow your example. And then there's another part of it,

which is I think you talked about failing together. So The

way that most most technology is more complicated than one person

made one error. It's normally lots of people made lots of

decisions that have all coalesced into a bad thing.

There was a famous one company, I used to work out where a

junior engineer had kind of gone on to there, they'd written some

code that was supposed to send an email, telling people who

weren't paying that they should pay basically kind of trying to

prompt and increase conversion. And the logic was a bit wrong.

And it was actually targeting all the people who were paying.

And they ran it in staging. And customer support got inundated

with requests. And the studio engineer is sitting there being

like, put it in staging, I'm really confused. This is very

stressful, like the team jumps in, like they go to support

support, then is sort of told this the mistake, don't worry,

your billings, fine. And they start to look back. And it turns

out that somebody has seeded staging with production data to

run some load tests, and they didn't anonymize the emails. And

so they've got they've just ran. Basically, they've run their

code in production, but they didn't know. And something like

that the junior engineer is, is really mortified because they've

done this thing. And they've they clicked a button and a

bunch of emails went out. And that's really bad. But I think

it's important to look at that as a group and be like, Well,

how would you have known that? Possibly, right? What Why did we

put production data in staging, why did we not anonymize it? Why

is staging setup so that it can send unlimited emails to

unlimited numbers of people? And there are a whole load of other

questions, right, and you can start to look at it as a

systemic problem. Or you can look at it, there's like the

Swiss cheese analogy of like, all the holes have to line up.

And I think that if you talk about things like that a lot,

then people get it, and people buy into it. And at that point,

it's much more comfortable to admit your mistake, because you

know that your team is going to gather around you and you know

that your team is going to take accountability. And so if you'd

like if you succeed together, if you fail together, you can build

this blameless culture. But if you hang people out to dry, if

you mock people, if you're mean, that's just going to reinforce

the shame that that person is already worried about.

Jason Baum: So if I if I'm hearing you, it's, it's that

proactive honesty, it's the mistake is made calling it out.

But saying, basically, it's calling it out for what it is

this happened. How do we address it? What do we do coming

together, getting everybody to rally around it? Without

pointing the finger?

Lisa Karlin Curtis: Yeah, I think that's exactly it. And

then you need to combine that with incident shouldn't be a big

scary monster. So I think there's a blog post on our blog

about like incidents and no bad thing, you should be declaring

more incidents. And there are there are, there are lots of

organizations who sort of measure their success on number

of incidents, which I think is is a really perverse incentive.

But I think that if if incidents become the norm, then mistakes

become the norm. And if you're all talking about incidents, and

if that information about that those incidents is really

accessible to people in your organization, then you've made a

mistake, just like everybody else on the team has made a

mistake has made hundreds of mistakes. Whereas if that is all

kept hush hush within the team, and it's not broadcast, then all

of a sudden, like that's the first mistake anyone in the

company has ever made as far as you're concerned. And that's a

really terrifying place to be.

Jason Baum: Yeah, and we all know that's not true. But yet we

feel it's just inherent human nature, right? To think that

your mistake is the worst mistake ever made, and oh my

god, they're gonna fire me or they're gonna like, black list

me or something bad is gonna happen. I'm never gonna work

again in this at the end of my life. And like, we just have

this habit, I think even as like from like kids through

adulthood, I always hope that that feeling would go away, and

it never has. Why?

Lisa Karlin Curtis: I think it's kind of it's partly the imposter

syndrome thing of like, the more you progress, the more

responsibility to have you have, the more you can see all the

things that you don't have to do. But I think that also there

is a there's another part of it, which is we, I think we inherent

we net, we inherently strive for perfection. And we want

ourselves to be perfect, because it's quite inconvenient that

we're not, because every single decision you make, you're like,

I think I'm right. You as a software engineer, you spend

your entire day going. Yeah, I think this is right, let's do

it. And if you've got a little voice in your head all the time

going, what if you're not what if you're not, that becomes

really stressful and really tiring. And so what we do is we

go, yeah, I'm right. Most of the time, this is kind of fine. And

then when something goes wrong, that punctures that sense of

confidence, and then that's really destructive. And then we

get really stressed because we think that the only reason

anybody has hired us is because we're right all the time. And

they haven't. But because because that makes our day to

day easier. I think it's very easy to kind of fall into that

and fall into the trap of almost believing your own rubbish that

you are, in fact, perfect and flawless.

Jason Baum: It's so funny because it's even, I would say

probably, if not the most common one. have the most common

questions that come up in an interview processes. Name and

incident that happened that were you like, for example, where you

made a mistake, but how did you handle it? And how did you

overcome it? How did you overcome it? Right? That's like

one of the most common questions, I think. If you

haven't, if you haven't had it, I don't know, maybe you haven't

ever applied for a job before in your life? Because I think it

must be the most common one. And yet, when you're hired to do a

job, it's almost like yeah, we do strive for perfection. And

that question went out the window. It's like, now I can't

make a mistake.

Lisa Karlin Curtis: Yeah, it's, it's really, it feels like one

of those things that there should be a better answer to as

well. But I don't think there is I don't think there's a silver

bullet. I think like, you talk about it, you lead from the

front. You you try and catch it when it does happen. And, and

you hope that slowly but surely, people, you know that that

feeling of wanting to hide it, that feeling of shame just

becomes less and less strong, and the muscle, you develop this

muscle of overriding it. But you know, I'm talking about this in

evangelizing, I still absolutely have that instinct. The only

thing that I've learned is I have a muscle that I can now be

like, I can recognize it. And I can look it in the face and be

like, we're not doing that today. Because that's not

useful. But it's it's definitely an active thing. It's not it's

not a sort of default.

Jason Baum: Yeah, yeah. And, and we have to look at it, it's

based on what you're saying. It's like isolated in each

incident is an incident, isolated, we have to take care

of it, learn from it, and just move past that. If I'm gathering

that.

Lisa Karlin Curtis: Think emotionally, absolutely. I think

there's there's another side to that, right, which is, if you as

an organization, I think incidents are really important

source of data to understand where you should be putting your

chips. So if you have good reporting, if you can look at

your incidents in aggregate, if you have like a good way of

recording them, and categorizing them, then you can start to use

that data and start to go, oh, this bit of tech seems to cause

us a lot of problems or, you know, this process seems to be

really risky for us. And now that will help that will help us

decide like, where are we going to invest next. So I think from

that point of view, you don't want to kind of leave them

behind the tool. And that's in direct conflict with what you

want people to do emotionally, which is to, you know, be there

in the moment, solve the problem, close it and not worry

about it. And I think that that's quite difficult to

manage, because you simultaneously are telling

people to leave it behind, and also telling them to constantly

be thinking about them and be you know, sitting in a quarterly

review being like, what went wrong this quarter? What do we

want to invest in to help make our platform more reliable,

right?

Jason Baum: You know, what it makes me think of, and apologies

ahead of time, American football, we have a quarterback

and the quarterback, throws interceptions. It's a given

thing. Everyone knows their quote, no matter who it is Tom

Brady is I mean, they're going to throw interceptions. And yet,

they strive for perfection. Because what sport what athlete

doesn't, right? And then when they happen, the one thing that

I would say is in that culture is they get on the phone, or

they go next to the offensive coordinator coach, and they look

at what happened in that play. Here's what happened. Here's why

he didn't see it. This is what happens. And then they are

supposed to forget it. Forget it ever happened and move on?

Because how do you move on with the rest of the game, if all

you're thinking about is the one big mistake you made? And I just

I think that for me, when you're when you're talking about kind

of forgetting it, that that instantly popped into my head?

So many things could be applied to that, I think.

Lisa Karlin Curtis: Yeah, I think also when we talk about

incidents, there's nothing that's specific to engineering

about them. Really. I think the engineers talk about them a lot.

We have a language that we discussed them in. But there are

loads of examples of incidents that are not engineering. And I

think almost all the stuff that we discuss around incidents

being a chance to build your network with other people being

a chance to touch things that you don't normally interact

with, you know, being a chance to watch what your system does

when it fails. Like that feels very engineering. But actually,

if you're a customer success team, you know what happens when

your processes fall over? What happens when the person who's

doing all of the glue work has gone on holiday and all of a

sudden something bad happens? And like you're still stress

testing, you're still finding the edges. It's just a slightly

different environments.

Ad: Are you looking to get DevOps certified? Demonstrate

your DevOps knowledge and advance your career with a

certification from DevOps Institute? get certified in

DevOps leader, SRE or dev SEC ops, just to name a few. Learn

anywhere, anytime. The choice is yours. Choose to get certified

through our vast partner network self study programs, or our new

skillup elearning videos. The exams are developed in

collaboration with industry thought leaders, and subject

matter experts in the DevOps space and Learn more at DevOps

institute.com/certifications.

Lisa Karlin Curtis: I think what's what I find interesting

about that is that you you start at, you're like, oh, when things

go wrong when it's bad. And we've had a number of incidents

where I would say the net impact on our company has been

positive. Because somebody reports it, we see it, we've got

some really quite good observability. So often, we can

like, find it pretty quickly fix it, turn it around and say half

an hour. And the customer ends that interaction, actually

feeling better about us than when they started, which is

probably quite counterintuitive, because really, if we just

hadn't broken it in the first place, maybe that would have

been better for them. But we we've joked internally about

maybe we should deliberately come up with bugs because of how

how much like great feedback we get when we fix things from

people,

Jason Baum: right? I feel like that's the evolution, right of

any good product is the feedback. So, you know, in

thinking about letting it go and thinking about these

improvements. I feel like there must be obstacles, though,

besides ourselves, right? And our own internal turmoil that we

put ourselves through, when we make a mistake, or when an

incident happens. There's deadlines, and you need to hit

them, you need to meet them. And when you miss them, that's a big

deal. So how does that play into when incidents happen? How does

that how does that, I guess, impact that feeling that we're

already feeling right, the shame that you talked about, and then

we have this deadline looming over our heads?

Lisa Karlin Curtis: I think that's really interesting. I

think it's very, very difficult because you have a trade off

generally. So normally, there's that there's a triangle of like,

speed, and quality, and the common warts on the other end of

the triangle. But the idea being you have to, you have to trade

off something number of people, maybe, maybe we should just

scrap all of that, I'm gonna start again, that's fine. I

think there's a trade off here. So when something goes wrong,

the first thing is I need to fix the things broke. And that takes

you however long it takes you and basically nothing else

matters. Generally, there is there is a sort of a type of

failure mode, where you're just trying to bring your system back

up, or resolve the bug or stop anything getting any worse. And

that's a really easy decision, because it's there, it's on

fire, we got to fix it. And then you get to a sort of second

stage of an incident, which maybe is like follow ups. Or

maybe you're still kind of in the incident mode where nothing

is on fire anymore. But there's a lot of things that you could

do that would make it less likely to happen or resolve it

in a neater way. And that's where you need your strong

engineers to come in and make those trade offs. And it's like,

what is the value of this piece of work? How long is it going to

take us? How much in the wrong direction? Is it from what we

thought we were doing? And can we afford to punt it? Can we

afford to delay it? And you get to this point where you've got a

deadline, people are set of problems. And you need to make a

decision about basically which is more important. And that is a

that is a strange shootout trade off often because it's like, we

have four people in our team, we have two weeks, what shall we

do? And the answer to that is not worth 60 Nowadays, because

in all likelihood, you won't get anything more than in my

experience. And so instead it has to be right which of these

which of these is more of a risk to us? What happens if we missed

the deadline. And that's a decision that needs to get

escalated to someone who has the authority to make that call and

the information. So that person that means that you have to make

that information really available to them in terms of,

you know, what, what is the work that we could do to mitigate it?

What would it be mitigating? And versus how far behind? Are we on

deadline? What does it mean, if we don't get the deadline? And

that is one of those. I think the lots of people have this,

oh, we'll find a creative solution. And sometimes there's

a creative solution and somebody's overskirt something

and actually, it's all gonna be fine. And sometimes there isn't.

There isn't enough time, and you have to pick something. And I

think identifying that is really important and being really

honest, from a kind of motivation and human point of

view. I think the the times where I've seen that go badly is

when people either kind of try and have their cake and eat it

and sort of say, Oh, I know you said you can't do these two

things. But what if I told you you could. And then there's

another problem where there is a trade off and nobody makes the

decision. And then you just end up in a situation where both

like the team is all kind of looking at each other being

like, do we make the decision now because we don't think it's

our choice, but but I guess no one's telling us what to do. And

then somebody shouts at them afterwards because they made the

wrong decision.

Jason Baum: I think that leads us right to my next question.

It's what can the leaders do to make this mindset part of the

culture of the organization and encourage it across all teams.

Lisa Karlin Curtis: So I think the first thing to look for is

look out with anybody playing the hero. In all organizations

that I've ever seen, there are a group of people who take on more

than their fair share of the burden of dealing with

incidents. And we could call them heroes. And that is really

good until it's really bad. And it's good, because they're

probably very good at dealing with incidents, because they've

had a lot of practice. And they're often the people who've

been at the company for a long time. But it's bad because it

means that nobody else is learning how to do it. And so

all of those benefits that we talked about right at the start

on, no one else is getting that. So they're kind of gatekeeping,

the skill needed to debug these issues. And that's very

problematic, because it stunts other people's growth. And when

that person burns out, or when that person goes on holiday, or

when that person leaves the company, you're suddenly in a

really bad situation. So you end up with these really bad key man

dependencies. And as a leader, I think it's really important to

identify those patterns. And if you're, if you're using tooling

you can look at who's answering who's getting paged, who's

taking your on call load, you can look at your incident, who's

leading your incidents, you know, have you got somebody

who's leading 50% of your incidents? That's probably not a

good sign. And you can use that data to find those people and

then chat to them Be like, why are you doing that? And probably

the answer is, well, because I think it's useful. And that's

like, great. But now we're gonna have a conversation about why we

need to spread this load out of the team. And it's a combination

of like protecting you and your mental health, frankly, but

also, it's about spreading the knowledge and spreading the

experience. So I think that that kind of pattern of having those

superheroes is really damaging. And it restricts your ability to

scale your incident response. And then, as a leader, the other

things you can do is encourage people to show that working. So

if you want people to learn from incidents you to make that

information available to them. And then you need to make it

accessible by which I mean available is like, have your

conversations in a public Slack channel. Ideally, use some

incident tooling so that you can curate those conversations and

build a timeline that somebody can interact with, write a post

mortem, share the post mortem. And then by accessible, I mean,

try and make it really easy for people to get that information.

So have it in a knowledge base that people can look at to find

something that they're interested in. And if you're,

it's like step one, write the thing. But if you just write a

post mortem that goes into draw, no one's really one at that

point. So push it out to people and make it clear to people that

reading those materials is part of their job and considered a

very good use of their time. And that's a difficult balance,

because there are some, sometimes you need people to

ship stuff. But I think it's important to talk about this

explicitly, and talk about the fact that if you look at what

other people did in their incidents, you can build better

software, you're going to have less, fewer incidents or less

fewer, you're gonna have fewer incidents, or your incidents are

going to be less severe and easier to debug. And that then

means that you'll be able to sort of teach the next

generation to the next generation, and you get this

great positive feedback loop, if everybody's talking about it,

and learning from each other, as opposed to the negative feedback

loop, where people are keeping it very secret where people are

gatekeeping it and where not everyone is getting involved.

Jason Baum: I feel like the word of the day, the word of the day

is transparency. I feel like that that's pretty much what

you're saying. Not like not to put a word in your mouth,

because I don't think you've said it specifically. But what

I'm hearing is transparency, transparency, and transparency.

As someone who works with the engineering team, or you know,

I'm just relaying information from people to people, on a on a

leadership team, for example, and half the time with a

deadline. It's because the leadership doesn't necessarily

understand it. You know, they they don't necessarily know,

what is the specific issue? What is the specific reason why a

deadline isn't being hit? Or? Or, you know, and that's where I

feel like the culture part can sometimes go awry, right. We

allow it to happen when there isn't transparency.

Lisa Karlin Curtis: Yeah, I think that when, when you lack

transparency that gives you it gives humans a lot more remit to

try and get what they want from bad, bad ways, basically. And if

you don't have transparency, you can lie. And you can put forward

an argument that suits whatever you think your goal is. And in

an ideal world, everybody in your organization has exactly

the same goal, and they're all pulling in the same direction.

But in reality, often people view their goals as being

slightly like in conflict with other people's goals, because

they're trying to get more resources for their team,

because they think their problem is the most important thing. And

if you if you don't have transparency, it's very

difficult to hold people accountable to those things.

Whereas if you do and if people are very honest and open, then

you The organization can make the right choice for the

organization. And if you think about a sort of individual first

versus organization, first type culture, ideally, as an

organization, you should be putting your chips and the thing

that is most important, not on the thing that has the best

argument. And the way that you don't make that mistake is to

use transparency, and to be open and honest and like generate

that culture and generate the culture of it being okay to make

mistakes. And also it being okay to say, I don't think this is so

important. And that's not know that that's not gonna impact

your career. And I think that, that's one of the reasons why

people don't do that. If because there is this view that like to

get promoted, or to get that job that you really want to get

influenced, you need to be the most important person, you need

to be doing the most important thing. And none of us are always

doing the most important thing. i This week I work I'm not doing

the most important thing that is very clear. And I and that's

kind of fun. But it also means that if somebody else needs an

extra pair of hands, I'm going to jump on their thing I'm not

going to just pursue with mine, because I want to look good. And

that's the sort of team first thinking that I think you need

to try and get into your cultural DNA as an organization.

Jason Baum: I love that I would love to hear in a in a team

update with the company. This week, I'm not working on the

most important thing. You know, I don't think we ever hear that.

Because everyone who does want to be the most important, I

think. So, you know, you you said accountability. And I

hadn't planned on asking this question, but it does now

trigger something in me. Incidents are okay. And they are

learning experiences. We just spent the past 30 minutes

talking about it. But when is how does accountability play

into this? When our incidents? Not that they're not okay. But

when do we need to hold accountability? Because I think

there needs to be an element of accountability. How does that

play in? Especially in a blameless culture,

Lisa Karlin Curtis: I think it's a really difficult balance. And

I think I'd come back to the stuff I was saying about failing

together. So I think that you, you can, as a team, you take

accountability for what happens in your team. There are

obviously occasions where somebody goes rogue and does

something that the team thinks was terrible idea that that's

sort of a HR issue, frankly, and I think it's very separate. But

generally, it's you as a team have made some some choices. And

you're now looking at the consequences of those choices. I

think that the way to hold teams accountable around incidents is

the same way that you hold teams accountable for any kind of

delivery. So if you imagine an incident is normally something

that that team maintains, has broken in some way. And so that

team has a kind of agreement or a contract with the rest of the

org that they will have this service and it will do this

stuff. And sometimes it won't, because incidents happen and

mistakes happen. I think you make people accountable by by

making them transparent, and by making them expose the trade

offs that they're making. And so as an example, if you're in a

team, which is under loads and loads of pressure for time, and

they're like, you've got to shut this thing as quickly as you can

as quickly as you can, if you as the as the senior engineer, or

the tech leader are looking at them saying, cool, we'll do

that. But there's risk. And these are the things that might

go wrong. If we do that, are we comfortable with that risk? Then

you're accountable. Because if something goes wrong, it's

either like, yeah, I said, these things will go wrong. And we

talked about it. And we decided we were okay with the risk, or

it's something completely different has gone wrong, that

is maybe significantly more severe than the things that we

thought my right let's talk about why we thought this was

safe. And why we thought why that wasn't in our risk

assessment. And so you're you're holding people accountable for

the trade offs that they're making. You're not holding

people accountable for an individual thing that went

wrong. It's not like why did this happen? It's why did you

think that we should take this risk, what pressure was put on

you, and let's look at it like at a system level, as opposed to

some person pressed a button on a backfill, and it made the

database really sad. And now we're going to run around

screaming saying that they should be fired. And I think the

other thing about accountability is it's about time. So I think

incidents are generally a lagging indicator, as opposed to

a leading indicator, if that's terminology people are familiar

with. leading indicator basically means you find out

pretty quickly whether a choice you're making is good or bad.

And a lagging indicator is something where a choice that

you make has impact sometime in the future. And because

incidents are a lagging indicator, often the people

handling the incidents are not the people who made those trade

offs. And that's really important to recognize when that

is true. And to understand what is the what are the root causes

to have that kind of discussion whether whether you go down the

five why's route or some other route But to have a really

meaningful discussion about what were the choices we could have

made to avoid this. And why didn't we make them? Was it

because we had loads of pressure on delivery? Was it because no

one was thinking about it, and we just didn't think it was a

risk. And then that's the problem that you have to solve.

And those are the things that you can hold people accountable

for think.

Jason Baum: Awesome. Thank you for answering that. One that I

don't know. It just came when you said accountability. It's it

just popped into my head because I feel like all we've ever heard

for years was accountability, accountability, who's

accountable for this? who's accountable for that? All the

accountability, stuff that people say? And, yeah, it's hard

to be blameless, when, when that's what the buzzword was

before blameless. So now, we're at the point of the podcast,

where I like to ask sort of a fun question of you personally,

because this is the humans of DevOps, and we're all about the

humans. So if there was one thing that you could be

remembered for, what would that be?

Lisa Karlin Curtis: I think I would like to be remembered as

someone who he made systems work better for people.

Jason Baum: Great. I think that's, that's certainly

applicable for today's conversation. Well, thank you,

Lisa, so much for joining me today. It was an absolute

pleasure.

Lisa Karlin Curtis: Thanks so much. I really enjoyed it. And

thank

Jason Baum: you for listening to this episode of the humans of

DevOps Podcast. I'm going to end this episode the same way I

always do encouraging you to become a member of DevOps

Institute to get access to even more great resources just like

this one. Until next time, stay safe, stay healthy, and most of

all, stay human, live long and prosper.

Narrator: Thanks for listening to this episode of the humans of

DevOps podcast. Don't forget to join our global community to get

access to even more great resources like this. Until next

time, remember, you are part of something bigger than yourself.

You belong