Lightnews — Scholar-powered news

John Allspaw @allspaw.bsky.social · 5d

Does anyone have first-hand experience with using an AI agent that has “participated” in the response to an actual incident in a genuinely diagnostic — or otherwise contextually-specific way?

2 3 3

John Allspaw @allspaw.bsky.social · 5d

Comedy is about taking chances, Av. You took a chance.

1 6

John Allspaw @allspaw.bsky.social · 6d

There are a couple of phrases I genuinely despise. One of them is "...we're at an inflection point..."

1. That's not how inflection points work.
2. Hindsight's a helluva drug, isn't it?
3. Just say you are at a loss to explain some recent surprises. It's honest and not unnecessarily dramatic.

2 9

John Allspaw @allspaw.bsky.social · 11d

@staysaasy.bsky.social I’m not on X very much, finally collected notes on the post docs.google.com/document/d/1...

Misunderstanding the blameless postmortem post

Accountability vs. Punishment The biggest misunderstanding is that it seems like you’re conflating "blameless" with "no accountability." Your post claims that blameless cultures mean "people believe...

docs.google.com

7

John Allspaw @allspaw.bsky.social · Sep 3

Hi, 👋 that right there, is me.

2

John Allspaw @allspaw.bsky.social · Aug 21

For leaders with expectations about how more productive, efficient, etc. the software engineers in their org will be with AI:

1. How are you handling the "Left-Over Principle" challenges?

2. Also: customer comms about the incidents involving code produced AI?

(Seems clear #2 is depends on #1)

1 6

Reposted by John Allspaw

Woooooo Ghoul-lego @gallego.bsky.social · Aug 1

Another real life demonstration of Ironies of Automation

Sleepin Dogo @sleepindogo.bsky.social · Jul 27

This is pretty funny. The teleoperated Optimus popcorn server appears to have a full-time sweeper. How efficient....

1 4 18

John Allspaw @allspaw.bsky.social · Jul 30

2 4

Reposted by John Allspaw

Lorin Hochstein @norootcause.surfingcomplexity.com · Jul 28

Incidents happen because people do things that have always worked successfully, up until the incident. Doing something that always worked in the past is completely rational!

1 5 16

John Allspaw @allspaw.bsky.social · Jul 22

Come on, @gergely.pragmaticengineer.com. You and I talked about the hindsight bias trap on your podcast. Don’t also fall for this trap as well:

www.adaptivecapacitylabs.com/2021/08/22/w...

What makes public posts about incidents different from analysis write-ups

www.adaptivecapacitylabs.com

13

John Allspaw @allspaw.bsky.social · Jul 19

Yes, and when inevitably: “Oh…huh…whoa, how come you’re doing it like that?”

They’ve got a story for you. 😀

3

John Allspaw @allspaw.bsky.social · Jul 18

As if there was any other route!

2 6

John Allspaw @allspaw.bsky.social · Jul 10

I'm now talking with Hamed Silatani, Alex Hibbitt, and Beth Adele Long about the various Catch-22 situations leaders find themselves in when responding to incidents!

www.linkedin.com/events/incid...

[Incident Fest] - The Catch-22 of Executives in Incidents | LinkedIn

Join us for a short-but-sweet live session that focuses on the role of executives during incident management. Although the session is prerecorded, our experts will participate in a Q&A in the comment...

www.linkedin.com

1 3

John Allspaw @allspaw.bsky.social · Jul 9

Watching www.youtube.com/live/Uxe_mw8...

[Incident Fest] - What Does Great Incident Response Look Like?

YouTube video by Uptime Labs

www.youtube.com

1

Reposted by John Allspaw

Lorin Hochstein @norootcause.surfingcomplexity.com · May 16

New blog post: surfingcomplexity.blog/2025/05/15/l...

Labeling a root cause is predicting the future, poorly

Why do we retrospect on our incidents? Why spend the time doing those write-ups and holding review meetings? We don’t do this work as some sort of intellectual exercise for amusement. Rather,…

surfingcomplexity.blog

2 11

Reposted by John Allspaw

Lorin Hochstein @norootcause.surfingcomplexity.com · May 10

New blog post: surfingcomplexity.blog/2025/05/10/w...

When a bad analysis is worse than none at all

One of the most famous physics experiments in modern history is the double-split experiment, originally performed by the English physicist Thomas Young back in 1801. You probably learned about this…

surfingcomplexity.blog

3 9 16

John Allspaw @allspaw.bsky.social · Mar 18

1. PEOPLE keep things working.
2. When things break down, PEOPLE work to make the consequences much less than they might have been otherwise.

Both dynamics are, for the most part, invisible to management.
(via @lauramaguire.bsky.social's dissertation)

As Richard Cook notes, in reflecting on the data collected from the SNAFU Catchers Consortium:
"It's hard to know who is monitoring the system, who is looking at things, and who is available.
We know for certain in (some organizations) that there is a pretty well defined group of people who are in high-frequency update mode more or less continuously while awake. This is a huge contributor to reducing the costs of coordination and it is invisible to everyone and therefore not on any dashboard." (Cook, personal communication)

2 16 62

Reposted by John Allspaw

Erika Rowland @erikareads.bsky.social · Mar 7

I recently read the paper "Towards Joint Activity Design Heuristics: Essentials for Human-Machine Teaming" which I loved so much I wanted to make it easier to share. To that end, I've excerpted the Ten Heuristics from the paper here: human-machine.team with anchors for each heuristic.

Ten Machine Requirements To Satisfy Essentials Of Joint Activity

human-machine.team

10 16

Reposted by John Allspaw

John Allspaw @allspaw.bsky.social · Feb 25

What a great podcast! Honored to be talking about Resilience Engineering with @colettecello.bsky.social and @spamaps.org!

This is Fine! @thisisfinepod.com · Feb 25

@allspaw.bsky.social joined us last week to help contrast ITIL's approach of "Counting and tabulating incidents" with the resilience engineering way of looking directly at the messy reality underneath them.

www.youtube.com/watch?v=cimc...

Episode 10 - When They go Full ITIL on You w/special guest John Allspaw

YouTube video by thisisfinepodcast

www.youtube.com

1 7 16

Reposted by John Allspaw

Adaptive Capacity Labs @adaptivecapacity.bsky.social · Mar 2

Two years ago, at the first LFI Conference, we spoke alongside a client of ours (Indeed) about the amazing progress they had made in learning effectively from incidents.

This is what "good" looks like with respect to learning effectively from incidents.

www.adaptivecapacitylabs.com/2025/02/28/w...

What Progress In Learning From Incidents Actually Looks Like

www.adaptivecapacitylabs.com

1 3 5

Reposted by John Allspaw

This is Fine! @thisisfinepod.com · Feb 25

@allspaw.bsky.social joined us last week to help contrast ITIL's approach of "Counting and tabulating incidents" with the resilience engineering way of looking directly at the messy reality underneath them.

www.youtube.com/watch?v=cimc...

Episode 10 - When They go Full ITIL on You w/special guest John Allspaw

YouTube video by thisisfinepodcast

www.youtube.com

3 4

Reposted by John Allspaw

mharrigan @mharrigan.bsky.social · Feb 25

Preach! youtu.be/cimcogNc02I?... 13:30-16:30

1 1

John Allspaw @allspaw.bsky.social · Feb 25

What a great podcast! Honored to be talking about Resilience Engineering with @colettecello.bsky.social and @spamaps.org!

This is Fine! @thisisfinepod.com · Feb 25

@allspaw.bsky.social joined us last week to help contrast ITIL's approach of "Counting and tabulating incidents" with the resilience engineering way of looking directly at the messy reality underneath them.

www.youtube.com/watch?v=cimc...

Episode 10 - When They go Full ITIL on You w/special guest John Allspaw

YouTube video by thisisfinepodcast

www.youtube.com

1 7 16

Reposted by John Allspaw

Erika Rowland @erikareads.bsky.social · Feb 21

The Resilience in Software Foundation blog included notes I wrote for the paper "Four Concepts for Resilience Engineering" in their post today: resilienceinsoftware.org/news/1149720

Three Takes on Four Concepts for Resilience Engineering

Ed note: The first time I read Dr. David Woods' paper Four Concepts for Resilience Engineering, I felt so many things click in my brain. While the field of Resilience Engineering is not new, those of ...

resilienceinsoftware.org

8 16

John Allspaw @allspaw.bsky.social · Feb 12

Very few know we've (@adaptivecapacity.bsky.social) been developing tools to help support us. After 7 years & 6 patents (!) we realized others were interested in integrating/commercializing "Churchkey."

We’re looking for partners interested in licensing Churchkey's IP. churchkey.info

4 16