incident.io

Your engineers know what's slowing them down. Proving it is the hard part.

@TrueLayer used incident.io to track why every incident happened. That evidence got reliability work onto the roadmap, and cut a whole category of incidents by about 70%. 👇

incident.io/customers/true…

AI tooling is becoming critical infrastructure. But are we treating it like it is?

On Oct 14, we’re talking with Skyscanner about what happens when models fail, get rate-limited, or change behavior, and how engineering teams can prepare.

Join us live: incident.io/events/how-do-…

If you’re going to stare at incident data, it should at least look good. 📊

Custom dashboards just got a big upgrade: build your own charts, rearrange layouts, group data, add rich text, customize colors, and filter down to exactly what you need.

Reliability gets harder when nobody knows who owns what.

Very excited to have @GetCortexApp sponsoring SEV0 in London + SF. 🔥

They’re helping teams make sense of services, ownership, standards, and operational health. A natural fit for the SEV0 crowd.

#SEV02026

AI RCA gets interesting when it stops being a very confident guess.

A useful hypothesis needs evidence: what broke, why, confidence, and the sources behind it.

Then have another agent try to prove it wrong.

For production AI, knowing why it believes something matters.

Pssst... we’re giving away free tickets to SEV0 in SF. 👀🔥

SEV0 is the reliability conference, exploring the impact of AI on incident response with engineers from @netflix, @OpenAI, @RobinhoodApp, @WorkOS + more.

Grab your free ticket: go.incident.io/ncdqna7

The promise of AI for incidents isn’t finding signals, but connecting them.

A PR in one repo. An error in another service. A recent deploy.

Humans hunt for those relationships manually. Agents can follow the threads across systems and reason about how they fit together.

Natural language might be the most interesting new interface for observability.

Engineers shouldn’t need the right query syntax while prod is burning. 🔥

Ask what you need to know. Let an agent query the telemetry, pull the evidence, and follow the thread.

🫶

There’s a big gap between AI getting it right and trusting it to get it right.

On Sept 17, go behind the scenes of Investigations: how Nexus reasons across signals, challenges its own conclusions, and what we’ve learned building AI for incident response.

incident.io/events/inside-…

Finding an anomaly is one thing. Knowing if anyone should care is harder.

AI for incidents needs to reason about blast radius: which services are affected, how far the failure spread, and how serious it is.

Detection tells you what changed. Context tells you what it means. 💥

The interesting part of AI for incidents isn’t explaining what broke. It’s deciding what to do next.

If AI can reason across code, telemetry, deployments, & incident history, the output shouldn’t stop at a diagnosis.

It should recommend the next move.

@turbopuffer's billing service started running out of MySQL connections, and no one knew why.

Investigations found the change and the engineer with the most context.

A correlation that could take a human hours, found in minutes.

Read more: incident.io/customers/turb…

One failure is manageable. A chain reaction? That’s where things get interesting. 👀

At SEV0 London on Oct 20, Sam Newman is unpacking progressive collapse: how failures cascade through distributed systems, & how to stop them.

Get your ticket: sev0.com

#SEV02026

Modern systems don't fail in one place.

Understanding what changed often means connecting signals across repositories, services, deployments, and telemetry.

See how it works: incident.io/investigations

Introducing Investigations.

Agentic root cause analysis tackles the hardest part of an incident, getting you from alert to resolution faster.

Hypotheses backed by evidence, conclusions supported by reasoning, and Nexus helps get smarter over time.

🔥incident.io/investigations

Every fast-growing company usually ends up rebuilding the same enterprise features from scratch.

Michael Grinich founded @WorkOS to solve this, powering enterprise readiness for hundreds of leading companies.

He'll be at SEV0 SF, Oct 27: sev0.com
#SEV02026

Our database got so fast it started gaslighting itself.

Workers could publish + process faster than our replica could catch up, so we got phantom 404s for stuff that definitely existed.

LSN stamping + nack-and-retry fixed it. Primary CPU dropped by half: incident.io/blog/dont-add-…

A satellite that won't respond is a different kind of incident. 🛰️

Orbital Sidekick's GHOSt-1 launched with a funding round riding on the first images. It couldn't transmit them.

Andrew Guenther explains at SEV0 London, Oct 20 🔥

Register: sev0.com

#SEV02026

Go with the flow: an afternoon on the Thames with engineering leaders.

The other week, we joined a small group of engineering leaders at the Henley Royal Regatta for afternoon tea and a boat down the course.

Enjoyed getting to know people away from the pager for a few hours!