Most companies’ incident processes work well for routine incidents. An engineer gets paged, a few colleagues join the response, someone takes on the incident commander (IC) role, and the team works the problem together in a Slack channel. The IC can keep track of what everyone is doing because “everyone” is five or six active responders. Communication flows naturally. The process feels lightweight because it is.

The first scaling step: naming a tech lead

The first time this setup gets strained is usually when the IC has too many things competing for their attention. They’re trying to guide the technical investigation while also fielding questions from stakeholders, coordinating communication, and keeping the response organized. The IC’s attention fragments, and both the technical work and the coordination suffer.

The answer at this stage is naming a tech lead (TL) for the incident: someone who takes over direct management of the technical work while the IC handles everything outward-facing. Together, an IC and TL can effectively coordinate more responders than either could alone, because they’ve split the two biggest demands on attention into separate roles. Adding a TL helps you scale beyond what the IC alone could handle, but it only takes you so far.

When even IC and TL aren’t enough

When an incident has enough active responders working on different aspects of the problem simultaneously, even an IC and TL working together can’t directly coordinate all of them. The single channel becomes congested with interleaved conversations about different problem areas. Responders spend more time trying to follow the scroll than working the problem. People start working at cross purposes because nobody has visibility into what other responders are doing.

To be clear: it’s fine to have dozens of people watching an incident channel. Spectators following along for situational awareness is a sign of healthy transparency, not a coordination problem. The challenge is the active responders, the people actually working the incident, and what happens when there are more of them than an IC and TL can directly coordinate.

What experienced responders do next

In many companies, the experienced responders sense this breakdown before anyone names it. They’ve seen it before. They get frustrated with the chaos, quietly break off into smaller groups, and start working the problem in DMs, side huddles, or breakout channels where they can actually focus.

What they’re doing, whether they’d use the word or not, is forming workstreams: small teams organized around specific aspects of the problem. For example, in an incident involving data corruption across multiple systems, one workstream might focus on stopping the corruption, another on assessing the blast radius, and a third on coordinating customer communication. Each workstream has a natural focus and can work without wading through the noise of the other workstreams’ conversations.

This instinct is sound. The main channel has become too congested for focused technical work, and breaking into smaller, objective-focused workstream groups is exactly the right response. But when it happens informally, it creates its own problems. The IC and TL may not know the workstreams have formed. There’s no explicit coordination between the workstreams. Information that matters to one workstream gets trapped in another workstream’s side conversation. And at its worst, it shades into freelancing: experienced people working on what they think is most important, outside the IC’s and TL’s awareness, with nobody having the big picture across the effort. The whole thing only works when those particular experienced people happen to be responding. When they’re not, the incident stays in the chaotic single-channel mode, and the response suffers for it.

Making the instinct explicit

The fix is to formalize what experienced responders already do naturally. When an incident grows beyond what an IC and TL can directly coordinate (roughly eight to ten active responders, in my experience), break the response into named workstreams, each with a designated workstream lead who coordinates the work within that workstream.

The TL’s role shifts at this point. Instead of directly managing all the technical responders, the TL coordinates across workstream leads: connecting dots between workstreams, making sure one workstream’s fix isn’t creating problems for another, and maintaining the technical big picture that no single workstream can see. The IC continues to handle organizational coordination, stakeholder communication, and the overall direction of the response. It’s the same IC/TL partnership, scaled up one level.

One detail matters more than it might seem: name workstreams after what they’re trying to accomplish, not which team the people came from. “Stop the data corruption” is a workstream objective. “Database team” is an org chart label. In a complex incident, the work rarely falls along team boundaries; the whole point of the incident response program is to be able to form an ad hoc team to address the problems. A workstream focused on stopping data corruption might need engineers from the database team, the networking team, and the application team all working together. Naming by objective keeps everyone oriented toward the same goal and avoids the trap of siloing along reporting lines when the problem doesn’t respect those lines.

How fire departments scale their coordination

Fire departments face the same scaling challenge, and the way they handle it is instructive. The Incident Command System (ICS) explicitly defines how the organizational structure scales with the size of the incident. A single-engine response to a dumpster fire has one officer managing a 3-4 person crew. A multi-alarm apartment building fire has an IC at the top of an org chart spanning multiple engine and truck companies, rescue teams, ambulances, and other specialized units organized across multiple floors and faces of the building. As units arrive to join the response, the organizational structure scales up in well-defined ways to absorb them, and firefighters train for these transitions before they ever face a live fire.

The specific structure from ICS doesn’t translate directly to software incidents (we don’t need strike team leaders, for example). But the principle does: the coordination model must evolve in predictable, preplanned ways, and people need to practice making those changes before they’re in the middle of a crisis.

Getting started with workstreams

Most companies design their incident process for the incidents they have most often, and that’s reasonable. Routine incidents with a handful of responders don’t need workstream coordination. But if you’ve ever had an incident where the response felt scattered and chaotic, where experienced people quietly broke away to work independently, and where nobody had the big picture across all the parallel efforts (or even knew what all those efforts were), the issue was probably that your coordination model didn’t scale with the incident.

The shift to workstreams doesn’t happen all at once in an incident. When the IC or TL recognizes that the response has outgrown direct coordination, they might start by spinning up a single workstream for the most clearly defined problem area, while other responders continue working in the main channel under the TL’s direct coordination. As more distinct problem areas emerge, more workstreams form. It’s a gradual transition, not an instant cutover. And the reverse applies as well: as workstreams accomplish their objectives, they can be dissolved and their responders absorbed into the remaining workstreams or stood down. The coordination structure should contract as the incident winds down, just as it expanded when the incident grew.

A few practical starting points: when you spin up a workstream, give it its own channel so the work is visible, not DMs or side huddles where it disappears from the IC’s and TL’s view. Name it by its objective. Each workstream should have an identified lead who coordinates the work within the workstream and reports back to the TL. The TL connects the dots and coordinates across workstreams, making sure one workstream’s approach isn’t undermining another’s. And just like with Slack threads, key discoveries and decisions made within a workstream need to get shared back to the main channel immediately, so the IC, the TL, and other workstreams all have the full picture.

There’s more to managing workstreams well than a blog post can cover: how workstream leads communicate status, how the TL balances attention across workstreams, how to handle responders who need to move between workstreams as the situation evolves. I cover these in a chapter on large, long-running, and other special-situation incidents in my forthcoming book, Incident Management for DevOps and SRE. Sign up for publication updates at im4ds.com. If your company needs help building this into your incident process right now, my consulting practice is greatcircle.com/im.

The experienced responders on your team already instinctively self-organize when an incident gets big. Your process should be supporting and channeling that instinct into something reliable and repeatable, not leaving it to chance.