Don’t restrict your incident commander pool to engineering managers

Many companies build their incident commander pool from their engineering managers and stop there. The assumption that the role belongs to engineering managers often goes unquestioned, but it limits the pool unnecessarily. The incident commander is a subject matter expert for a very special subject: how to run an incident. That expertise isn’t limited to engineering managers; some of the best incident commanders I’ve worked with were individual contributors, or from program management or customer care.

What the incident commander actually needs to know

Being an effective incident commander means knowing roughly how the systems fit together and when to ask a clarifying question. They need to be technical enough to follow the discussion at a high level, but that’s sufficient; they don’t need to be able to argue technical points with the subject matter experts.

The incident commander’s most valuable knowledge is organizational. Which teams own which components and services. Who to page for what. How decisions actually get escalated in the organization, as opposed to how the documentation says it’s supposed to work. Who the right liaison is for legal or support or a key account. What the company’s business context is this particular week: Is the annual user conference next week? Is a key customer in the middle of a renewal negotiation? These considerations shape the priorities, the urgency, the communications strategy, and the decisions about acceptable tradeoffs. That’s the incident commander’s territory, and it depends on knowing the organization, not the code.

In fact, having deep technical expertise in the affected system can work against an incident commander. The temptation to get down in the weeds with the tech lead and the subject matter experts is real, and the moment the incident commander starts troubleshooting, they stop coordinating. Status updates stop going out. Incoming questions from executives go unanswered. New responders arrive and don’t know who to check in with. An incident commander who isn’t a deep expert on the broken system doesn’t have that temptation. They stay in the command role, which is exactly where they should be.

And a non-expert can ask the questions that everyone else is too close to the problem to think of. “How does this system normally work?” “What changed recently?” “Have we seen anything like this before?” “Who else should we loop in?” Those sound basic, but they routinely surface information that the experts have been taking for granted, and they help everyone following along understand what’s happening without having to interrupt and ask.

This isn’t a new idea. In fire departments, the incident commander is an officer standing back from the scene, not someone on a hose line; their key expertise is commanding the incident rather than performing the tactical work. Once you recognize that the same separation applies in tech, the incident commander’s skill set turns out to have very little to do with engineering.

Who to recruit

Facilitation, communication, tracking multiple workstreams, and deciding under pressure are the incident commander’s key work during an incident. And those are daily skills for several non-incident roles that most companies never think to recruit incident commanders from.

For example, program managers and technical program managers coordinate cross-functional efforts for a living; they track multiple workstreams, communicate status to stakeholders, and navigate organizational complexity. Customer care leaders know the escalation paths from the other side, have strong communication instincts about customer impact, and are often better at stakeholder updates than engineers because talking to customers is what they do every day.

These folks bring something else, too. They aren’t as tempted to jump in and start troubleshooting, so they stay more firmly in the incident commander role. Their focus doesn’t drift to the technical details. And because cross-team coordination is their daily work, they often know the organizational map better than the engineers do: how the organization fits together, who to page for what, how to get the right people engaged and keep them working together. That organizational knowledge turns out to be among the incident commander’s most important assets.

Even within engineering, the incident commander doesn’t have to be a manager. A non-manager engineer with strong facilitation and organizational skills can be an excellent incident commander, and they’re often more available than their managers are. The title on the org chart isn’t the qualification; the skills are.

At companies where I’ve helped expand the incident commander pool beyond engineering managers, the results have generally been positive. That’s not a knock on engineering managers in the role; some of them are excellent incident commanders. The point is that the pool should be wider than engineering managers alone.

The ask is real

Expanding the incident commander pool beyond engineering managers isn’t free. These folks are joining an on-call rotation, and that’s a genuine ask. They need training, practice, and a steady flow of real incidents to stay sharp. The company needs to compensate and recognize the work the same way it does for engineers on call, which means incident commander service shows up in performance reviews and career development conversations, not as a side project that nobody notices.

There’s a prerequisite to making this work: a separate tech lead role within your incidents. If your incident commander is also your technical decision-maker, expanding the pool beyond engineering managers doesn’t work, because the incident commander would need deep technical expertise after all. Fix the role separation first, then widen the pool. I’ve written elsewhere about why that separation matters.

Start recruiting

Many companies restrict the incident commander pool to engineering managers without ever making a conscious decision to do so. It’s just the default, and nobody questions it. If the skills that matter most for incident command are facilitation, communication, organizational knowledge, and decision-making under pressure, then some of your strongest candidates are probably in your program office, your customer care organization, or on your own engineering team in non-management roles. Go talk to them. You’ll never know until you ask.