Security and availability incidents share many similarities, but also have key differences that, if understood and addressed, can significantly improve your incident management.
Similarities:
- Both types of incidents require forming and coordinating ad hoc teams of responders.
- Challenges include identifying the right responders, fostering effective collaboration among them, and keeping stakeholders informed.
Key Differences:
- Duration: Availability incidents typically get resolved in hours (e.g., when I led incident management at Slack, about half of our availability incidents were resolved in under 2 hours, and two-thirds in under 4 hours). Security incidents often last days or weeks.
- Secrecy: Availability incidents are usually handled openly within a company. Security incidents are often handled secretly, especially if an intruder might be monitoring communications.
Ideally, a single incident management process and unified tools should be used for both availability and security incidents. You don’t want your engineering teams to have to learn two different processes and tool sets for the different types of incidents. This requires adapting the process and tools to accommodate the differences, though:
- Extended Duration of Security Incidents
- Develop robust handoff procedures for responsibilities and information as responders transition in and out over time.
- Refine methods for providing ongoing progress updates, as a single resolution summary at the end of the incident (days or weeks later) will not suffice.
- Secrecy/Visibility of Security Incidents:
- Enable incident commanders and responders to collaborate securely while keeping details (or even the incident’s existence) confidential from those without a legitimate need to know.
- This may involve using restricted communication channels (e.g., a private Slack channel instead of a public one) and tightly controlling access to documents (e.g., access permissions in Google Drive/Docs/Sheets).
One tip: consider sharing “placeholder” information about security incidents publicly within the company. Even if details are secret, acknowledge the incident’s existence and direct inquiries to a point of contact (e.g., the Incident Commander or a senior security manager). This can be posted in the incident’s automatically created public Slack channel while responders work in a parallel private channel. However, this approach is not suitable if the mere existence of the incident must be kept secret (e.g., when chasing an active intruder who may have access to your Slack workspace). For most security incidents, even simply acknowledging the incident can address many concerns and improve security engagement within the company.
Finally, even if the security incident needs to be kept secret while it is underway, don’t assume that it must be kept secret forever. There might be valuable lessons to learn from broader visibility once the incident is over.
—
IT incidents can be incredibly costly for you with both your customers (resulting in lost revenue, missed sales, and damaged reputation) and your staff (resulting in decreased productivity, reduced morale, and increased turnover). That’s why it’s crucial to be proactive, prepare for future incidents, and learn as much as you can from every incident. As an expert incident management consultant, advisor, and coach, I can guide your organization in developing these critical skills and help you avoid expensive mistakes. Contact me today to learn how I can help your team.
Recent Comments