Incident Resolution

The Incident Resolution workflow is an installable workflow template designed to help you tailor xMatters Incident Management to your incident resolution process. It starts as the same workflow used by default for the Incident Management feature, but lets you adapt how incidents are created, what happens when they're created, and the notifications that are sent to resolvers when the incident is initiated and when you notify additional resolvers to engage.

This customizable workflow allows you to do the following:

  • Add collaboration channels to incidents.
  • Include automated steps in the incident resolution flow to create incidents or send incident information to other systems.
  • Customize the settings for new incidents.
  • Tailor notifications sent about the incident when it is initiated, including information from other tools that might help resolvers triage and address the issue.
  • Tailor notifications sent about the incident when you notify additional resolvers to engage, including current details about the incident its collaboration channels.
  • Allow your team or your tools to initiate incidents using email or an HTTP trigger.
  • Control who can initiate incidents with the workflow.

For more information on initiating and working incidents, see our help on the Incidents list and Incident Console.

Use the workflow

After you install the workflow, anyone you give sender permissions to the flow trigger form can use it to initiate incidents from the Incident Console and the Initiate Incident dashboard widget. If you enable the flow trigger form in the mobile app, it will also appear as an option for initiating incidents in the xMatters mobile app. If you have the Slack integration set up, you can initiate incidents using the /xmatters initiate command. If you have the Microsoft Teams integration set up, you can initiate incidents within a chat or Teams channel.

Once you select the incident you want to initiate, a form appears where you can fill in the incident details. Here's what this looks like in the Incident Console:

Learn more in our topic on initiating incidents in the web user interface, or learn about changing the trigger for automatic incident initiation.

Configure the default workflow

The default workflow allows you to make customizations to the xMatters Incident Management workflow.

The workflow includes a flow trigger form that lets you initiate an incident by submitting the form. The only modifications you can make to the flow trigger form are whether it's enabled and who has permission to use it.

You can customize the 'New Incident Notification' Create Alert step to modify the notifications sent to resolvers when the incident is initiated. You can also customize the 'Engage Additional Resolvers' Create Alert step associated with the Notify to Engage trigger to modify the notifications sent when you 'Notify to Engage' resolvers from the Resolvers section of the Incident Console or a service info card in the console or incident service dependencies map.

The workflow includes steps that map the incident severity level to alert priority levels in xMatters. You can modify which severity levels correspond to each alert priority. The workflow also includes a step to retrieve the details of any collaboration channels that were created for the incident so you can include them in your messages to resolvers when you notify them to engage in the incident.

The following sections walk through the changes you can make to these steps. Depending on your incident resolution process, you might do one, some, or all of the steps below. For instructions on adding collaboration channels, see the following instructions dedicated to that.

Since it's always a good idea to test what you develop, initiate a test incident after you make changes to the workflow to check that the incident that gets created meets the needs of your incident resolution framework.

Add collaboration channels

One of the most common — and most useful — additions to the basic workflow is to automatically create collaboration channels when you initiate an incident.

Communication amongst your team and with stakeholders is key to resolving an incident efficiently and preventing similar incidents from happening in future. But setting up collaboration channels can easily fall through the cracks when you're dumped in the middle of an incident.

By adding steps to the initiate incident flow, the channels that suit your incident resolution framework get created and associated with the incident automatically. For example, you can create separate chats with different purposes (such as one for cross-team communication, one for the Ops team, and one for the team communicating about the issue to customers) or create different channels depending on the severity of the incident (critical incidents get all the channels while medium ones just get a single chat).

If you add additional steps to enrich and automate the resolution process, you can feed information from those steps into chat channels.

Add collaboration channels to an incident resolution workflow

To automatically add a collaboration channel to an incident, add and configure one of the following steps in your flow:

If your resolution team prefers a channel not listed above, you can use the following step to add any type of channel that has a URL:

When configuring the steps, make sure to drag the Incident ID variable from the Initiate Incident step into the Incident ID field — this associates the channel with the incident:

You can add the incident details to Slack and Teams channels using the associated Post to Channel step, or invite bots you use in your incident discussions using the Invite to Channel step: see our topics on configuring Slack steps or configuring Teams steps for more information.

Since it's always a good idea to test what you develop, initiate a test incident after you've added your collaboration channels to make sure they get created and appear in the Collaboration section in the Incident Console.

Automate and enrich

Collaboration steps aren't the only steps you can add to the flow — the flexibility of Flow Designer lets you connect to all the tools in your toolchain to automate resolution and enrich incident notifications.

Change the trigger

If you want to initiate incidents in ways that don't require users to fill in a form, you can change the trigger for the flow, allowing the workflow to initiate an incident by HTTP trigger or email, or when a user selects a specific response to a notification.

  • Swap out the form trigger for an HTTP trigger to automatically initiate an incident when xMatters receives an alert.
  • Use an Email trigger to initiate incidents via an email to xMatters.
  • Connect the Initiate Incident step to a Response trigger to let your people create an incident at the touch of a button.

Add additional steps to gather information from or take action in other systems

Some steps you might add before the Initiate Incident step are:

  • Create issues in other systems such as Jira or ServiceNow and add that information to the incident summary or description.
  • Set the Incident ID or severity to match a value passed in from another system. For example, you can drag the issue ID or incident ID output from a create issue step into the Incident ID field in the Initiate Incident step.
  • Run a playbook or rebuild a Jenkins job, and then use a switch step to only create the incident if that didn't solve the problem.

After the Initiate Incident step, you could add steps to:

  • Update issues in other systems. For example, add a comment to that Jira or ServiceNow issue you created before the initiate incident step to update playbooks with any lessons learned from the incident.
  • Update records in the signal source. For example, if the incident was created because of an incident in ServiceNow, you can update the ServiceNow record with information about the xMatters incident.
  • Post a message to a status page to let people know you're aware of the issue and are working on it.