The Incident Resolution workflow is an installable workflow template designed to help you tailor xMatters Incident Management to your incident resolution process. It starts as the same workflow used by default for the Incident Management feature, but lets you adapt how incidents are created, what happens when they're created, and the notifications that are sent to resolvers when the incident is initiated.
This customizable workflow lets you:
- add collaboration channels to incidents.
- include automated steps in the incident resolution flow to create incidents or send incident information to other systems.
- customize the settings for new incidents.
- tailor notifications sent about the incident when it is initiated, including information from other tools that might help resolvers triage and address the issue.
- allow your team or your tools to initiate incidents using email or an HTTP trigger.
- control who can initiate incidents with the workflow.
For more information on initiating and working incidents, see our help on the Incidents list and Incident Console.
- Click the Incident Resolution tile on the Workflows Templates page.
- On the Install Workflow dialog box, give the workflow a name that identifies its purpose (this must be unique in your instance) and add an optional description.
- You can edit these later, if needed.
- Click Install.
- After the workflow installs, the screen shows next steps plus an installation log. The installation log gives you additional information about the installation, such as if you have languages in your instance that aren't configured for the messages in the workflow.
You're now ready to initiate incidents with the workflow or tailor it to your incident resolution processes.
How to use the workflow
After you install the workflow, anyone you give sender permissions to the flow trigger form can use it to initiate incidents from the Incident Console and the Initiate Incident dashboard widget. If you enable the flow trigger form in the mobile app, it will also appear as an option for initiating incidents in the xMatters mobile app. If you have the Slack integration set up, you can initiate incidents using the /xmatters initiate command.
Once you select the incident you want to initiate, a form appears where you can fill in the incident details:
Configure the default workflow
The default workflow functions the same as the xMatters Incident Management workflow, but allows you to make customizations.
The workflow includes two forms: a messaging form to notify resolvers about new incidents and a flow trigger form that lets you initiate an incident by submitting the form. Customize the form associated with the Create Alert Using Form step (the New Incident Notification form) to modify the notification that are sent to resolvers when the incident is initiated. The only modifications you can make to the flow trigger form are whether it's enabled and who has permission to use it.
The workflow also has two flow canvases — the Trigger an Incident canvas contains the flow that initiates incidents. That flow has three steps:
The following sections walk through the changes you can make to each of those steps. Depending on your incident resolution process, you might do one, some, or all of the steps below. For instructions on adding collaboration channels, see the following instructions dedicated to that.
When you first install the workflow, you're the only person with permission to edit it or initiate incidents using it. To allow other users to initiate incidents, you need to set Sender Permissions on the form trigger in the flow. The ability to limit who can initiate incidents using the workflow is one of the reasons to install it.
- Open the Trigger an Incident canvas on the Flow Designer tab.
- Double-click the Form Trigger (or hover and click the pencil icon).
- Form triggers are now known as Incident Initiation triggers, but existing triggers (or ones included in existing workflow templates) will retain the old name until they are updated or changed.
- Make sure Web UI is set to Enable; if not click the slider to toggle it. This sets whether the incident form appears as an option for the Initiate Incident button or the widget. If you want users to be able to initiate incidents from the mobile app
, Slack, or Microsoft Teams, set Mobile App to Enabled.
- Click Sender Permissions.
- Add the users or roles you want to have permission to initiate incidents for this workflow (type two spaces to see the list of options you can select).
- For example, if you select an "Incident Manager" role, anyone with that role can initiate incidents using this form.
- Click Save Changes.
- Click Done, and then click Save on the canvas and close it.
- Switch to the Forms tab, click the drop-down beside the New Incident Notification form, and select Sender Permissions.
- Add the same users and roles you added for the Form Trigger in step 5 — this allows the person who initiates the form to also add and notify resolvers.
- Click Save Changes.
To customize the settings for new incidents, update the inputs in the Initiate Incident step.
- Double-click the Initiate Incident step (or hover and click the pencil icon).
- Update the inputs to fit with your incident processes.
- You can populate the inputs with text, outputs from previous steps, or constants; however, at runtime, the severity needs to be one of the following: Critical, High, Medium, Low, or Minimal.
- Click Done, and then click Save on the canvas.
In this example, we've added the reporter to the summary:
In a more complex workflow, you could set the severity based on the path in a switch step, map the severity levels from an external system to xMatters with a Map Incident Severity Step, or use the outputs of an HTTP trigger to populate the details, including setting the incident ID to match that in another system. For more information on adding the Initiate Incident step to other workflows, see the step instructions.
Edit the Create Alert Using Form step (formerly known as the Create Event step) to add additional resolvers and customize alert properties for the notification send to resolvers when the incident is initiated. If you want to edit the contents of the notification, see the following section.
- Double-click the Create Alert Using Form step (or hover and click the pencil icon).
- Depending on your workflow template, the step may be named "Create Event".
- Make sure the Incident ID field is mapped to the "Incident ID" output of the Initiate Incident step — this associates the alert with the incident.
- Make sure the Recipient field includes the "Resolvers" output from the Form Trigger. This sets any recipients added when the incident form is used as resolvers and sends them notifications.
- You can add additional recipients to the field if you always want an individual or group involved in the incident resolution process (for example, an incident review team or the support team).
- Update the other fields as required — the available fields are determined by the workflow properties added to the New Incident Notification form layout.
- Update the Devices and Handling & Overrides tabs if required. See the step instructions for details.
- Click Done, and then click Save on the canvas.
In this example, we've added the Antares support group to the recipients since we always want someone from that team involved in our incident resolution:
If you add additional steps to enrich and automate the resolution process, you can feed information from those steps into the notification by adding outputs from those steps to the inputs of the Create Alert Using Form step.
You can change the content of notifications sent to resolvers when the incident is first initiated.
- Open the New Incident Notification form.
- Make sure the properties you want to include in the message are on the form layout.
- Click the Message tab and edit the message you want to change.
For example, we added the incident ID directly into the subject of the email message:
For full instructions on editing forms and messages, see Design forms and Define message content. If you decide to modify the response choices presented to users, only responses with Positive response contribution values will show the responder as "Engaged" in the Incident Console and add them as a resolver to the incident. None, Neutral,and Negative contribution values do not affect the resolver status.
When you add resolvers to an existing incident, they're notified using the default Incident Management message.
Since it's always a good idea to test what you develop, initiate a test incident after you make changes to the workflow to check that the incident that gets created meets the needs of your incident resolution framework.
One of the most common — and most useful — additions to the basic workflow is to automatically create collaboration channels when you initiate an incident.
Communication amongst your team and with stakeholders is key to resolving an incident efficiently and preventing similar incidents from happening in future. But setting up collaboration channels can easily fall through the cracks when you're dumped in the middle of an incident.
By adding steps to the initiate incident flow, the channels that suit your incident resolution framework get created and associated with the incident automatically. For example, you can create separate chats with different purposes (such as one for cross-team communication, one for the Ops team, and one for the team communicating about the issue to customers) or create different channels depending on the severity of the incident (critical incidents get all the channels while medium ones just get a single chat).
If you add additional steps to enrich and automate the resolution process, you can feed information from those steps into chat channels.
To automatically add a collaboration channel to an incident, add and configure one of the following steps in your flow:
- Slack Create Channel
- Microsoft Teams Create Channel
- Microsoft Teams Create Online Meeting
- Zoom Create Meeting
- Create xMatters Conference
If your resolution team prefers a channel not listed above, you can use the following step to add any type of channel that has a URL:
- Add Collaboration Channel to Incident to add any type of collaboration channel with a URL
When configuring the steps, make sure to drag the Incident ID variable from the Initiate Incident step into the Incident ID field — this associates the channel with the incident:
You can add the incident details to Slack and Teams channels using the associated Post to Channel step, or invite bots you use in your incident discussions using the Invite to Channel step: see our topics on configuring Slack steps or configuring Teams steps for more information.
Since it's always a good idea to test what you develop, initiate a test incident after you've added your collaboration channels to make sure they get created and appear in the Collaboration section in the Incident Console.
Collaboration steps aren't the only steps you can add to the flow — the flexibility of Flow Designer lets you connect to all the tools in your toolchain to automate resolution and enrich incident notifications.
If you want to initiate incidents in ways that don't require users to fill in a form, you can change the trigger for the flow, allowing the workflow to initiate an incident by HTTP trigger or email, or when a user selects a specific response to a notification.
- Swap out the form trigger for an HTTP trigger to automatically initiate an incident when xMatters receives an alert.
- Use an Email trigger to initiate incidents via an email to xMatters.
- Connect the Initiate Incident step to a Response trigger to let your people create an incident at the touch of a button.
In this example, a monitoring system sends an alert to xMatters, which triggers an alert. This sends a notification to the on-call member of the Stellar App support team, Ali Samara. When Ali reviews the details of the issue, she realizes this is a major incident and selects the "SEV-1" response option, launching the organization's major incident response process.
Selecting that response option runs a flow that initiates an incident, creates a Jira ticket with the details, creates a Slack channel for the team working the incident to collaborate in (and invites their scribe bot to the channel), creates a conference call for the customer liaison team to plan their response, and adds the details about the incident and the Jira ticket to the channel.
If you trust the machine, you can also initiate an incident directly via HTTP trigger, without a human making the decision — just connect an Initiate Incident step to the HTTP trigger.
When a signal is received from the monitoring tool that it's noticed a problem, an incident is initiated, a notification is sent to the service team, and a channel is created in Teams with a message about the incident.
Add additional steps to gather information from or take action in other systems
Some steps you might add before the Initiate Incident step are:
- Create issues in other systems such as Jira or ServiceNow and add that information to the incident summary or description.
- Set the Incident ID or severity to match a value passed in from another system. For example, you can drag the issue ID or incident ID output from a create issue step into the Incident ID field in the Initiate Incident step.
- Run a playbook or rebuild a Jenkins job, and then use a switch step to only create the incident if that didn't solve the problem.
After the Initiate Incident step, you could add steps to:
- Update issues in other systems. For example, add a comment to that Jira or ServiceNow issue you created before the initiate incident step to update playbooks with any lessons learned from the incident.
- Update records in the signal source. For example, if the incident was created because of an incident in ServiceNow, you can update the ServiceNow record with information about the xMatters incident.
- Post a message to a status page to let people know you're aware of the issue and are working on it.
In this example, we add a step to automatically create an incident in ServiceNow when an incident is initiated in xMatters:
Then we add a Post to Channel step and set the message to include a link to both the ServiceNow record and the xMatters incident, with the addition of a few constants to hold the initial portion of the URLs.
Here's the message posted to the channel: