Moogsoft Incidents
Moogsoft is an AIOps powered monitoring and observability platform that enables IT & DevOps teams to manage and resolve incidents. xMatters uses Moogsoft’s AIOps to relay critical information to teams and prevent system issues from becoming service incidents.
This workflow lets you send actionable alerts to on-call resources when xMatters gets a signal from Moogsoft. Responders can initiate an incident with the press of a button, or you can build on the flow to perform automated resolution tasks.
How it works
When an alert is generated in Moogsoft, it sends a JSON-formatted webhook to xMatters, based on user-defined alert rules. A Moogsoft trigger in xMatters parses the webhook and initiates a flow that either creates a new incident or updates an existing one. The Moogsoft incident is updated with the xMatters Incident ID and any change to the incident in xMatters is sent back to Moogsoft and appended to the incident as a note.
Install the workflow
- Go to the Workflow Templates page and click the Moogsoft Incidents tile.
- On the Set up the Workflow tab, give the workflow a name that identifies its purpose (this must be unique in your instance), add an optional description, and set the default incident type (if applicable). Any built-in Initiate Incident steps in the workflow will automatically be set to the selected incident type.
- You can edit these later, if needed.
- Click Next to set up the connection.
- Choose the authentication method. A trigger URL is generated based on the selected authentication method.
- Copy the trigger URL — you’ll use this to configure the webhook in Moogsoft.
- The trigger URL includes the recipients parameter, which specifies who should be notified. By default, this parameter is set to notify you (the logged in user), but you can set it to target any user or group you want.
- Copy the Configuration Payload to configure the signal in Moogsoft.
- Click Open Workflow to view and customize the workflow, or Close to return to the Workflows page.
Configure Moogsoft to send requests to the trigger URL
To have Moogsoft send alerts to the flow trigger, you need to configure a webhook and set it to use the trigger URL, and create a custom user property to connect your Moogsoft and xMatters accounts. You can also set the Moogsoft correlation engine to combine similar alerts into Moogsoft incidents.
- In Moogsoft, go to Integrations > Outbound Integrations and select Webhook from the menu.
- Click Add a Webhook.
- In the Name and Scope section fill in the following fields:
- Name: Give the webhook a name (for example, xMatters).
- Type: Select Incident from the drop-down menu.
- In the CREATE Operation and HTTP Configuration section fill in the following fields:
- Request Method: Select POST from the drop-down menu.
- URL: Paste the trigger URL you copied from Flow Designer. Add the target names of any recipients you want to notify when the alert fires.
- For URL authentication, use an ampersand to attach recipients. For example, if you want to notify Emma Pearson and the on-call members in the group responsible for the Antares service, you'd add &recipients=epearson,antares to the URL.
- For other authentication types, use a question mark to attach recipients. For example, if you want to notify Barry Gull and the on-call members in the group responsible for the Cassiopeia service, you'd add ?recipients=bgull,cassiopeia to the URL.
- You must URL-encode any special characters or spaces in the target names.
- In the Payload Body for CREATE Operation section, set the payload as: Copy
{
"timestamp": "$created_at",
"location": "$tags.region",
"severity": "$severity",
"id": "$id",
"description": "$description",
"services": $services,
"status": "$status",
"totalalerts": "$total_alerts",
"signalMode": "New",
"assignee": "$assignee",
"correlationDefinition": "$correlation_definition",
"incidentURL": "$incident_url"
} - In the UPDATE notifications section fill in the following fields:
- Turn on the Enable update notifications toggle.
- Request Method: Select POST from the drop-down menu.
- URL: Paste the trigger URL you copied from Flow Designer. Add the target names of any recipients you want to notify when the alert fires.
- For URL authentication, use an ampersand to attach recipients. For example, if you want to notify Emma Pearson and the on-call members in the group responsible for the Antares service, you'd add &recipients=epearson,antares to the URL.
- For other authentication types, use a question mark to attach recipients. For example, if you want to notify Barry Gull and the on-call members in the group responsible for the Cassiopeia service, you'd add ?recipients=bgull,cassiopeia to the URL.
- You must URL-encode any special characters or spaces in the target names.
- Triggers: turn on the toggles for type of updates you'd like to receive notifications for.
- In the Payload Body for UPDATE Operation section, set the payload as: Copy
{
"timestamp": "$created_at",
"location": "$tags.region",
"severity": "$severity",
"id": "$id",
"description": "$description",
"services": $services,
"status": "$status",
"totalalerts": "$total_alerts",
"signalMode": "Update",
"assignee": "$assignee",
"correlationDefinition": "$correlation_definition",
"incidentURL": "$incident_url"
} - Optional: To test the webhook, go to the top of the Outbound Webhook page and click Test.
- Click Save.
You're ready to use the webhook to trigger automated flows, including steps such as sending alerts and initiating incidents, though we always recommend testing before putting things into use. To keep things tidy in Moogsoft, we recommend using the Correlation Engine to combine similar incoming alerts into Moogsoft incidents.
- In Moogsoft go to Correlate & Automate > Correlation Engine.
- Click Add Correlation Definition.
- In the Construct Your Incident Description section fill in the following fields:
- Correlation Name: Give the correlation a unique name.
- Incident Description: Set the correlation description. The description displays for all incidents created by this correlation definition.
- In the Definition section, set the Scope of the correlation to match all alerts or only alerts that match the filter parameters, then click Add Field.
- Optional: In the Advanced section you can set duration for the Correlation Time Window and select the minimum number of similar alerts before an incident is created.
- Click Save.
The correlation engine will create Moogsoft incidents from the incoming alerts based on your configuration settings.
Configure xMatters to send updates to Moogsoft
To send updates to Moogsoft, the workflow requires a valid Moogsoft email address, API key credentials, and a token endpoint. The following instructions describe how to create a custom property for the email address, generate the API key, and configure a token endpoint.
The xMatters custom user property associates the email address of your Moogsoft account with your xMatters account. Once the property is created, any user can add their Moogsoft email address to their xMatters profile to receive notifications. The email address stored in the custom user property is used in the Create Alert - Assign to User step of the workflow.
- Open the xMatters Admin menu and select Custom User Properties.
- On the Custom User Properties window, click Add Property and fill in the following fields:
- Name: Moogsoft User ID
- For the workflow to successfully send updates to Moogsoft, the name of the property must be Moogsoft User ID. If you give this property a different name, you must update the workflow constant with the new name.
- Type: Select Text from the drop-down menu.
- Name: Moogsoft User ID
- Click Save.
Now that the property is saved in your system, you need to populate the field with the users' Moogsoft email addresses. There are two ways to do this:
- Batch upload:
- Manually add each user's email address to their account.
Here's how to add a Moogsoft email address to an xMatters account profile.
-
To access your user profile, click on your username at the top-right corner of the xMatters, then select Profile.
- Click Edit Profile.
- Scroll to the bottom of the Edit Profile screen to the new Moogsoft User ID field.
- Enter the email address associated with the Moogsoft account into the Moogsoft User ID field.
- Click Save.
The Moogsoft API key sets the authentication method for the endpoint so that information can flow between Moogsoft and xMatters. Creating an API key and adding it to your flow as a constant means there's no configuration required for the Moogsoft endpoint.
Go to your Moogsoft account and use Moogsoft's instructions to create an API key. Save the API key to a secure location as it is only displayed once. If you don't save the API key, or forget it, you must create a new one.
You can use a pre-existing endpoint or configure a new endpoint with the following information:
- Name: Type a name that will identify your endpoint.
- Base URL: Enter https://api.moogsoft.ai.
- Endpoint Type: Select 'Token'.
- Header: Select 'Custom'.
- Token Header: Type 'apikey'.
- Token: Enter your organization's Moogsoft API key.
- ID Header: Leave blank.
- ID: Leave blank.
Set recipients in the trigger URL
The trigger expects the recipients in the trigger URL. When you copy the URL from xMatters, it includes the recipients parameter: recipients=<yourname>. Of course, you don’t want to receive all the alerts.
To change the recipients for alerts from this webhook, swap out your name for the people or groups you want to target.
- For URL authentication, use an ampersand to attach recipients. For example, if you want to notify Emma Pearson and the on-call members in the group responsible for the Antares service, you'd add &recipients=epearson,antares to the URL.
- For other authentication types, use a question mark to attach recipients. For example, if you want to notify Barry Gull and the on-call members in the group responsible for the Cassiopeia service, you'd add ?recipients=bgull,cassiopeia to the URL.
Remember to URL-encode any special characters, including spaces, in your group names.
We recommend using groups so you can take advantage of the xMatters group features — rotations, escalations, and absences — to reach the right on-call people to jump on an issue.
How to use the workflow
The Moogsoft Incidents workflow performs two-way communication between xMatters and Moogsoft. When condition you've set fires, it sends a signal to xMatters, which either creates a new incident or updates an existing one. When there is a change to the incident in xMatters, an update is sent back to Moogsoft.
When a signal is received from Moogsoft and a new incident is created, the person responding to the notification has the following response options available:
- Assign to me: Acknowledges the notifications and stops escalations. The Moogsoft incident is updated with a note containing the xMatters incident ID and information about the authenticating user.
- Escalate: Immediately escalates the alert to the next on-call resolver in a targeted group.
- Resolve: Ends the xMatters alert and stops notifying all targeted recipients. The Moogsoft incident is resolved and a note is added for the change in status.
- Close: Ends the xMatters alert and stops notifying all targeted recipients. Both the xMatters and Moogsoft incidents are closed and a note is added for the change in status.
Next Steps
Now that you've installed the workflow, you can use it as-is, or customize it to suit your needs better. Here are some examples of things you can add to the workflow to customize it:
- Use Slack, Zoom, and Microsoft Teams steps to add collaboration channels to the flow.
- Change the severity of incidents created when a recipient selects the Initiate Incident response.
- Update the message sent to resolvers to include the information most relevant to your team.
- Use the Moogsoft Incidents trigger to build your own custom flows.