Splunk Infrastructure Monitoring

Splunk Infrastructure Monitoring (formerly SignalFx) is a real-time operational intelligence platform that discovers and collects metrics across every component in the cloud. The service provides real-time visibility into today’s dynamic environments for data-driven DevOps teams.

How it works

When a test meets a specified threshold in Splunk Infrastructure Monitoring, it sends a JSON-formatted webhook to xMatters. A Splunk Infrastructure Monitoring trigger in xMatters parses the webhook and initiates a flow. The webhook includes essential alert data you can use to enrich notifications to users or when building automated tasks.

Install the workflow

Here's how to install the Splunk Infrastructure Monitoring workflow through the xMatters one-click installation process. If you already installed the previous, built-in version of the integration, you can find instructions here.

  1. Go to the Workflow Templates page and click the Splunk Infrastructure Monitoring tile.
  2. On the Set up the Workflow tab, give the workflow a name (this must be unique in your instance) and add an optional description.
    • You can edit these later, if needed.

  3. Click Next to set up the connection.
  4. From the Select a method drop-down menu, choose the authentication method you want to use to generate the trigger URLs.
  5. Click Next.
  6. Copy the trigger URLs — you’ll use these to configure the webhooks in Splunk Infrastructure Monitoring.
    • The trigger URLs include the recipients parameter, which specifies who should be notified. By default, this parameter is set to notify you (the logged in user), but you can set it to target any user or group you want.

  7. Click Open Workflow.

Configure Splunk to send requests to the trigger URL

To have Splunk send alerts to the flow trigger, you need to configure a webhook and set it to use the trigger URL. You can either set up a new detector and alert rule using the Splunk Infrastructure Monitoring documentation, or modify one of your existing alert rules.

Set recipients in the trigger URL

The trigger expects the recipients in the trigger URL. When you copy the URL from xMatters, it includes the recipients parameter: recipients=<yourname>. Of course, you don’t want to receive all the alerts.

To change the recipients for alerts from this webhook, swap out your name for the people or groups you want to target. For example, to target the 'DatabaseTeam', 'Antares Service Team', and 'HR & Marketing' groups, add recipients=databaseteam,antares%20service%20team,HR%20%26%20Marketing to the trigger URL. Remember to URL-encode any special characters, including spaces, in your group names.

We recommend using groups so you can take advantage of the xMatters group features — rotations, escalations, and absences — to reach the right on-call people to jump on an issue.

How to use the workflow

When a condition you've set fires, it sends a signal to xMatters, which creates an alert and notifies the individual or the on-call members of the people or groups you set as recipients in the webhook URL. When the trigger receives a signal saying the issue is resolved, it automatically terminates related alerts in xMatters.

Next Steps

Now that you've installed the Splunk Infrastructure Monitoring workflow, you can use it as-is, or customize it to suit your needs better. Here are some examples of things you can add to the workflow to customize it:

Previous versions

While the previous, built-in version of this integration (known as SignalFx) is no longer available, the instructions for it are included below for anyone who has it installed in their system already.

Do more with this integration

Incorporate this integration into your orchestrated incident resolution workflows with Flow Designer. After you create a configuration, the Flow Designer tab appears. From there, you can build your flows — add new response options, connect to other apps, or even create new xMatters alerts based on activities in the flow — enriching the information injected by this integration along the way.

Get started with SignalFx

How to set up a SignalFx configuration

After you give your configuration a name and description, type a name to use when sending alerts (so you can easily tell which configuration or integration the alerts are from).

This integration also needs your SignalFx Access Token to authenticate xMatters requests.

Enter your SignalFx Access Token in the field on the configuration page, then select whether or not you want to add notes to SignalFx when the integration creates an alert in xMatters or receives a device delivery update. Add all the users and groups you want to be notified when the integration is triggered, then click Save.

Once you save the configuration, xMatters displays the URL you need to configure SignalFx.

You can now create a webhook in SignalFx targeting this URL.

Your integration is now complete! You can now add the xMatters integration as an alert recipient to an alert rule in SignalFx.

Use your integration

When an alert rule (with the xMatters webhook as the alert recipient) is triggered in SignalFx, the webhook integration sends details to xMatters, which creates an alert and notifies the recipients.

Extending your integration

Looking to do more with xMatters and SignalFx? If you want to tailor the settings and notifications for the integration, you can convert it to a custom workflow.