Lightstep

Lightstep combines metric and tracing data to provide observability into your system. This workflow lets you send actionable alerts to on-call resources when xMatters gets a signal from either Lightstep Metric, or Lightstep Stream alerts. Responders can initiate an incident with the press of a button, or you can build on the flow to perform automated resolution tasks.

How it works

When an alert is generated in Lightstep, it sends a JSON-formatted webhook to xMatters, based on user-defined alert rules. A Lightstep Alert trigger in xMatters parses the webhook and initiates a flow. The webhook includes essential alert data you can use to enrich notifications to users or when building automated tasks.

Install the workflow

The following instructions describe how to install the workflow through the xMatters one-click installation process.

  1. Go to the Workflow Templates page and click the Lightstep tile.
  2. On the Set up the Workflow tab, give the workflow a name that identifies its purpose (this must be unique in your instance), add an optional description, and set the default incident type (if applicable). Any built-in Initiate Incident steps in the workflow will automatically be set to the selected incident type.
    • You can edit these later, if needed.

  3. Click Next to set up the connection.
  4. Choose the authentication method. A trigger URL is generated based on the selected authentication method.
  5. Copy the trigger URL — you’ll use this to configure the webhook in Lightstep.
    • The trigger URL includes the recipients parameter, which specifies who should be notified. By default, this parameter is set to notify you (the logged in user), but you can set it to target any user or group you want.

  6. Send a test signal to the trigger URL to test the connection.
  7. Click Open Workflow to view and customize the workflow, or Close to return to the Workflows page.

Configure Lightstep Metric to send requests to the trigger URL

The following instructions describe how to configure Lightstep Metric to send signals to your triggers.

To have Lightstep Metric send alerts to the flow trigger, you need to configure a webhook and set it to use the trigger URL. Ensure you've added the Lightstep Metric Alert trigger to your flow and that you've copied the URL from the trigger on the Flow Designer canvas. Lightstep Metric Alert URLs and Lightstep Stream Alert URLs are not interchangeable.

  1. In Lightstep, navigate to Alerts > Notification destinations.

  2. From the Create a destination drop-down, select Webhook.
  3. On the Create a webhook destination window, fill in the following fields:
    • Name: Give the webhook a unique name.
    • URL: Paste the trigger URL you copied from Flow Designer. Add the target names of any recipients you want to notify when the alert fires to the end of the URL.
      • For URL authentication, use an ampersand to attach recipients. For example, if you want to notify Emma Pearson and the on-call members in the group responsible for the Antares service, you'd add &recipients=epearson,antares to the URL.
      • For other authentication types, use a question mark to attach recipients. For example, if you want to notify Barry Gull and the on-call members in the group responsible for the Cassiopeia service, you'd add ?recipients=bgull,cassiopeia to the URL.
      • You must URL-encode any special characters or spaces in the target names.

  4. Optional: If you use Basic Authentication, click New Header.
    • Set the Headers to authorization and enter basic followed by a Base64 encoded xMatters username and password (for example basic aBCdeF12g34hiJKL):

      You can create a Base64 encoded username and password at https://www.base64encode.org/ or by typing echo “username:password” | base64 into a terminal.

  5. Click Create to save the webhook.
  6. To test the webhook, click Test.

Now that you’ve created the webhook, it needs to be added to a metric alert.

  1. Select the Alerts tab.

  2. Click Create an alert, then add a Metric alert. The Create alert window opens.
  3. Fill in the following fields: 
    • Enter an alert name: Type a unique name for the alert
    • Query (Optional): Add metric names and formulas.
    • Alert configuration: Set the parameters for when the alert sends a signal to xMatters.

  4. In the Notification destinations section:
    • From the Select a destination type drop-down, select Webhook.
    • Adjust the Renotify every timeframe, if required.
    • In the Search for a destination name field, type the name of the webhook you created.
  5. Click Save at the top of the Create alert window.

You're ready to use the webhook to trigger automated flows, including steps such as sending alerts and initiating incidents, though we always recommend testing before putting things into use.

Configure Lightstep Stream to send requests to the trigger URL

The following instructions describe how to configure Lightstep Stream to send signals to your triggers.

To have Lightstep Stream send alerts to the flow trigger, you need to configure a webhook and set it to use the trigger URL. Ensure you've added the Lightstep Stream Alert trigger to your flow and have copied the URL from the trigger on the Flow Designer canvas. Lightstep Stream Alert URLs and Lightstep Metric Alert URLs are not interchangeable.

  1. In Lightstep, navigate to Alerts > Notification destinations.

  2. From the Create a destination drop-down, select Webhook.
  3. On the Create a webhook destination window, fill in the following fields:
    • Name: Give the webhook a unique name.
    • URL: Paste the trigger URL you copied from Flow Designer.

  4. Add the target names of any recipients you want to notify when the alert fires to the end of the URL.
    • For URL authentication, use an ampersand to attach recipients. For example, if you want to notify Emma Pearson and the on-call members in the group responsible for the Antares service, you'd add &recipients=epearson,antares to the URL.
    • For other authentication types, use a question mark to attach recipients. For example, if you want to notify Barry Gull and the on-call members in the group responsible for the Cassiopeia service, you'd add ?recipients=bgull,cassiopeia to the URL.
    • You must URL-encode any special characters or spaces in the target names.
  5. Optional: If you use Basic Authentication, click New Header.
    • Set the Headers to authorization and enter basic followed by a Base64 encoded xMatters username and password (for example basic aBCdeF12g34hiJKL):

      You can create a Base64 encoded username and password at https://www.base64encode.org/ or by typing echo “username:password” | base64 into a terminal.

  6. Click Create to save the webhook.
  7. To test the webhook, click Test.

Now you’ve created the webhook, it needs to be added to a stream query through a stream alert.

  1. From the left-hand menu, select Streams

  2. Open the stream you want to associate with the new xMatters webhook.
  3. Select the Alerts tab.
  4. Click Create an alert.

  5. On the Create an alert page, fill in the following fields: 
    • Signal: Use the drop-down to select a signal type.
    • Name: Give the alert a unique Name.
    • Threshold: Set the threshold value.
    • Evaluation Window: Set the evaluation window time span.

  6. Under Notification destinations, click Add a destination.
  7. When the Notification destinations window opens, fill in the following fields:
    • From the Integration drop-down, select Webhook.
    • From the Destination drop-down, select the new xMatters webhook you created.
    • Set the Update Interval.

  8. Click Add a destination to save the destination and return to the Create an Alert page.
  9. Click Create to save the alert.

You can add multiple destinations to the alert, if required.

You're ready to use the webhook to trigger automated flows, including steps such as sending alerts and initiating incidents, though we always recommend testing before putting things into use.

Set recipients in the trigger URL

The trigger expects the recipients in the trigger URL. When you copy the URL from xMatters, it includes the recipients parameter: recipients=<yourname>. Of course, you don’t want to receive all the alerts.

To change the recipients for alerts from this webhook, swap out your name for the people or groups you want to target.

  • For URL authentication, use an ampersand to attach recipients. For example, if you want to notify Emma Pearson and the on-call members in the group responsible for the Antares service, you'd add &recipients=epearson,antares to the URL.
  • For other authentication types, use a question mark to attach recipients. For example, if you want to notify Barry Gull and the on-call members in the group responsible for the Cassiopeia service, you'd add ?recipients=bgull,cassiopeia to the URL.

Remember to URL-encode any special characters, including spaces, in your group names.

We recommend using groups so you can take advantage of the xMatters group features — rotations, escalations, and absences — to reach the right on-call people to jump on an issue.

How to use the workflow

When a condition you've set fires, it sends a signal to xMatters, which creates an alert and notifies the individual or the on-call members of the people or groups you set as recipients in the webhook URL. When the trigger receives a signal saying the issue is resolved, it automatically terminates related alerts in xMatters.

The person responding to the notification has the following response options:

  • Acknowledge: Acknowledges the notifications and stops escalations.
  • Escalate: Immediately escalates the alert to the next on-call resolver in a targeted group.
  • Close: Ends the xMatters alert and stops notifying all targeted recipients.
  • Initiate Incident: Initiates an incident in xMatters.

Next Steps

Now that you've installed the workflow, you can use it as-is, or customize it to suit your needs better. Here are some examples of things you can add to the workflow to customize it: