Event Flood Control
During an event flood, the first events to arrive typically contain the details you need to identify which of your systems needs attention. Subsequent events that are generated during a flood add a lot of extra noise and they can result in processing delays if you exceed your licensed number of events per minute.
Event flood control reduces the number of events that are generated in xMatters when you experience a flood or event storm from one or more of your systems. This feature compares incoming event requests to recent events and suppresses correlated events that occur in too close succession to one another. These suppressed events aren't queued for processing and don't count against your licensed number of events per minute, which allows events from other sources to flow through xMatters without lengthy processing delays.
The initial release of event flood control includes a company-level default Event Rate Filter for all inbound integrations. This default filter is based on our analysis of actual customer event flood data and provides a balance of effective protection against sudden influxes of events, while allowing real traffic to proceed as normal.
This feature relies on improvements we're making to our hosting service, which means it will be available on different dates depending on your hosting region. Customers in North and South America will be able to opt-in to this feature in January 2019. For customers in all other regions, to enable event flood control for your company, contact xMatters Client Assistance.
The initial release of event flood control includes a default Event Rate Filter that limits the number of events generated by the same integration and targeting the same recipients to four events per minute.
Events from the same integration to the same recipients that occur at or below a rate of four events per minute are generated as normal in xMatters and are listed in the Recent Events report.
If events from an integration with the same recipients exceed the rate specified in the Event Rate Filter, event flood control is triggered and the additional event requests are suppressed beneath the most recently processed event to which they correlate (the "parent" event). This is indicated on the Events report by a stacked icon with a running badge count of the number of suppressed events.
Suppression remains in effect until the incoming rate of correlated event requests drops to four or fewer events per minute. When the flood is over, xMatters resumes generating events as normal.
When event flood control is triggered, the recipients who would otherwise have received a flood of event notifications receive a single notification from xMatters to inform them that an event flood was detected. They also receive periodic updates while the flood is ongoing.
These 'Event flood detected' notifications are associated with system events initiated by xMatters, which you'll see listed in the Recent Events report.
Notifications that xMatters sends to inform recipients that a flood has occurred or is ongoing include the following information:
- The name of the communication plan and form used to create the event.
- The name of the event flood control filter that triggered event suppression.
- The time the first event was suppressed.
- The total number of events that have been suppressed so far.
- The event ID of the parent event.
The notification includes the following response options:
- Acknowledge: Stops notifying other users about the flood.
- Close: Stops notifying all recipients and terminates the flood notification event.
- Escalate: Stops notifying the current user and immediately escalates the event to the next scheduled recipient.
As long as the flood continues to meet the conditions of the Event Rate filter, xMatters will initiate a new system event and send an updated version of the notification every 15 minutes, or for every 1000 suppressed events, whichever occurs first.
During a flood, xMatters automatically notifies and updates the targeted recipients of the event that triggered the flood. If you're not a targeted recipient and want to be informed when event floods occur and are ongoing in your system, you can subscribe to receive these notifications.
xMatters includes an internal communication plan with a built-in one-way FYI subscription for event floods. When you create a subscription, you can select a specific communication plan or integration that you'd like to receive event flood notifications about.
To subscribe to event flood control alerts:
- Click your user name at the top-right corner of the page. A drop-down menu appears.
- In the drop-down menu, click Subscriptions.
- On the Subscriptions page, click Add Subscription, and from the drop-down list select Event Flood Detected.
- On the Subscription Details page, give your subscription a name and description.
- Set the planName and integrationName criteria to specify the name of the communication plan and integration to which you want to subscribe.
- To be notified about event floods from any of the integrations in a communication plan, specify the planName and leave the integrationName field blank.
- Set the remaining options to customize the subscription, and then click Subscribe.
xMatters will now send you a notification whenever event flood control is triggered for the specified integration, and updates if the flood is ongoing.