This section describes some techniques for monitoring and troubleshooting the xMatters Agent, including how to verify that the xMatters Agent is communicating with xMatters and how to troubleshoot configuration issues so you can ensure that your integrations are processed successfully.
Status update alerts notify selected users when communication from the xMatters Agent has not been detected for more than five minutes. This can happen when the xMatters Agent service stops running, the host system is turned off or disconnected from the network, or when there is another event such as a network outage.
To configure status update recipients, see Set status alert recipients.
You can use an internal communication plan's built-in subscription form to create a subscription that will notify you when a specific integration encounters a processing error. This can happen if there's a syntax error in your script or your integration received unexpected inputs.
To subscribe to notifications when your integration encounters a processing error, see Subscribe to integration alerts.
If your xMatters Agent is not appearing in the list of agents or appears to be disconnected, try the following troubleshooting steps.
Once you have installed an xMatters Agent, it may take a few minutes before it appears in the list of installed agents in the web user interface.
Ensure that the host system is turned on, connected to the network, and that the xMatters Agent service is running. The xMatters Agent service is configured to restart automatically when the host computer is rebooted.
For more information about starting the xMatters Agent service, see Stop, start, or restart an xMatters Agent.
It is recommended to use the most up-to-date version of the xMatters Agent. To see if there is a newer version of the xMatters Agent, log in to the xMatters user interface and view the status of the agent, as described in View details about an xMatters Agent.
To update the xMatters Agent, see Update an xMatters Agent to the latest version.
The xMatters Agent can experience connectivity issues when the clock on the host system becomes inaccurate. This situation is prone to occur when an xMatters Agent is installed on a virtual machine that experiences time drift. To prevent the xMatters Agent from disconnecting, you may want to use a service such as NTP to synchronize the time on your host system with internet time.
If a previously-connected xMatters Agent becomes disconnected even though the xmatters-xa service is running and appears healthy, ensure that the time on the host system is accurate. System clock errors appear as a warning in the logs with the following structure:
2017-04-12 16:08:45,608 5422 [qtp726379593-159363] com.xmatters.cloudagent.WebSocketConfiguration WARN --- An otherwise valid token failed the timestamp check with expiry=1492013171 domain="acmeco.na1.xmatters.com",trace="d9ab8e88-aba1-41e4-a6a8-0d7214ca3a11"
If your host system uses a proxy server to connect with xMatters, you can define its proxy configuration settings during installation of the xMatters Agent. To update existing proxy configuration settings, you'll need to re-install the xMatters Agent.
If the above troubleshooting steps do not work, you can restart the xMatters Agent service. For more information, see Stop, start, or restart an xMatters Agent.
If the above actions don't fix the issues, take a look at the monitoring endpoints and system logs for more information.
You can monitor the status of the xMatters Agent service on the host system by accessing some built-in URLs. You can access these URLs by typing them in to your web browser or by making a GET request from a system that supports making RESTful web requests. The URLs return information such as whether the agent is connected, the time of the last successful connection to xMatters, and other information such as the version of the agent and the number of jobs that are queued for processing.
The URL structure is in the format http://<localhost>:8081/<endpoint>, where <localhost> is the IP address of the host system and <endpoint> is the name of the monitoring endpoint you want to access.
Monitoring endpoints are configured to run on port 8081 by default. This value can be changed by editing the SERVER_PORT setting in the xa.conf configuration file on Linux installations, and by setting a global SERVER_PORT environment variable on Windows installations. If you are running both the xMatters Agent and the Integration Agent on your system, you will need to change the default port for one of these agents.
The following sections describe endpoints that are available for monitoring the xMatters Agent.
Returns information about the connection between the xMatters Agent and the xMatters server. Example: http://localhost:8081/health
|connected||True if the agent is currently connected to the xMatters server, false otherwise.|
|lastConnectedTime||Timestamp of the most recent time a connection between the xMatters Agent and the xMatters server was created.|
|lastDisconnectedTime||Timestamp of the most recent time the connection between the xMatters Agent and the xMatters server was lost.|
|lastHeartbeatTime||Timestamp of the most recent time the xMatters Agent sent a packet to the xMatters server.|
Returns information about the xMatters Agent. Example: http://localhost:8081/info
|version||Version of the xMatters Agent.|
Returns information about the number of local queued jobs and threads in the xMatters Agent. Example: http://localhost:8081/metrics
|xa.maxNumOfWorkers||The maximum number of threads that can be used by the xMatters Agent.|
|xa.numOfWorkers||The current number of threads used by the xMatters Agent.|
|xa.numOfActiveJobs||The number of processing jobs that are queued locally.|
|xa.numOfJobsInWaiting||The maximum number of jobs that may be queued locally.|
The xMatters Agent logs information in a file on the host's file system and contains status information including the content of the console.log statements in scripts. You can access this file at the following locations:
- Linux: /var/log/xmatters/xmatters-xa/agent-communication-xmatters.log
- Windows: C:\Program Files\xa\xalog\agent-communication-xmatters.log