Best practices for creating ZipSync data files

This section presents best practices for transforming your system data into a set of comma-separated value (CSV) files that can be loaded into xMatters using ZipSync and the EPIC tool. These concepts are independent of the tools that you use to perform the data transformation. You can use one of the many available Extract, Transform, and Load (ETL) tools, or your own custom-designed process.

You should be familiar with basic ETL concepts and the tools you will use to extract and transform your data. You should understand how your xMatters system is configured, including how Sites, Groups and Teams are used. You should also understand the workflow that people will use to update information that is stored in both your system and in xMatters so you can determine whether to overwrite these values during the ETL process.

These best practices should be used in conjunction with ZipSync mode Included Files, which includes detailed information about the structure of the CSV files in the ZipSync archive.

These best practices include suggestions for completing the following tasks:

  • preventing data loss by checking system thresholds
  • logging and error tracking
  • data cleansing and validation
  • mapping your system data to xMatters values
  • creating prerequisite fields before dependent fields

Technical context

Before you begin the data transformation process, ensure that you understand how your xMatters installation is configured and whether you want updates to overwrite xMatters values or leave them unchanged.

Know your xMatters system structure

Become familiar with how your xMatters system is configured so you can map your data to the existing xMatters configuration. In particular, you need to understand how the following areas are configured:

  • how sites are structured
  • how users are assigned to sites
  • how roles are assigned to users
  • how groups and teams are structured

You may be required to provide supplementary information to xMatters that is not stored in your system. For example, if your system doesn't store roles that map directly to xMatters roles, you will need to add that information to your ETL process. This could be as simple as creating a file that maps user IDs to roles and using that file as a lookup table during the ETL process.

Know which system owns the data

It is important to decide whether your system or xMatters will contain the master copy of each piece of data. If the data changes in one system, you need to instruct the EPIC tool whether to update the value in xMatters.

To decide which system should own a particular field, consider how it will be updated over time. When your system owns a field, it is responsible for keeping the field updated and xMatters copies this value from your system. In this case, a user's personal mobile phone number is updated on your system, the change is automatically applied to xMatters the next time the ETL process is run. If the mobile phone number is updated in xMatters without being updated in your system, it will be overwritten by the next ETL update.

If a field is owned by xMatters, then it needs to be updated in both your system and xMatters, because they keep independent copies of the data. However, having xMatters own a value enables you to update it in xMatters without waiting for the ETL process to run, and users are able to provide contact information to xMatters that they do not provide to your system. For example, a user may not want to provide you with their personal mobile phone number, but they might want to add it to xMatters so they can be notified in an emergency.

Decide whether updates will be made incrementally or in mirror mode

When you perform the ETL process, you specify whether you are updating xMatters incrementally or in mirror mode: 

  • Incremental : When you make updates incrementally, your system provides only the records that have changed since the last update. This means that less data is transmitted on each update, and that records are only deleted if they are explicitly marked for deletion. However, updating incrementally can become more complicated if a daily update is missed or if changes need to be rolled back.
  • Mirror mode : When you make updates in mirror mode, your system provides all of the records in the system, and xMatters makes an exact copy of the data. Any records that are not provided by the update are deleted from xMatters. This makes it simple to rollback changes, since the xMatters system reflects the data in your system. However, you need to be careful that you don't accidentally transmit only part of your data set and delete the rest of your users.