Migrations Design

Objectives

We want to allow for safe and reversible ABS database migrations.

A user should be allowed to install arbitrary versions of ABS server on top of arbitrary database versions.

At server startup, if the server detects that the current server version changed, it should automatically attempt to migrate the database upwards or downwards to the closest migration:

• if the server version is higher than the database version, it should apply all the relevant upward migrations up to the server version, in order.
• if the server version is lower than the database version, it should apply all the relevant downward migrations down to the server version, in reverse order.

Design

Some terminology:

• ABS server has a server_version.
  • The server version is determined by the version field in the project's package.json.
• On server startup, ABS also reads and updates the server version on the database. Let's call this the database_version.
• A version change occurs when database_version is different from server_version.
• A database migration is usually some significant change to the ABS database schema, but can also involve other significant configuration changes.
• A database migration is performed by running a migration script (a piece of code that usually runs various SQL queries to update the ABS database).
• A reversible database migration has both an upwards and a downwards migration script.
  • An upwards migration script performs the database migration (runs code to perform the required database changes).
  • A downwards migration script reverses the database migration (runs code to reverse the changes made by the upwards migration script).

This design makes the following assumptions:

• ABS uses semantic versioning for its versions.
• Database migrations only occur on version changes.
• Every database migration has to be reversible.
• Each database migration is associated with a single server version (the one in which it was introduced).
• Each server version may have at most one database migration associated with it.

We plan to use umzug to run and keep track of migrations.

• Each migration will be kept as a separate module file under server/migrations, and will follow the naming convention <version>-<migration_name>.js (e.g. v2.13.0-unique_series.js).
• Each migration module will implement and export two functions, up and down, implementing the upwards and downwards migration script, as in the example below:

async function up({context: queryInterface}) {
  // Upwards migration script
  ...
}

async function down({context: queryInterface}) {
  // Downward migration script
  ...
}

module.exports = {up, down}

Migrations will have a natural order determined by the order of their associated server versions.

In the new design, the server settings stored in the database will contain a new field, max_version. That field will hold the maximal server code ever installed with this database.

At server startup, if a version change (server_version != database_version) occurred, the following will happen, after Database.connect(), and before Database.buildModels():

1. if server_version > max_version (or if max_version doesn't exist)
   1. all migration scripts are copied to /config/migrations
   2. max_version is updated to be server_version
2. if server_version > database_version
   1. perform any pending upwards migrations in order, up to and including the closest migration with version <= server_version
      1. (done by calling umzug.up({to: version}))
3. if server_version < database_version
   1. perform downwards migrations in reverse order, down to and including the closest migration with version > server_version
      1. (done by calling umzug.down({to: version}))