mongodb-migrations

A Node.js migration framework for MongoDB with both programmatic and CLI API.

Table of Contents generated with DocToc

• Installation
• Common Usage (CLI)
  • Configuration
  • Creating Migrations
    • Migration functions
  • Sample migration file
  • Running migrations
• Programmatic usage
  • Creating Migrator object
    • Custom logging
  • Adding migrations
    • migrator.add
    • migrator.bulkAdd
  • migrator.migrate
  • migrator.runFromDir
  • migrator.rollback
  • migrator.create
  • migrator.dispose

Installation

  npm install mongodb-migrations --save

Common Usage (CLI)

The package installs a single CLI executable — mm.

When installing locally to your project this executable can be found at ./node_modules/.bin/mm.

When installing globally (normally not recommended) the executable should automatically become accessible on your PATH.

Configuration

The CLI app expects a configuration file to be present in the directory where the app is being ran.

By default (if the configuration file is not set through the command-line argument) the app checks mm-config.json, mm-config.js, and mm-config.coffee files for existence.

File name can be passed through the means of --config parameter. The path is relative to the current directory:

  mm --config=configs/mm.json

In case of json the file should contain valid JSON representation of the configuration object.

In case of js or coffee the file should be a CommonJS module exporting the configuration object. This is useful when you already have configuration data (potentially in a different format) and want to avoid duplication. See test/mm-config.coffee for an example of this usage.

In case of coffee the coffee-script >= 1.7.0 package must be importable from the current directory (include it as your project's dependency).

The configuration object can have the following keys:

• host — MongoDB host (optional when using replicaset),
• port — MongoDB port (optional when using replicaset),
• db — MongoDB database name,
• ssl [optional] - boolean, if true, '?ssl=true' is added to the MongoDB URL,
• user [optional] — MongoDB user name when authentication is required,
• password [optional] — MongoDB password when authentication is required,
• collection — The name of the MongoDB collection to track already ran migrations,
• directory — the directory (path relative to the current folder) to store migration files in and read them from,
• timeout [optional] — time in milliseconds after which migration should fail if done() is not called (use 0 to disable timeout)
• poolSize - the size of the mongo connection pool,
• replicaset [optional] - if using replica sets should be an object of the following structure:

name: 'rs-ds023680',
members: [
  {
    host: 'bee.boo.bar',
    port: 23680
  }
  {
    host: 'choo.choo',
    port: 24610
  }
]

Creating Migrations

The app simplifies creating migration stubs by providing a command

  mm create MIGRATION-NAME [--coffee|-c]

This creates automatically numbered file NNN-migration-name.js (or .coffee if -c of --coffee flag provided) inside of the directory defined in the configuration file.

The migration file must be a CommonJS module exporting the following:

• id — a string that's used to identify the migration (filled automatically when creating migrations through mm create).
• up [optional] — a function used for forward migration.
• down [optional] — a function used for backward migration (rollback).

See Configuration if your config file has non-standard name.

Migration functions

The up and down functions take a single parameter — a Node-style callback:

module.exports.up = function (done) {
  // call done() when migration is successfully finished
  // call done(error) in case of error
}

The up and down functions are executed with the scope providing 2 convenient properties:

• this.db is an open MongoDB native driver connection. Useful if you are not using any ODM library.
• this.log is a function allowing you to print informative messages during the progress of your migration. By default these messages are printed to stdout with proper indentation. See Custom logging for advanced usage.

Sample migration file

exports.id = 'create-toby';

exports.up = function (done) {
  var coll = this.db.collection('test');
  coll.insert({ name: 'tobi' }, done);
};

exports.down = function (done) {
  var coll = this.db.collection('test');
  coll.remove({}, done);
};

Running migrations

Run all migrations from the directory (specified in Configuration) by simply calling

mm

or

mm migrate

The library only runs migrations that:

1. have up function defined,
2. were not ran before against this database.

Successfully ran migrations are recorded in the collection specified in Configuration.

The migration process is stopped instantly if some migration fails (returns error in its callback).

See Configuration if your config file has non-standard name.

If you have .coffee migration files, coffee-script >= 1.7.0 package must be importable from the current directory.

Debugging migrations

DEBUG=true mm

Running with a DEBUG=true will print out the error stack on the console.

Programmatic usage

The library also supports programmatic usage.

Start with require'ing it:

var mm = require('mongodb-migrations');

Creating Migrator object

Next, you have to create a Migrator object. The syntax is:

var migrator = new mm.Migrator(config, [customLogFn]);

Where config is an object with the keys defined in the Configuration section (except of the directory which does not make sense in this scenario).

Custom logging

By default when migrations are ran migrator will log it's progress to console — 1 line for each migration added, indicating the status (skipped, succeeded or failed). It will also print any custom messages you pass to this.log inside of the up / down function.

To suppress this logging pass customLogFn = null to the Migrator constructor (undefined won't do the trick).

If you want to handle the logging on your own (save it to file, or whatever else) you can pass your custom function having this signature:

function customLogFn(level, message),

where level is either "system" (migration status message) or "user" (when you call this.log inside of your migration), and message is the actual message string.

Adding migrations

Once you have the migrator object you can add migrations definitions to it.

migrator.add

To add a single migration, call

migrator.add(migrationDef),

where migrationDef is an object with id, up [optional], and down [optional] keys, all having the same meaning as described in Creating Migrations.

migrator.bulkAdd

To add multiple migrations at once, call

migrator.bulkAdd(migrationDefsArray),

where migrationDefsArray is an array of objects explained in migrator.add.

migrator.migrate

Once you have one or more migrations added, run them with calling

migrator.migrate(doneFn, [progressFn]).

Migrations are ran in order they were added to the migrator.

The doneFn is called once all migrations are ran or once one of them fails. The function has the signature

function doneFn(error, results),

where error is null if everything is OK, or is an error returned by the failed migration (if any).

The results object is always passed (even in case of error).

Its keys are ids of the added migrations (till the one that failed, if any, or till the last one). The values are result objects having the following properties:

• status — 'ok', 'skip', or 'error',
• error - the error object returned from the failed migration,
• reason — the reason why the migration was skipped, can be "no migration function for direction up", "no migration function for direction down", "migration already ran", "migration wasn't in the recent migrate run". See Rollback for the explanation of the last case.

The optional progressFn function is called once per each migration and has the signature

function progressFn(id, result),

where id is migration's ID, and result object is explained above.

migrator.runFromDir

In case your migrations are modules in specific directory (see Running migrations) there's a convenience method that reads them in order and then runs.

The files must conform to the following rules:

1. have their names starting with one or more digits (this number defines the migrations order) — proper naming is held when Creating Migrations using the CLI mm tool,
2. be CommonJS modules and export id, up [optional], and down [optional] — see Creating Migrations for explanation,
3. if the migration file has .coffee extension, the coffee-script >= 1.7.0 package must be importable from the current directory.

To run the migrations from the directory call

migrator.runFromDir(directory, doneFn, [progressFn])

The doneFn and progressFn have the same meaning as in migrator.migrate.

migrator.rollback

If you decide that current migration run was unsuccessful, you can roll back all recently ran transactions. Currently this operation is only supported through programatic interface.

Do so by calling

migrator.rollback()

This runs all the migrations added to the migrator in the reverse order, and follows these rules:

1. migrations now having down method are skipped,
2. migrations not ran recently (potentially those after the failed one) are skipped.

migrator.create

To programmatically create a migration stub file, call

migrator.create(directory, id, doneFn, coffeeScript=false),

where directory is the directory to save the file to, id is migration's ID, doneFn is a callback that gets passed the error object in case of error, and optional coffeeScript flag tells the library to create the stub in CoffeeScript instead of plain JavaScript.

The ID is lowercased and then dasherized. It's your responsibility to assure it's unique.

The method automatically handles files numbering and naming, and sets the ID inside of the generated file.

migrator.dispose

When you are done with the migrator you should call

migrator.dispose(cb)

to release the MongoDB connections pool. Once disposed the migrator cannot be used anymore.

The cb is a Node-style callback:

function cb(error).