A suite of tools to discover, download, normalise, enhance and republish data that uses the OpenActive specification.
Set up a Postgres database and user. Record the details.
These instructions assume Linux.
Set environment variable:
DATABASE_URL=postgres://user:pass@host:port/database
$ npm install
$ npm run migrate
Run each of these in turn to run the full pipeline. See
$ node ./src/bin/spider-data-catalog.js
Fetch feeds$ node ./src/bin/download-raw.js
Download raw data$ node ./src/bin/validate-raw-data.js
Validate Raw data$ node ./src/bin/normalise-data.js
Create normalised version of raw data$ node ./src/bin/profile-normalised-data.js
Profile normalised data (make sure the git submodule conformance-profiles is up to date before running this)
$ npm run start-webserver
Runs webserver providing JSON output of certain data for use by status
Note : Tests are destructive to any existing data. Create a dedicated database before running them.
To run the tests:
$ DATABASE_URL=postgres://test:test@localhost/testing ./node_modules/mocha/bin/mocha