UMich ArcLight (Finding Aids)

Discovery and access application for archival material at University of Michigan Libraries.

A front-end for archival finding aids / collection guides built on the ArcLight Rails Engine.

The application currently runs at https://findingaids.lib.umich.edu.

Development Quick Start

Handy Dandy Aliases (Optional)

alias dc="docker-compose"
alias dce="dc exec --"
alias abe="dce app bundle exec"

NOTES

• To revert back to the old way just copy docker-compose.bak over docker-compose.yml file. a.k.a. cp docker-compose.bak docker-compose.yml

Build application image

The default is amd64 architecture a.k.a. Intel

docker-compose build app

For Apple Silicon use arm64 architecture

docker-compose build --build-arg ARCH=arm64 app

Bring up development environment

docker-compose up -d

NOTES

• The resque and resque-web containers will exit because we have yet to do a bundle install!.

Install bundler

docker-compose exec -- app gem install 'bundler:~>2.2.21'
docker-compose exec -- app bundle config --local build.sassc --disable-march-tune-native

This was moved into the Dockerfile so it is no longer is necessary but is left here as a reminder so it will not be forgotten.

Need to revisit why this is necessary in the first place!

Bundle install

docker-compose exec -- app bundle install

NOTES

• Environment variable BUNDLE_PATH is set to /var/opt/app/gems in the Dockerfile.

Yarn install

docker-compose exec -- app yarn install

NOTES

• Investigate using a volume for node_modules directory like we do for gems

Setup databases

docker-compose exec -- app bundle exec rails db:setup

If you need to recreate the databases run db:drop and then db:setup.

docker-compose exec -- app bundle exec rails db:drop
docker-compose exec -- app bundle exec rails db:setup

NOTES

• Names of the databases are defined in ./config/database.yml
• The environment variable DATABASE_URL takes precedence over configured values.

Create solr cores

docker-compose exec -- solr solr create_core -d umich-arclight -c umich-arclight-development 
docker-compose exec -- solr solr create_core -d umich-arclight -c umich-arclight-test 

If you need to recreate a core run delete and create_core (e.g. umich-arclight-test)

docker-compose exec -- solr solr delete -c umich-arclight-test
docker-compose exec -- solr solr create_core -d umich-arclight -c umich-arclight-test 

NOTES

• Names of the solr cores are defined in ./config/blacklight.yml file.
• The environment variable SOLR_URL takes precedence over configured values.

Restart Resque and Resque-Web

docker-compose restart resque
docker-compose restart resque-web

NOTES

• Resque.redis is assigned in the ./config/initializers/resque.rb file.
• The REDIS_URL is referenced in the ./config/cable.yml file.
• The environment variable REDIS_URL takes precedence over configured values.
• Environment variable REDIS_URL is set to redis://redis:6379 in the docker-compose.yml file.

Start development rails server

docker-compose exec -- app bundle exec rails s -b 0.0.0.0

Verify the application is running http://localhost:3000/

Bring it all down then back up again

docker-compose down

docker-compose up -d

docker-compose exec -- app bundle exec rails s -b 0.0.0.0

The gems, database, solr, and redis use volumes to persit between the ups and downs of development. When things get flakey you have the option to simply delete any or all volumes after you bring it all down. If you remove all volumes just repeat the Development quick start, otherwise you'll need to run the appropriate steps depending on which volumes you deleted:

• For gems run the Bundle install and Restart Resque and Resque-Web steps.
• For database run the Setup databases step.
• For solr run the Create solr cores step.
• For redis there is nothing else to do.

Indexing Encoded Archival Description (EAD) Files

Initially the application data directory contains only the ead2002 directory which contains the 2002 specifications.

/var/opt/app/data/ead2002
  /related_optional
  dtd2schema.xsl
  ead.dtd
  ead2002.xsd

The convention is for finding aid EAD XML files to be located in the appropriate repository subdirectory under the ead directory in the application data directory where the repository slug is used as the subdirectory name.

/var/opt/app/data
  /ead
    /bhl
    /clarke
    /clements
    ...

If you don't have any particular finding aid EAD XML files you can just copy the sample-ead/ead directory into the application data directory.

docker-compose exec -- app cp -r sample-ead/ead /var/opt/app/data

or equivalently (because of the way the volume is mapped in the docker-compose.yml file)

cp -r ./sample-ead/ead ./data

NOTES

• Environment variable FINDING_AID_DATA is set to /var/opt/app/data in the Dockerfile.
• The docker volume data is mapped to ${PWD}/data (a.k.a. /opt/app/data) in the docker-compose.yml file.

Index all the EAD files.

docker-compose exec -- app bundle exec rake dul_arclight:reindex_everything

Background processing jobs for indexing may be monitored via resque-web http://localhost:8080/overview

Index a repository's EAD files (e.g. Bently Historical Library; repository ID bhl).

docker-compose exec -- app bundle exec rake dul_arclight:reindex_repository REPOSITORY_ID=bhl

Index a single EAD file (e.g. Bently Historical Library; repository ID bhl).

docker-compose exec -- app bundle exec rake arclight:index REPOSITORY_ID=bhl FILE=/var/opt/app/data/ead/bhl/umich-bhl-032.xml

Clear the Solr index.

docker-compose exec -- app bundle exec rake arclight:destroy_index_docs

NOTES

• Repositories are define in ./config/repositories.yml
• The default application data directory is /data and is declared in ./lib/dul_arclight.rb.
• The environment variable FINDING_AID_DATA takes precedence over the default value.
• Environment variable FINDING_AID_DATA is set to /var/opt/app/data in the Dockerfile.

Continuous Integration

Continuous integration is the default rake task and is defined in the ./lib/tasks/default.rake file.

docker-compose exec -- app bundle exec rake

It actually does nothing but is dependent on these rake tasks:

JavaScript Linting

docker-compose exec -- app bundle exec rake lint

Ruby Linting

docker-compose exec -- app bundle exec rake rubocop

Tests

docker-compose exec -- app bundle exec rake test

Specs

docker-compose exec -- app bundle exec rake spec

You may also call these commands directly

docker-compose exec -- app yarn lint
docker-compose exec -- app bundle exec rubocop
docker-compose exec -- app bundle exec test
docker-compose exec -- app bundle exec rspec

List all rake tasks

docker-compose exec -- app bundle exec rake -T

Environment Variables

Name	Comment
BUNDLE_PATH	Path to application bundle gems directory
DATABASE_URL	Database connection URL
FINDING_AID_DATA	Path to application data directory
GOOGLE_ANALYTICS_DEBUG	Google Analytics debug flag
GOOGLE_ANALYTICS_TRACKING_ID	Google Analytics tracking ID
RAILS_ENV	Rails enviroment: development, test, or production
REDIS_URL	Redis endpoint URL
SOLR_URL	Solr core URL

List all environment variables

docker-compose exec -- app env

Local Ports

Port	Container	Comment	Endpoint
3000	app	Rails Application	http://localhost:3000/
1234	app	RubyMine IDE
26162	app	RubyMine IDE
5432	db	Postgres Server
8983	solr	Solr Server	http://localhost:8983/solr
6579	redis	Redis Server
8282	resque	Resque Workers
8080	resque-web	Resque Web	http://localhost:8080/overview

Volumes

Volume	Container	Mount
umich-arclight_data	app, resque	/var/opt/app/data
umich-arclight_gems	app, resque, resque-web	/var/opt/app/gems
umich-arclight_db-data	db	/var/lib/postgresql/data/db
umich-arclight_solr-conf	solr	/opt/solr/server/solr/configsets/umich-arclight:ro
umich-arclight_solr-data	solr	/var/solr
umich-arclight_redis-data	redis	/data

Resources

• ArcLight on GitHub
• ArcLight project wiki
• Docker
• Rails
• Ruby