Issue/279 heroku background worker

[iturgeon]

The main goal of this PR is to add a background worker mode to UDOIT. This allows UDOIT to support more simultaneous users and much larger courses. This is ideal for small Heroku accounts or web servers that support multiple applications.

The worker mode is an opt-in upgrade. UDOIT can be run in classic mode w/o any modifications by your server admin, but it will continue to be limited in the same way prior versions were limited.

Limitations present in Classic mode (and all versions prior to 2.3.0):

• NGINX/Apache timeouts can easily be exceeded before a course scan finishes.
• Load Balancers timeouts can easily be exceeded before a course scan finishes.
• PHP.ini max_execution_time can easily be exceeded before a course scan finishes.
• A single user consumes an entire Heroku dyno during a scan.

Changes in this PR:

• Adds background worker mode. (UFixit still needs to be backroundized)
• Splits up scanning into multiple "jobs"
• Simplifies scanning and api code
• Adds composer's autoloading for classes in lib/
• Cleans up some of the request vars for process.php
• Separates tasks for starting a scan, checking progress, and loading results
• Adds some early support for sqlite (mostly for testing)
• Adds a lot more unit tests
• Adds lint scripts to make sure all php compiles
• Adds phpCS code sniffing and code standard scripts
• Removes fractured pagination api code
• Adds php 7.0 and 7.1 to the supported list and Travis tests
• clean up canvas_api_domain to prevent multiple trailing slashes
• update logo to use svg
• moves key in api requests from get params into request header
• adds logging of each api request via monolog's addInfo()
• add user id checking to process.php to make sure jobs aren't started without the needed vars
• change process.php errors to use json and display them in page
• adds UDOIT_VERSION and displays it in the header
• now ensures the report shown to the user is sorted in a specific order
• docker setup updated to aid testing in multiple php versions
• adds another table for job queue (and matures the migrations slightly)
• adds a background worker for new heroku installs
• opens videos found in module_urls in a new window (the canvas links to them stopped working, or never did)

Fixes #279

[iturgeon]

This isn't ready to merge yet, just opened it up while I work through a few remaining issues.

[iturgeon]

This ended up being pretty huge and pushing the codebase forward quite a bit. I updated the pull request overview above.

[bagofarms]

I'm reviewing the PR now, but please merge in dev/v2.3.0 and resolve any conflicts.

[bagofarms]

Additionally, Heroku doesn't seem to see the app.json, meaning the database never gets built. Judging by the build log, the composer script gets run, installing all of the dependencies, but the database never gets set up. I've added you to my Heroku instance so you can take a look.

[iturgeon]

I'm working on resolving the conflicts.

Looks like I need to make a migration script to move to the worker version

[iturgeon]

Migration added for the job_queue table and fixed all the problems from the merge. We should be able to get back to testing

[bagofarms]

Detecting YouTube videos without captions doesn't work in Module URLs.
Intermittent error where it only scans module_urls and unscannable, producing zero errors and zero suggestions. The JSON looks like this in the database:
{"total_results":{"errors":0,"warnings":0,"suggestions":0},"content":{"module_urls":{"title":"module_urls","items":[],"amount":0,"time":0},"unscannable":{"title":"unscannable","items":[],"amount":0}},"course":null}

[iturgeon]

Alright, that last commit contained a bunch of fixes for module_urls and other things.

I updated the bulleted list of features in the pull PR.

I think this is buttoned up at this point. Recommend final testing 🎉

[bagofarms]

I'm unable to complete a scan, receiving this error: Warning: Invalid argument supplied for foreach() in /app/lib/UdoitUtils.php on line 255

During the scan, announcements, assignments, etc. stacked up in the scan button without ever seeming to complete.

I updated the composer.lock file since there were updates to the composer.json that heroku wasn't taking into account, but that didn't fix the issue.

[iturgeon]

I was able to fix the files being missed in the scanner. Also found syllabus wasn't scanning and was able to fix up the composer.json/lock files so composer update will work again.