-
Notifications
You must be signed in to change notification settings - Fork 251
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Maze Runner scaffolding for Lambda tests #1297
Add Maze Runner scaffolding for Lambda tests #1297
Conversation
This will: 1. Pack all relevant Bugsnag packages 2. Install them into any fixtures in `features/fixtures` 3. Build each fixture 4. Tidy up after itself
This stops a ruby backtrace from showing, which can be confusing - the error happened in the 'build-fixtures' script rather than env.rb
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
I have a small concern regarding similar work being done in slightly different ways in different languages. The packing of fixtures is now done in Ruby and JS for these tests and the browser/node end to end tests. Maybe the context is too different that we can't consolidate the work, but I think it's worth a look.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
I didn't realise we had packing in JS land too; I thought that was done by the dockerfiles and so had to be done differently for these tests as they can't run in docker. It's still a bit different as this is run by MR automatically rather than being a manual script I think there's still an advantage to this being in Ruby; the buildkite boxes that will run these tests won't need Node installed as the JS execution is done in docker by SAM, but they do need Ruby to run Maze Runner so it cuts out a dependency We could rewrite it in JS but I think it'd be more code & dependencies (e.g. for the YAML parsing) and be less robust, e.g. |
Maybe we rewrite the JS version in ruby then? This is the script I'm talking about https://github.com/bugsnag/bugsnag-js/blob/next/bin/local-test-util It builds the browser notifier(s) and test fixtures outside of docker for quicker iteration in development (I was mistaken when I said node too). |
I would back that - it could move us a step closer to a "Maze Builder". |
Created https://bugsnag.atlassian.net/browse/PLAT-6086 for rewriting |
Goal
This PR adds the scaffolding necessary to be able to run Maze Runner tests against Lambda functions. The fixture and tests are by no means complete, but will be expanded on as further work. This PR is purely to prove that we can build, and run tests against, a Lambda fixture
Each commit is standalone so can be reviewed separately if that's easier
Testing
As we can't run tests using SAM on Buildkite yet, this has to be run locally for now:
Install Maze Runner
Run the tests
$ bundle exec maze-runner
Note: the tests don't assert against Bugsnag payloads as we don't guarantee delivery in Lambda functions (yet!) so they assert against the Lambda responses instead