Run e2e tests against deployed environments

[nickclyde]

Description

Run e2e tests against deployed environments. This should run against an environment that's deployed against the regular flow, and uses Okta Preview and the Experian demo account. We'll need to add dev back to the deployment pipeline for this to work.

Proposed solution

After deploying to training or demo, add a step to run the e2e tests, configured to point at the dev URL

Additional context

This ticket was created in response to this postmortem

We may need to add different profiles for running the e2e tests locally vs in deployed environments. Locally we'll want to use Wiremock instead of Okta preview, for instance.

This should be run before deploying a PR to production, rather than just doing it daily. We may need to add a flag enabling users to bypass e2e tests to respond to production incidents.

[emmastephenson]

It's worth deferring this ticket until the e2e tests are written/rewritten, so we have a ballpark idea of how long these tests will take.

[zdeveloper]

Add a deployment to prod blocking step (parallel to deployment to stg) that will run e2e against a real env deployed on azure,
Run against dev at the moment until we stand up a new env specifically for this purpose

[emmastephenson]

Next steps on this:

• test environment will be used for e2es (and other automated tests) going forward. As such it needs to be put back in the deployment pipeline.
• stg can't be used here because it's on prod Okta and we don't want an e2e Okta user in prod.
• Rin to talk to CDC Azure folks and ask for new resource groups for:
  • dev II
  • feature branches
    This ticket is blocked until dev II is up and running, so that we continue to have the same number of environments available for local development.