feat(ci): Add e2e tests against latest docker images

[alexvy86]

Description

Adds a new stage for e2e tests against a local routerlicious environment spun up with docker in the build agent. This exercises our e2e tests against the current version (head of main) of all server code (vs the tests we run against the internal routerlicious deployment in AKS, which runs not-terribly-old-but-never-head-of-main docker images of the server components), which should help get us some confidence that the current version of server isn't breaking something (which in fact already seems to have happened; keep reading).

For this, a few adjustments (and fixes?) were necessary in the docker setup, server configuration, and the e2e tests pipeline:

• Add an nginx container to the docker environment spun up from server/docker-compose.yml, with the nginx configuration defined in the repo. This is necessary so recent changes that cause alfred to redirect certain requests to nexus are in place. Otherwise older FF versions do not work against this environment.
• Split the image tag used for historian from the ones used for gitrest and gitssh. They come from different pipelines from different build numbers so with a single variable we would not be able to find all of them at the same time.
• Enable whole summary upload by default in alfred. Without this, I was able to run things correctly using the latest routerlicious + gitrest + gitssh images, but when adding the latest historian image into the mix, I started seeing many errors in the historian logs, and the tests ran, but they did it way slower. I don't understand the interaction here, but we're moving away from shredded summary upload anyway, so changing the default in the docker images to be whole summary upload seems to make sense regardless.
  • In my opinion, this basically means that we (kind of?) broke the server environment provided by our docker images at some point. The tests still work, but something seems to not be working as intended, if one were to spin up a server environment with the latest docker images available internally. It has had no public impact because the image versions that are available publicly for historian, gitrest, and gitssh, are pretty old (and things seem to work fine with those, but they're far from being the latest).
• Update the template for e2e test runs in the pipeline so it can take pre/post steps (post-steps capability was already there but unused), where the new stage can spin up and spin down the docker environment.

Reviewer Guidance

The review process is outlined on this wiki page.

This sample run (msft internal) was done with only the new stage. Runtimes are pretty good and no tests failed. And this run is more recent (18a1ae0) and with a more final version of this PR; all tests still passing.

[alexvy86]

Server folks (@znewton , @pradeepvairamani, tag anyone else that could/should chime in), I'm interested in your thoughts on the changes to the docker compose files and the routerlicious config. I'm not familiar with any workflows you guys might have with these files, but I think everything here is safe and/or necessary for a fully functional local environment (including supporting older FF client versions).

[alexvy86]

Used this locally to validate running e2e tests against the latest server image, but with whole summary upload enabled (the image itself doesn't do it). This will go away before merging.

[alexvy86]

Will remove too, this was also for local testing.

[alexvy86]

The new stage doesn't use it right now, and if nothing is passed this ends up being a syntax error with the current code.