Run tests on GKE spot instances

[cardil]

GKE provides, so-called spot instances. The main benefit, of using them, is a massive cost savings. The GKE with Spot instances doc describes how to use them.

Although, it would be a good idea to run tests on those nodes, some changes are probably needed in Knative:

• ensure the tests could survive node termination
• reconfigure test clusters to use more, but smaller nodes

[jasonsmithio]

/assign

[jasonsmithio]

A question with regards to this. What are the outcomes we are looking for in this test? I am happy to run it but want to know what would be considered a success wrt this issue.

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

[cardil]

/remove-lifecycle stale

@jasonsmithio Thanks, you wanna tackle this one. However, It is not a matter of writing some new tests for spot instances. The current Knative e2e tests have to work on a test cluster made of spot instances (the nodes of the cluster could be evicted at any time). We need to make sure that the tests do not produce false negatives due to random node evictions.

[jasonsmithio]

gotcha. It may take a while to test as we need the nodes to evict and do it enough times to collect enough data.