GitHub Actions Failures

[ryanjbaxter]

@wind57 this failure consistently happens, I am not sure what happens that causes the zip to not be there
https://github.com/spring-cloud/spring-cloud-kubernetes/actions/runs/5824481925/job/15794694156#step:35:395

[wind57]

The previous step needs to download some artifacts, the timeout to do that is 30 minutes... The next step fails to find it, of course. Either way, that is very annoying indeed :( I'll take a closer look.

[wind57]

a few things for history here, as I have started working on it. The process is this atm:

• build docker images needed for our integration tests
• save these images (via actions/cache@v3)
• start matrix steps to run the tests (we do this in parallel)
• in each step of the matrix, download those docker images (via same actions/cache@v3)

Of course, the same exact images are downloaded for each matrix step. One might think that this is at least sub-optimal, and we could download them once, per all matrix steps. Something like this in pseudo-code:

  test_when_cache_present:
    needs: [ build ]
    runs-on: ubuntu-latest
    env:
      SEGMENT_DOWNLOAD_TIMEOUT_MINS: 30
    # only run this one if there is a previous cache of test times
    if: needs.build.outputs.test_times_cache_present == 'true'
    timeout-minutes: 60

    #####################################################
    # have some sort of a "step" here that would download docker images
    #####################################################

    strategy:
      fail-fast: true
      matrix:
        current_index: [ "${{ fromJSON(needs.build.outputs.matrix_array) }}" ]
        number_of_jobs: [ "${{ fromJSON(needs.build.outputs.number_of_matrix_instances) }}" ]

Unfortunately, github actions does not support such a thing, and this is why we download the same images in every single matrix step. If at least one step timeouts (which happens in our case), the entire flow breaks and this is exactly what we see.

[ryanjbaxter]

What does it mean for a step to timeout?

[wind57]

this

Basically download of a cache is "stuck", as seen in our logs:

Received 3351248896 of 3790969212 (88.4%), 24.0 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 23.8 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 23.6 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 23.5 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 23.3 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 23.1 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 23.0 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 22.8 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 22.6 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 22.5 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 22.3 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 22.2 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 21.9 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 21.7 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 21.6 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 21.4 MBs/sec
Received 3351248896 of 3790969212 (88.4%), 21.4 MBs/sec
Warning: Failed to restore: The operation cannot be completed in timeout.
Cache not found for input keys: docker-images-cache-5824481925, docker-images-cache-5824481925

[wind57]

we can close this one btw