Single sketches reports workflow artifact creation via arduino/compile-sketches job matrix not allowed by v4 of actions/upload-artifact

[per1234]

Describe the problem

It is common for Arduino projects to target multiple boards. In this case, the project maintainer may wish to monitor the memory usage deltas for several boards. This is accomplished by using the arduino/compile-sketches action to compile the sketches for each targeted board. A separate sketches report file is generated for each board and the arduino/report-size-deltas action is designed to consume such a collection of multiple report files.

The common practice is to use a matrix for the jobs that run the arduino/compile-sketches action. The sketches report file generated by the arduino/compile-sketches step of each matrix job is uploaded to a single workflow artifact via the actions/upload-artifact action.

For example:

jobs:
  compile:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        fqbn:
          - "arduino:avr:uno"
          - "arduino:samd:mkrzero"
    steps:
      - uses: actions/checkout@v4

      - uses: arduino/compile-sketches@v1
        with:
          fqbn: ${{ matrix.fqbn }}
          enable-deltas-report: true
          sketches-report-path: sketches-reports

      - name: Upload sketches report to workflow artifact
        uses: actions/upload-artifact@v3
        with:
          name: sketches-reports
          path: sketches-reports

In the use case (as described here) where the arduino/report-size-deltas action is run from a workflow triggered by a schedule event, the action downloads a single workflow artifact for each pull request that requires a deltas report. That artifact must contain all the sketches reports from which the size deltas report will be generated. There is no provision for downloading multiple artifacts per pull request.

Starting from version 4.0.0 of actions/upload-artifact, uploading multiple times to a single artifact (as is the common practice in the arduino/compile-sketches matrix jobs) is not possible:

https://github.com/actions/upload-artifact#breaking-changes

2. Uploading to the same named Artifact multiple times.

Due to how Artifacts are created in this new version, it is no longer possible to upload to the same named Artifact multiple times. You must either split the uploads into multiple Artifacts with different names, or only upload once. Otherwise you will encounter an error.

The actions/upload-artifact step of the workflow will fail if a job attempts this:

Run actions/upload-artifact@v4
With the provided path, there will be 1 file uploaded
Artifact name is valid!
Root directory input is valid!
Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

For the arduino/compile-sketches matrix jobs, the best way to adapt to this change would be configuring the workflow so that each job uploads its sketches report file to a different workflow artifact (resulting in multiple artifacts):

jobs:
  compile:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        board:
          - fqbn: "arduino:avr:uno"
            sketches-report-artifact: arduino-avr-uno
          - fqbn: "arduino:samd:mkrzero"
            sketches-report-artifact: arduino-samd-mkrzero
    steps:
      - uses: actions/checkout@v4

      - uses: arduino/compile-sketches@v1
        with:
          fqbn: ${{ matrix.board.fqbn }}
          enable-deltas-report: true
          sketches-report-path: sketches-reports

      - name: Upload sketches report to workflow artifact
        uses: actions/upload-artifact@v3
        with:
          name: ${{ matrix.board.sketches-report-artifact }}
          path: sketches-reports

🐛 The schedule event-triggered action's artifact download system doesn't support downloading multiple sketches report artifacts, making it incompatible with the best solution for the problem of uploading artifacts using version 4.x of actions/upload-artifact.

To reproduce

1. Create a workflow that uses a job matrix to upload multiple sketch report files to a single workflow artifact using actions/upload-artifact@v3:

   on: [push, pull_request]
   jobs:
     compile:
       runs-on: ubuntu-latest
       strategy:
         matrix:
           board:
             - fqbn: "arduino:avr:uno"
             - fqbn: "arduino:samd:mkrzero"
       steps:
         - uses: actions/checkout@v4
   
         - uses: arduino/compile-sketches@v1
           with:
             fqbn: ${{ matrix.board.fqbn }}
             libraries: |
               - Servo
             sketch-paths: |
               - ~/Arduino/libraries/Servo/examples
             enable-deltas-report: true
             sketches-report-path: sketches-reports
   
         - name: Upload sketches report to workflow artifact
           uses: actions/upload-artifact@v3
           with:
             name: sketches-reports
             path: sketches-reports

   🙂 The workflow run is successful
2. Bump the actions/upload-artifact action dependency to v4 in the workflow.
   🙁 The workflow run fails:

   Run actions/upload-artifact@v4
   With the provided path, there will be 1 file uploaded
   Artifact name is valid!
   Root directory input is valid!
   Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run
   

3. Reconfigure the workflow so that it uploads each sketches report to a separate workflow artifact:

   on: [push, pull_request]
   jobs:
     compile:
       runs-on: ubuntu-latest
       strategy:
         matrix:
           board:
             - fqbn: "arduino:avr:uno"
               sketches-report-artifact-suffix: arduino-avr-uno
             - fqbn: "arduino:samd:mkrzero"
               sketches-report-artifact-suffix: arduino-samd-mkrzero
       steps:
         - uses: actions/checkout@v4
   
         - uses: arduino/compile-sketches@v1
           with:
             fqbn: ${{ matrix.board.fqbn }}
             libraries: |
               - Servo
             sketch-paths: |
               - ~/Arduino/libraries/Servo/examples
             enable-deltas-report: true
             sketches-report-path: sketches-reports
   
         - name: Upload sketches report to workflow artifact
           uses: actions/upload-artifact@v3
           with:
             name: sketches-report-${{ matrix.board.sketches-report-artifact-suffix }}
             path: sketches-reports

4. Submit a pull request that triggers the workflow run.
   Two workflow artifacts are generated, with the following names
   • sketches-report-arduino-avr-uno
   • sketches-report-arduino-samd-mkrzero
5. Attempt to reconfigure the schedule event triggered arduino/report-size-deltas workflow to produce a deltas report from all the sketches reports generated by the arduino/compile-sketches workflow run:

   on:
     schedule:
       - cron:  '*/5 * * * *'
   jobs:
     build:
       runs-on: ubuntu-latest
       steps:
         - uses: arduino/report-size-deltas@v1
           with:
             # Globs/patterns are not supported:
             # sketches-reports-source: sketches-report-*
   
             # Lists of artifact names are not supported:
             # sketches-reports-source: |
             #   - sketches-report-arduino-avr-uno
             #   - sketches-report-arduino-samd-mkrzero
   
             # So I guess I can only get a report for one of the boards? :(
             sketches-reports-source: sketches-report-arduino-avr-uno

🐛 It is not possible to reconfigure the schedule event-triggered arduino/report-size-deltas workflow to produce a deltas report from all the sketches reports generated by the arduino/compile-sketches workflow run.

Expected behavior

The arduino/report-size-deltas action supports convenient/efficient practices for generating sketches reports with the modern versions of the actions/upload-artifact action.

This means it must have the capability to download multiple sketches report workflow artifacts per pull request.

arduino/report-size-deltas version

34c4c04

Additional context

I think the best solution will be to change the action so that the sketches-reports-source input is used to specify an artifact name pattern. All artifacts with a name matching that pattern will be downloaded and used to generate the size deltas report. This would not be a breaking change since an exact single artifact name input value, as used with the current action design will continue to match the single artifact's name just as before even after the input value is treated as a pattern instead of an exact name.

The decision will need to be made about which pattern syntax to use (regular expression vs. glob).

Workaround

Use version 3.x of the actions/upload-artifact action to upload the sketches reports to the artifact in your sketch compilation job (as shown in the example from the readme):

- uses: actions/upload-artifact@v3
  with:
    name: sketches-reports
    path: sketches-reports

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the action from the latest version
• My report contains all necessary details