actions/upload-artifact & actions/download-artifact v4 support

[Gedochao]

Describe

The popular https://github.com/actions/upload-artifact & https://github.com/actions/download-artifact recently got a major update to v4, which is incompatible with v3 actions.
The v4 release offers a considerable boost in the efficiency of artifacts' uploads & downloads, so it's generally a much desired bump.
By the look of things, this breaks the flow for using the test-reporter with public repositories, as test-reporter uses those with v3.

Example of a bump which seems to break the test reports flow (Scala CLI repo)

I did not do in-depth research on this topic, but it seems that bumping to v4 in the Scala CLI repo breaks the test reports:

• Bump actions/upload-artifact & actions/download-artifact from 3 to 4 VirtusLab/scala-cli#2701

We use the public repo setup like this:
https://github.com/VirtusLab/scala-cli/blob/3dcfb3357d9c35bd71be27548e7e4e82fc6ef528/.github/workflows/test-report.yml
So we generate test reports with Mill, use our custom script to change them into the JUnit XML format and then upload the artifacts for the test report workflow to pick it up, nothing special. Should be seen as the standard java-junit flow for a public repo.

Proposed solution

Bump test-reporter actions to v4 & probably release a v2 version of the test-reporter action, to prevent breaking old workflows and let them bump their stuff in peace.

Alternatives considered

Stay at v3, I guess?

[Gedochao]

Extra context: this is sneaky, as it's very easy to bump to v4 without reading the release notes, thus only realising the test reports are broken when inspecting them.
Users may also be confused when setting up the action for the first time and realising it doesn't work, not knowing why (so as a temporary measure, maybe it'd be useful if the test-reporter@v1 docs mentioned the incompatibility 🤔

[ScribbleTAS]

Just updated to v4 and indeed, test-reporter is causing issues...

Error: HTTPError: Response code 400 (Authentication information is not given in the correct format. Check the value of Authorization header.)

Will downgrade again, but I'm looking forward to v2...

And I agree, having a

Warning

actions/upload-artifact & actions/download-artifact version 4 is currently not supported, downgrade to v3

or something like that, would really help clear up any confusion

[lauxjpn]

As a workaround, we manually download and extract our test results containing artifact in a previous step (that we uploaded before using our CI workflow and actions/upload-artifact@v4) using actions/download-artifact@v4 or an actions/github-script@v7 script, and then use dorny/test-reporter@v1 without the artifact option. That way, path just represents the local file system instead the one contained in an artifact.

test-reporter/src/main.ts

Lines 82 to 92 in eaa763f

	const inputProvider = this.artifact
	? new ArtifactProvider(
	this.octokit,
	this.artifact,
	this.name,
	pattern,
	this.context.sha,
	this.context.runId,
	this.token
	)
	: new LocalFileProvider(this.name, pattern)

[jozefizso]

The #438 should fix the problem reported by @ScribbleTAS. The original issue will need another fix I think.

[JojOatXGME]

The #438 should fix the problem reported by @ScribbleTAS. The original issue will need another fix I think.

@jozefizso and @Gedochao, what is the original issue? If there is another issue here, it is not clear to me what it is. I think the action should work with Artifacts v4 now.

Background: As far as I know, this action doesn't use @actions/artifacts, but instead communicates with the GitHub REST API. The backward incompatible changes which have been announced by GitHub for Artifacts v4 should not affect the GitHub REST API, but only some Runner-specific API to access Artifacts while the workflow was still in progress. Note that before Artifacts v4, Artifacts were not available over the GitHub REST API before the workflow has finished. Besides all of that, I suspect that GitHub made some internal architectural changes with Artifact v4 which caused the error reported by @ScribbleTAS. I am not aware of any other issue related to Artifacts v4.

[ScribbleTAS]

Hi @jozefizso and @JojOatXGME ,
thank you for working on this issue.

I can confirm that it works now!

[JojOatXGME]

@ScribbleTAS Since you have edited your comment and removed the block about your issue, I assume you have resolved it.

I have been testing the new version and it fixes the error, but I'm getting a new error:

Error: HttpError: Resource not accessible by integration

For everyone else, the error did occur after the artifacts have already been downloaded and when the report was about to be created. While I don't know much about the case of @ScribbleTAS, this error usually means that your job doesn't have the checks: write permission. (Either because you configured restricted permissions by default and forgot to specify the permission in the workflow file, or because the build was triggered from a fork and you did not follow the recommended setup for public repos.)

PS: Also note that dorny/test-reporter@v1 still serves 1.9.0 which does not contain the fix. See #444. I just wanted to mention it as it might cause some confusion if someone wants to try out the new version.

[ScribbleTAS]

this error usually means that your job doesn't have the checks: write permission.

This was exactly the issue here. I checked the readme, saw the section with the permissions and immediately put two and two together on why my workflow was failing, even for upload v3.

[AliakseiT]

For all who is affected by the CVE-2024-42471, fix permissions as explained here https://stackoverflow.com/a/78679516/5608789

[JojOatXGME]

FYI, dorny/test-reporter@v1 runs together with actions/upload-artifact@v4 at JetBrains/gradle-changelog-plugin. All the runs started to fail after actions/upload-artifact has been updated to version 4, but after #444 has been resolved, all the runs were green again.

In short: I still believe this issue and #343 is already resolved since version 1.9.1.

[HofmeisterAn]

Thanks, @JojOatXGME, for the info. After pinning the version to 1.9.1, I was able to remove actions/upload-artifact@v3 and update it to version 4 as well 👍.