Support for Harbor

[Chaosflo]

Hey folks!
I really like the work you have done here, I tried to use this with combination of habor.

https://github.com/goharbor/harbor

But it doesn't really work sadly, the api calls seem to be off and authorization seems to be broken too.
I have done the authorization via environment variables to test if it works at all (found them in source code)

This is the api error
/opt/azure_repos/home_dir_tgw_user/.nuget/packages/microsoft.net.build.containers/0.4.0/build/Microsoft.NET.Build.Containers.targets(195,5): error CONTAINER1013: Failed to push to the output registry: System.ApplicationException: CONTAINER1001: Failed to upload blob to POST****

[baronfel]

I dug into this a bit on my demo repo: https://github.com/baronfel/sdk-container-demo/actions/runs/4755870576/jobs/8450631677

After some searching (and thanks to the error dump) it seems that Harbor is sending a session token and CSRF token. It appears that their server implementation does CSRF validation even on these endpoints if session information is provided. Based on the Docker API conformance changes here I think we need to update our use of HTTPClient to not automatically handle 'Set-Cookie' headers on responses.

[thompson-shaun]

Is this possibly affecting other registries? I'm currently having issues pushing to our GitLab registries via dotnet publish. Failures are consistent in CI and locally. Using docker push directly works as expected. Error output below

/Users/dev/.nuget/packages/microsoft.net.build.containers/0.4.0/build/Microsoft.NET.Build.Containers.targets(195,5): error CONTAINER1013: Failed to push to the output registry: System.ApplicationException: CONTAINER1001: Failed to upload blob to POST https://registry.gitlab.com/v2/my-user/sandbox/dotnet-container-app/blobs/uploads/; received Unauthorized with detail Docker-Distribution-Api-Version: registry/2.0 [/Users/dev/git/gitlab/dotnet-container-app/src/Worker1/DotNet.ContainerImage1.csproj]
/Users/dev/.nuget/packages/microsoft.net.build.containers/0.4.0/build/Microsoft.NET.Build.Containers.targets(195,5): error CONTAINER1013: WWW-Authenticate: Bearer realm="https://gitlab.com/jwt/auth",service="container_registry",scope="repository:my-user/sandbox/dotnet-container-app:pull,push",error="insufficient_scope" [/Users/dev/git/gitlab/dotnet-container-app/src/Worker1/DotNet.ContainerImage1.csproj]
/Users/dev/.nuget/packages/microsoft.net.build.containers/0.4.0/build/Microsoft.NET.Build.Containers.targets(195,5): error CONTAINER1013: X-Content-Type-Options: nosniff [/Users/dev/git/gitlab/dotnet-container-app/src/Worker1/DotNet.ContainerImage1.csproj]
/Users/dev/.nuget/packages/microsoft.net.build.containers/0.4.0/build/Microsoft.NET.Build.Containers.targets(195,5): error CONTAINER1013: Date: Tue, 09 May 2023 13:34:35 GMT [/Users/dev/git/gitlab/dotnet-container-app/src/Worker1/DotNet.ContainerImage1.csproj]
/Users/dev/.nuget/packages/microsoft.net.build.containers/0.4.0/build/Microsoft.NET.Build.Containers.targets(195,5): error CONTAINER1013:  [/Users/dev/git/gitlab/dotnet-container-app/src/Worker1/DotNet.ContainerImage1.csproj]
/Users/dev/.nuget/packages/microsoft.net.build.containers/0.4.0/build/Microsoft.NET.Build.Containers.targets(195,5): error CONTAINER1013: {"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":[{"Type":"repository","Class":"","Name":"my-user/sandbox/dotnet-container-app","Action":"pull"},{"Type":"repository","Class":"","Name":"my-user/sandbox/dotnet-container-app","Action":"push"}]}]} [/Users/dev/git/gitlab/dotnet-container-app/src/Worker1/DotNet.ContainerImage1.csproj]
....

Or this issue with GitLab solved by dotnet/sdk#32301?

[Pyroluk]

I am experiencing the same issue with the official docker registry 2.0 image using basic authentication (username and password):

2>Failed to push to the output registry: CONTAINER1001: Failed to upload blob to POST https://XXX.XXX:50000/v2/XXX/blobs/uploads/; received Unauthorized with detail Docker-Distribution-Api-Version: registry/2.0

I added docker support to my project via right-click in the project file explorer, then added a publish config with the VS wizard.
Using the Docker Desktop option, it all works perfectly fine. The new .NET SDK option fails with the error message from above.

[baronfel]

@autotmp gitlab.com pushes are working (see https://github.com/baronfel/sdk-container-demo/actions/runs/4810532615/jobs/8563289773) as best as I can see. gitlab.com doesn't use cookies in the way that Harbor does in this issue, so I don't think your report is related to the Cookies issue at the heart of this report. Could you raise a separate issue to track/investigate what's going on with your particular setup so that this issue doesn't get cluttered?

[baronfel]

@Pyroluk can you look at #314 and see if that matches your problem? We don't have a fix for it yet.

[Pyroluk]

@baronfel Yes, same problem. Thank you for pointing me to the issue.

[djheap]

Pushing images to Harbor is an important scenario for us as well and it still appears to be broken. I tried the latest dotnet 8 daily .20x sdk build and still received 'Forbidden' status code failures. From detailed logs I can see:

Response content: {"errors":[{"code":"FORBIDDEN","message":"CSRF token invalid"}]}

The newer demo repo seems to indicate that it has been fixed, however (https://github.com/baronfel/sdk-container-demo/actions/runs/5172500858). Are there fixes not yet included in the daily SDK builds?

[baronfel]

Looks like the cookie disablement didn't make it in - that would need to happen before Harbor support lights up.

[djheap]

Is there an issue for that we can track? I haven't found one that seems to be it.

[baronfel]

@djheap can you try this package and see if that addresses the cookie issue? It's built from the PR I just sent up.

Microsoft.NET.Build.Containers.8.0.200-dev.nupkg.zip

[djheap]

Yes, that worked well, thank you - we are looking forward to significantly simplifying our pipelines!

[baronfel]

Great to hear!

[baronfel]

Closing as it's resolved :)