HTTPError: 400 Bad Request from https://test.pypi.org/legacy/ Bad Request for Github Actions on Test PyPI

[thibmonsel]

This is maybe a bug but i'm not sure. I've been following the tutorial for automating the push of latest packages on pip from here. I'm trying to do it on Test PyPI for now.

I have the following workflow to release the distribution package on Test PyPI :

name: Publish Python 🐍 distribution 📦 to PyPI and TestPyPI

on: push

jobs:
  build:
    name: Build distribution 📦
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4
    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: "3.11"
    - name: Install pypa/build
      run: >-
        python3 -m
        pip install
        build
        --user
    - name: Install dependencies and Run tests
      run: |
        python -m pip install --upgrade pip
        python -m pip install .
        python -m pip install -r test/requirements.txt
        pytest test
    - name: Build a binary wheel and a source tarball
      run: python3 -m build
    - name: Store the distribution packages
      uses: actions/upload-artifact@v4
      with:
        name: python-package-distributions
        path: dist/

  publish-to-testpypi:
    name: Publish Python 🐍 distribution 📦 to TestPyPI
    needs:
    - build
    runs-on: ubuntu-latest

    environment:
      name: testpypi
      url: https://test.pypi.org/p/torchdde

    permissions:
      id-token: write

    steps:
    - name: Download all the dists
      uses: actions/download-artifact@v4
      with:
        name: python-package-distributions
        path: dist/
    - name: Publish package distributions to TestPyPI
      uses: pypa/gh-action-pypi-publish@release/v1
      with:
        repository-url: https://test.pypi.org/legacy/
        verbose: true

I have also followed to add the trusted publishers from the PyPI website so there is no need to specify the API tokens apparently.

The issue arises at Publish package distributions to TestPyPI :

Run pypa/gh-action-pypi-publish@release/v1
  with:
    repository-url: https://test.pypi.org/legacy/
    verbose: true
    user: __token__
    repository_url: https://upload.pypi.org/legacy/
    packages_dir: dist
    verify_metadata: true
    skip_existing: false
    print_hash: false
    attestations: false
/usr/bin/docker run --name aa8f0f61634924446291bdd2b2eb26b6d6_75c6e7 --label 1599aa --workdir /github/workspace --rm -e "INPUT_REPOSITORY-URL" -e "INPUT_VERBOSE" -e "INPUT_USER" -e "INPUT_PASSWORD" -e "INPUT_REPOSITORY_URL" -e "INPUT_PACKAGES-DIR" -e "INPUT_PACKAGES_DIR" -e "INPUT_VERIFY-METADATA" -e "INPUT_VERIFY_METADATA" -e "INPUT_SKIP-EXISTING" -e "INPUT_SKIP_EXISTING" -e "INPUT_PRINT-HASH" -e "INPUT_PRINT_HASH" -e "INPUT_ATTESTATIONS" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_ID_TOKEN_REQUEST_URL" -e "ACTIONS_ID_TOKEN_REQUEST_TOKEN" -e "ACTIONS_RESULTS_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/torchdde/torchdde":"/github/workspace" 1599aa:8f0f61634924446291bdd2b2eb26b6d6  "__token__" "" "https://test.pypi.org/legacy/" "" "" "" "true" "" "false"
Checking dist/torchdde-0.1.0-py3-none-any.whl: PASSED
Checking dist/torchdde-0.1.0.tar.gz: PASSED
Showing hash values of files to be uploaded:
/github/workspace/dist/torchdde-0.1.0.tar.gz

SHA256: 1d1a8a94a684850e7b9d3461ff9b03e17d9322f1af51603db5a5d08bc55fb043
MD5: a5ac5d5a817ec5af613f89336242b2a0
BLAKE2-256: 21fbeaa243b4d5ac3ec3d9b37988853cd49d145730068bdd5258f97b4134997d

/github/workspace/dist/torchdde-0.1.0-py3-none-any.whl

SHA256: e3c8ed0569cda55144e4cf0656f6e9d3d131830b34d52defedcccb4cf664fdd3
MD5: dbaf6c6828101e07b01b9c14a2488ebb
BLAKE2-256: 1380db838e6daac381ea0f28e7328692fee475ef4576563c49d0e4ae66e309d5

Uploading distributions to https://test.pypi.org/legacy/
INFO     dist/torchdde-0.1.0-py3-none-any.whl (35.0 KB)                         
INFO     dist/torchdde-0.1.0.tar.gz (22.8 KB)                                   
INFO     password set by command options                                        
INFO     username: __token__                                                    
INFO     password: <hidden>                                                     
Uploading torchdde-0.1.0-py3-none-any.whl
INFO     Response from https://test.pypi.org/legacy/:                           
         400 Bad Request                                                        
INFO     <html>                                                                 
          <head>                                                                
           <title>400 Bad Request</title>                                       
          </head>                                                               
          <body>                                                                
           <h1>400 Bad Request</h1>                                             
           The server could not comply with the request since it is either      
         malformed or otherwise incorrect.<br/><br/>                            
         POST body may not contain duplicate keys                               
                                                                                
                                                                                
          </body>                                                               
         </html>                                                                
ERROR    HTTPError: 400 Bad Request from https://test.pypi.org/legacy/          
         Bad Request                                                            

I'd like also to note that my current master is on version 0.1.0 and that the first PyPI test was done successfully by doing the CLIs with a version 0.0.1 so i'm not sure that the version bumping is the problem here.

The github action provides the url link to the release 0.0.1 and not 0.1.0 :

0s
Evaluate and set environment url
Evaluated environment url: https://test.pypi.org/p/torchdde
Cleaning up orphan processes

This seems to be related to #16732 but I'm not sure.

[di]

Thanks for the issue! Currently trying to debug this in #16755.

[di]

Accidentally closed, I'm not sure we've figured out what the issue is here yet.

[di]

I still can't reproduce this myself but I've merged #16755 which might help, can you retry and see if this is still happening?

[thibmonsel]

On my end, it doesn't seem so !
I'll with the regular pypi and not test pypi to see if its test pypi problem or not.

[quinnmil]

I've encountered this via a github action as well. I just tried re-running the failed action, but getting the same error. I am also using the pypa/gh-action-pypi-publish@release/v1 action. Seeing this on two repositories, one setup to use Trusted Publisher and the other using a token, both using the skip-existing
Interesting, one repo is seemingly successful uploading the .whl, but fails on the tarball. Here's some verbose output if it's any help.

Uploading distributions to https://test.pypi.org/legacy/
INFO     dist/scout_apm_logging-0.1.2-py3-none-any.whl (4.5 KB)                 
INFO     dist/scout_apm_logging-0.1.2.tar.gz (3.2 KB)                           
INFO     password set by command options                                        
INFO     username: __token__                                                    
INFO     password: <hidden>                                                     
Uploading scout_apm_logging-0.1.2-py3-none-any.whl
INFO     Response from https://test.pypi.org/legacy/:                           
         200 OK                                                                 
INFO     <html>                                                                 
          <head>                                                                
           <title>200 OK</title>                                                
          </head>                                                               
          <body>                                                                
           <h1>200 OK</h1>                                                      
           <br/><br/>                                                                                                              
          </body>                                                               
         </html>                                                                
Uploading scout_apm_logging-0.1.2.tar.gz
INFO     Response from https://test.pypi.org/legacy/:                           
         [40](https://github.com/scoutapp/scout_apm_python_logging/actions/runs/10930758049/job/30389277563#step:4:41)0 Bad Request                                                        
INFO     <html>                                                                 
          <head>                                                                
           <title>400 Bad Request</title>                                       
          </head>                                                               
          <body>                                                                
           <h1>400 Bad Request</h1>                                             
           The server could not comply with the request since it is either      
         malformed or otherwise incorrect.<br/><br/>                            
         POST body may not contain duplicate keys (URL:                         
         &#x27;https://test.pypi.org/legacy/&#x27;)
         </body>                                                               
         </html>                                                                
ERROR    HTTPError: 400 Bad Request from https://test.pypi.org/legacy/          
         Bad Request

Thanks for taking a look at this. The issue seems to only be effecting to our devel (testPyPI) workflow, and only occurs when trying to push a version already uploaded to testPyPI (Hence the skip-existing). Bumping the version bypasses this error.

[di]

We've identified the issue, #16759 will fix this once it's merged & deployed.

[thibmonsel]

I didn't have any version of the package on PyPI but by using the workflow :

  publish-to-pypi:
      name: >-
        Publish Python 🐍 distribution 📦 to PyPI
      if: startsWith(github.ref, 'refs/tags/')  # only publish to PyPI on tag pushes
      needs:
      - build
      runs-on: ubuntu-latest
      environment:
        name: pypi
        url: https://pypi.org/p/torchdde  # Replace <package-name> with your PyPI project name
      permissions:
        id-token: write  # IMPORTANT: mandatory for trusted publishing

      steps:
      - name: Download all the dists
        uses: actions/download-artifact@v4
        with:
          name: python-package-distributions
          path: dist/
      - name: Publish distribution 📦 to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1

The release of the distribution package was successful. (maybe the latest commit come through)

So this is either a problem probably coming from test PyPI itself or the update of packages.