Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Pinned-Dependencies fails for jobs with complex matrix-defined OS #3075

Open
pnacht opened this issue May 26, 2023 · 3 comments
Open

BUG: Pinned-Dependencies fails for jobs with complex matrix-defined OS #3075

pnacht opened this issue May 26, 2023 · 3 comments
Labels
kind/bug Something isn't working

Comments

@pnacht
Copy link
Contributor

pnacht commented May 26, 2023

Describe the bug
Jobs with matrix-defined OS can cause an internal error in Pinned-Dependencies.

Reproduction steps
Steps to reproduce the behavior:

  1. scorecard --repo apache/beam --checks Pinned-Dependencies --show-details --format json
Error: check runtime error: Pinned-Dependencies: internal error: internal error: unable to determine OS for job: Build python wheels on ${{matrix.arch}} for ${{ matrix.os_python.os }}
...
  "checks": [
    {
      "details": null,
      "score": -1,
      "reason": "internal error: internal error: unable to determine OS for job: Build python wheels on ${{matrix.arch}} for ${{ matrix.os_python.os }}",
      "name": "Pinned-Dependencies",
      ...
    }
  ],
...

The relevant job is here and is defined below. Note that the OS isn't simply a value of the matrix, but a component of a JSON object defined by the matrix:

  build_wheels:
    name: Build python wheels on ${{matrix.arch}} for ${{ matrix.os_python.os }}
    needs:
      - check_env_variables
      - build_source
    env:
      CIBW_ARCHS_LINUX: ${{matrix.arch}}
    runs-on: ${{ matrix.os_python.os }}
    strategy:
      matrix:
        os_python: [
          {"os": "ubuntu-latest", "python": "${{ needs.check_env_variables.outputs.py-versions-full }}" },
          {"os": "macos-latest", "python": "${{ needs.check_env_variables.outputs.py-versions-test }}" },
          {"os": "windows-latest", "python": "${{ needs.check_env_variables.outputs.py-versions-test }}" },
        ]
        arch: [auto]
        include:
          - os_python: {"os": "ubuntu-latest", "python": "${{ needs.check_env_variables.outputs.py-versions-test }}" }
            arch: aarch64
    # ...

Expected behavior
The case should be handled normally (why does Scorecard need to determine the OS?).

If this edge case is too complex to be worth the effort, Scorecard should then ignore failed jobs (with a warning in the details) but run the rest of the Pinned-Dependencies check normally.

@pnacht pnacht added the kind/bug Something isn't working label May 26, 2023
@abs007
Copy link

abs007 commented Jul 2, 2023

GetOSesForJob() checks the OS'es a particular job's steps runs on. If it's all windows, then the pwsh shell doesnt get matched here.

For taking care of the edge case (which seems to be an issue in parsing the workflow yaml), the err message: unable to determine OS for job can be checked over here and then continue if its this error.

Of course, the better soln would be to improve yaml parsing inside the GetOSesForJob() func.

@diogoteles08
Copy link
Contributor

diogoteles08 commented Sep 1, 2023

Same issue is happening when running Scorecard for github.com/AcademySoftwareFoundation/openexr. For this project, the bug is happening because of this job.

This was discussed in AcademySoftwareFoundation/openexr#1535 , cc @cary-ilm

@diogoteles08
Copy link
Contributor

Also, this issue is related to #3316

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
Status: Backlog - Bugs
Development

No branches or pull requests

3 participants