Skip to content

Commit

Permalink
pipelines: Run CodeQL analysis for Go on 1ES Hosted Pool
Browse files Browse the repository at this point in the history
Vendoring the Microsoft Graph SDK for Go causes memory consumption
during CodeQL analysis to double due to its enormous API surface,
putting it well beyond the memory limit of standard GitHub Action
runners.

I inquired with the Azure organization admins about provisioning
larger GitHub runners, but was directed instead to use the 1ES
Hosted Pool which runs our other CI checks. Since ARO controls
the VM type for Hosted Pool agents, we can use a VM type with
adequate memory for CodeQL analysis with the Graph SDK.

Note: Implemented CodeQL commands in a template in case we
      ever decide to move Javascript or Python analysis to
      1ES Hosted Pool as well.
  • Loading branch information
mbarnes committed Jun 9, 2023
1 parent e36318a commit 6a7b18d
Show file tree
Hide file tree
Showing 3 changed files with 64 additions and 1 deletion.
2 changes: 1 addition & 1 deletion .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
strategy:
fail-fast: false
matrix:
language: ['go', 'javascript', 'python']
language: ['javascript', 'python']
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ]
# Learn more:
# https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#changing-the-languages-that-are-analyzed
Expand Down
27 changes: 27 additions & 0 deletions .pipelines/ci.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,17 @@
# Azure DevOps Pipeline running CI
#
# Note: This pipeline uses a secret variable "github_codeql_upload_token".
# This is a GitHub Personal Access Token (Classic) owned by mbarnes.
# It has no expiration and only has the "security_events" scope for
# the purpose of uploading CodeQL results.
#
# However, for this secret to be available to pull requests from
# forked ARO-RP repositories, the pipeline option "Make secrets
# available to builds of forks" is enabled.
#
# More information:
# https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/github#contributions-from-forks
#
trigger:
branches:
include:
Expand Down Expand Up @@ -31,6 +44,20 @@ variables:
- template: vars.yml

jobs:
- job: Golang_CodeQL
pool:
name: 1es-aro-ci-pool
variables:
HOME: $(Agent.BuildDirectory)
steps:
- template: ./templates/template-checkout.yml
- template: ./templates/template-codeql.yml
parameters:
language: go
target: golang
github_token: $(github_codeql_upload_token)
timeoutInMinutes: 120

- job: Python_Unit_Tests
pool:
name: 1es-aro-ci-pool
Expand Down
36 changes: 36 additions & 0 deletions .pipelines/templates/template-codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
parameters:
- name: language
type: string
values:
# Based on "codeql resolve languages"
- cpp
- csharp
- csv
- go
- html
- java
- javascript
- properties
- python
- ruby
- xml
- name: target
type: string
default: host
- name: github_token
type: string

# Based on "Use CodeQL in CI system" documentation:
# https://docs.github.com/en/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system
steps:
- script: |
set -xe
sarif_file=codeql-results-${{ parameters.language }}.sarif
wget --quiet --output-document=- https://github.com/github/codeql-action/releases/latest/download/codeql-bundle-linux64.tar.gz | tar --extract --gunzip
./codeql/codeql database create ./codeql-db --language=${{ parameters.language }}
./codeql/codeql database analyze ./codeql-db --format=sarif-latest --sarif-category=no --output=${sarif_file}
./codeql/codeql github upload-results --sarif=${sarif_file} --ref=$(Build.SourceBranch)
env:
GITHUB_TOKEN: ${{ parameters.github_token }}
displayName: ⚙️ CodeQL Analysis (${{ parameters.language }})
target: ${{ parameters.target }}

0 comments on commit 6a7b18d

Please sign in to comment.