Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Propose CI infrastructures in github action #1494

Merged
merged 27 commits into from
Aug 13, 2021
Merged

Conversation

laserprec
Copy link
Contributor

@laserprec laserprec commented Aug 6, 2021

Description

To improve developer experience working with this repo, we will begin migrating our DevOps pipelines into Github Action, along with consolidating documentations and leveraging popular DevOps tools like tox and flake8. This will be a sequence of updates to our DevOps infrastructures:

  1. Propose and draft the CI pipeline in Github Action <---- (Current PR)
  2. Optimize build time for unit tests
  3. Enforce flake8 (coding style checks) on the build and clean up coding styles to pass flake8 checks
  4. Setup self-hosted machines to run our GPU workloads
  5. Run tests on the appropriate dependency subsets
  6. Create feature parity with the existing CI pipelines (pr-gates & night-build)
  7. Deprecate CI pipelines in ADO and switch to Github Actions

In THIS PR

we have the following changes:

  1. Add a draft of PR-gate implemented in GitHub Action executing test suite in parallel ([FEATURE] Move the tests to hosted machines #1460)

    • Status badge to the CI pipeline: pr-gate
      image
  2. Add __init__.py to subfolders of tests (resolves issue [BUG] testing within vscode #1395)

  3. Deprecate .pytest.ini in favor of tox.ini and ensure repo is tox-runnable.

    • Tox is great for establishing parity between local and build environment (less "this works locally but not in build ")
    • For example: you can just run tox -- tests/unit -m "not spark and not notebooks" to run cpu tests after installation.
    • For more convincing words on tox, please read this 😃

Related Issues

  1. [FEATURE] Move the tests to hosted machines #1460
  2. [BUG] testing within vscode #1395

Checklist:

  • I have followed the contribution guidelines and code style for this project.
  • I have added tests covering my contributions.
  • I have updated the documentation accordingly.
  • This PR is being made to staging branch and not to main branch.

Copy link
Collaborator

@gramhagen gramhagen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few questions, and small typo suggestion

.github/workflows/pr-gate.yml Outdated Show resolved Hide resolved
.github/workflows/pr-gate.yml Outdated Show resolved Hide resolved
.github/workflows/pr-gate.yml Outdated Show resolved Hide resolved
.github/workflows/pr-gate.yml Show resolved Hide resolved
tox.ini Outdated Show resolved Hide resolved
.github/workflows/pr-gate.yml Outdated Show resolved Hide resolved
.github/workflows/pr-gate.yml Outdated Show resolved Hide resolved
Copy link
Collaborator

@gramhagen gramhagen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@codecov-commenter
Copy link

Codecov Report

❗ No coverage uploaded for pull request base (staging@cab9a2a). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             staging    #1494   +/-   ##
==========================================
  Coverage           ?   34.36%           
==========================================
  Files              ?       84           
  Lines              ?     8368           
  Branches           ?        0           
==========================================
  Hits               ?     2876           
  Misses             ?     5492           
  Partials           ?        0           

@laserprec laserprec merged commit 6e67f3a into staging Aug 13, 2021
@miguelgfierro miguelgfierro deleted the laserprec/ghaction-ci branch August 24, 2021 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Move the tests to hosted machines [BUG] testing within vscode
3 participants