test: add provider verification CI job #326
Conversation
Codecov Report
@@ Coverage Diff @@
## master #326 +/- ##
==========================================
- Coverage 94.59% 94.56% -0.04%
==========================================
Files 27 27
Lines 2997 2998 +1
Branches 161 161
==========================================
Hits 2835 2835
- Misses 144 145 +1
Partials 18 18
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
.github/workflows/ci.yml
Outdated
| - name: Publish Provider Verification Results | ||
| if: matrix.python-version == '3.8' && matrix.toxenv == 'django32' | ||
| run: | | ||
| export PACT_BROKER_BASE_URL=https://edx.pactflow.io | ||
| export PUBLISH_VERSION=`git rev-parse --short HEAD` | ||
| export PUBLISH_VERIFICATION_RESULTS=true | ||
| pytest edxval/pacts/verify_pact.py --ds=edxval.settings.pact |
There was a problem hiding this comment.
This job should be run separately and not part of unit tests. In VEM, publish pact job is run with unit tests because the pact is created/updated during the unit test suite. The same is not true for verification. This job will read pact from the broker/local file and then run the verification.
Ali-D-Akbar
force-pushed
the
aakbar/PROD-2552
branch
3 times, most recently
from
4512b92
to
dfda6a7
Compare
.github/workflows/ci.yml
Outdated
|
|
||
|
|
||
| provider-verification: | ||
| name: Publish Provider Verification Results |
There was a problem hiding this comment.
nit: Pact Provider Verification. No need to add publish in start.
.github/workflows/ci.yml
Outdated
| pip install -r requirements/ci.txt | ||
| pip install -r requirements/test.txt | ||
|
|
||
| - name: Publish Results |
.github/workflows/ci.yml
Outdated
| export PACT_BROKER_TOKEN=${{ secrets.PACT_FLOW_ACCESS_TOKEN }} | ||
| export PUBLISH_VERSION=`git rev-parse --short HEAD` | ||
| export PUBLISH_VERIFICATION_RESULTS=true | ||
| pytest edxval/pacts/verify_pact.py --ds=edxval.settings.pact |
There was a problem hiding this comment.
Add -s flag so that logs/console statements are visible.
There was a problem hiding this comment.
Wouldn't that diable all the capturing? https://docs.pytest.org/en/6.2.x/capture.html#setting-capturing-methods-or-disabling-capturing
There was a problem hiding this comment.
Oh. On local, using -s yielded in a lot more log output, so I thought it was giving more output. I didn't consult the documents.
Ali-D-Akbar
force-pushed
the
aakbar/PROD-2552
branch
from
cdcc32c
to
845b3ab
Compare
PROD-2552
Adds a separate pact provider verification job in CI. The pacts are retrieved from pactflow, verification is performed and after that, the results are published with dev/prod tags and provider version.
Also, we're allowing Pending Pacts in edx-val as not doing that may block the edxval from merging the new changes if one or more consumers pacts are failing (even when the provider is compatible with the deployed version of the consumer)