You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Run the python script to generate the tidy CSV & scraper.json file.
Run the csvqb CLI to generate the CSV-W taking the tidy CSV, info.json & scraper.json as inputs.
As mentioned in Issue Create CLI for csvqb #108 there will be a command line option for the CSV-qb CLI to force outputting validation errors into a file called validation-errors.json. Ensure that this is kept as an artifact of this pipeline.
Run csvlint on the outputted CSV-Ws to ensure they're valid.
Ensure CSV-W (and all associated codelist dependencies) are registered as build artifacts.
Test/Ensure the build is marked as a failure if validation errors fail.
Make sure to write units tests to cover this pipeline (so we can debug locally) - a few rough integration tests to ensure the main branches/paths of the code succeess. Maybe we could check one of these for some of the RDF outputs we expect?
N.B. don't go crazy on the tests since we'd be duplicate a lot of testing we'll already have elsewhere.
The text was updated successfully, but these errors were encountered:
Rough steps:
validation-errors.json
. Ensure that this is kept as an artifact of this pipeline.N.B. don't go crazy on the tests since we'd be duplicate a lot of testing we'll already have elsewhere.
The text was updated successfully, but these errors were encountered: