Follow instructions in the documentation to set up your development environment.
For an enhanced experience, we provide the extensions, settings, and tasks for VSCode in the .vscode
folder.
-
Install recommended extensions
bash .vscode/install_extensions.sh
-
Open the project in VSCode
code .
We use pre-commit hooks to ensure that the code as well as non-code text (such as JSON, YAML, and Markdown) is formatted and linted before committing. First, install pre-commit
:
rye install pre-commit
To install viadot
's pre-commit hooks, run the following command:
pre-commit install
The best way to fix pre-commit errors is to avoid them in the first place. In the case of Python checks, the easiest way is to set Ruff as your formatter in VSode. The formatter will automatically format whatever it can, and the linter will highlight the areas that need fixing, so you get the feedback immediately as you're writing the code.
To run unit tests, simply execute:
pytest tests/unit
For integration tests, you need to set up some environment variables and viadot config.
NOTE: Configs used internally within dyvenia are stored in our Passbolt instance.
You can find all used environment variables in the tests' dotenv file. The env file must be located at tests/.env
.
You can find all used viadot config entries in the example config file.
Code should be formatted and linted with ruff. All settings are defined in the pyproject file.
The easiest way to format your code as you go is to use the VSCode extension and the provided VSCode settings - your code will be automatically formatted and linted on each save, and the linter will highlight areas in your code which need fixing.
Alternatively, the pre-commit hook runs ruff check
, so you can also wait till you commit to receive the formatting/linting feedback.
Commit messages should:
-
begin with an emoji (we recommend using the gitmoji VSCode extension (it's already included in viadot's
.vscode/extensions.list
)) -
start with one of the following verbs, capitalized, immediately after the summary emoji: "Add", "Update", "Remove", "Fix", "Rename", and, sporadically, other ones, such as "Upgrade", "Downgrade", or whatever you find relevant for your particular situation
-
contain a useful summary of what the commit is doing
See this article to understand the basics of commit naming.
-
Uncheck the "Copy the
main
branch only" box -
Follow the setup outlined above
-
Checkout a new branch
# Make sure that your base branch is `2.0`. git switch 2.0 && git checkout -b <name>
-
Add your changes
-
Sync your fork with the
dyvenia
repogit remote add upstream https://github.com/dyvenia/viadot.git git fetch upstream 2.0 git switch 2.0 git rebase upstream/2.0
-
Push the changes to your fork
git push --force
-
Submit a PR into the
2.0
branch.Make sure to read & check all relevant checkboxes in the PR description!
Before creating a release, either add a commit with a version bump to the last PR included in the release, or create a specific release PR. To bump package version, simply run:
rye version major.minor.patch
for example:
rye version 2.1.0
This will update the version in pyproject.toml
accordingly.
NOTE: Make sure to follow semantic versioning.
Once the modified pyproject.toml
is merged to 2.0
, a version tag will be automatically created, and the release workflow will be triggered.
The release workflow will:
- create a release on GitHub with auto-generated changelog
- publish the package to PyPI
- publish Docker images to ghcr.io
You can execute actions manually with GitHub CLI:
gh workflow run <workflow_name>.yml
If you need to pass parameters to the workflow, you can do so with the --json
flag:
echo '{"name":"scully", "greeting":"hello"}' | gh workflow run workflow.yml --json
To works on actions, you can use act.
act -W .github/workflows/detect-and-tag-new-version.yml -s GITHUB_TOKEN="$(gh auth token)"
NOTE for actions that implicitly {{ github.token }}
, you need to pass your token as the GITHUB_TOKEN
act secret, as shown in the example above.