Update contributing docs for running integration tests #638
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
👷 Deploy Preview for amazing-pothos-a3bca0 processing.
|
tatiana
pushed a commit
that referenced
this pull request
Nov 6, 2023
To run the integration tests locally without errors, developers have to
set more environment variables than the ones that are currently in the
docs. If these were excluded, they would get errors like:
```shell
tests/test_example_dags_no_connections.py:59: in <module>
@pytest.mark.parametrize("dag_id", get_dag_ids())
tests/test_example_dags_no_connections.py:54: in get_dag_ids
dag_bag = get_dag_bag()
tests/test_example_dags_no_connections.py:49: in get_dag_bag
assert not db.import_errors
E AssertionError: assert not {'/Users/justin.bandoro/astronomer-cosmos/dev/dags/example_cosmos_python_models.py': 'Traceback (most recent call last...ema.yml from project jaffle_shop: Parsing Error\n Env var required but not provided: \'DATABRICKS_CLUSTER_ID\'\n\n'}
E + where {'/Users/justin.bandoro/astronomer-cosmos/dev/dags/example_cosmos_python_models.py': 'Traceback (most recent call last...ema.yml from project jaffle_shop: Parsing Error\n Env var required but not provided: \'DATABRICKS_CLUSTER_ID\'\n\n'} = <airflow.models.dagbag.DagBag object at 0x1579cd000>.import_errors
```
(cherry picked from commit 156e51d)
tatiana
added a commit
that referenced
this pull request
Nov 6, 2023
Bug fixes * Support ProjectConfig.dbt_project_path = None & different paths for Rendering and Execution by @MrBones757 in #634 * Fix adding test nodes to DAGs built using LoadMethod.DBT_MANIFEST and LoadMethod.CUSTOM by @edgga in #615 Others * Add pre-commit hook for McCabe max complexity check and fix errors by @jbandoro in #629 * Update contributing docs for running integration tests by @jbandoro in #638 * Fix CI issue running integration tests by @tatiana in #640 and #644 * pre-commit updates in #637
Merged
tatiana
pushed a commit
that referenced
this pull request
Nov 6, 2023
To run the integration tests locally without errors, developers have to
set more environment variables than the ones that are currently in the
docs. If these were excluded, they would get errors like:
```shell
tests/test_example_dags_no_connections.py:59: in <module>
@pytest.mark.parametrize("dag_id", get_dag_ids())
tests/test_example_dags_no_connections.py:54: in get_dag_ids
dag_bag = get_dag_bag()
tests/test_example_dags_no_connections.py:49: in get_dag_bag
assert not db.import_errors
E AssertionError: assert not {'/Users/justin.bandoro/astronomer-cosmos/dev/dags/example_cosmos_python_models.py': 'Traceback (most recent call last...ema.yml from project jaffle_shop: Parsing Error\n Env var required but not provided: \'DATABRICKS_CLUSTER_ID\'\n\n'}
E + where {'/Users/justin.bandoro/astronomer-cosmos/dev/dags/example_cosmos_python_models.py': 'Traceback (most recent call last...ema.yml from project jaffle_shop: Parsing Error\n Env var required but not provided: \'DATABRICKS_CLUSTER_ID\'\n\n'} = <airflow.models.dagbag.DagBag object at 0x1579cd000>.import_errors
```
(cherry picked from commit 156e51d)
tatiana
added a commit
that referenced
this pull request
Nov 6, 2023
Bug fixes * Support ProjectConfig.dbt_project_path = None & different paths for Rendering and Execution by @MrBones757 in #634 * Fix adding test nodes to DAGs built using LoadMethod.DBT_MANIFEST and LoadMethod.CUSTOM by @edgga in #615 Others * Add pre-commit hook for McCabe max complexity check and fix errors by @jbandoro in #629 * Update contributing docs for running integration tests by @jbandoro in #638 * Fix CI issue running integration tests by @tatiana in #640 and #644 * pre-commit updates in #637
tatiana
added a commit
that referenced
this pull request
Nov 7, 2023
Bug fixes * Support ProjectConfig.dbt_project_path = None & different paths for Rendering and Execution by @MrBones757 in #634 * Fix adding test nodes to DAGs built using LoadMethod.DBT_MANIFEST and LoadMethod.CUSTOM by @edgga in #615 Others * Add pre-commit hook for McCabe max complexity check and fix errors by @jbandoro in #629 * Update contributing docs for running integration tests by @jbandoro in #638 * Fix CI issue running integration tests by @tatiana in #640 and #644 * pre-commit updates in #637 (cherry picked from commit fa0620a)
tatiana
added a commit
that referenced
this pull request
Dec 5, 2023
[Justin Bandoro](https://www.linkedin.com/in/justin-bandoro-592b14a7/) (@jbandoro) is a Data Engineer at Kevala Inc. He's based in San Francisco (USA) and has been an early adopter of Cosmos, using it regularly at his company. Not only has he been using Cosmos since the early stages, but he has consistently improved Cosmos since January 2023:  Some of his contributions include new features, code quality, documentation and overall improvements. Some examples: * Speed up integration tests in 67% #732 * Prevent override of dbt profile fields #702 * Add support for env vars in `RenderConfig` in #690 * Use symbolic links to run local tasks, avoiding to copy potentially huge dbt project folders in #660 * Improve documentation in #638 * Automated and improved the code complexity checks in #629 * Added `DbtDocsGCSOperator` in #616 * Added support for Python 3.7 in #88 and #214 Additionally, he has been interacting with users in the #airflow-dbt Slack channel in a very collaborative and supportive way. We want to promote him as a Cosmos committer and maintainer for all these, recognising his constant efforts and achievements towards our community. Thank you very much, @jbandoro !
arojasb3
pushed a commit
to arojasb3/astronomer-cosmos
that referenced
this pull request
Jul 14, 2024
Bug fixes * Support ProjectConfig.dbt_project_path = None & different paths for Rendering and Execution by @MrBones757 in astronomer#634 * Fix adding test nodes to DAGs built using LoadMethod.DBT_MANIFEST and LoadMethod.CUSTOM by @edgga in astronomer#615 Others * Add pre-commit hook for McCabe max complexity check and fix errors by @jbandoro in astronomer#629 * Update contributing docs for running integration tests by @jbandoro in astronomer#638 * Fix CI issue running integration tests by @tatiana in astronomer#640 and astronomer#644 * pre-commit updates in astronomer#637 (cherry picked from commit fa0620a)
arojasb3
pushed a commit
to arojasb3/astronomer-cosmos
that referenced
this pull request
Jul 14, 2024
[Justin Bandoro](https://www.linkedin.com/in/justin-bandoro-592b14a7/) (@jbandoro) is a Data Engineer at Kevala Inc. He's based in San Francisco (USA) and has been an early adopter of Cosmos, using it regularly at his company. Not only has he been using Cosmos since the early stages, but he has consistently improved Cosmos since January 2023:  Some of his contributions include new features, code quality, documentation and overall improvements. Some examples: * Speed up integration tests in 67% astronomer#732 * Prevent override of dbt profile fields astronomer#702 * Add support for env vars in `RenderConfig` in astronomer#690 * Use symbolic links to run local tasks, avoiding to copy potentially huge dbt project folders in astronomer#660 * Improve documentation in astronomer#638 * Automated and improved the code complexity checks in astronomer#629 * Added `DbtDocsGCSOperator` in astronomer#616 * Added support for Python 3.7 in astronomer#88 and astronomer#214 Additionally, he has been interacting with users in the #airflow-dbt Slack channel in a very collaborative and supportive way. We want to promote him as a Cosmos committer and maintainer for all these, recognising his constant efforts and achievements towards our community. Thank you very much, @jbandoro !
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
In order to run the integration tests locally without errors I had to set more environment variables than the ones that are currently in the docs. If these were excluded I would get errors like:
Let me know if I missed something that was already in the docs, this PR might help anyone else trying to run integration tests for the first time.
Related Issue(s)
None
Breaking Change?
None
Checklist