Refactor partial parsing cache to be per dbt project #1130
Labels
area:parsing
Related to parsing DAG/DBT improvement, issues, or fixes
dbt:parse
Primarily related to dbt parse command or functionality
Milestone
In #904, we started caching the partial parsing and manifest files. However, that happened at a
DbtDag
andDbtTaskGroup
level. By then, we considered configuring at the dbt project level, but there were concerns regarding concurrent tasks running in the same Airflow worker node.This ticket aims to refactor the caching to be set at a dbt project level. We should consider using the
_safe_copy
method created in #1086. We should create alphas and have as many users as possible trying out this.This refactor will improve with filling up users' disk: #1042
The text was updated successfully, but these errors were encountered: