Bug on Cosmos 1.5 No such file or directory "/usr/local/aiflow/dags/dbt/dbt_venv/bin/python"
#1096
Comments
dosubot
bot
added
the
execution:virtualenv
|
After lots of time trying to reproduce and understand this issue, I narrowed it down:
Cosmos was raising an exception while trying to parse the file and create a version hash to represent the version of that dir. I validated a possible solution with one of the customers earlier today, available in Cosmos 1.5.1a3 - and they confirmed it solved the problem. I'll make a quick PR with the solution and unittests today |
tatiana
added a commit
that referenced
this issue
Jul 17, 2024
Two Astronomer customers reported issues while trying to upgrade to Cosmos 1.5: ``` No such file or directory "/usr/local/aiflow/dags/dbt/dbt_venv/bin/python" ``` The issue is that their dbt project folder had a symbolic link to a file that no longer existed. When Cosmos attempted to iterate over that directory to create a hash representing the cache version, it could not open the file referenced by the symbolic link. This PR reproduces the issue and fixes the behaviour. Users are also advised to check if there are any broken sym links on their dbt folder and delete them. Closes #1096
|
We fixed the issue in #1097, and, in parallel to this, we would like to encourage users to have Python virtual environments outside the dbt project folder and outside of the Airflow DAGs folder. |
tatiana
added a commit
that referenced
this issue
Jul 17, 2024
Two Astronomer customers reported issues while trying to upgrade to Cosmos 1.5: ``` No such file or directory "/usr/local/aiflow/dags/dbt/dbt_venv/bin/python" ``` The issue is that their dbt project folder had a symbolic link to a file that no longer existed. When Cosmos attempted to iterate over that directory to create a hash representing the cache version, it could not open the file referenced by the symbolic link. This PR reproduces the issue and fixes the behavior. Users are also advised to check if there are any broken symlinks on their dbt folder and delete them. We also discourage having Python virtual environments inside the dbt project folder and/or the Airflow DAGs folder. Closes: #1096 (cherry picked from commit 0bafe88)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Two Astro customers reported that when updating to Cosmos 1.5, they started seeing the error:
As illustrated in the two screenshots below:
The text was updated successfully, but these errors were encountered: