Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor dbt ls to run from a temporary directory #414

Merged
merged 8 commits into from
Aug 14, 2023

Commits on Aug 10, 2023

  1. Refactor dbt ls to run from a temporary directory

    As of Cosmos 1.0.0, `LoadMode.DBT_LS` ran `dbt ls` from within the original dbt project directory.
    
    The `dbt ls` outputs files to the directory it's running from unless the environment variables `DBT_LOG_PATH` and `DBT_TARGET_PATH` are specified.
    
    Depending on the deployment, the Airflow worker does not have write permissions to the dbt project directory. This PR changes the behavior of `dbt ls` to make a copy of the original project directory into a temporary directory and run the command `dbt ls` from there.
    
    Closes: #411
    tatiana committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    dbef093 View commit details
    Browse the repository at this point in the history
  2. Fix for Python==3.7

    tatiana committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    cd5a07f View commit details
    Browse the repository at this point in the history
  3. Set test as integration

    tatiana committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    21f9b9c View commit details
    Browse the repository at this point in the history
  4. (unstable) dbt 1.6 solution for dbt ls outputting logs and target to …

    …a separate dir
    
    Unfortunately this does not work in dbt 1.5 or previous versions
    tatiana committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    6d52ba4 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    4360fd1 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    cdb633e View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2023

  1. Improve test

    tatiana committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    731cceb View commit details
    Browse the repository at this point in the history
  2. Remove comment

    tatiana committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    7bd4e30 View commit details
    Browse the repository at this point in the history