Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug where dbt_executable_path was not passed to operators #397

Merged
merged 1 commit into from
Jul 26, 2023

Conversation

jlaneve
Copy link
Collaborator

@jlaneve jlaneve commented Jul 26, 2023

Description

This PR fixes a bug with Cosmos, where previously the dbt_executable_path didn't get passed all the way down to the underlying operators. This meant that the operators didn't have an executable, and would result in an error like so:

[2023-07-26, 22:21:06 UTC] {subprocess.py:70} INFO - Running command: [None, 'seed', '--models', 'raw_customers', '--profiles-dir', '/tmp/tmpde8e9703', '--profile', 'airflow_db', '--target', 'dev']
[2023-07-26, 22:21:06 UTC] {taskinstance.py:1824} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/cosmos/operators/local.py", line 242, in execute
    result = self.build_and_run_cmd(context=context, cmd_flags=cmd_flags)
  File "/usr/local/lib/python3.10/site-packages/cosmos/operators/local.py", line 188, in build_and_run_cmd
    return self.run_command(cmd=dbt_cmd, env=env, context=context)
  File "/usr/local/lib/python3.10/site-packages/cosmos/operators/local.py", line 172, in run_command
    result = self.run_subprocess(
  File "/usr/local/lib/python3.10/site-packages/cosmos/operators/local.py", line 125, in run_subprocess
    subprocess_result: FullOutputSubprocessResult = self.subprocess_hook.run_command(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/cosmos/hooks/subprocess.py", line 72, in run_command
    self.sub_process = Popen(
  File "/usr/local/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.10/subprocess.py", line 1738, in _execute_child
    and os.path.dirname(executable)
  File "/usr/local/lib/python3.10/posixpath.py", line 152, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Related Issue(s)

Breaking Change?

Nope!

Checklist

  • I have made corresponding changes to the documentation (if required)
  • I have added tests that prove my fix is effective or that my feature works

@jlaneve jlaneve requested a review from a team as a code owner July 26, 2023 22:24
@jlaneve jlaneve requested a review from a team July 26, 2023 22:24
@netlify
Copy link

netlify bot commented Jul 26, 2023

👷 Deploy Preview for amazing-pothos-a3bca0 processing.

Name Link
🔨 Latest commit 7243910
🔍 Latest deploy log https://app.netlify.com/sites/amazing-pothos-a3bca0/deploys/64c19d218f2e2f00082e4b8e

@codecov
Copy link

codecov bot commented Jul 26, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.06% 🎉

Comparison is base (554f250) 91.21% compared to head (7243910) 91.28%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #397      +/-   ##
==========================================
+ Coverage   91.21%   91.28%   +0.06%     
==========================================
  Files          48       48              
  Lines        1684     1686       +2     
==========================================
+ Hits         1536     1539       +3     
+ Misses        148      147       -1     
Files Changed Coverage Δ
cosmos/converter.py 97.14% <100.00%> (+0.08%) ⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@harels harels merged commit de9af5c into main Jul 26, 2023
39 checks passed
@harels harels deleted the fix-executable-bug branch July 26, 2023 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants