DbApiHook import error in common.sql

[phi-friday]

Apache Airflow version

main (development)

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Traceback (most recent call last):
  File "/opt/airflow/airflow/models/taskinstance.py", line 766, in _execute_task
    result = _execute_callable(context=context, **execute_callable_kwargs)
  File "/opt/airflow/airflow/models/taskinstance.py", line 732, in _execute_callable
    return ExecutionCallableRunner(
  File "/opt/airflow/airflow/utils/operator_helpers.py", line 252, in run
    return self.func(*args, **kwargs)
  File "/opt/airflow/airflow/models/baseoperator.py", line 406, in wrapper
    return func(self, *args, **kwargs)
  File "/opt/airflow/airflow/providers/common/sql/operators/sql.py", line 1182, in execute
    record = self.get_db_hook().get_first(self.sql, self.parameters)
  File "/opt/airflow/airflow/providers/common/sql/operators/sql.py", line 194, in get_db_hook
    return self._hook
  File "/usr/local/lib/python3.8/functools.py", line 967, in __get__
    val = self.func(instance)
  File "/opt/airflow/airflow/providers/common/sql/operators/sql.py", line 156, in _hook
    from airflow.hooks.dbapi_hook import DbApiHook as _DbApiHook
ModuleNotFoundError: No module named 'airflow.hooks.dbapi_hook'

What you think should happen instead?

Now that we've removed the deprecated imports from #41368 , we need to modify the following code

airflow/airflow/providers/common/sql/operators/sql.py

Lines 155 to 178 in fcc9f3d

	if not isinstance(hook, DbApiHook):
	from airflow.hooks.dbapi_hook import DbApiHook as _DbApiHook
	if isinstance(hook, _DbApiHook):
	# This case might happen if user installed common.sql provider but did not upgrade the
	# Other provider's versions to a version that supports common.sql provider
	class_module = hook.__class__.__module__
	match = _PROVIDERS_MATCHER.match(class_module)
	if match:
	provider = match.group(1)
	min_version = _MIN_SUPPORTED_PROVIDERS_VERSION.get(provider)
	if min_version:
	raise AirflowException(
	f"You are trying to use common-sql with {hook.__class__.__name__},"
	f" but the Hook class comes from provider {provider} that does not support it."
	f" Please upgrade provider {provider} to at least {min_version}."
	)
	raise AirflowException(
	f"You are trying to use `common-sql` with {hook.__class__.__name__},"
	" but its provider does not support it. Please upgrade the provider to a version that"
	" supports `common-sql`. The hook class should be a subclass of"
	" `airflow.providers.common.sql.hooks.sql.DbApiHook`."
	f" Got {hook.__class__.__name__} Hook with class hierarchy: {hook.__class__.mro()}"
	)

How to reproduce

Here's the test where I found the error.

airflow/tests/providers/common/sql/operators/test_sql.py

Lines 615 to 622 in fcc9f3d

	def test_not_allowed_conn_type(self):
	with mock.patch(
	"airflow.providers.common.sql.operators.sql.BaseHook.get_connection",
	return_value=Connection(conn_id="sql_default", conn_type="postgres"),
	) as mock_get_conn:
	mock_get_conn.return_value = Connection(conn_id="sql_default", conn_type="airbyte")
	with pytest.raises(AirflowException, match=r"You are trying to use `common-sql`"):
	self._operator._hook

airflow/tests/providers/common/sql/operators/test_sql.py

Lines 1020 to 1033 in fcc9f3d

	@pytest.mark.db_test
	def test_unsupported_conn_type(self):
	"""Check if BranchSQLOperator throws an exception for unsupported connection type"""
	op = BranchSQLOperator(
	task_id="make_choice",
	conn_id="redis_default",
	sql="SELECT count(1) FROM INFORMATION_SCHEMA.TABLES",
	follow_task_ids_if_true="branch_1",
	follow_task_ids_if_false="branch_2",
	dag=self.dag,
	)
	with pytest.raises(AirflowException):
	op.run(start_date=DEFAULT_DATE, end_date=DEFAULT_DATE, ignore_ti_state=True)

Operating System

Not required

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct