[ADAP-522] Three-part identifiers (catalog.schema.table)

[dbeatty10]

Spark >= 3.0 can discover tables/views from multiple catalogs, such as a Hive or Glue catalog. This is a prerequisite to enable so-called three-part identifiers for dbt-spark (catalog.schema.table).

To keep PRs concise for any refactoring plus the implementation, we are splitting this over multiple issues.

Three-part identifiers

Beta Give feedback

1. [ADAP-518] Convert information to a dict #751
   enhancement +1

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

[mukeshkumarkulmi]

Hi Everyone, Can someone suggest how to use Apache Nessie catalog with Spark adapter in dbt. If we modify generate_schema_name to add nessie catalog name, then list_None_* API fails as it creates show table extended in nessie.<schema_name> like '*' query to get table names in the schema. And we cannot use "generate_database_name" for customisation in Spark adapter as it fails with "Cannot set database in spark!" error.
Also, in case of incremental models, DBT only creates insert into command, if the table already exists in the schema when it run "show table extended" query but show table extended in nessie.<schema_name> like '*' this query is failing, so DBT always create new table in default catalog.
Please suggest.