[ADAP-518] Convert information to a dict

[Fokko]

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt-spark functionality, rather than a Big Idea better suited to a discussion

Describe the feature

In preparation for having three-part identifiers catalog.schema.table (#755), I would like to change the information attribute on the SparkRelation into a dict:

dbt-spark/dbt/adapters/spark/relation.py

Lines 36 to 37 in cb41ab0

	# TODO: make this a dict everywhere
	information: Optional[str] = None

Describe alternatives you've considered

The current way is unmaintainable with the regex that extracts useful information from the big blob of text. Also, I noticed that the types are missing currently:

Who will this benefit?

Mostly the developers because it is hard to maintain right now, and hard to extend the current situation

Are you interested in contributing this feature?

Yes!

Anything else?

I wanted to add the database to the configuration. In Spark, this is called a catalog: https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.Catalog.html

Since Spark 3.0 it can discover tables/views from multiple catalogs, such as a Hive or Glue catalog. I would love to add this, but this refactor needs to be done first and I also want to keep the PRs concise.

[dbeatty10]

Thanks for kicking this off @Fokko 🏆

I started a tasklist to track each of the refactor(s) + feature implementation(s) needed for three-part identifiers:

• [ADAP-522] Three-part identifiers (catalog.schema.table) #755

As you create more issues for this, just let me know and we'll add them to that tasklist.

[Fokko]

@dbeatty10 Thanks! Much appreciated

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

[dbeatty10]

@Fokko I just noticed that this and #755 were closed as stale, so I'm re-opening them now.