feat(glue): allow resource links to be ignored

[YusufMahtab]

Description

This PR adds a config option to the Glue recipe to detect when database is actually a resource link and automatically filter it out.

The get_all_tables_and_databases() method has also been refactored (and renamed) so that the boto3 calls are in their own methods.

Context

There are at least 2 ways of sharing Glue databases across accounts:

• external catalog sharing
• resource linking - the resource link may have a different name to the original database

If

1. a database is shared via both methods
2. the resource link has a different name to the original database
3. the Glue recipe filter allows for both of them

then a KeyError is thrown. An example is shown in this Slack thread: https://datahubspace.slack.com/archives/CUMUWQU66/p1674044495369769.

Checklist

• The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues (if applicable)
• Tests for the changes have been added/updated (if applicable)
• Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
• For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated
docs-website	✅ Ready (Inspect)	Visit Preview	💬 Add your feedback	Mar 21, 2023 at 3:48PM (UTC)