Skip to content

Commit

Permalink
deltalake _delta_lake.py: Allow Glue catalog cross account access (#2113
Browse files Browse the repository at this point in the history
)

This PR sets the account Id into glue call, so users can access catalog
in another account.
Tested in our dev environment.
  • Loading branch information
pang-wu authored Apr 15, 2024
1 parent 0fe3df7 commit 9c441bb
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion daft/io/_delta_lake.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,14 @@ def read_delta_lake(
aws_secret_access_key=s3_config.access_key,
aws_session_token=s3_config.session_token,
)
glue_table = glue.get_table(DatabaseName=table.database_name, Name=table.table_name)
if table.catalog_id is not None:
# Allow cross account access, table.catalog_id should be the target account id
glue_table = glue.get_table(
CatalogId=table.catalog_id, DatabaseName=table.database_name, Name=table.table_name
)
else:
glue_table = glue.get_table(DatabaseName=table.database_name, Name=table.table_name)

# TODO(Clark): Fetch more than just the table URI from Glue Data Catalog.
table_uri = glue_table["Table"]["StorageDescriptor"]["Location"]
elif table.catalog == DataCatalogType.UNITY:
Expand Down

0 comments on commit 9c441bb

Please sign in to comment.