Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix S3Hook get_key method by removing load extra args #35951

Closed
wants to merge 1 commit into from

Conversation

hussein-awala
Copy link
Member

S3 Object.load() doesn't accept any argument, it just returns the metadata of the object. However, the download method should provide the transfer configuration.

@Taragolis
Copy link
Contributor

Actually it does accept, it just not well documented, maybe because boto3.resoure abandoned, at least how it look from my perspective as a regular user of boto3.

And example how to check that it accept you could have a look into the discussion in PR which start propagate this extra values to this method: #35037 (comment)

@hussein-awala
Copy link
Member Author

Actually it does accept, it just not well documented, maybe because boto3.resoure abandoned, at least how it look from my perspective as a regular user of boto3.

And example how to check that it accept you could have a look into the discussion in PR which start propagate this extra values to this method: #35037 (comment)

Hummm interesting 🤔

I will check if these configurations are used where we call other methods on the object.

@Taragolis
Copy link
Contributor

Long term solution would be, get rid of (deprecate) boto3.resource usage in favor of boto3.client:

  1. Looks like this objects frozen in a time and do not support all of the features which provided by a client
  2. boto3.resource, especially in S3Hook implementation, consume quite a few amount of memory. Reduce memory usage in S3Hook #35449

Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Jan 14, 2024
@github-actions github-actions bot closed this Jan 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:amazon-aws AWS/Amazon - related issues stale Stale PRs per the .github/workflows/stale.yml policy file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants