ResourceWarning: unclosed ssl.SSLSocket when using a session

[GergelyKalmar]

Describe the bug

I'm getting the following warning when using a boto3 session to interact with the AWS API despite explicitly calling close() on the client:

.../lib/python3.8/site-packages/botocore/credentials.py:667: ResourceWarning: unclosed <ssl.SSLSocket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=(...), raddr=(...)>
  response = self._get_credentials()
ResourceWarning: Enable tracemalloc to get the object allocation traceback

Expected Behavior

I was expecting to see no warning.

Current Behavior

The error described above.

Reproduction Steps

Create a script called test.py as follows:

import boto3

client = boto3.session.Session(profile_name='my-profile').client('secretsmanager')
client.put_secret_value(SecretId='test', SecretString='test')
client.close()

Run the script as python3 -Wall test.py and observe the warning.

Possible Solution

No response

Additional Information/Context

The issue is similar to #454. The fix described there does not seem to work with sessions however.

SDK version used

boto3==1.26.47

Environment details (OS name and version, etc.)

Ubuntu 20.04 LTS

[tim-finnigan]

Hi @GergelyKalmar thanks for reaching out. By "The fix described there" in #454, which fix were you referring to?

You may have been referencing this comment but I wanted to confirm:

...the warnings themselves are not an issue. It's an issue in the standard library incorrectly warning because urllib3 keeps connections alive in our ConnectionPool. They are managed and will be reclaimed when appropriate. For users who want to avoid the warnings, but not suppress them with a filter, close() is now available on the client to clear the pool.

Have you also tried this approach mentioned there?

from contextmanager import closing
with closing(boto3.client(service_name='s3')) as client:
    buckets = client.list_buckets()

[nateprewitt]

The info Tim linked above is largely correct but from the traceback this appears to be unrelated to the client itself. The warning is stemming from the credentials being used by the client which would not necessarily be cleaned up with a client.close(). The context manager is just syntactic sugar for managing the closure of the client.

@GergelyKalmar could you provide some more information about how you're sourcing credentials for this profile and if you're using EC2 in this case?

[GergelyKalmar]

Sure, they are AWS SSO named profile credentials stored in .aws/credentials. I'm running the script locally on my laptop.

Note that if I use the AWS SSO credentials as environment variables and call boto3.client(...) directly (instead of using a session object) it works fine without the warning (with client.close()).

[nateprewitt]

Ah, alright so it seems we may be creating a urllib3 connection pool, outside of our client connection pool, when retrieving current SSO credentials. I'm not immediately seeing where that's happening, but I'm going to flag this as a bug.

We'll need to do a deeper dive to find the source and see if we can more gracefully close the connection. This shouldn't have any tangible performance impact as they'll be cleaned up as the connection pool flags them, but we likely can do better with the noise it generates.

Thanks for the info, @GergelyKalmar! We'll update once we have more information.

[gilbd]

Hi, I have the same issue with python3.9.
Neither contextlib.closing nor client.close() solved the warning.

[kortsi]

I'm getting this with just:

import boto3

session = boto3.session(profile_name='my-profile')
credentials = session.get_credentials()
print(credentials.access_key)

Versions:

• boto3==1.26.93
• botocore==1.29.93