[BUG] s3 module fails when IMDSv2 is enforced on ec2 instances

[ryanm-sq]

Description
An ec2 minion with IMDSv2 enabled cannot properly use it's IAM instance profile with the s3 module. The s3 module fails with the message Error running 's3.get': Failed s3 operation. InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.

Setup
Enforce IMDSv2 on a standalone minion running in AWS EC2:

$ aws ec2 modify-instance-metadata-options --instance-id {INSTANCE_ID} --http-tokens required --http-endpoint enabled

Steps to Reproduce the behavior
With IMDSv2 enforced, attempt to use the s3 module either on its own or in a state (i.e. file.managed with s3://{bucket} as the source). The call will fail with:

[WARNING ] Failed to decode JSON from instance metadata.
Error running 's3.get': Failed s3 operation. InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.

Expected behavior
The s3 module should be able to retrieve proper credentials for subsequent calls to s3 api and the s3 module should work with no additional setup.

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

Salt Version:
          Salt: 3003.1

Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: 2.6.1
     docker-py: 2.5.1
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.10
       libgit2: 0.26.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 0.5.6
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: Not Installed
  pycryptodome: 3.4.7
        pygit2: 0.26.2
        Python: 3.6.9 (default, Jan 26 2021, 15:33:00)
  python-gnupg: 0.4.1
        PyYAML: 3.12
         PyZMQ: 17.1.2
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.2.5

System Versions:
          dist: ubuntu 18.04 Bionic Beaver
        locale: UTF-8
       machine: x86_64
       release: 5.4.0-1041-aws
        system: Linux
       version: Ubuntu 18.04 Bionic Beaver

Additional context
See also #57514

[welcome]

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey.
Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

• Community Wiki
• Salt’s Contributor Guide
• Join our Community Slack
• IRC on LiberaChat
• Salt Project YouTube channel
• Salt Project Twitch channel

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar.
If you have additional questions, email us at [email protected]. We’re glad you’ve joined our community and look forward to doing awesome things with you!

[ryanm-sq]

It's been a year since this was opened. Any updates on this?

[block-erichter]

Bumping this as its impacting my team as well. Our AWS policy forces IMDSv2 so this is crucial.

[jdelnano]

Bumping the bump: Being able to use IMDSv2 is a best security practice, and having this support is critical for encouraging use of IMDSv2.

[Ch3LL]

Closed by #63067

[Ch3LL]

Actually looking a bit closer at the code I'm not certain it was fixed by this PR. Can someone confirm if this is still broken on the latest version of Salt?

[lorengordon]

Yeah I tested it, it works.