Add NotFound retries when tagging a new VPC Peering connection

[tremble]

SUMMARY

Prior to botocore 1.17.24 it wasn't possible to tag the connection as a part of the creation call. As such a separate create_tags call is performed after the creation. However, this leads to a race condition when "I(wait=False)".

Rather than adding hard to test logic, add InvalidVpcPeeringConnectionID.NotFound retries to the tagging.

Also move from custom tag handling code over to the shared code in ansible_collections.amazon.aws.plugins.module_utils.ec2

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

ec2_vpc_peer

ADDITIONAL INFORMATION

TASK [ec2_vpc_peer : Create local account VPC peering Connection] **************
task path: /home/zuul/.ansible/collections/ansible_collections/community/aws/tests/integration/targets/ec2_vpc_peer/tasks/main.yml:403
<testhost> ESTABLISH LOCAL CONNECTION FOR USER: zuul
<testhost> EXEC /bin/sh -c 'echo ~zuul && sleep 0'
<testhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/zuul/.ansible/tmp `"&& mkdir "` echo /home/zuul/.ansible/tmp/ansible-tmp-1624925621.1931164-8629-186765425497910 `" && echo ansible-tmp-1624925621.1931164-8629-186765425497910="` echo /home/zuul/.ansible/tmp/ansible-tmp-1624925621.1931164-8629-186765425497910 `" ) && sleep 0'
Using module file /home/zuul/.ansible/collections/ansible_collections/community/aws/plugins/modules/ec2_vpc_peer.py
<testhost> PUT /home/zuul/.ansible/tmp/ansible-local-8151h1f3kbcc/tmplyozom8u TO /home/zuul/.ansible/tmp/ansible-tmp-1624925621.1931164-8629-186765425497910/AnsiballZ_ec2_vpc_peer.py
<testhost> EXEC /bin/sh -c 'chmod u+x /home/zuul/.ansible/tmp/ansible-tmp-1624925621.1931164-8629-186765425497910/ /home/zuul/.ansible/tmp/ansible-tmp-1624925621.1931164-8629-186765425497910/AnsiballZ_ec2_vpc_peer.py && sleep 0'
<testhost> EXEC /bin/sh -c 'ANSIBLE_DEBUG_BOTOCORE_LOGS=True /home/zuul/venv/bin/python3.6 /home/zuul/.ansible/tmp/ansible-tmp-1624925621.1931164-8629-186765425497910/AnsiballZ_ec2_vpc_peer.py && sleep 0'
<testhost> EXEC /bin/sh -c 'rm -f -r /home/zuul/.ansible/tmp/ansible-tmp-1624925621.1931164-8629-186765425497910/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
  File "/tmp/ansible_ec2_vpc_peer_payload_bbdh2ojz/ansible_ec2_vpc_peer_payload.zip/ansible_collections/community/aws/plugins/modules/ec2_vpc_peer.py", line 559, in create_tags
  File "/tmp/ansible_ec2_vpc_peer_payload_bbdh2ojz/ansible_ec2_vpc_peer_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py", line 288, in deciding_wrapper
    return retrying_wrapper(*args, **kwargs)
  File "/tmp/ansible_ec2_vpc_peer_payload_bbdh2ojz/ansible_ec2_vpc_peer_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 154, in retry_func
    raise e
  File "/tmp/ansible_ec2_vpc_peer_payload_bbdh2ojz/ansible_ec2_vpc_peer_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 144, in retry_func
    return f(*args, **kwargs)
  File "/home/zuul/venv/lib/python3.6/site-packages/botocore/client.py", line 386, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/zuul/venv/lib/python3.6/site-packages/botocore/client.py", line 705, in _make_api_call
    raise error_class(parsed_response, operation_name)
fatal: [testhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "aws_access_key": "ASIA6CCDWXDOPPPI445H",
            "aws_ca_bundle": null,
            "aws_config": null,
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "debug_botocore_endpoint_logs": true,
            "ec2_url": null,
            "peer_owner_id": null,
            "peer_region": null,
            "peer_vpc_id": "vpc-0deec00ddeb3584bf",
            "peering_id": null,
            "profile": null,
            "region": "us-east-1",
            "security_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "state": "present",
            "tags": {
                "Name": "Peering connection for VPC vpc-018bc7cc64b0731f9 to VPC vpc-0deec00ddeb3584bf"
            },
            "validate_certs": true,
            "vpc_id": "vpc-018bc7cc64b0731f9",
            "wait": false
        }
    },
    "msg": "An error occurred (InvalidVpcPeeringConnectionID.NotFound) when calling the CreateTags operation: The vpcPeeringConnection ID 'pcx-06ed26ea8993aba5b' does not exist",
    "resource_actions": [
        "ec2:DeleteTags",
        "ec2:CreateTags",
        "ec2:CreateVpcPeeringConnection",
        "ec2:DescribeVpcPeeringConnections"
    ]
}

[ansibullbot]

cc @jillr @mmochan @s-hertel @wimnat
click here for bot help

[alinabuzachis]

LGTM. Thanks @tremble.