Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kinesis_stream: Don't mark kstreams changed when no encryption actions taken #27

Merged

Conversation

Tyler-2
Copy link
Contributor

@Tyler-2 Tyler-2 commented Apr 7, 2020

Fixes ansible/ansible#65928

SUMMARY

Fixes ansible/ansible#65928.

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

kinesis_stream

ADDITIONAL INFORMATION

Do a few extra conditional checks when running to avoid reporting changes if the stream already matches the desired configuration.

ok: [localhost] => {
    "changed": false, 
    "invocation": {
        "module_args": {
            "aws_access_key": null, 
            "aws_config": null, 
            "aws_secret_key": null, 
            "debug_botocore_endpoint_logs": false, 
            "ec2_url": null, 
            "encryption_state": "disabled", 
            "encryption_type": "KMS", 
            "key_id": "alias/aws/kinesis", 
            "name": "tyler-test", 
            "profile": null, 
            "region": null, 
            "retention_period": 24, 
            "security_token": null, 
            "shards": 1, 
            "state": "present", 
            "tags": null, 
            "validate_certs": true, 
            "wait": true, 
            "wait_timeout": 300
        }
    }, 
    "msg": "Kinesis Stream tyler-test encryption already stopped.", 
    "success": true

@tremble
Copy link
Contributor

tremble commented Apr 8, 2020

I know the tests don't currently exist for the kinesis module, would you be willing to try and add some integration tests too?

https://docs.ansible.com/ansible/latest/dev_guide/platforms/aws_guidelines.html#integration-tests-for-aws-modules

@Tyler-2
Copy link
Contributor Author

Tyler-2 commented Apr 9, 2020

That seems like it would be a feature, not really related to this fix. Would prefer that be added as an issue to be taken on. I tend to make changes during work hours to support my employer's use of Ansible, not sure if I can justify adding tests!

Do agree, it's a good idea though.

@tremble
Copy link
Contributor

tremble commented Apr 17, 2020

Unfortunately I'm not familiar enough with Kinesis to be willing to +1 this without test suites. Let's see if we can get ansible/ansible#62114 cleaned up and pulled in.

@Tyler-2
Copy link
Contributor Author

Tyler-2 commented Apr 17, 2020

Thankyou Tremble!

@tremble
Copy link
Contributor

tremble commented Apr 22, 2020

After some digging I suspect there's something more systemic wrong with the encryption pieces. All of the return values seem to change when encryption is being manipulated.

This change on it's own looks sane (now I understand a little about the module). I'll probably poke the code more thoroughly once this change has landed.

@Tyler-2
Copy link
Contributor Author

Tyler-2 commented Jun 2, 2020

Any other steps I can take to get this merged?

@jillr jillr changed the base branch from master to main July 2, 2020 19:48
@ansibullbot
Copy link

@ansibullbot ansibullbot added affects_2.10 bug This issue/PR relates to a bug community_review module module stale_ci CI is older than 7 days, rerun before merging labels Aug 19, 2020
@tremble
Copy link
Contributor

tremble commented Aug 20, 2020

@Tyler-2 One quick thing while we wait on the integration tests please add a changelog fragment: https://docs.ansible.com/ansible/latest/community/development_process.html#changelogs-how-to

@Tyler-2
Copy link
Contributor Author

Tyler-2 commented Aug 20, 2020

Added.

@ansibullbot

This comment has been minimized.

@ansibullbot ansibullbot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR plugins plugin (any type) and removed community_review labels Aug 28, 2020
@tremble
Copy link
Contributor

tremble commented Oct 8, 2020

Looks like something's a little funky with the unit tests here. Given the results of my integration testing I suspect the unit test is buggy rather than your code.

@gundalow gundalow added the pr_day Has been reviewed during a PR review Day. https://github.com/ansible/community/issues/407 label Dec 1, 2020
@jillr
Copy link
Collaborator

jillr commented Dec 1, 2020

The unit tests for this module pass on main, but do not pass for me locally if I rebase this PR onto latest main. @Tyler-2 the unit tests here probably need to be updated for the code changes, would you be able to take a look at that please?

@tremble tremble self-assigned this Dec 1, 2020
@tremble tremble force-pushed the 65928-fix-kinesis-changed-status branch from aee36ce to b0a009d Compare March 13, 2021 18:59
@tremble
Copy link
Contributor

tremble commented Mar 13, 2021

I've dug more into the unit tests. They're intrinsically flawed, they relied on find_stream returning a fixed value in check mode. (The failing test was failing because the test was bad, not the change - find_stream returned an unencrypted stream so no change should have been made)

I've fixed up various additional issues with kinesis_stream and removed the unit tests, testing is now based on integration tests instead.

@tremble tremble force-pushed the 65928-fix-kinesis-changed-status branch from 6d49eb4 to 9cb596a Compare March 29, 2021 11:10
@tremble tremble changed the title fix: Don't mark kstreams changed when no encryption actions taken kinesis_stream: Don't mark kstreams changed when no encryption actions taken Mar 29, 2021
@tremble tremble force-pushed the 65928-fix-kinesis-changed-status branch from 9cb596a to 29fe3b0 Compare March 29, 2021 18:40
@ansibullbot ansibullbot added community_review and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR labels Mar 31, 2021
@tremble tremble merged commit 44cf4be into ansible-collections:main Mar 31, 2021
@tremble
Copy link
Contributor

tremble commented Mar 31, 2021

@Tyler-2 it's taken almost a year but your fix is finally merged! I'm sorry it's taken so long. Thanks for your PR.

@Tyler-2
Copy link
Contributor Author

Tyler-2 commented Mar 31, 2021

Hallelujah! Thanks Tremble for really keeping on it.

alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this pull request Jul 19, 2021
…ons taken (ansible-collections#27)

* fix: Don't mark kstreams `changed` when no encryption actions taken

Fixes ansible/ansible#65928

* doc: add changelog fragment

* Move disable_stream_encryption test to integration test

* Update descriptions/fetch calls to still run on check_mode

* use standard helpers to convert tags to/from boto3 format

* use camel_dict_to_snake_dict helper

* use standard compare_aws_tags helper

* Fix tag handling and use standard helpers

* Format results and add tags when manipulating encryption settings

* Move kinesis_stream tests over to just integration tests

* changelog

* lint

Co-authored-by: Tyler Schwend <[email protected]>
Co-authored-by: Mark Chappell <[email protected]>
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this pull request Jul 19, 2021
…ons taken (ansible-collections#27)

* fix: Don't mark kstreams `changed` when no encryption actions taken

Fixes ansible/ansible#65928

* doc: add changelog fragment

* Move disable_stream_encryption test to integration test

* Update descriptions/fetch calls to still run on check_mode

* use standard helpers to convert tags to/from boto3 format

* use camel_dict_to_snake_dict helper

* use standard compare_aws_tags helper

* Fix tag handling and use standard helpers

* Format results and add tags when manipulating encryption settings

* Move kinesis_stream tests over to just integration tests

* changelog

* lint

Co-authored-by: Tyler Schwend <[email protected]>
Co-authored-by: Mark Chappell <[email protected]>
danielcotton pushed a commit to danielcotton/community.aws that referenced this pull request Nov 23, 2021
…ons taken (ansible-collections#27)

* fix: Don't mark kstreams `changed` when no encryption actions taken

Fixes ansible/ansible#65928

* doc: add changelog fragment

* Move disable_stream_encryption test to integration test

* Update descriptions/fetch calls to still run on check_mode

* use standard helpers to convert tags to/from boto3 format

* use camel_dict_to_snake_dict helper

* use standard compare_aws_tags helper

* Fix tag handling and use standard helpers

* Format results and add tags when manipulating encryption settings

* Move kinesis_stream tests over to just integration tests

* changelog

* lint

Co-authored-by: Tyler Schwend <[email protected]>
Co-authored-by: Mark Chappell <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects_2.10 bug This issue/PR relates to a bug community_review integration tests/integration module module plugins plugin (any type) pr_day Has been reviewed during a PR review Day. https://github.com/ansible/community/issues/407 stale_ci CI is older than 7 days, rerun before merging tests tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kinesis_stream marks all encryption operations as changes, even if encryption is already enabled
7 participants