Integration testing of this cookbook in the CI

[artem-sidorenko]

Currently we only run unit tests/lints. Its not easily possible to test this cookbook in the same way like we do with chef-ssh-hardening (kitchen-dokken): we change here tonns of OS parameters.

What about to have a proper integration testing via IaaS?

• I already did it with digitalocean and it works just fine
• Another option would be maybe the Google Cloud with an advantage - its billed per minute. I do not want to consider the ec2/azure, they are a bit complexer for this simple job (and require a bit more configuration/setup)

My suggested way:

• to go with DigitalOcean (I'll provide a token within my account)
• to go with travis environment variables for storing of access token

Via this way we get following:

• integration tests of main repository
• people with forks can configure their own DO access token in travis and get integration tests too
• in case of PRs without integration tests, we can repush them to our forks/main repo and get them tested

@atomic111 @chris-rock opinions?

[chris-rock]

I like the idea, especially since not all parameters can be tested properly in a docker container. Do you think we should activate the current kitchen-dokken setup for this cookbook too?
This would allow us to have basic os coverage via containers and extensive testing via digital ocean. In our travis tests, we could make digital ocean optional so that we see the failure, but do not need to merge red PRs.

[artem-sidorenko]

Do you think we should activate the current kitchen-dokken setup for this cookbook too?

I'm not sure this will work properly: we change a lot of really system related things, which are probably not covered by kernel namespaces (e.g. some sysctl flags related to tcp/ip)

We can try that and see if it works

[ehaselwanter]

it would be cool to be able to switch off the kernel related tests anyways. I run the inspec suite against docker images (as the target) and at the moment I switch dem off with a profile.

[chris-rock]

@artem-sidorenko we could add an inspec attribute if required to our baseline tests. Kitchen supports inspec attributes already. Nevertheless, lets split this into two issues:

• add digitalocean tests
• add docker tests

[artem-sidorenko]

@chris-rock It makes definitely sense to add some flag(s) to the chef-os-hardning and to the linux-baseline, which would allow use-case like @ehaselwanter described and enable a partly testing with containers. However I'm not sure about the short-term perspective of this.

What about following suggestion?

• right now we add DO "full-vm" integration testing support like described (its straight-forward way and does not require architecture changes/new flags within chef-os-hardening and linux-baseline)
• we release chef-os-hardening 2.0.0
• for chef-os-hardening 3.0.0 we make the required changes in order to allow it to be more "container-friendly"

[chris-rock]

@artem-sidorenko lets go with that approach. Lets have a discussion, what we expect from a bare container. cc @atomic111

[ehaselwanter]

sounds good to me :-)

[artem-sidorenko]

@chris-rock @ehaselwanter @atomic111 please have a look and review the #144

[artem-sidorenko]

I'll close this as we have now integration tests