Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add helper to get hosts on Go Metricbeat integration tests #13055

Merged
merged 38 commits into from
Aug 5, 2019

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Jul 24, 2019

Takes from #13039 and #7957 the required changes to allow
Metricbeat integration tests to obtain the hosts of the containers
from the framework, instead of needing to use environment
variables and tricky setups. This allows to run integration tests
for specific modules with go run -tags=integration, without
previous setup.

This is similar to the change introduced for python tests in #11925.

<SERVICE>_HOST environment variable can be set to the
host:port of a running service to avoid using compose, similarly
to old NO_COMPOSE environment variable.

This PR modifies many files, but most of the relevant changes
are in libbeat, and in the metricbeat docker compose. In the docker
compose file, env files are removed and metricbeat is run in the host
network. The rest of changes are to adapt the integration tests to
don't depend on environment variables to get the hosts information.

@jsoriano jsoriano added in progress Pull request is currently in progress. Metricbeat Metricbeat :Testing [zube]: In Progress Team:Integrations Label for the Integrations team labels Jul 24, 2019
@jsoriano jsoriano self-assigned this Jul 24, 2019
@jsoriano jsoriano force-pushed the metricbeat-integration-tests-host branch from 04489a8 to 2d8cdaf Compare July 26, 2019 10:27
@jsoriano jsoriano force-pushed the metricbeat-integration-tests-host branch 5 times, most recently from 5522efd to a3ce2e2 Compare July 29, 2019 19:21
@jsoriano
Copy link
Member Author

jenkins, test this again please

1 similar comment
@jsoriano
Copy link
Member Author

jenkins, test this again please

@jsoriano jsoriano force-pushed the metricbeat-integration-tests-host branch from f12a31b to 8284004 Compare July 30, 2019 18:53
@jsoriano
Copy link
Member Author

jenkins, test this again please

@jsoriano jsoriano force-pushed the metricbeat-integration-tests-host branch from 82ee7f4 to e8941df Compare July 31, 2019 09:12
@jsoriano jsoriano force-pushed the metricbeat-integration-tests-host branch from 1982ad0 to b523145 Compare July 31, 2019 11:57
@jsoriano jsoriano marked this pull request as ready for review July 31, 2019 14:43
@jsoriano jsoriano requested review from a team as code owners July 31, 2019 14:43
@jsoriano jsoriano added [zube]: In Review review and removed [zube]: In Progress in progress Pull request is currently in progress. labels Jul 31, 2019
@zube zube bot added the in progress Pull request is currently in progress. label Jul 31, 2019
@jsoriano jsoriano removed the in progress Pull request is currently in progress. label Aug 1, 2019
Copy link
Contributor

@exekias exekias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great enabler! did a first pass and left some questions, I will look into it more tomorrow

libbeat/tests/compose/compose.go Show resolved Hide resolved
}

// EnsureUpWithTimeout starts all the requested services (must be defined in docker-compose.yml)
// Wait for `timeout` seconds for health
func EnsureUpWithTimeout(t *testing.T, timeout int, services ...string) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what’s the reason to accept only one service? not saying it’s a bad idea, just want to know the reasoning

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two main reasons:

  • It simplifies the returned information. With an only service, only an object with the host information for a service is returned.
  • It doesn't seem to be needed, we only had an scenario with two services (Kibana, that also needs Elasticsearch), and it can also be done with docker compose dependencies. If access to both hosts were needed, we could have two EnsureUp.

If wouldn't be hard to implement it though, by returning a struct with all the services info and adding a method like HostForServicePort(service, port) HostInfo. But I didn't consider it neccessary by now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good to me, thanks for the explanation!

libbeat/tests/compose/wrapper.go Outdated Show resolved Hide resolved
@exekias
Copy link
Contributor

exekias commented Aug 2, 2019

I'm happy if CI is happy

@jsoriano
Copy link
Member Author

jsoriano commented Aug 2, 2019

jenkins, test this again please

}

// Wait ensures all wanted services are healthy. Wait loop (60s timeout)
func (c *Project) Wait(seconds int, services ...string) error {
func (c *Project) Wait(seconds time.Duration, services ...string) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@jsoriano
Copy link
Member Author

jsoriano commented Aug 5, 2019

CI is happy, we all are happy 🙂

@jsoriano jsoriano merged commit 01d7bfd into elastic:master Aug 5, 2019
@jsoriano jsoriano deleted the metricbeat-integration-tests-host branch August 5, 2019 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Metricbeat Metricbeat review Team:Integrations Label for the Integrations team :Testing v7.4.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants