Issue 34213 : Fixing Docker port mapping to avoid Integration test case failure #34221

agithomas · 2023-01-10T05:42:53Z

Type of change

Bug

What does this PR do?

Fixed the integration testcases for sql and oracle modules

Why is it important?

Fixed the integration testcases for sql and oracle modules. Build will fail if the fix is not present.
Example : #34213

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Relates Build 1179 for main with status FAILURE #34213

Use cases

Screenshots

agikthomas@service-integration-dev-idc-01:~/go/src/github.com/elastic/beats/x-pack/metricbeat$ mage build
>> build: Building metricbeat
agikthomas@service-integration-dev-idc-01:~/go/src/github.com/elastic/beats/x-pack/metricbeat$ MODULE=sql mage goIntegTest
Generated fields.yml for metricbeat to /home/agikthomas/go/src/github.com/elastic/beats/x-pack/metricbeat/fields.yml
>> Building docker images
Building beat
Building elasticsearch
Building kibana
Creating metricbeat_8_7_0_30087202c4-snapshot_beat_run ... done
go: downloading gotest.tools/gotestsum v1.7.0
go: downloading gotest.tools v2.2.0+incompatible
go: downloading github.com/fatih/color v1.13.0
go: downloading github.com/dnephin/pflag v1.0.7
go: downloading golang.org/x/tools v0.1.12
go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/jonboulle/clockwork v0.2.2
go: downloading golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde
go: downloading golang.org/x/term v0.2.0
go: downloading github.com/mattn/go-colorable v0.1.12
go: downloading github.com/mattn/go-isatty v0.0.14
go: downloading github.com/adriansr/fsnotify v1.4.8-0.20211018144411-a81f2b630e7c
go: downloading golang.org/x/sys v0.2.0
go: downloading golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4
>> go test: Integration-sql Testing
exec: gotestsum --no-color -f standard-quiet --junitfile build/TEST-go-integration-sql.xml --jsonfile build/TEST-go-integration-sql.out.json -- -tags  integration ./module/sql/...
go: downloading github.com/godror/godror v0.33.2
go: downloading github.com/go-sql-driver/mysql v1.6.0
go: downloading github.com/jmoiron/sqlx v1.3.1
go: downloading github.com/elastic/elastic-agent-libs v0.2.15
go: downloading github.com/lib/pq v1.10.3
go: downloading github.com/stretchr/testify v1.8.1
go: downloading github.com/joeshaw/multierror v0.0.0-20140124173710-69b34d4ec901
go: downloading go.uber.org/zap v1.23.0
go: downloading github.com/gofrs/uuid v4.2.0+incompatible
go: downloading github.com/docker/docker v20.10.12+incompatible
go: downloading github.com/elastic/elastic-agent-autodiscover v0.5.0
go: downloading github.com/mitchellh/hashstructure v0.0.0-20170116052023-ab25296c0f51
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/elastic/go-sysinfo v1.9.0
go: downloading github.com/elastic/go-ucfg v0.8.6
go: downloading github.com/go-logfmt/logfmt v0.5.1
go: downloading go.uber.org/atomic v1.10.0
go: downloading go.uber.org/multierr v1.8.0
go: downloading github.com/go-logr/logr v1.2.3
go: downloading github.com/godror/knownpb v0.1.0
go: downloading github.com/hashicorp/go-multierror v1.1.1
go: downloading go.elastic.co/ecszap v1.0.1
go: downloading github.com/gorilla/mux v1.8.0
go: downloading go.elastic.co/apm/v2 v2.0.0
go: downloading github.com/elastic/elastic-agent-client/v7 v7.0.3
go: downloading golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e
go: downloading golang.org/x/net v0.2.0
go: downloading github.com/docker/go-connections v0.4.0
go: downloading google.golang.org/protobuf v1.28.1
go: downloading github.com/hashicorp/errwrap v1.1.0
go: downloading github.com/golang/protobuf v1.5.2
go: downloading google.golang.org/grpc v1.49.0
go: downloading howett.net/plist v1.0.0
go: downloading github.com/prometheus/procfs v0.8.0
go: downloading go.elastic.co/fastjson v1.1.0
go: downloading github.com/armon/go-radix v1.0.0
go: downloading github.com/docker/go-units v0.4.0
go: downloading github.com/opencontainers/image-spec v1.0.2
go: downloading github.com/gogo/protobuf v1.3.2
go: downloading github.com/docker/distribution v2.8.1+incompatible
go: downloading github.com/containerd/containerd v1.5.13
go: downloading github.com/opencontainers/go-digest v1.0.0
go: downloading github.com/sirupsen/logrus v1.8.1
go: downloading google.golang.org/genproto v0.0.0-20220829175752-36a9c930ecbf
go: downloading golang.org/x/text v0.4.0
?       github.com/elastic/beats/v7/x-pack/metricbeat/module/sql        [no test files]
ok      github.com/elastic/beats/v7/x-pack/metricbeat/module/sql/query  366.608s

=== Skipped
=== SKIP: x-pack/metricbeat/module/sql/query TestMySQL/data (0.00s)
    data_generator.go:87: skip data generation tests
    --- SKIP: TestMySQL/data (0.00s)

=== SKIP: x-pack/metricbeat/module/sql/query TestMySQL/data#01 (0.00s)
    data_generator.go:87: skip data generation tests
    --- SKIP: TestMySQL/data#01 (0.00s)

=== SKIP: x-pack/metricbeat/module/sql/query TestPostgreSQL/data (0.00s)
    data_generator.go:87: skip data generation tests
    --- SKIP: TestPostgreSQL/data (0.00s)

=== SKIP: x-pack/metricbeat/module/sql/query TestPostgreSQL/data#01 (0.00s)
    data_generator.go:87: skip data generation tests
    --- SKIP: TestPostgreSQL/data#01 (0.00s)

DONE 20 tests, 4 skipped in 372.207s
>> go test: Integration-sql Test Passed

agikthomas@service-integration-dev-idc-01:~/go/src/github.com/elastic/beats/x-pack/metricbeat$ MODULE=oracle mage goIntegTest
Generated fields.yml for metricbeat to /home/agikthomas/go/src/github.com/elastic/beats/x-pack/metricbeat/fields.yml
>> Building docker images
Building beat
Building elasticsearch
Building kibana
WARNING: Found orphan containers (metricbeat_8_7_0_30087202c4-snapshot_oracle_1, metricbeat_8_7_0_30087202c4-snapshot_postgresql_1, metricbeat_8_7_0_30087202c4-snapshot_mysql_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Creating metricbeat_8_7_0_30087202c4-snapshot_beat_run ... done
go: downloading gotest.tools/gotestsum v1.7.0
go: downloading gotest.tools v2.2.0+incompatible
go: downloading github.com/fatih/color v1.13.0
go: downloading github.com/dnephin/pflag v1.0.7
go: downloading golang.org/x/tools v0.1.12
go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/jonboulle/clockwork v0.2.2
go: downloading golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde
go: downloading golang.org/x/term v0.2.0
go: downloading github.com/mattn/go-colorable v0.1.12
go: downloading github.com/mattn/go-isatty v0.0.14
go: downloading github.com/adriansr/fsnotify v1.4.8-0.20211018144411-a81f2b630e7c
go: downloading golang.org/x/sys v0.2.0
go: downloading golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4
>> go test: Integration-oracle Testing
exec: gotestsum --no-color -f standard-quiet --junitfile build/TEST-go-integration-oracle.xml --jsonfile build/TEST-go-integration-oracle.out.json -- -tags  integration ./module/oracle/...
go: downloading github.com/elastic/elastic-agent-libs v0.2.15
go: downloading github.com/stretchr/testify v1.8.1
go: downloading github.com/godror/godror v0.33.2
go: downloading github.com/gofrs/uuid v4.2.0+incompatible
go: downloading github.com/joeshaw/multierror v0.0.0-20140124173710-69b34d4ec901
go: downloading github.com/elastic/go-sysinfo v1.9.0
go: downloading github.com/elastic/go-ucfg v0.8.6
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/gorilla/mux v1.8.0
go: downloading go.uber.org/multierr v1.8.0
go: downloading github.com/elastic/elastic-agent-client/v7 v7.0.3
go: downloading go.elastic.co/apm/v2 v2.0.0
go: downloading github.com/go-logfmt/logfmt v0.5.1
go: downloading github.com/go-logr/logr v1.2.3
go: downloading github.com/godror/knownpb v0.1.0
go: downloading go.uber.org/zap v1.23.0
go: downloading golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e
go: downloading golang.org/x/net v0.2.0
go: downloading github.com/hashicorp/go-multierror v1.1.1
go: downloading go.elastic.co/ecszap v1.0.1
go: downloading go.uber.org/atomic v1.10.0
go: downloading github.com/golang/protobuf v1.5.2
go: downloading google.golang.org/grpc v1.49.0
go: downloading google.golang.org/protobuf v1.28.1
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/hashicorp/errwrap v1.1.0
go: downloading howett.net/plist v1.0.0
go: downloading github.com/prometheus/procfs v0.8.0
go: downloading go.elastic.co/fastjson v1.1.0
go: downloading github.com/armon/go-radix v1.0.0
go: downloading google.golang.org/genproto v0.0.0-20220829175752-36a9c930ecbf
go: downloading golang.org/x/text v0.4.0
?       github.com/elastic/beats/v7/x-pack/metricbeat/module/oracle     [no test files]
?       github.com/elastic/beats/v7/x-pack/metricbeat/module/oracle/performance [no test files]
ok      github.com/elastic/beats/v7/x-pack/metricbeat/module/oracle/sysmetric   0.012s
ok      github.com/elastic/beats/v7/x-pack/metricbeat/module/oracle/tablespace  0.016s

DONE 16 tests in 4.735s
>> go test: Integration-oracle Test Passed

Logs

…lastic#31628) This prevents "monitored" from being written into event.outcome which does not allow this value according to ECS.

mergify · 2023-01-10T05:43:27Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @agithomas? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

elasticmachine · 2023-01-10T05:50:19Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-01-17T04:22:09.852+0000
Duration: 55 min 9 sec

Test stats 🧪

Test	Results
Failed	0
Passed	1383
Skipped	95
Total	1478

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

agithomas · 2023-01-11T03:27:25Z

@efd6 , please help by reviewing the code changes

efd6

The commit message should probably have an explanation similar to the line in the changelog as editted.

efd6 · 2023-01-15T22:03:37Z

CHANGELOG.next.asciidoc

@@ -107,6 +107,7 @@ https://github.com/elastic/beats/compare/v8.2.0\...main[Check the HEAD diff]
 - Add tags to events based on parsed identifier. {pull}33472[33472]
 - Support Oracle-specific connection strings in SQL module {issue}32089[32089] {pull}32293[32293]
 - Remove deprecated metrics from controller manager, scheduler and proxy {pull}34161[34161]
+- Fixed the integration testcases for sql and oracle modules {pull}34221[34221]


Suggested change

- Fixed the integration testcases for sql and oracle modules {pull}34221[34221]

- Fix the integration testcase docker port mapping for sql and oracle modules {pull}34221[34221]

Though this is not user facing; is this changelog line needed?

Thanks @efd6 for your review. I updated CHANGELOG-developer.next.asciidoc to instead of CHANGELOG.next.asciidoc

…ndent change

… issue-34213

…se failure (#34221) *Fix the integration testcase docker port mapping for sql and oracle modules

agithomas and others added 22 commits May 18, 2022 11:10

x-pack/filebeat/module/cisco: remove invalid values from ECS fields (e…

4c62bba

…lastic#31628) This prevents "monitored" from being written into event.outcome which does not allow this value according to ECS.

Merge branch 'elastic:main' into main

3c39232

Merge branch 'elastic:main' into main

d1c97af

Merge branch 'elastic:main' into main

aa4f464

Merge branch 'elastic:main' into main

d9e070c

Merge branch 'elastic:main' into main

a089576

Merge branch 'elastic:main' into main

5173fda

Merge branch 'elastic:main' into main

f429eaf

Merge branch 'elastic:main' into main

4640072

Merge branch 'elastic:main' into main

c795711

Merge branch 'elastic:main' into main

ea7b9cf

Merge branch 'elastic:main' into main

746a767

Merge branch 'elastic:main' into main

f75a9ab

Merge branch 'elastic:main' into main

7723142

Merge branch 'elastic:main' into main

1bad1ac

Merge branch 'elastic:main' into main

be6298a

Merge branch 'elastic:main' into main

047a198

Merge branch 'elastic:main' into main

85a89d8

Merge branch 'elastic:main' into main

6655dda

Merge branch 'elastic:main' into main

3008720

Fixed the integration testcases for sql and oracle modules

cebf92c

Added changelog entry

c016979

agithomas requested a review from a team as a code owner January 10, 2023 05:42

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 10, 2023

mergify bot assigned agithomas Jan 10, 2023

Updated changelog

419033c

agithomas changed the title ~~Issue 34213~~ Issue 34213 : Fixing Docker port mapping to avoid Integration test case failure Jan 10, 2023

agithomas requested a review from efd6 January 10, 2023 06:39

Merge branch 'main' into issue-34213

d382f4c

agithomas added the Team:Service-Integrations Label for the Service Integrations team label Jan 10, 2023

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 10, 2023

efd6 reviewed Jan 15, 2023

View reviewed changes

agithomas and others added 3 commits January 17, 2023 09:43

Merge branch 'main' into issue-34213

3e91952

Updated the changelog file and its content as it is not customer depe…

6b1bf0d

…ndent change

Merge branch 'issue-34213' of https://github.com/agithomas/beats into…

8dd0c5b

… issue-34213

efd6 approved these changes Jan 17, 2023

View reviewed changes

agithomas merged commit 98e0bab into elastic:main Jan 17, 2023

chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023

Issue 34213 : Fixing Docker port mapping to avoid Integration test ca…

b3e2f73

…se failure (#34221) *Fix the integration testcase docker port mapping for sql and oracle modules

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue 34213 : Fixing Docker port mapping to avoid Integration test case failure #34221

Issue 34213 : Fixing Docker port mapping to avoid Integration test case failure #34221

agithomas commented Jan 10, 2023 •

edited

Loading

mergify bot commented Jan 10, 2023

elasticmachine commented Jan 10, 2023 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

agithomas commented Jan 11, 2023

efd6 left a comment

efd6 Jan 15, 2023

agithomas Jan 17, 2023 •

edited

Loading

	- Fixed the integration testcases for sql and oracle modules {pull}34221[34221]
	- Fix the integration testcase docker port mapping for sql and oracle modules {pull}34221[34221]

Issue 34213 : Fixing Docker port mapping to avoid Integration test case failure #34221

Issue 34213 : Fixing Docker port mapping to avoid Integration test case failure #34221

Conversation

agithomas commented Jan 10, 2023 • edited Loading

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

mergify bot commented Jan 10, 2023

elasticmachine commented Jan 10, 2023 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

agithomas commented Jan 11, 2023

efd6 left a comment

Choose a reason for hiding this comment

efd6 Jan 15, 2023

Choose a reason for hiding this comment

agithomas Jan 17, 2023 • edited Loading

Choose a reason for hiding this comment

agithomas commented Jan 10, 2023 •

edited

Loading

elasticmachine commented Jan 10, 2023 •

edited by jenkins-beats-ci bot

Loading

agithomas Jan 17, 2023 •

edited

Loading