Move openmetrics module to oss #26561

ChrsMark · 2021-06-29T12:00:51Z

What does this PR do?

This PR moves openmetrics Metricbeat module under oss directory.

elasticmachine · 2021-06-29T12:00:53Z

Pinging @elastic/integrations (Team:Integrations)

elasticmachine · 2021-06-29T12:16:06Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #26561 updated
Start Time: 2021-06-30T14:56:07.230+0000
Duration: 141 min 17 sec
Commit: 1f05045

Test stats 🧪

Test	Results
Failed	0
Passed	47281
Skipped	5259
Total	52540

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	47281
Skipped	5259
Total	52540

jsoriano · 2021-06-29T14:45:36Z

CI failure seems related.

It'd be nice to add a changelog entry for this.

mergify · 2021-06-29T20:18:38Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b move_openmetrics_to_oss upstream/move_openmetrics_to_oss
git merge upstream/master
git push upstream move_openmetrics_to_oss

Signed-off-by: chrismark <[email protected]>

kuisathaverat · 2021-06-30T09:45:03Z

Jenkinsfile

+    if(!fileExists(module)) {
+      module = ''
+    }


the getCommonModuleInTheChangeSet(String directory) search for a pattern in a directory path, fi the pattern does not exist will return ''. I am not sure of the goal of check a folder with the name of the module exists in the current folder, probably will never exist.

cc @v1v

Is pattern checked for the current(=new) state or the old one? In this particular case we have removed the openmetrics module completely from x-pack but the tester still understands that it has changes and thus it kicks the tests for it which leads to test's failure later on since the module does not actually exists under x-pack.

A bit of context:

Unfortunately this is an edge case when moving files between x-pack and oss beats.

elastic/apm-pipeline-library#535 didn't implement a git diff by excluding files that have been deleted/moved.

So somehow it tries to run the module in x-pack when that particular module was moved to the oss.

Therefore, the validation if the module folder exists could be a good workaround.

I am not sure of the goal of check a folder with the name of the module exists in the current folder, probably will never exist.

if the module name exists in the pwd then it should work, unless module is a nested folder somewhere else, if so, it's required to consider that particular case.

I've just confirmed what Ivan said, and there is something missing in this logic, something like the below snippet might work,

Suggested change

if(!fileExists(module)) {

module = ''

}

if(!fileExists("${directory}/module/${module}")) {

module = ''

}

~~though I don't know if module is always a nester folder under <beat_name_folder> or x-pack/<beat_name_folder>~~

Actually yes:

https://github.com/elastic/beats/blob/master/Jenkinsfile#L19

https://github.com/elastic/beats/blob/master/Jenkinsfile#L25

Co-authored-by: Victor Martinez <[email protected]>

jsoriano · 2021-06-30T10:53:13Z

Failing test is flaky, opened issue (#26608) and PR to skip (#26609).

ChrsMark · 2021-06-30T12:30:53Z

Ah now another flaky test...

@v1v @kuisathaverat are you ok with the change on Jenkinsfile?

jsoriano

Once the changelog entry is added, ship it!

Signed-off-by: chrismark <[email protected]>

* master: [MetricBeat] [AWS] Fix aws metric tags with resourcegroupstaggingapi paginator (elastic#26385) (elastic#26443) Move openmetrics module to oss (elastic#26561) Skip flaky test TestFilestreamMetadataUpdatedOnRename (elastic#26609) [filebeat][fortinet] Use default add_locale for fortinet.firewall (elastic#26524) Enroll proxy settings (elastic#26514)

ChrsMark · 2021-07-01T08:22:10Z

Wondering why mergify didn't open the backport PR for this 🤔

(cherry picked from commit 92143fe)

v1v · 2021-07-01T08:54:19Z

Wondering why mergify didn't open the backport PR for this 🤔

@ChrsMark , backports are automated with mergify as long as the backport-v7.<number> is added.

<number> can be changing with the upcoming versions and therefore .mergify.yml should have those changes in place.

backport-v7.15 does not exist in the .mergify.yml file.

There is some automation to include new versions in the .mergify.yml, but I'm not sure those PRs were merged by the person who bumped those release versions.

In fact, the PR is open, #26620 so , I'll leave that operation to the release manager in the beats team, either that PR can be merged if possible, or the mergify change to be cherry-picked and push to the master branch.

v1v · 2021-07-01T09:20:27Z

I've just merged #26620 and created #26641 to automate the auto-merge.

ChrsMark · 2021-07-01T09:22:30Z

Thanks @v1v !

…stage-failed-within-same-build * upstream/master: (36 commits) Revert "[CI] fight the flakiness with some retry option in the CI only for the Pull Requests (elastic#26617)" (elastic#26704) Packaging: linux/armv7 is not supported (elastic#26706) Cyberarkpas: Link to official docs on how to setup TLS (elastic#26614) Make network_direction, registered_domain and convert processors compatible with ES older than 7.13.0 (elastic#26676) Disable armv7 packaging (elastic#26679) [Heartbeat] use --params flag for synthetics (elastic#26674) Update dependent package to avoid downloading a suspicious file (elastic#26406) [mergify] set title and allow bp in any direction (elastic#26648) Fix memory leak in SQL helper when database is not available (elastic#26607) [CI] fight the flakiness with some retry option in the CI only for the Pull Requests (elastic#26617) [mergify] automate PRs that change the backport rules (elastic#26641) [Metricbeat] Add Airflow module in xpack (elastic#26220) chore: add-backport-next (elastic#26620) [metricbeat] Add state_job metricset (elastic#26479) CI: jenkins labels are less time consuming now (elastic#26613) [MetricBeat] [AWS] Fix aws metric tags with resourcegroupstaggingapi paginator (elastic#26385) (elastic#26443) Move openmetrics module to oss (elastic#26561) Skip flaky test TestFilestreamMetadataUpdatedOnRename (elastic#26609) [filebeat][fortinet] Use default add_locale for fortinet.firewall (elastic#26524) Enroll proxy settings (elastic#26514) ...

ChrsMark added the Team:Integrations Label for the Integrations team label Jun 29, 2021

ChrsMark self-assigned this Jun 29, 2021

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 29, 2021

ChrsMark requested a review from a team as a code owner June 30, 2021 08:37

botelastic bot added the Team:Automation Label for the Observability productivity team label Jun 30, 2021

ChrsMark added 5 commits June 30, 2021 11:48

Move openmetrics module to oss

06ff876

Signed-off-by: chrismark <[email protected]>

Restore doc.go file

01c6f22

Signed-off-by: chrismark <[email protected]>

Fix restored file

0c04a48

Signed-off-by: chrismark <[email protected]>

Fix docker-compose file

e584964

Signed-off-by: chrismark <[email protected]>

try to fix ci

cedb7f7

Signed-off-by: chrismark <[email protected]>

ChrsMark force-pushed the move_openmetrics_to_oss branch from 61f2a65 to cedb7f7 Compare June 30, 2021 08:48

check if dir exists

9d84c0b

Signed-off-by: chrismark <[email protected]>

kuisathaverat reviewed Jun 30, 2021

View reviewed changes

jsoriano mentioned this pull request Jun 30, 2021

Flaky Test [Build&Test / filebeat-build / TestFilestreamMetadataUpdatedOnRename – github.com/elastic/beats/v7/filebeat/input/filestream] #26608

Closed

Update Jenkinsfile

d07f483

Co-authored-by: Victor Martinez <[email protected]>

v1v approved these changes Jun 30, 2021

View reviewed changes

jsoriano approved these changes Jun 30, 2021

View reviewed changes

Add changelog entry

1f05045

Signed-off-by: chrismark <[email protected]>

jsoriano approved these changes Jun 30, 2021

View reviewed changes

ChrsMark added the backport-v7.15.0 Automated backport with mergify label Jun 30, 2021

ChrsMark merged commit 92143fe into elastic:master Jun 30, 2021

ChrsMark mentioned this pull request Jul 1, 2021

Cherry-pick #26561 to 7.x: Move openmetrics module to oss #26639

Merged

ChrsMark added the v7.14.0 label Jul 1, 2021

ChrsMark added a commit to ChrsMark/beats that referenced this pull request Jul 1, 2021

Move openmetrics module to oss (elastic#26561)

836a4f5

(cherry picked from commit 92143fe)

v1v mentioned this pull request Jul 1, 2021

[mergify] automate PRs that change the backport rules #26641

Merged

ChrsMark added a commit that referenced this pull request Jul 1, 2021

Cherry-pick #26561 to 7.x: Move openmetrics module to oss (#26639)

4b6b03c

ChrsMark mentioned this pull request Jul 6, 2021

[7.x](backport #26479) [metricbeat] Add state_job metricset #26671

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move openmetrics module to oss #26561

Move openmetrics module to oss #26561

ChrsMark commented Jun 29, 2021

elasticmachine commented Jun 29, 2021

elasticmachine commented Jun 29, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

jsoriano commented Jun 29, 2021

mergify bot commented Jun 29, 2021

kuisathaverat Jun 30, 2021

ChrsMark Jun 30, 2021 •

edited

Loading

v1v Jun 30, 2021

v1v Jun 30, 2021 •

edited

Loading

jsoriano commented Jun 30, 2021

ChrsMark commented Jun 30, 2021

jsoriano left a comment

ChrsMark commented Jul 1, 2021

v1v commented Jul 1, 2021 •

edited

Loading

v1v commented Jul 1, 2021

ChrsMark commented Jul 1, 2021

Move openmetrics module to oss #26561

Move openmetrics module to oss #26561

Conversation

ChrsMark commented Jun 29, 2021

What does this PR do?

elasticmachine commented Jun 29, 2021

elasticmachine commented Jun 29, 2021 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

jsoriano commented Jun 29, 2021

mergify bot commented Jun 29, 2021

kuisathaverat Jun 30, 2021

Choose a reason for hiding this comment

ChrsMark Jun 30, 2021 • edited Loading

Choose a reason for hiding this comment

v1v Jun 30, 2021

Choose a reason for hiding this comment

v1v Jun 30, 2021 • edited Loading

Choose a reason for hiding this comment

jsoriano commented Jun 30, 2021

ChrsMark commented Jun 30, 2021

jsoriano left a comment

Choose a reason for hiding this comment

ChrsMark commented Jul 1, 2021

v1v commented Jul 1, 2021 • edited Loading

v1v commented Jul 1, 2021

ChrsMark commented Jul 1, 2021

elasticmachine commented Jun 29, 2021 •

edited by jenkins-beats-ci bot

Loading

ChrsMark Jun 30, 2021 •

edited

Loading

v1v Jun 30, 2021 •

edited

Loading

v1v commented Jul 1, 2021 •

edited

Loading