[build][packaging] Add resilience when docker build #22050

v1v · 2020-10-21T14:08:45Z

What does this PR do?

Retry to build docker images when something bad happened.

We could potentially include a retry step in the CI but it might add some overhead, another approach could be creating a specific mage goal to run the docker build rather than just using the mage goal for everything.

Why is it important?

Sometimes some third party resources are not accessible temporarily.

Also the Unified Release team will enjoy this particular approach to avoid any issues when pulling docker images or building from scratch.

Related issues

Caused by #21563
Closes #22020

elasticmachine · 2020-10-21T14:08:51Z

Pinging @elastic/ingest-management (Team:Ingest Management)

x-pack/elastic-agent/magefile.go

elasticmachine · 2020-10-21T14:17:03Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: [Pull request #22050 updated]
Start Time: 2020-10-22T09:05:26.950+0000
Duration: 92 min 42 sec

Test stats 🧪

Test	Results
Failed	0
Passed	16369
Skipped	1342
Total	17711

Co-authored-by: Manuel de la Peña <[email protected]>

kuisathaverat · 2020-10-21T14:25:58Z

probably it resolves also #22020, @jsoriano Do package target uses the same class/function to build the Docker images?

This reverts commit 81e447a.

jsoriano · 2020-10-21T17:15:03Z

libbeat/tests/compose/wrapper.go

+			fmt.Printf(">> Building docker images failed using docker-compose (let's try now with docker-compose up --build): %s", stderr.String())
+		}
+	}
+


There are already some retries in the higher-level EnsureUp method, that is the one used out of this package.

Also, we shouldn't add a explicit build here, we should take into account opts.Create.Build and so on.

I think this code is not needed.

…e retry happens in https://github.com/elastic/beats/blob/addadf2f692949a3b90c2825adb150acf952c7d0/libbeat/tests/compose/compose.go\#L99

…uild-retry-docker-if-error-packaging * upstream/master: Fix the url of reviewdog (elastic#21981) revert WSS process reporting for windows (elastic#22055) Fix typo (elastic#19585) (elastic#22061)

jsoriano

👍

* upstream/master: (27 commits) chore: use ubuntu 18 as linux agent (elastic#22084) docs: Prepare Changelog for 7.9.3 (elastic#22073) (elastic#22075) docs: Prepare Changelog for 6.8.13 (elastic#22072) (elastic#22079) [build][packaging] Add resilience when docker build (elastic#22050) Fix the url of reviewdog (elastic#21981) revert WSS process reporting for windows (elastic#22055) Fix typo (elastic#19585) (elastic#22061) [Ingest Manager] Use ML_SYSTEM to detect if agent is running as a service (elastic#21884) Prevent log input from sending duplicate messages due to file renaming (elastic#21911) feat: package aliases for snapshots (elastic#21960) [DOC] Add firewall as possible troubleshooting issue (elastic#21743) [Filebeat] Add max_number_of_messages config parameter for S3 input (elastic#21993) [Elastic Agent] Fix missing elastic_agent event data (elastic#21994) Document auditbeat system process module config (elastic#21766) Update links (elastic#22012) dynamically find librpm (elastic#21936) Fix Istio docs (elastic#22019) [beats-tester][packaging] store packages in another location (elastic#21903) [Kubernetes] Remove redundant dockersock volume mount (elastic#22009) [Ingest Manager] Always try snapshot repo for agent upgrade (elastic#21951) ...

…beats into feature/support-flaky-test-analyser * 'feature/support-flaky-test-analyser' of github.com:v1v/beats: (40 commits) [CI] support windows-10 (elastic#19804) Use default config when creating the input (elastic#22126) Change x509 mappings from file. to tls.server. (elastic#22097) Add fleet settings image (elastic#22065) Edit 7.9.3 changelog (elastic#22117) Edit 6.8.13 release notes (elastic#22120) Incorporate librpm fix feedback (elastic#22098) [libbeat] Add more disk queue unit tests and fix a size-check bug (elastic#22107) docs: move kerberos include (elastic#22109) Check context.Canceled and fix s3 input config (elastic#22036) Add max_number_of_messages into aws filebeat fileset vars (elastic#22057) Remove suricata.eve.timestamp alias (elastic#22095) [Ingest Manager] Use symlink path for reexecutions (elastic#21835) chore: use ubuntu 18 as linux agent (elastic#22084) docs: Prepare Changelog for 7.9.3 (elastic#22073) (elastic#22075) docs: Prepare Changelog for 6.8.13 (elastic#22072) (elastic#22079) [build][packaging] Add resilience when docker build (elastic#22050) Fix the url of reviewdog (elastic#21981) revert WSS process reporting for windows (elastic#22055) Fix typo (elastic#19585) (elastic#22061) ...

…lastic#22083)

[build][packaging] Add resilience when docker build

1ccaf2a

v1v added Packaging automation Team:Automation Label for the Observability productivity team build-system Issue or change affecting Mage, Make, or build scripts. labels Oct 21, 2020

v1v self-assigned this Oct 21, 2020

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label Team:Ingest Management and removed needs_team Indicates that the issue/PR needs a Team:* label labels Oct 21, 2020

v1v requested review from andrewkroh, jsoriano, urso and a team October 21, 2020 14:09

mdelapenya reviewed Oct 21, 2020

View reviewed changes

x-pack/elastic-agent/magefile.go Outdated Show resolved Hide resolved

v1v and others added 2 commits October 21, 2020 15:19

Update x-pack/elastic-agent/magefile.go

ba580ab

Co-authored-by: Manuel de la Peña <[email protected]>

[docker-compose] Add resilience when docker-compose up --build

a1a6668

kuisathaverat approved these changes Oct 21, 2020

View reviewed changes

v1v added 6 commits October 21, 2020 15:31

fix linting

dba421c

fix linting

81e447a

Revert "fix linting"

1c26720

This reverts commit 81e447a.

fix linting redeclared

8211f25

Use printf

1b94c05

Fix type

addadf2

cachedout approved these changes Oct 21, 2020

View reviewed changes

jsoriano requested changes Oct 21, 2020

View reviewed changes

v1v added 2 commits October 22, 2020 10:04

As stated in the review this change is not required anymore, since th…

109932a

…e retry happens in https://github.com/elastic/beats/blob/addadf2f692949a3b90c2825adb150acf952c7d0/libbeat/tests/compose/compose.go\#L99

Merge remote-tracking branch 'upstream/master' into feature/support-b…

5ae1ae8

…uild-retry-docker-if-error-packaging * upstream/master: Fix the url of reviewdog (elastic#21981) revert WSS process reporting for windows (elastic#22055) Fix typo (elastic#19585) (elastic#22061)

jsoriano approved these changes Oct 22, 2020

View reviewed changes

v1v merged commit 69cddaa into elastic:master Oct 22, 2020

v1v deleted the feature/support-build-retry-docker-if-error-packaging branch October 22, 2020 10:38

v1v added a commit to v1v/beats that referenced this pull request Oct 22, 2020

[build][packaging] Add resilience when docker build (elastic#22050)

3b420d2

This was referenced Oct 22, 2020

Cherry-pick to 7.x: [build][packaging] Add resilience when docker build (#22050) #22081

Merged

Cherry-pick to 7.10: [build][packaging] Add resilience when docker build (#22050) #22082

Merged

v1v added a commit to v1v/beats that referenced this pull request Oct 22, 2020

[build][packaging] Add resilience when docker build (elastic#22050)

3cd2565

v1v added a commit to v1v/beats that referenced this pull request Oct 22, 2020

[build][packaging] Add resilience when docker build (elastic#22050)

c3f255c

v1v mentioned this pull request Oct 22, 2020

Cherry-pick to 7.9: [build][packaging] Add resilience when docker build (#22050) #22083

Merged

v1v added a commit that referenced this pull request Oct 22, 2020

[build][packaging] Add resilience when docker build (#22050) (#22082)

6a780e7

v1v added a commit that referenced this pull request Oct 22, 2020

[build][packaging] Add resilience when docker build (#22050) (#22081)

efe6f88

v1v added a commit that referenced this pull request Oct 26, 2020

[build][packaging] Add resilience when docker build (#22050) (#22083)

7cc1544

leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023

[build][packaging] Add resilience when docker build (elastic#22050) (e…

15eedf0

…lastic#22083)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[build][packaging] Add resilience when docker build #22050

[build][packaging] Add resilience when docker build #22050

v1v commented Oct 21, 2020 •

edited

Loading

elasticmachine commented Oct 21, 2020

elasticmachine commented Oct 21, 2020 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

kuisathaverat commented Oct 21, 2020 •

edited

Loading

jsoriano Oct 21, 2020

jsoriano left a comment

[build][packaging] Add resilience when docker build #22050

[build][packaging] Add resilience when docker build #22050

Conversation

v1v commented Oct 21, 2020 • edited Loading

What does this PR do?

Why is it important?

Related issues

elasticmachine commented Oct 21, 2020

elasticmachine commented Oct 21, 2020 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

kuisathaverat commented Oct 21, 2020 • edited Loading

jsoriano Oct 21, 2020

Choose a reason for hiding this comment

jsoriano left a comment

Choose a reason for hiding this comment

v1v commented Oct 21, 2020 •

edited

Loading

elasticmachine commented Oct 21, 2020 •

edited by jenkins-beats-ci bot

Loading

kuisathaverat commented Oct 21, 2020 •

edited

Loading