Entrypoint: Support for multiple pods and multiple stages #1824
Conversation
|
Example of stages: |
|
Calling this a WIP. There's a bit to do here, but putting this up early. |
There was a problem hiding this comment.
The commit message is good...but there's so much going on here I have trouble understanding how it all works; basically need to try to see some porting of existing cosa usage.
One high level thought - maybe it would be clearer if this was a separate quay.io/coreos/coreos-assembler-openshift container from a separate git repo or something?
Is there a particular section of this you think needs more review?
💯 that why there is an |
|
Lifting the WIP. @cgwalters there's nothing specific I need reviewed other than the "make sure I'm not doing something stupid" sniff check. This is the minimum viable work needed to start draining the pipelines. This should be the last massive drop of code and this works on OpenShift 3 and OpenShift 4. I anticipate the usual amount of follow-on PR's and I've tried to be really verbose in comments. |
- Added GoDeps - Added schema generator. - Fix CI by using ../tools/bin in the path
To aide in the coordination of artifacts between pods, entrypoint now understands how to find a build. Entrypoint also will use the schema to determine what artifacts its capable of building.
In a previous PR, an unrelated question triggered a line of thinking
that illustrated that `entrypoint` was not being creative enough. This
is the result.
Previously to this change, COSA expected the orchestrator to know enough
about COSA to both setup and run COSA. When running COSA in various
environments this a high bar. This change moves the knowledge of "how
to run COSA" into COSA itself, namely:
- Extends the JobSpec to support "stages". Each stage can run arbitrary
commands such as "cosa build." Stages are run sequentially, but the
commands can be run concurrently.
- Full live-cycling of worker pods. Worker pods are created, monitored
and deleted.
- Defines, but is not implemented, the ability to create non-blocking
pods for stages.
- Enables serving of files from the buildconfig pod to the worker pods.
Worker pods can fetch sources and configs from the buildconfig pod.
- Full streaming of logs to the buildconfig and saving of logs.
- Worker pods return the results to starting log via Minio.
- Stages can require an artifact, which requires something (i.e. a
worker) to place it on the origin pod.
- Stages support prep and post commands.
Worker pods are tightly coupled to the buildconfig pod, by design. A
worker pod is created from the same pod definition of the buildconfig.
`entry builder` determines whether its running in buildconfig or worker
mode by checking for an OpenShift buildconfig spec or a COSA entrypoint
`workSpec`. When a worker comes up:
- initializes /srv
- auto-discovers secrets
- runs the stage(s) assigned to it.
- returns the results over Minio
The JobSpec now supports short-hand artifact creation. For example:
job:
build_name: me
strict: true
recipe:
git_ref: testing-devel
git_url: https://github.com/coreos/fedora-coreos-config
stages:
- id: base
description: "building"
build_artifacts:
- base
- metal
- metal4k
- id: live-iso
own_pod: true
description: "live ISO"
require_artifacts:
- base
- metal
- metal4k
build_artifacts:
- installer
|
|
||
| .PHONY: clean | ||
| clean: | ||
| @go clean . | ||
| @rm -rf bin | ||
|
|
||
| .PHYON: schema |
| } | ||
| } | ||
|
|
||
| if buildID == "" { |
There was a problem hiding this comment.
Can this code be moved to the block above?
|
There is a lot going on here but overall looks sane to me. There are some minor things that can be handled in a followup PR. I will leave it up for a little longer to give more time for others to review. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bh7cw, cgwalters, darkmuggle The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/override ci/prow/sanity Tests past, PROW showed success, but the status was not showing up here. Status show everythings okay....forcing the merge. |
|
@darkmuggle: Overrode contexts on behalf of darkmuggle: ci/prow/sanity, tide In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
FWIW this shouldn't be necessary - tide is the thing that merges once the other PR contexts go green.
Usually I'd go with at least a |
|
Looks like the COSA builds in Quay started failing right about the time this was merged:
|
|
|
Looks like the build issue was fixed by #1856 |
In a previous PR, an unrelated question triggered a line of thinking
that illustrated that
entrypointwas not being creative enough. Thisis the result.
Previously to this change, COSA expected the orchestrator to know enough
about COSA to both setup and run COSA. When running COSA in various
environments this a high bar. This change moves the knowledge of "how
to run COSA" into COSA itself, namely:
commands such as "cosa build." Stages are run sequentially, but the
commands can be run concurrently.
and deleted.
pods for stages.
Worker pods can fetch sources and configs from the buildconfig pod.
worker) to place it on the origin pod.
Worker pods are tightly coupled to the buildconfig pod, by design. A
worker pod is created from the same pod definition of the buildconfig.
entry builderdetermines whether its running in buildconfig or workermode by checking for an OpenShift buildconfig spec or a COSA entrypoint
workSpec. When a worker comes up:The JobSpec now supports short-hand artifact creation. For example: