Add RFC for extension layers #301

c0d1ngm0nk3y · 2023-10-09T12:42:34Z

buildpack-bot · 2023-10-09T12:42:50Z

Maintainers,

As you review this RFC please queue up issues to be created using the following commands:

/queue-issue <repo> "<title>" [labels]...
/unqueue-issue <uid>

Issues

(none)

loewenstein · 2023-10-19T17:04:31Z

@natalieparellano how do we manage to get feedback? We don't seem to be able to request a review.

natalieparellano

Thank you @c0d1ngm0nk3y for the RFC. I've added a few comments to try to understand it better.

natalieparellano · 2023-10-23T17:00:17Z

text/0000-extension-layer.md

+
+This change allows extensions to possess their own layers to utilize during the generation/extend process. Additionally, it ensures that extension output does not inadvertently interfere with other extension or buildpack layers during the build, and it does not unintentionally become part of the final application image.
+
+This would allow distroless run images to be extended.


IIUC this would allow an extension to use the package manager from the build image to download packages for the run image (by copying the package manager to the layers dir).

I would love to see n example extension laid out in the RFC

For instance, if we try to extend a distroless image, there's no way to extend it meaningfully with additional packages/files by the means available in the run image itself (sometimes even coreutils are not present). The extension could populate the extension layer with necessary files/packages/binaries and manipulate them from the generated Dockerfile in the context of a run image.

Sounds great - lets get that example added into the RFC. An example extension + Dockerfile that makes use of the files in the context. It'll help communicate the idea to a broader group

text/0000-extension-layer.md

pbusko · 2023-10-30T13:21:13Z

@jabrown85 @natalieparellano the RFC has been updated with the proposed changes.

natalieparellano

Thanks for the updates @c0d1ngm0nk3y. The proposed changes make sense and should be relatively easy to implement in the lifecycle. I wonder if there's a better term compared to "layers" so as not to create confusion with respect to buildpack layers. The purpose, lifecycle, etc of these context directories is very different.

text/0000-extension-layer.md

loewenstein · 2023-11-20T16:05:09Z

I wonder what the state of this is? @natalieparellano is there anything missing from your point of view? @pbusko are we on top of all provided review feedback?

text/0000-extension-layer.md

natalieparellano · 2023-11-21T14:45:41Z

I wonder what the state of this is? @natalieparellano is there anything missing from your point of view? @pbusko are we on top of all provided review feedback?

I left two new comments (here and here) but otherwise I think this is ready. If we don't evolve it any further after this week we can put it up for votes.

pbusko · 2023-11-22T09:58:20Z

I left two new comments (here and here) but otherwise I think this is ready. If we don't evolve it any further after this week we can put it up for votes.

The comments have been addressed

loewenstein · 2023-11-27T07:05:30Z

I wonder what the state of this is? @natalieparellano is there anything missing from your point of view? @pbusko are we on top of all provided review feedback?

I left two new comments (here and here) but otherwise I think this is ready. If we don't evolve it any further after this week we can put it up for votes.

@natalieparellano are you putting up for vote?

loewenstein · 2023-11-27T13:39:46Z

@jabrown85 could you help bringing this to a vote and getting it merged. That would be great. I hesitate to immediately start working on spec and implementation pull requests - inviting people to vote might lead to more review comments that lead to changes, but I would really like to see buildpacks/spec#378 closed and 0.13 released as soon as possible.

text/0000-extension-layer.md

natalieparellano · 2023-11-29T20:04:18Z

text/0000-extension-layer.md

+[how-it-works]: #how-it-works
+
+- Before execution of the `./bin/generate`, the lifecycle will create a distinct writable layer `$CNB_GENERATED_DIR/<extension-id>` for each extension which passed detection.
+- The `$CNB_GENERATED_DIR/<extension-id>` is provided to the `./bin/generate` as `<output>` (`$CNB_OUTPUT_DIR`) directory.


Currently the extender is looking for Dockerfiles in <generated>/run/<image extension ID>/Dockerfile and <generated>/build/<image extension ID>/Dockerfile (where <generated> defaults to <layers>/generated). Should we change the Platform API so that the extender looks for <generated>/<image extension ID>/run.Dockerfile, or should the lifecycle re-arrange the files after the generate phase? The former might be simpler in the long run, but platforms such as pack look for the presence of files in <generated>/build and <generated>/run to infer that extension is needed.

I would go with writing to <generated>/<image extension ID>/run.Dockerfile and/or <generated>/<image extension ID>/build.Dockerfile, as suggested in this RFC. Platforms such as pack might have a fallback mechanism, instead of adding more complex logic on the lifecycle side.

@pbusko I'm in favor of removing this complexity from the lifecycle. Could we add an extend boolean to the [build-image] table in analyzed.toml, to match [run-image]:

[run-image] image = "<image>" reference = "<image reference>" extend = false [build-image] reference = "<image reference>" extend = false

This will give platforms a simple way to check if extension is needed.

text/0000-extension-layer.md

pbusko · 2023-12-01T12:04:52Z

@natalieparellano please let us know if there's something still unclear/blocking the RFC from being merged

loewenstein · 2023-12-05T19:47:52Z

@natalieparellano @jabrown85 anything blocking a vote, merge and implementation?

natalieparellano · 2023-12-12T14:40:32Z

@jabrown85 could you give this one a read?

Co-authored-by: Ralf Pannemans <[email protected]> Signed-off-by: Ralf Pannemans <[email protected]> Co-authored-by: Pavel Busko <[email protected]> Signed-off-by: Pavel Busko <[email protected]>

Signed-off-by: Pavel Busko <[email protected]>

Co-authored-by: Natalie Arellano <[email protected]> Signed-off-by: Pavel Busko <[email protected]>

Signed-off-by: Pavel Busko <[email protected]> Co-authored-by: Pavel Busko <[email protected]>

natalieparellano · 2023-12-14T15:14:17Z

Let's merge this one, it was moved into voting exactly one week ago...

c0d1ngm0nk3y force-pushed the extension-distroless branch from 4f90632 to a1cec3d Compare October 9, 2023 12:44

pbusko force-pushed the extension-distroless branch 2 times, most recently from ff21dd8 to 17369c5 Compare October 13, 2023 13:39

natalieparellano reviewed Oct 23, 2023

View reviewed changes

natalieparellano added team/implementation spec/platform scope/project type/rfc status/steward-assigned labels Oct 23, 2023

natalieparellano self-assigned this Oct 23, 2023

natalieparellano requested review from jabrown85 and a team October 23, 2023 17:33

pbusko force-pushed the extension-distroless branch from 5ee23a7 to 15a3d5c Compare October 25, 2023 12:34

natalieparellano reviewed Oct 30, 2023

View reviewed changes

natalieparellano reviewed Oct 31, 2023

View reviewed changes

text/0000-extension-layer.md Outdated Show resolved Hide resolved

loewenstein mentioned this pull request Nov 3, 2023

Making builpack image extensions feature non-experimental buildpacks/spec#378

Closed

natalieparellano added status/ready-for-review and removed status/steward-assigned labels Nov 14, 2023

natalieparellano reviewed Nov 21, 2023

View reviewed changes

text/0000-extension-layer.md Show resolved Hide resolved

pbusko force-pushed the extension-distroless branch from ea6c575 to c80e600 Compare November 22, 2023 09:57

pbusko force-pushed the extension-distroless branch from c80e600 to 0208c59 Compare November 22, 2023 10:00

natalieparellano reviewed Nov 29, 2023

View reviewed changes

text/0000-extension-layer.md Outdated Show resolved Hide resolved

natalieparellano reviewed Nov 29, 2023

View reviewed changes

loewenstein reviewed Nov 29, 2023

View reviewed changes

text/0000-extension-layer.md Outdated Show resolved Hide resolved

pbusko force-pushed the extension-distroless branch from 0208c59 to f55475c Compare December 1, 2023 12:03

jabrown85 added status/voting and removed status/ready-for-review labels Dec 7, 2023

natalieparellano approved these changes Dec 12, 2023

View reviewed changes

pbusko and others added 7 commits December 14, 2023 13:33

Add RFC for extension layers

91a8eaa

Co-authored-by: Ralf Pannemans <[email protected]> Signed-off-by: Ralf Pannemans <[email protected]> Co-authored-by: Pavel Busko <[email protected]> Signed-off-by: Pavel Busko <[email protected]>

add examples

54fd7b0

Signed-off-by: Pavel Busko <[email protected]>

Update text/0000-extension-layer.md

3b23720

Co-authored-by: Natalie Arellano <[email protected]> Signed-off-by: Pavel Busko <[email protected]>

Update text/0000-extension-layer.md

990d715

Co-authored-by: Natalie Arellano <[email protected]> Signed-off-by: Pavel Busko <[email protected]>

Update text/0000-extension-layer.md

4130a0e

Co-authored-by: Natalie Arellano <[email protected]> Signed-off-by: Pavel Busko <[email protected]>

Update text/0000-extension-layer.md

5aa99d6

Co-authored-by: Natalie Arellano <[email protected]> Signed-off-by: Pavel Busko <[email protected]>

Reuse CNB_GENERATED_DIR and allow image-specific folders

d627627

Signed-off-by: Pavel Busko <[email protected]> Co-authored-by: Pavel Busko <[email protected]>

pbusko force-pushed the extension-distroless branch from f55475c to d627627 Compare December 14, 2023 12:33

natalieparellano merged commit b5c8bac into buildpacks:main Dec 14, 2023
7 checks passed

pbusko deleted the extension-distroless branch December 15, 2023 08:25

pbusko mentioned this pull request Jan 8, 2024

Platform API changes for image extension build contexts buildpacks/spec#382

Closed

This was referenced Jan 9, 2024

Changes for image extension build contexts buildpacks/spec#383

Merged

Platform API changes for image extension build contexts buildpacks/spec#384

Merged

natalieparellano mentioned this pull request Feb 6, 2024

Run image extension does not install RPMs that are already installed in the builder buildpacks/lifecycle#1285

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add RFC for extension layers #301

Add RFC for extension layers #301

c0d1ngm0nk3y commented Oct 9, 2023 •

edited

Loading

buildpack-bot commented Oct 9, 2023

loewenstein commented Oct 19, 2023

natalieparellano left a comment

natalieparellano Oct 23, 2023

jabrown85 Oct 23, 2023

pbusko Oct 24, 2023

jabrown85 Oct 24, 2023

pbusko commented Oct 30, 2023

natalieparellano left a comment

loewenstein commented Nov 20, 2023

natalieparellano commented Nov 21, 2023

pbusko commented Nov 22, 2023

loewenstein commented Nov 27, 2023

loewenstein commented Nov 27, 2023

natalieparellano Nov 29, 2023

pbusko Dec 1, 2023

natalieparellano Dec 12, 2023

pbusko commented Dec 1, 2023

loewenstein commented Dec 5, 2023

natalieparellano commented Dec 12, 2023

natalieparellano commented Dec 14, 2023


		This change allows extensions to possess their own layers to utilize during the generation/extend process. Additionally, it ensures that extension output does not inadvertently interfere with other extension or buildpack layers during the build, and it does not unintentionally become part of the final application image.

		This would allow distroless run images to be extended.

Add RFC for extension layers #301

Add RFC for extension layers #301

Conversation

c0d1ngm0nk3y commented Oct 9, 2023 • edited Loading

buildpack-bot commented Oct 9, 2023

Issues

loewenstein commented Oct 19, 2023

natalieparellano left a comment

Choose a reason for hiding this comment

natalieparellano Oct 23, 2023

Choose a reason for hiding this comment

jabrown85 Oct 23, 2023

Choose a reason for hiding this comment

pbusko Oct 24, 2023

Choose a reason for hiding this comment

jabrown85 Oct 24, 2023

Choose a reason for hiding this comment

pbusko commented Oct 30, 2023

natalieparellano left a comment

Choose a reason for hiding this comment

loewenstein commented Nov 20, 2023

natalieparellano commented Nov 21, 2023

pbusko commented Nov 22, 2023

loewenstein commented Nov 27, 2023

loewenstein commented Nov 27, 2023

natalieparellano Nov 29, 2023

Choose a reason for hiding this comment

pbusko Dec 1, 2023

Choose a reason for hiding this comment

natalieparellano Dec 12, 2023

Choose a reason for hiding this comment

pbusko commented Dec 1, 2023

loewenstein commented Dec 5, 2023

natalieparellano commented Dec 12, 2023

natalieparellano commented Dec 14, 2023

c0d1ngm0nk3y commented Oct 9, 2023 •

edited

Loading