[builder] add docker buildx bake as a builder

[saulshanabrook]

First off, I wanted to say thank you for putting together this tool. The ability to specify local build context and have they transformed into images is great!

I have been using Docker buildx bake as a similar tool, using the Kubernetes executor to build images on k8 and push images.

I have been appreciating the ability to parallelize the build DAG as well as add caching for certain steps easily. See my generated docker bake json, for an example of the caching and tags.

So I am curious about two points:

1. Are you using buildkit to parallelize the builds and to support caching?
2. Have you considered using the same JSON syntax as Docker Bake or working with them to come up with a common form? It seems like the tools are trying to specify very similar features. Or could you just use that tool directly?

These are larger discussion points, so I more just wanted to get a sense from you all how you are thinking about the relationship between the two tools.

I am very thankful that there is so much great work going on this space right now!

[cppforlife]

Are you using buildkit to parallelize the builds and to support caching?

kbld just calls docker build CLI command. layer caching is done by docker itself so no work for us there.

Have you considered using the same JSON syntax as Docker Bake or working with them to come up with a common form? It seems like the tools are trying to specify very similar features. Or could you just use that tool directly?

hmm, i havent really looked at docker buildx bake, i probably should. one of the goals for kbld was to wrap familiar building tools (docker build, pack build) so that they can be more easily used in a dev/prod workflow. i could see integration with buildx, but i havent personally used it so dont have opinion how best use it. is it more effecient compared to plain docker build?

[saulshanabrook]

kbld just calls docker build CLI command. layer caching is done by docker itself so no work for us there.

Just FYI that docker buidkit is not enabled by default in the docker build command but is enabled by default in the docker buildx command. It has some extra parallelization functionality over the traditional docker build command as well as the ability use build time caching for RUN commands. this combination makes rebuilding a number of images that have some shared layers much faster, I have found, than with the traditional docker builder.

i could see integration with buildx, but i havent personally used it so dont have opinion how best use it. is it more effecient compared to plain docker build?

AFAIK It's a higher level semantics to express a number of images to build and possibly tags for them, which is similar to the syntax you provide. It uses docker buildx under the covers. But yes, if you are building more than one image it's more efficient than docker build.

However, I just started using it a month or so ago and would refer to maybe @tonistiigi to give more details.

[cppforlife]

However, I just started using it a month or so ago

do you typically write out something like this (https://github.com/data-apis/python-record-api/blob/4fa06df6c0eac85e9f54d70f7172dde9e1ef3125/k8/Makefile#L123-L169) and feed it into bake command? if so it sounds like this might be an interesting backend for kbld (as an alternative to docker build and pack backends) for more advanced users of docker. i could imagine instead of taking Dockerfile it could find file x and call out to docker buildx bake. then grab images that were built. (im not sure how that would fit though with making a "single" build graph from multiple image sources though).

[saulshanabrook]

do you typically write out something like this (https://github.com/data-apis/python-record-api/blob/4fa06df6c0eac85e9f54d70f7172dde9e1ef3125/k8/Makefile#L123-L169) and feed it into bake command?

Yeah. Here is the generated JSON, which is a bit easier to reason about:

{
  "group": {
    "default": {
      "targets": [
        "dask",
        "matplotlib",
        "networkx",
        "pandas",
        "sample-usage",
        "scipy",
        "skimage",
        "sklearn",
        "xarray"
      ]
    }
  },
  "target": {
    "base-base": {
      "dockerfile": "Dockerfile"
    },
    "base": {
      "context": "images/base",
      "inherits": [
        "base-base"
      ],
      "cache-to": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/base:cache,mode=max"
      ],
      "cache-from": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/base:cache"
      ],
      "args": {
        "PYTHON_PACKAGE_VERSION": "1.2.1"
      },
      "tags": [
        "registry.digitalocean.com/python-record-api/base:1.2.1-0"
      ]
    },
    "xarray": {
      "context": "images/xarray",
      "inherits": [
        "base-base"
      ],
      "args": {
        "FROM": "registry.digitalocean.com/python-record-api/base:1.2.1-0"
      },
      "cache-to": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/xarray:cache,mode=max"
      ],
      "cache-from": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/xarray:cache"
      ],
      "tags": [
        "registry.digitalocean.com/python-record-api/xarray:1.2.1-0-1"
      ]
    },
    "sklearn": {
      "context": "images/sklearn",
      "inherits": [
        "base-base"
      ],
      "args": {
        "FROM": "registry.digitalocean.com/python-record-api/base:1.2.1-0"
      },
      "cache-to": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/sklearn:cache,mode=max"
      ],
      "cache-from": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/sklearn:cache"
      ],
      "tags": [
        "registry.digitalocean.com/python-record-api/sklearn:1.2.1-0-1"
      ]
    },
    "skimage": {
      "context": "images/skimage",
      "inherits": [
        "base-base"
      ],
      "args": {
        "FROM": "registry.digitalocean.com/python-record-api/base:1.2.1-0"
      },
      "cache-to": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/skimage:cache,mode=max"
      ],
      "cache-from": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/skimage:cache"
      ],
      "tags": [
        "registry.digitalocean.com/python-record-api/skimage:1.2.1-0-1"
      ]
    },
    "scipy": {
      "context": "images/scipy",
      "inherits": [
        "base-base"
      ],
      "args": {
        "FROM": "registry.digitalocean.com/python-record-api/base:1.2.1-0"
      },
      "cache-to": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/scipy:cache,mode=max"
      ],
      "cache-from": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/scipy:cache"
      ],
      "tags": [
        "registry.digitalocean.com/python-record-api/scipy:1.2.1-0-0"
      ]
    },
    "sample-usage": {
      "context": "images/sample-usage",
      "inherits": [
        "base-base"
      ],
      "args": {
        "FROM": "registry.digitalocean.com/python-record-api/base:1.2.1-0"
      },
      "cache-to": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/sample-usage:cache,mode=max"
      ],
      "cache-from": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/sample-usage:cache"
      ],
      "tags": [
        "registry.digitalocean.com/python-record-api/sample-usage:1.2.1-0-0"
      ]
    },
    "pandas": {
      "context": "images/pandas",
      "inherits": [
        "base-base"
      ],
      "args": {
        "FROM": "registry.digitalocean.com/python-record-api/base:1.2.1-0"
      },
      "cache-to": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/pandas:cache,mode=max"
      ],
      "cache-from": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/pandas:cache"
      ],
      "tags": [
        "registry.digitalocean.com/python-record-api/pandas:1.2.1-0-1.1.0-1"
      ]
    },
    "networkx": {
      "context": "images/networkx",
      "inherits": [
        "base-base"
      ],
      "args": {
        "FROM": "registry.digitalocean.com/python-record-api/base:1.2.1-0"
      },
      "cache-to": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/networkx:cache,mode=max"
      ],
      "cache-from": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/networkx:cache"
      ],
      "tags": [
        "registry.digitalocean.com/python-record-api/networkx:1.2.1-0-7"
      ]
    },
    "matplotlib": {
      "context": "images/matplotlib",
      "inherits": [
        "base-base"
      ],
      "args": {
        "FROM": "registry.digitalocean.com/python-record-api/base:1.2.1-0"
      },
      "cache-to": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/matplotlib:cache,mode=max"
      ],
      "cache-from": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/matplotlib:cache"
      ],
      "tags": [
        "registry.digitalocean.com/python-record-api/matplotlib:1.2.1-0-1"
      ]
    },
    "dask": {
      "context": "images/dask",
      "inherits": [
        "base-base"
      ],
      "args": {
        "FROM": "registry.digitalocean.com/python-record-api/base:1.2.1-0"
      },
      "cache-to": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/dask:cache,mode=max"
      ],
      "cache-from": [
        "type=registry,ref=registry.digitalocean.com/python-record-api/dask:cache"
      ],
      "tags": [
        "registry.digitalocean.com/python-record-api/dask:1.2.1-0-1"
      ]
    }
  }
}

They also support some HCL version of the file, to generate this JSON as well, but I needed more flexibility than that, so I just switched to using jq generate the JSON instead.

[cppforlife]

👍 im going to keep this issue open for anyone who may be interested in contributing support for this type of builder.

[cppforlife]

related: added docker buildx build support in https://github.com/vmware-tanzu/carvel-kbld/releases/tag/v0.34.0