install riff on-prem with images in non-public container registry or localhost docker

[jldec]

For enterprise users, running riff system install should work also if the install is launched on-prem from files in a downloaded archive, and all the installable images have been pushed to a private container registry or loaded to the localhost docker daemon.

Probably should investigate whether parameterizing with helm would be helpful, rather than patching all the install yaml on the fly.

cc: @trisberg

[glyn]

I'd like to explore whether install yaml can be made to work, before considering another approach such as helm.

Patching install yaml would have the downside that the images installed would not then be those referred to in the original yaml, which would seem to be a security issue. Also, some coordinates of the private container registry would need to be passed to the riff cli so that the yaml could be patched correctly.

I think it would be cleaner to use yaml files that correctly refer to images in the private container registry and to have some way of telling the riff cli to use those yaml files. We could for instance provide switches to specify alternative URLs for the yaml files. This would make for a longer command line, but if the user is typically cutting and pasting the command line, e.g. from Pivnet, then that needn't be a problem.

[trisberg]

I think we need to provide some kind of manifest for the installation as outlined here: #648

[jldec]

I have clarified the requirement in the issue name and original post.
All installation artifacts (including a manifest, and the release yaml) would live in a tarball and extracted to a local directory before the install is launched. Images could be docker loaded to a private container registry. The install process would need to be parameterizable to reference the local yaml.

[glyn]

Why would the tarball need to contain both a manifest and the yaml files that the manifest refers to? One or other should be sufficient.

[glyn]

Also, with custom yaml files, why would it be necessary to patch the yaml on the fly?

[jldec]

The tarball and its contents would not be custom - customers all download the same tarball.

The coordinates of the private registry would only be known at install time, so references to images in the yaml would probably still need to be tweaked.

I agree that the existence of a manifest is a separate question, but i think it's generally a good idea for testing or customizing the install so that only one location needs to be parameterized in the code.

[glyn]

Actually, the manifest turns out to be necessary so that the cli can tell the yaml files apart without resorting to file name matching. Also, it provides an ordering in case that turns out to be significant.

Going back to the tarball contents, let's take an example. Suppose Pivotal wants to ship its own release and we put a tarball on Pivnet and some images in a Pivnet docker repo. Why wouldn't we make the yaml files in the tarball refer to the location of the images on Pivnet rather than having to tweak them later?

[jldec]

Images will be pushed into customer-owned (private) registries prior to installation to allow for air-gapped operation.

[glyn]

Thanks, that makes sense. I'll implement the manifest support first before continuing with this issue.

[glyn]

The current issue seems to break down into two independent features, although the two would probably be used together most of the time:

1. Install from local archive containing manifest and all the YAML files referenced by the manifest.
2. Map the image coordinates to a private registry.

Using 1 without 2, the images would need to be in the normal registry.
Using 2 without 1, all the images referenced by the default YAML files would need to be in a private registry, suitably mapped.

@jldec do you agree with this decomposition?

[jldec]

yes I agree @glyn

A future enhancement may automate pre-loading images from files in the archive to a private registry.

[glyn]

Breaking out separate features for clarity:

• Install from local archive
• Map image coordinates to a private registry

[jldec]

@glyn, how about targeting the following scenario as a (mostly manual) proofpoint to give us confidence that our design makes sense:

• start with a local directory containing a manifest and hand-edited copies of release yaml
• identify and copy all required images into a private (riff team owned) registry like gcr
• validate that riff system install works (onto minikube or docker, not gke if the registry is gcr)
• validate that the resulting Knative environment can run functions

[glyn]

@jldec That's a good "sniff test". Probably best to split out a separate issue for that testing.

[glyn]

@jldec On second thoughts, the above test wouldn't be sufficient because if we missed some images that are in a public docker registry, the test will successfully pull them. I think we need to be disconnected from the internet to be safe.

[jldec]

Yes, you're right @glyn - but since testing fully air-gapped appears hard to accomplish short term, i think we should start with "best effort" and at least test that the image mappings work for those images which we've identified.

[glyn]

See the design document entitled Private docker Registries and riff.

[glyn]

@jldec I think this issue is complete. If you agree, would you like to close it?

[jldec]

I'm going to be selfish (sorry) and modify the requirement to include localhost docker.

[glyn]

@jldec No problem, but please could you elaborate the steps you expect to work (like you did above)?

[jldec]

(with the benefit of hindsight now that we have implemented riff image relocate)

local install from archive

• start with a local download archive/directory which includes the release yaml and image binaries
• load the images into the local docker
• retag the images (in local docker) and modify the release yaml to match retagged images
• riff system install (using modified release yaml) onto local k8s (docker or minikube)

cloud k8s install from archive with images pushed to private repo/registry

• push the retagged images from the local docker to the private repo/registry
• riff system install (again using modified release yaml) onto cloud k8s

if necessary i would be ok with the local-install tags being different from the relocated repo/registry tags - the most important thing is that both scenarios result in runnable Knative installs.

[glyn]

Thanks @jldec.

local install from archive

Breaking out features for clarity:

• riff image load (load & tag but not push)
• riff image relocate --flatten

cloud k8s install from archive with images pushed to private repo/registry

This already works using the following steps:

1. Download and explode archive
2. riff image relocate to private repo/registry
3. riff image push
4. riff system install --manifest

[glyn]

@jldec All the pieces have now landed. Please close this issue when you are ready.

[jldec]

well done!