[Enhancement] Cross-egeria container image

[planetf1]

Is there an existing issue for this?

• I have searched the existing issues

Please describe the new behavior that that will improve Egeria

Proposal:

Build an 'uber' egeria docker container image -that assembles artefacts from mulitple git repositories

• Use existing egeria image as the base
• Pull in latest/ version specific connectors to add to the image (and any other artefacts we deem useful)
• publish to docker.io / quay.io as 'egeria-full'
• align with egeria releases only - trying to track every snapshot with varying release versions and update cycles would be too much overhead

This would not replace the egeria image, which is 'just' egeria

This would require a new git repository, but any future cross-repo-cutting docker images could be built here also. But we wouldn't extend to other artefact types

This approach would 'replace' that currently taken in the postgres connector, and sas connector where an egeria base image is used, then that single connector is added. However those approaches could still be useful to individual communities, and as part of automated testing.

This would also be in addition to odpi/egeria-docs#532 which proposed that in our helm charts we also specify additional connectors via URLs, and they are dynamically downloaded at runtime. So we end up with 2 approaches, one is a hardened image (including security scans), the other a more flexible, developer-friendly approach.

However there are some limitations with this approach - especially when each connector may have it's own lifecycle - when versions change, and so the dynamic approach is probably the priority -- but this could be useful.

Alternatives

• Pulling additional connectors & other files at runtime as identified above
• Building many additional docker images, one per connector - but no way to combine

Any Further Information?

No response

Would you be prepared to be assigned this issue to work on?

• I can work on this

[planetf1]

As an alternative

• Build a new egeria filesystem layout on disk, using published build artifacts (ie not reusing the egeria image) - this might require changes to the existing ci/cd, perhaps maven/gradle
• package this up and publish both as an archive (jar or more likely gzip)
• Also build a docker image using a similar approach to used in egeria

The end result would also be a combined docker image, but also an archive file where we might include utilities from egeria-dev-projects for example. Though one tension is that between keeping it suitably streamlined & not including 'non-production' code. This could mean have a few different assembly combinations, with docker images to match

[planetf1]

One of the challenges with such an image is that each connector may have different dependencies, and they may overlap or clash.

If connectors are built with shaded jars, this might be manageable. However an alternative approach, is to minimize the connectors deployed on a particular image, and then chose the right image for the hosted servers.

For example the platform running a postgres integration service would run an egeria image with just the postgres connector
A platform running the strimzi integration service would run an egeria image with just the strimzi integration service

The pros/cons of this approach should be considered, and documented.

It may be appropriate to just do both
If we do adopt a combined approach also it will likely surface again the challenge of dependencies and not using shaded jars

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[planetf1]

#7224 covers the same principle, so closing this one as a dup