Cloud independent storage | option to specify endpoint for aws-java-sdk-s3 client

[hydridity]

Feature description 💡

Greetings,

I've been following development on this project for quite a while now, and I'm happy to see DEX being implemented
it's great step towards achieving private cloud environments compatibility for the project.

With that in mind, I'd like to suggest feature for registry storage module (and any other module using only cloud storage as for now) allowing ability to use self hosted S3 compatible storage solution such as MinIO which I'm not afraid to say is de-facto standard for self hosted S3 storage

Since this project uses aws-java-sdk-s3 implementation can be achieved using the AmazonS3ClientBuilder class as described in: https://nm-muzi.com/docs/how-to-use-aws-sdk-for-java-with-minio-server.html

The ClientBuilder allows for setting custom S3 endpoint which allows aws-java-sdk-s3 client to communicate with any compatible s3 storage solution which MinIO is, while from the app standpoint it still functions as if communicating with the AWS S3, so no separate client connector implementation is required and all the api calls for the s3client are the same, with the benefit of being able to store objects inside private cloud environment.

I'd like to test this project in real world environment and provide feedback but unfortunately I'm unable to do so as long as any of the critical part is public cloud dependent

[alfespa17]

This sounds very interesting and it could be quite simple to implement I will check it during this week. Thanks for the feedback @hydridity

[hydridity]

Just noticed, the link I've send is not official documentation but some kind of independent snapshot mirror of it, MinIO updated their docs page and unfortunately the new documentation does not have that example anymore, the only official(as of the source) docs I've been able to retrieve is via wayback machine here, and there are few references for that technique scattered on the stackoverflow

as far as I know, it should still work fine and stable with the official aws-s3 client

[alfespa17]

Hello @hydridity, MinIO support was added in version 2.7.0 that we released yesterday and we also did the update in the helm chart now you should be able to use Terrakube with MinIO using this configuration in the helm chart.

storage:
  aws:
    accessKey: "XXXX"
    secretKey: "XXXX"
    bucketName: "XXXX"
    region: "us-east-1"
    endpoint: "hello.com"

By the way feel free to open a new issue if you find any kind of problem, all feedback is welcome