Add additional storage providers: Google Drive, Storj, MinIO, FTP, etc.

[cedricve]

Currently we only have Kerberos Vault and Kerberos Hub integration implemented. However we would like to embrace other 3rd party tools such as Google Drive, Storj, Minio, Dropbox and others. With this open issue, we would like to request help from our community to implement more storage/persistence providers. We'll discuss here what is required to contribute to this feature.

Supported persistence:

• Kerberos Vault (by @cedricve)
• Kerberos Hub (by @cedricve)
• Storj
• MinIO
• Dropbox (by @cedricve) - currently not working with long lifetime tokens.
• Google Drive
• FTP

How to contribute

A detailed description is found below, describing the different parts that need to be added and modified. However a good PR (#96) can be found here.

Machinery (Golang/Backend/API)

Before implementing a new function make sure we know all the credentials/information that is required for creating a new provider. Have a look at the ./machinery/src/models/Config.go file, and add the additional information that you require. Also note that you'll need a new short name for your provider. For example Storj > storj or Dropbox > dropbox.

Create a new file in the machinery project: machinery/src/cloud.

Implement a single upload function with following function description. The function expects a configuration object which holds the credentials of the relevant provider, and the fileName which refers to the file that we want to upload. The return statement (bool, bool, error) defines if the upload was successful, if the file can be removed from disk (it might be that we need to try again false or we don't need to upload it anymore true), and if any error occurred.

func UploadDropbox(configuration *models.Configuration, fileName string) (bool, bool, error) {

Add a verify function, that will test if the configuration is working. This is called from the front-end by click the Verify connection button.

func VerifyDropbox(config models.Config, c *gin.Context) {

Once implemented we need to adapt the upload logic in ./machinery/src/cloud/Cloud.go. Extend the if statement and add your new function.

if config.Cloud == "s3" {
    uploaded, configured, err = UploadS3(configuration, fileName)
} else if config.Cloud == "kstorage" {
    uploaded, configured, err = UploadKerberosVault(configuration, fileName)
} else if config.Cloud == "dropbox" {
    uploaded, configured, err = UploadDropbox(configuration, fileName)
} else if config.Cloud == "gdrive" {
    // Todo: implement gdrive upload

UI (Front-end)

Open the ui/src/pages/Settings/Settings.jsx file. Most of the work is done here.

1. Append the storageTypes array with the new provider.
2. Add the required input fields to use the provider.
3. Provide translations in the ui/public/locales/ folder, make sure your labels/translation is copied to all the different translation files.

[LeoSpyke]

May I also suggest to add WebDAV to the supported protocols? It would unlock the whole potential of integration with fully open-source and self-hosted solutions like Nextcloud.

Edit: I noticed now that it is already taken into account due to this line in Cloud.go. Maybe the list of "supported persistence" in this issue should be upgraded as well.