New storage backends

[meln1k]

If I want to integrate a custom storage backend, what parts do I need to implement?

So far it looks like I can start with something similar to https://github.com/umputun/remark/blob/master/backend/app/store/engine/mongo.go, but it is not clear how to implement a backend for storing images.

Are there any recommendations or best practices?

[umputun]

Hi. As of now there is no documentation explaining how to deal with external storages. Example of mongo implementation is the closest one. In addition to the core storage (i.e. store/engine) you will need 2 more things:

• storage for uploaded images (implement store/image/image.go/Store interface)
• storage for user's avatars. This part is external library accepting user defined implementations - see https://github.com/go-pkgz/auth/blob/master/avatar/store.go

Another thing you may want is a store for admin data and this is in store/admin.

Generally I would recommend not to do it yet. The current way to add storages is not too friendly and need way too much efforts. I'm in the process of migrating all of this to some pluggable architecture allowing much simpler extensions. As a part of this process I'll move mongo to such a plugin (bolt will stay as a core implementation inside) and will provide an example of some trivial user-defined storage covering all areas.

[umputun]

The new backend interface has been merged into master. Instead of the insane amount of function we used to have, all low-level storage functionality consolidated into just a few generalized methods and several higher-level operations were moved to the common store/service.

In order to allow users to bring their own storages, I have added a special layer for RPC-like calls (see /backend/app/remote). For convenience, it also includes the code for user's plugin, i.e. server/remote/plugin side/dispatcher. Support of such custom storages was added to options (cli params and env) as well.

Both new interface and RPC protocol need some detailed documentation and examples (work in progress), but generally speaking, it should be clear from the code what the protocol is and how to use it. I'm planning to add full demo implementation of the custom storage backend (in-memory storage) this weekend.

[umputun]

I have added an example implementation of alternative storage.