Image enhancement via super-resolution machine learning

[LSgeo]

Hi, it's fairly straightforward to run super-resolution using pre-trained models in the web-browser via colab, or on your own computer using binaries/python these days. The quality of the photo enhancement is quite good, and I think it would be a stand-out feature to include in photoprism.

For example, load a photo, click an "enhance" button, the super-resolution executable is called targeting the image, and the result is displayed / saved with metadata copied across. All the data are kept on the local server, using local GPU or CPU resources. On a modern GPU, the inference time is typically only a few seconds.

It would require a little UI modification, a system to call the super-resolution function, and a way to review/save the result, as well a fair amount of testing hardware requirements.

Real-ESRGAN is BSD-3 licensed, and has a GitHub repository. A stand-alone executable version is available, which removes the PyTorch dependency. There may be a TensorFlow implementation that would tie in nicely to the existing platform.

[lastzero]

From what I understand, that could only benefit images with a bad original resolution? When you have a great camera, you generally don't need/want to upscale pictures with AI.

Note that with the current resources available, it will take a bit (next year?) until we can start thinking about adding such features. Even if it just takes a few days, we don't have them right now.

[LSgeo]

It's also useful when dealing with cropped or otherwise digitally zoomed photos (or mobile phones, old photos, etc). The linked model is also trained to reduce JPEG artefacts and other digital "clean-ups". It could be a good exercise in calling an executable binary from the UI too, for use with other possible off-the-shelf machine learning products.

Understandable re: capacity, but it's always good to have some ideas on the table. Cheers for a cool product! I'll dabble in my own space with it and see what it could look like.

[lastzero]

Feel free to add your learnings / recommendations to our Developer Guide for future reference.