9_Deploy demonstration image segmentation applications to Hugging Face Spaces

The demo app is live here

Build the model for deployment

Use the Segmentation Gym function gen_saved_model.py to create a model in .pb format from an existing model with model weights from h5 file

Create a Hugging Face account and create a new Space

1. Create a free account on Hugging Face
2. create a new Space for your app.
3. give it a name and a license
4. select 'gradio', and 'public', and then 'create space'

On your machine, clone it git clone https://huggingface.co/spaces/YourUserName/NameOfYourSpace

Create the app.py using Gradio

Imports:

import gradio as gr
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from skimage.transform import resize
from doodleverse_utils.prediction_imports import *
from doodleverse_utils.imports import *

These are the commands for loading and compiling the model

#load model
filepath = './saved_model'
model = tf.keras.models.load_model(filepath, compile = True)
model.compile

We use test-time augmentation consisting of 4 image transformations. Otsu thresholding is also used, with a small bias to ensure liberal coin identification

def segment(input_img, dims=(768, 768)):

    w = input_img.shape[0]
    h = input_img.shape[1]       
    img = standardize(input_img)
    
    img = resize(img, dims, preserve_range=True, clip=True) 
    img = np.expand_dims(img,axis=0)
    
    est_label = model.predict(img) 

    #Test Time Augmentation
    est_label2 = np.flipud(model.predict((np.flipud(img)), batch_size=1))
    est_label3 = np.fliplr(model.predict((np.fliplr(img)), batch_size=1))
    est_label4 = np.flipud(np.fliplr(model.predict((np.flipud(np.fliplr(img))))))

    #soft voting - sum the softmax scores to return the new TTA estimated softmax scores
    est_label = est_label + est_label2 + est_label3 + est_label4
    est_label /= 4
    
    pred = np.squeeze(est_label, axis=0)
    pred = resize(pred, (w, h), preserve_range=True, clip=True)
    
    bias=.1
    thres_land = threshold_otsu(pred[:,:,1])-bias
    print("Land threshold: %f" % (thres_land))
    mask = (pred[:,:,1]>=thres_land).astype('uint8')

    #pred = np.squeeze(est_label, axis=0)
    #pred = resize(pred, (w, h), preserve_range=True, clip=True)
    #mask = np.argmax(pred,-1)

   
    class_label_colormap = [
        "#3366CC",
        "#DC3912",
        "#FF9900",
        "#109618",
        "#990099",
        "#0099C6",
        "#DD4477",
        "#66AA00",
        "#B82E2E",
        "#316395",
    ]
    
    # add classes
    class_label_colormap = class_label_colormap[:2]

    color_label = label_to_colors(
        mask,
        input_img[:, :, 0] == 0,
        alpha=128,
        colormap=class_label_colormap,
        color_class_offset=0,
        do_alpha=False,
    )
    
    #overlay plot
    #plt.figure(figsize=(8,8))
    plt.clf()
    plt.imshow(input_img,cmap='gray')
    plt.imshow(color_label, alpha=0.4)
    plt.axis("off")
    plt.margins(x=0, y=0)
    return plt 

The app is built simply by defining a title and description

title = "Find coins in images of sand!" 
description = "This model segments beach sediment imagery into two classes: a) background, and b) coin" 

... creating a list of examples, create multiple outputs and a single input, then passing all arguments to the gr.Interface function

examples = [['examples/20190828_152930.jpg'],['examples/20191010_135020.jpg']] 

inp = gr.inputs.Image()
out1 = gr.outputs.Image(type='numpy')
out2 = gr.outputs.Plot(type='matplotlib')

Segapp = gr.Interface(segment, inp, [out1, out2], title = title, description = description, examples=examples, theme="grass", flagging_options=["bad", "ok", "good", "perfect"])

Segapp.launch()

Organize files

1. Add requirements.txt file to the root directory, including our very own doodleverse_utils package

tensorflow
numpy
matplotlib
scikit-image
doodleverse_utils

2. Add example imagery to a folder called examples

3. Add your model folder called saved_model that contains .pb file, as well as variables and assets folders. Note that assets may be empty

Deploy to Hugging Face Spaces

Git LFS is used for model and example imagery. You need to install it using git lfs install or a pre-built binary for your OS

Then you add, commit and push as normal like below:

git lfs track ".\examples\*.*"
git lfs track ".\saved_model\*.*"
git add .
git commit -m "msg"
git push

(input your HuggingFace username and password)

Hugging Face will then build the model and deploy it