Another implementation of image mask generator

[vfdev-5]

Hi,
I would like to propose another type of image mask generator.
I'm aware of this and of 'Example of transforming images and masks together', so I tried to mix both functions (transform image and mask, provide a pipeline of transformations).
Glad to hear any feedback on this proposition and the code.

[the-moliver]

Can you explain at a high level what you're trying to accomplish with this and why? What are the advantages, drawbacks, etc...

[vfdev-5]

Idea is to give more flexibility to image data generator and cover the case of image/mask augmentation (even it can be done by zipping two Keras ImageDataGenerators). Typical usage is :

def xy_provider(image_ids, infinite=True):
        while True:
            for image_id in image_ids:
                image = load_image(image_id)
                target = load_target(image_id)

                # Some custom preprocesssing: resize
                # ...
                yield image, target
            if not infinite:
                return

train_gen = ImageDataGenerator(pipeline=('random_transform', 'standardize'),
                             featurewise_center=True,
                             featurewise_std_normalization=True,
                             rotation_range=90.,
                             width_shift_range=0.15, height_shift_range=0.15,
                             shear_range=3.14/6.0,
                             zoom_range=0.25,
                             channel_shift_range=0.1,
                             horizontal_flip=True,
                             vertical_flip=True)
    
train_gen.fit(xy_provider(train_image_ids, infinite=False),
            len(train_image_ids),
            augment=True,
            save_to_dir=GENERATED_DATA,
            batch_size=4,
            verbose=1)

val_gen = ImageDataGenerator(featurewise_center=True,
                                 featurewise_std_normalization=True) # Just an infinite image/mask generator

val_gen.mean = train_gen.mean
val_gen.std = train_gen.std
val_gen.principal_components = train_gen.principal_components

history = model.fit_generator(
        train_gen.flow(xy_provider(train_image_ids), # Infinite generator is used
                       len(train_id_type_list),
                       batch_size=batch_size),
        samples_per_epoch=samples_per_epoch,
        nb_epoch=nb_epochs,
        validation_data=val_gen.flow(xy_provider(val_image_ids), # Infinite generator is used
                       len(val_image_ids),
                       batch_size=batch_size),
        nb_val_samples=nb_val_samples)

Advantages:

1. User decides how to get images with xy_provider and benefits of random transformations provided by Keras.
2. User can manage the pipeline of random transformations and add some custom functions (of certain signature) to randomly augment data.
3. ImageMaskGenerator similarly transforms image and mask (useful for a segmentation task).

Drawbacks:

1. Need to write some code before it works
2. Some parts of implementation code are just copied and adapted from keras.preprocessing.image

See also this example

[ahundt]

I've got two proposals that I think are related to this in upstream keras, comments would be appreciated:

1. generate_samples_from_disk API to replace ImageDataGenerator.flow_from_directory that should still be clear but now work for a wider cross-section of applications.
2. Functional Preprocessing and Augmentation API

While the discussion there is about segmentation labels, it should also apply equally well for masks.