ch6

Chapter 6: Enhancing and Segmenting Images

In Chapter 6 of the book, several paradigms for pixel-precise applications were covered. We introduced encoders-decoders and some specific architectures like U-Net and FCN. We presented how to apply them to multiple tasks from image denoising to semantic segmentation. We also demonstrated how different solutions can be combined to tackle more advanced problems like instance segmentation.

In this folder, readers can find several notebooks tackling these various notions. Addtionally, some of the key code snippets are compiled into reusable Python files.

Notebooks

• 6.1 - Introduction to Auto-Encoders
  • Implement a simple fully-connected auto-encoder, and apply it to the MNIST dataset. Use the encoder to embed and visualize the full dataset, applying t-SNE.
• 6.2 - Denoising with Auto-Encoders
  • Reuse the previous auto-encoder to recover corrupted MNIST images.
• 6.3 - Super-Resolution with Deep Auto-Encoders
  • Implement a basic convolutional deep auto-encoder, followed by a U-Net model. Apply both to improving the quality of downscaled images.
• 6.4 - Data Preparation for Smart Car Applications
  • Discover the Cityscapes dataset of urban scenes, and set up an efficient input pipeline to serve batches for semantic segmentation applications.
• 6.5 - Semantic Segmentation for Smart Cars with FCN-8s
  • Implement a FCN-8s model from scratch, and apply it to segmenting Cityscapes images for autonomous driving applications. Improve the results using per-class weighing of the loss.
• 6.6 - Object and Instance Segmentation for Smart Cars with U-Net
  • Apply our U-Net model to segmenting Cityscapes, training it with the Dice loss. Improve its results with CRF post-processing and use a pre-train model to achieve instance segmentaton.

Additional Files

• cityscapes_utils.py: functions to pre-process, serve, visualize the Cityscapes dataset (code presented in notebook 6.4).
• fcn.py: Keras implementation of the FCN-8s model (code presented in notebook 6.5).
• keras_custom_callbacks.py: Custom callbacks to monitor the training of Keras models (code presented in notebook 6.2).
• plot_utils.py: Plot helper functions (code presented in notebooks 6.1- 6.2).
• tf_losses_and_metrics.py: Custom TensorFlow/Keras losses and metrics for encoders-decoders (code presented in notebooks 6.1-6.2-6.5-6-6).
• tf_math.py: Advanced math/morphological functions in TensorFlow (code presented in notebooks 6.2-6.5).
• unet.py: Keras implementation of the U-Net model (code presented in notebook 6.3).