Seg2Map 🔎 🌌

An interactive web map app for applying Doodleverse/Zoo models to geospatial imagery

Overview:

• Seg2Map facilitates application of Deep Learning-based image segmentation models and apply them to high-resolution (~1m or less spatial footprint) geospatial imagery, in order to make high-resolution label maps. Please see our wiki for more information.

• The principle aim is to generate time-series of label maps from a time-series of imagery, in order to detect and assess land use/cover change. This project also demonstrates how to apply generic models for land-use/cover on publicly available high-resolution imagery at arbitrary locations.

• Imagery comes from Google Earth Engine via s2m_engine. Initially, we focus on NAIP time-series, available for the conterminious United States since 2003. In the future, Planetscope imagery may also be made available (for those with access, such as federal researchers).

• We offer a set of Segmentation Zoo models, especially created and curated for this project based on a publicly available datasets. These datasets have been selected because they are public, large (several hundred to several thousand labeled images), and provide broad class labels for generic land use/cover mapping needs.

Installation Instructions

In order to use seg2map you need to install Python packages in an environment. We recommend you use Anaconda to install the python packages in an environment for seg2map. After you install Anaconda on your PC, open the Anaconda prompt or Terminal in Mac and Linux and use the cd command (change directory) to go the folder where you have downloaded the seg2map repository.

1. Create an Anaconda environment

• This command creates an anaconda environment named seg2map and installs python 3.10 in it
• You can also use python 3.10
• We will install the seg2map package and its dependencies in this environment.

  conda create --name seg2map python=3.10 -y
  

2. Activate your conda environment

   conda activate seg2map

• If you have successfully activated seg2map you should see that your terminal's command line prompt should now start with (seg2map).

3. Install Conda Dependencies

• seg2map requires jupyterlab and geopandas to function properly so they will be installed in the seg2map environment.
• Geopandas has GDAL as a dependency so its best to install it with conda.
• Make sure to install geopandas from the conda-forge channel to ensure you get the latest version.
• Make sure to install both jupyterlab and geopandas from the conda forge channel to avoid dependency conflicts

  conda install -c conda-forge geopandas jupyterlab -y

4. Install the seg2map from PyPi

   pip install seg2map

5. Uninstall the h5py installed by pip and reinstall with conda-forge

   pip uninstall h5py -y
   conda install -c conda-forge h5py -y

Having Installation Errors?

Use the command conda clean --all to clean old packages from your anaconda base environment. Ensure you are not in your seg2map environment or any other environment by running conda deactivate, to deactivate any environment you're in before running conda clean --all. It is recommended that you have Anaconda prompt (terminal for Mac and Linux) open as an administrator before you attempt to install seg2map again.

Conda Clean Steps

conda deactivate
conda clean --all

How to Use Seg2Map

1. Sign up to use Google Earth Engine Python API

First, you need to request access to Google Earth Engine at https://signup.earthengine.google.com/. It takes about 1 day for Google to approve requests.

2. Activate your conda environment

   conda activate seg2map

• If you have successfully activated seg2map you should see that your terminal's command line prompt should now start with (seg2map).

3. Install the seg2map from PyPi

   cd <location you downloaded seg2map>
   ex: cd C:\1_repos\seg2map

4. Launch Jupyter Lab

• make you run this command in the seg2map directory so you can choose a notebook to use.

  jupyter lab

Features

1. Download Imagery from Google Earth Engine

Use google earth engine to download multiple years worth of imagery.

You can download multiple ROIs and years of data at lighting speeds 🌩️

2. Apply Models to Imagery

3. Load Segmented Imagery onto the Map

Generic workflow:

• Provide a web map for navigation to a location, and draw a bounding box
• Provide an interface for controls (set time period, etc)
• Download geospatial imagery (for now, just NAIP)
• Provide tools to select and apply a Zoo model to create a label image
• Provide tools to interact with those label images (download, mosaic, merge classes, etc)

Authors

• @dbuscombe-usgs
• @2320sharon

Contributions:

• @venuswku

We welcome collaboration! Please use our Discussions tab if you're interested in this project. We welcome user-contributed models! They must be trained using Segmentation Gym, and then served and documented through Segmentation Zoo - get in touch and we'll walk you through the process!

Roadmap / progress

V1

• Develop codes to create a web map for navigation to a location, and draw a bounding box
• Develop codes interface for controls (time period, etc)
• Develop codes for downloading NAIP imagery using GEE
• Put together a prototype jupyter notebook for web map, bounding box, and image downloads
• Create Seg2Map models
  • Coast Train / aerial / high-res. sat
    • 2 class dataset (water, other)
    • zenodo release for 768x768 imagery zenodo page
  • Coast Train / NAIP
    • 5 class dataset (water, whitewater, sediment, bare terrain, other terrain)
    • 8 class dataset (water, whitewater, sediment, bare terrain, marsh veg, terrestrial veg, ag., dev.)
    • zenodo release of 5-class ResUNet models for 768x768 imagery zenodo page
    • zenodo release of 8-class ResUNet models for 768x768 imagery zenodo page
    • zenodo release of 5-class Segformer models for 768x768 imagery zenodo page
    • zenodo release of 8-class Segformer models for 768x768 imagery zenodo page
  • Chesapeake Landcover (CCLC) / NAIP
    • 7 class dataset (water, tree canopy / forest, low vegetation / field, barren land, impervious (other), impervious (road), no data)
    • zenodo release of 7-class ResUNet models for 512x512 imagery page
    • zenodo release of 7-class SegFormer models for 512x512 imagery page
  • EnviroAtlas / NAIP
    • 6 class dataset (water, impervious, barren, trees, herbaceous, shrubland)
    • zenodo release of 6-class ResUNet models for 1024 x 1024 models zenodo page
  • OpenEarthMap / aerial / high-res. sat
    • 9 class dataset (bareland, rangeland, dev., road, tree, water, ag., building, nodata)
    • zenodo release of 9-class ResUNet models for 512x512 models zenodo page
  • DeepGlobe / aerial / high-res. sat
    • 7 class dataset (urban, ag., rangeland, forest, water, bare, unknown)
    • zenodo release for of 7-class ResUNet models 512x512 imagery zenodo page
  • Barrier Islands / orthomosaic / coastlines
    • Substrate data 6 class (dev, sand, mixed, coarse, unknown, water)
    • zenodo release of substrate models for 768x768 imagery
    • Vegetation type data 7 class (shrub/forest, shrub, none/herb., none, herb., herb./shrub, dev)
    • zenodo release of Vegetation type models for 768x768 imagery
    • Vegetation density data 7 class (dense, dev., moderate, moderate/dense, none, none/sparse, sparse)
    • zenodo release of Vegetation density models for 768x768 imagery
    • Geomorphic setting data 7 class (beach, backshore, dune, washover, barrier interior, marsh, ridge/swale)
    • zenodo release of Geomorphic setting models for 768x768 imagery
    • Supervised classification data 9 class (water, sand, herbaceous veg./low shrub, sparse/moderate, herbaceous veg/low shrub, moderate/dense, high shrub/forest, marsh/sediment, marsh/veg, marsh, high shrub/forest, development)
    • zenodo release of Supervised classification models for 768x768 imagery
  • AAAI / aerial / high-res. sat
    • 2 class dataset (other, building)
    • zenodo release for 1024x1024 imagery zenodo page
    • 2 class dataset (other, flooded building)
    • zenodo release for 1024x1024 imagery zenodo page
  • xBD-hurricanes / aerial / high-res. sat, a subset of the XView2 dataset
    • 4 class building dataset (other, no damage, minor damage, major damage)
    • zenodo release for 768x768 imagery zenodo page
    • 2 class building dataset (other, building)
    • zenodo release for 768x768 imagery zenodo page
  • Superclass models
    • 8 merged datasets for 8 separate superclass models (water, sediment, veg, herb. veg., woody veg., impervious, building, agriculture)
    • zenodo release for 768x768 imagery / water
    • zenodo release for 768x768 imagery / sediment
    • zenodo release for 768x768 imagery / veg
    • zenodo release for 768x768 imagery / herb. veg.
    • zenodo release for 768x768 imagery / woody veg.
    • zenodo release for 768x768 imagery / impervious
    • zenodo release for 768x768 imagery / building
    • zenodo release for 768x768 imagery / agriculture
• Develop codes/docs for selecting model
• Develop codes/docs for applying model to make label imagery
• Tool for mosaicing labels
• Tool for downloading labels in geotiff format

V2

• Tool for post-processing/editing labels
• Tool for detecting change
• Make Planetscope 3m imagery available via Planet API (federal researchers only)
• Include additional models/datasets (TBD)

Datasets

General Landcover

DeepGlobe

• paper
• challenge
• data
• Zenodo model release (512x512): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for DeepGlobe/7-class segmentation of RGB 512x512 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7576898

EnviroAtlas

• EnviroAtlas dataset
• EnviroAtlas paper
• paper using EnviroAtlasdata
• This dataset was organized to accompany the 2022 paper, Resolving label uncertainty with implicit generative models. More details can be found here
• Zenodo model release (512x512): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for EnviroAtlas/6-class segmentation of RGB 512x512 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7576909

OpenEarthMap

• website
• data
• paper
• Zenodo model release (512x512): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for OpenEarthMap/9-class segmentation of RGB 512x512 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7576894

Coastal Landcover

Chesapeake Landcover

• webpage
• Zenodo model release (512x512): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for Chesapeake/7-class segmentation of RGB 512x512 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7576904
• Zenodo SegFormer model release (512x512): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Segformer models for Chesapeake/7-class segmentation of RGB 512x512 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7677506

Coast Train

• paper
• website
• data
• preprint
• Zenodo model release, 2-class (768x768): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for CoastTrain water/other segmentation of RGB 768x768 orthomosaic images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7574784
• Zenodo model release, 5-class (768x768): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for CoastTrain/5-class segmentation of RGB 768x768 NAIP images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7566992
• Zenodo model release, 8-class (768x768): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for CoastTrain/8-class segmentation of RGB 768x768 NAIP images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7570583
• Zenodo SegFormer model release, 5-class (768x768): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map SegFormer models for CoastTrain/5-class segmentation of RGB 768x768 NAIP images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7641708
• Zenodo SegFormer model release, 8-class (768x768): Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map SegFormer models for CoastTrain/8-class segmentation of RGB 768x768 NAIP images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7641724

AAAI / Buildings / Flooded Buildings

• data
• data
• paper
• Zenodo model release (1024x1024) building / no building: Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for segmentation of buildings of RGB 1024x1024 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7607895
• Zenodo model release (1024x1024) flooded building / no flooded building: Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for segmentation of AAAI/flooded buildings in RGB 1024x1024 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7622733

XBD-hurricanes

• Xview2 challenge
• XBD-hurricanes code
• Zenodo SegFormer model release (768x768) building damage: Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map SegFormer models for segmentation of xBD/damaged buildings in RGB 768x768 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7613175
• Zenodo SegFormer model release (768x768) building presence/absence: Buscombe, Daniel. (2023). Doodleverse/Segmentation Zoo/Seg2Map SegFormer models for segmentation of xBD/buildings in RGB 768x768 high-res. images (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.7613212

Barrier Islands

• webpage
• paper
• Zenodo Substrate model release (768x768):
• Zenodo Vegetation type model release (768x768):
• Zenodo Vegetation density model release (768x768):
• Zenodo Geomorphic model release (768x768):
• Zenodo Supervised classification model release (768x768):

Superclasses

A. Water:

• Coast Train
• Chesapeake
• EnviroAtlas
• OpenEarthMap
• DeepGlobe
• Barrier Substrate
• NOAA
• [v2: Barrier Substrate]
• [v2: Elwha]

B. Sediment:

• Coast Train
• NOAA
• [v2: Barrier Substrate (sand, mixed, coarse)]
• [v2: Elwha]

C. Bare:

• Chesapeake (barren land)
• EnviroAtlas (barren)
• OpenEarthMap (bareland)

D. Vegetated:

• Coast Train (marsh veg, terrestrial veg, ag)
• FloodNet (tree, grass)
• Chesapeake (tree canopy / forest, low vegetation / field)
• EnviroAtlas (trees, herbaceous, shrubland)
• OpenEarthMap (rangeland, tree, ag)
• DeepGlobe (ag., rangeland, forest)
• NOAA (veg)
• [v2: Elwha]
• [v2: Barrier Substrate]

E. Impervious:

• FloodNet (Building-flooded, Building-non-flooded, Road-flooded, Road-non-flooded, vehicle)
• Chesapeake (impervious (other), impervious (road))
• EnviroAtlas (impervious)
• OpenEarthMap (dev, road, building)
• DeepGlobe (urban)
• NOAA (dev)
• [v2: Elwha]

F. Building:

• OpenEarthMap (building)
• AAAI (building)

G. Agriculture:

• OpenEarthMap (ag)
• DeepGlobe (ag)

H. Woody Veg:

• FloodNet (tree)
• Chesapeake (tree canopy / forest)
• EnviroAtlas (trees)
• OpenEarthMap (tree)
• DeepGlobe (forest)
• [v2: Elwha]
• [v2: Barrier Substrate]

References

Notes

• NLCD classes
• NAIP imagery

Classes:

	Coast Train 1	Coast Train 2	Coast Train 3	FloodNet	Chesapeake	EnviroAtlas	OpenEarthMap	DeepGlobe	AAAI	NOAA	Barrier Substrate
A. Water	X	X	X	X	X	X	X	X		X	X
a. whitewater		X	X
a. pool				X
---	---	---	---	---	---	---	---	---	---	---	---
B. Sediment		X	X							X
b. sand											X
b. mixed											X
b. coarse											X
---	---	---	---	---	---	---	---	---	---	---	---
C. Bare/barren		X	X		X	X	X	X
---	---	---	---	---	---	---	---	---	---	---
d. marsh			X
d. terrestrial veg			X							X
d. agriculture			X				X	X
d. grass				X
d. herbaceous / low vegetation / field					X	X
d. tree/forest				X	X	X	X	X
d. shrubland						X
d. rangeland						X	X	X
---	---	---	---	---	---	---	---	---	---	---	---
E. Impervious/urban/developed			X			X	X	X		X	X
e. impervious (other)					X
e. impervious (road)					X		X
e. Building-flooded				X
e. Building-non-flooded				X			X		X
e. Road-flooded				X
e. Road-non-flooded				X
e. Vehicle				X
---	---	---	---	---	---	---	---	---	---	---	---
X. Other	X	X							X