Skip to content
This repository has been archived by the owner on Oct 31, 2023. It is now read-only.

Dataset wrapper #1091

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Dataset wrapper #1091

wants to merge 2 commits into from

Conversation

botcs
Copy link
Contributor

@botcs botcs commented Sep 15, 2019

Wrapper

In cases when multiple datasets are combined to train the network, we may end up doing a lot of extra work finding out which index refers to which class (e.g. training on CityScapes with keeping the ROI heads from a COCO pre-training as in #259).

This auxiliary dataset helps to find common class names between
    `mimicked_dataset` and `wrapped_dataset` and wrap the latter to return
    ground truth that aligns with the indices of the former.

        A: mimicked_dataset
        B: wrapped_dataset

When multiple dataset are provided in a config file, the dataset builder checks if the datasets belong to different classes, and if possible it will match the 2nd, 3rd ... nth dataset's categories to the 1st dataset's in such a way that the corresponding class indices will be identical to the 1st dataset.
Instances belonging to classes that were left unmatched by this process are discarded on the fly.

The dataset comes with a pretty printing __str__ that helps debugging.

Example output

[WrapperDataset mimicks:Sim10kDataset wraps:CityScapesDataset]
 Mimicked index : Mimicked label       Wrapped index : Wrapped label  
              0 : __background__  ->               0 : __background__ 
              1 : aeroplane       ->            None : __unmatched__(aeroplane)
              2 : bicycle         ->              10 : bicycle        
              3 : bird            ->            None : __unmatched__(bird)
              4 : boat            ->            None : __unmatched__(boat)
              5 : bottle          ->            None : __unmatched__(bottle)
              6 : bus             ->               5 : bus            
              7 : car             ->               3 : car            
              8 : cat             ->            None : __unmatched__(cat)
              9 : chair           ->            None : __unmatched__(chair)
             10 : cow             ->            None : __unmatched__(cow)
             11 : diningtable     ->            None : __unmatched__(diningtable)
             12 : dog             ->            None : __unmatched__(dog)
             13 : horse           ->            None : __unmatched__(horse)
             14 : motorbike       ->            None : __unmatched__(motorbike)
             15 : person          ->               1 : person         
             16 : pottedplant     ->            None : __unmatched__(pottedplant)
             17 : sheep           ->            None : __unmatched__(sheep)
             18 : sofa            ->            None : __unmatched__(sofa)
             19 : train           ->               8 : train          
             20 : tvmonitor       ->            None : __unmatched__(tvmonitor)

@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Sep 15, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
CLA Signed Do not delete this pull request or issue due to inactivity.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants