You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
All GeoSamplers should take a GeoDataset index as input
Randomly choose a file, then randomly sample from within bounds of that file (solves sampling out of bounds problem)
Add new sampler (RandomBatchGeoSampler) that subclasses BatchSampler and returns a batch of random patches from a single tile
When using ZipDataset with random samplers, the index should come from whichever dataset is tile-based. When using ZipDataset with grid samplers, the index should come from whichever dataset is not tile-based. Not yet sure how to handle something like Landsat + Sentinel, but we can figure that out another day.
Class hierarchy:
Sampler
GeoSampler
RandomGeoSampler
GridGeoSampler
BatchGeoSampler
RandomBatchGeoSampler
Make sure to document the difference between samplers and batch samplers and when to use which. Should store samplers and batch samplers in different files and combine in __init__ like we do with datasets. Add utils.py for things like _to_tuple.
Question: if I'm using an LRU cache and BatchSampler and multiple workers, if something isn't yet in the cache, will PyTorch spawn multiple workers all trying to warp the entire tile? It may actually be faster to use a single worker in this case.
The text was updated successfully, but these errors were encountered:
Here are some ideas:
When using ZipDataset with random samplers, the index should come from whichever dataset is tile-based. When using ZipDataset with grid samplers, the index should come from whichever dataset is not tile-based. Not yet sure how to handle something like Landsat + Sentinel, but we can figure that out another day.
Class hierarchy:
Make sure to document the difference between samplers and batch samplers and when to use which. Should store samplers and batch samplers in different files and combine in
__init__
like we do with datasets. Addutils.py
for things like_to_tuple
.Question: if I'm using an LRU cache and BatchSampler and multiple workers, if something isn't yet in the cache, will PyTorch spawn multiple workers all trying to warp the entire tile? It may actually be faster to use a single worker in this case.
The text was updated successfully, but these errors were encountered: