Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Benchmark] unable to test the given dataset ILSVRC2012_img_val or owndataset to test the image_benchmark.py with. #1863

Open
karishmathumu opened this issue Aug 5, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@karishmathumu
Copy link

Ran the following command on CLI (Windows i7 PC)
albumentations/benkcmark> python image_benchmark.py --data-dir /path/to/image/directory --images 2000 --runs 5 --print-package-versions

Issue:
Executed the command on the terminal.
Observed the blinking underscore cursor with no output for over 10 minutes.

looked like stuck in infinite loop or a hang in the process.
Or could be incorrect or incomplete command execution.

Even Ctrl + C or Ctrl+Z does not seem to work .
Had to close the terminal and redo it again.

2024-08-05 14_52_26-Window

@karishmathumu karishmathumu added the bug Something isn't working label Aug 5, 2024
@karishmathumu karishmathumu changed the title [Benchmark] unable to test with the given dataset or owndataset to test the image_benchmark.py with. [Benchmark] unable to test the given dataset ILSVRC2012_img_val or owndataset to test the image_benchmark.py with. Aug 5, 2024
@ternaus
Copy link
Collaborator

ternaus commented Aug 6, 2024

I did not check the benchmark in Windows as I do not have it and I do not have issues in Linux

Could some specifics of that os.

  1. What happens if you use --images 10 --runs 1 ?
  2. Did you download images to the folder that you are pointing to?
  3. Can you check and put images outside of the albumentations folder?
 python -m benchmark.image_benchmark --data-dir ~/workspace/evo970/data/Imagenet1k/ILSVRC2015/Data/CLS-LOC/val --images 10 --runs 1 --markdown --show-std --print-package-versions -o df.csv
| Library | Version |
|---------|---------|
| Python | 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] |
| albumentations | 1.4.8 |
| imgaug | 0.4.0 |
| torchvision | 0.18.1+rocm6.0 |
| numpy | 1.26.4 |
| opencv-python-headless | 4.10.0.84 |
| scikit-image | 0.24.0 |
| scipy | 1.14.0 |
| pillow | 10.4.0 |
| kornia | 0.7.3 |
| augly | 1.0.0 |

Loading images for OpenCV: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 261.17it/s]
Loading images for Pillow: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 458.83it/s]
Loading images for Torch: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 534.07it/s]
Loading images for Kornia: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 341.54it/s]
Current benchmark: ColorJitter:  50%|████████████████████████████████████████████████████████████████████████████████████████████████▌                                                                                                | 11/22 [00:01<00:01,  6.80it/s]
/home/vladimir/anaconda3/envs/albumentations_benchmark/lib/python3.10/site-packages/pydantic/main.py:363: UserWarning: Pydantic serializer warnings:
  Expected `Union[float, tuple[float, float]]` but got `list` - serialized value may not be as expected
  Expected `Union[float, tuple[float, float]]` but got `list` - serialized value may not be as expected
  Expected `Union[float, tuple[float, float]]` but got `list` - serialized value may not be as expected
  Expected `Union[float, tuple[float, float]]` but got `list` - serialized value may not be as expected
  return self.__pydantic_serializer__.to_python(
Current benchmark: JpegCompression:  82%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋                                  | 18/22 [00:06<00:03,  1.25it/s]
/home/vladimir/anaconda3/envs/albumentations_benchmark/lib/python3.10/site-packages/torch/functional.py:512: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3587.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Current benchmark: Normalize: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22/22 [00:16<00:00,  1.35it/s]
|                 |albumentations<br><small>1.4.8</small>|torchvision<br><small>0.18.1+rocm6.0</small>|kornia<br><small>0.7.3</small>|augly<br><small>1.0.0</small>|imgaug<br><small>0.4.0</small>|
|-----------------|--------------------------------------|--------------------------------------------|------------------------------|-----------------------------|------------------------------|
|HorizontalFlip   |**17531 ± 0**                         |3394 ± 0                                    |1403 ± 0                      |8241 ± 0                     |9789 ± 0                      |
|VerticalFlip     |**19194 ± 0**                         |3586 ± 0                                    |1408 ± 0                      |17619 ± 0                    |14321 ± 0                     |
|Rotate           |876 ± 0                               |250 ± 0                                     |272 ± 0                       |**1367 ± 0**                 |625 ± 0                       |
|Affine           |**1635 ± 0**                          |250 ± 0                                     |266 ± 0                       |-                            |849 ± 0                       |
|Equalize         |1331 ± 0                              |505 ± 0                                     |103 ± 0                       |-                            |**1709 ± 0**                  |
|RandomCrop64     |16306 ± 0                             |15918 ± 0                                   |842 ± 0                       |**48968 ± 0**                |7620 ± 0                      |
|RandomResizedCrop|**2447 ± 0**                          |1277 ± 0                                    |194 ± 0                       |-                            |-                             |
|ShiftRGB         |1726 ± 0                              |-                                           |542 ± 0                       |-                            |**2098 ± 0**                  |
|Resize           |**2446 ± 0**                          |1438 ± 0                                    |204 ± 0                       |474 ± 0                      |1992 ± 0                      |
|RandomGamma      |**3006 ± 0**                          |294 ± 0                                     |278 ± 0                       |-                            |2434 ± 0                      |
|Grayscale        |**10285 ± 0**                         |2590 ± 0                                    |700 ± 0                       |3792 ± 0                     |1580 ± 0                      |
|ColorJitter      |**496 ± 0**                           |77 ± 0                                      |68 ± 0                        |286 ± 0                      |-                             |
|PlankianJitter   |818 ± 0                               |-                                           |**909 ± 0**                   |-                            |-                             |
|RandomPerspective|598 ± 0                               |178 ± 0                                     |153 ± 0                       |-                            |**627 ± 0**                   |
|GaussianBlur     |**2532 ± 0**                          |248 ± 0                                     |170 ± 0                       |209 ± 0                      |1572 ± 0                      |
|MedianBlur       |622 ± 0                               |-                                           |2 ± 0                         |-                            |**684 ± 0**                   |
|MotionBlur       |**2994 ± 0**                          |-                                           |186 ± 0                       |-                            |603 ± 0                       |
|Posterize        |2931 ± 0                              |**3591 ± 0**                                |509 ± 0                       |-                            |2179 ± 0                      |
|JpegCompression  |**1004 ± 0**                          |-                                           |68 ± 0                        |879 ± 0                      |571 ± 0                       |
|GaussianNoise    |94 ± 0                                |-                                           |-                             |82 ± 0                       |**175 ± 0**                   |
|Elastic          |**220 ± 0**                           |4 ± 0                                       |1 ± 0                         |-                            |170 ± 0                       |
|Normalize        |**1484 ± 0**                          |603 ± 0                                     |537 ± 0                       |-                            |-                             |

@karishmathumu
Copy link
Author

Thank you for your response.
I understand, but I have only windows system.
Tried as you suggested.
Seems like the problem yesterday still persists, like the infinite time it takes whilst nothing happens as the command is executed.

Here are the screenshots.

2024-08-06 08_17_37-Window

then changed the location of the extracted dataset's images location

2024-08-06 08_26_38-Window

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants