Skip to content
This repository has been archived by the owner on Aug 25, 2024. It is now read-only.

tests: mnist tests failing due to images missing? #1436

Open
johnandersen777 opened this issue Feb 9, 2023 · 3 comments
Open

tests: mnist tests failing due to images missing? #1436

johnandersen777 opened this issue Feb 9, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@johnandersen777
Copy link

johnandersen777 commented Feb 9, 2023 •
edited
Loading

Related: #1264

2023-02-08T06:37:23.3956144Z test_shell (MNIST.test_mnist.TestMNIST) ... Traceback (most recent call last):
2023-02-08T06:37:23.3965998Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/bin/dffml", line 8, in <module>
2023-02-08T06:37:23.3969940Z     sys.exit(CLI.main())
2023-02-08T06:37:23.3970408Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 272, in main
2023-02-08T06:37:23.3970844Z     result = loop.run_until_complete(cls._main(*argv[1:]))
2023-02-08T06:37:23.3972745Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
2023-02-08T06:37:23.3973199Z     return future.result()
2023-02-08T06:37:23.3973631Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 238, in _main
2023-02-08T06:37:23.3974006Z     return await cls.cli(*args)
2023-02-08T06:37:23.3974426Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 224, in cli
2023-02-08T06:37:23.3974801Z     return await cmd.do_run()
2023-02-08T06:37:23.3975464Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 201, in do_run
2023-02-08T06:37:23.3975867Z     return [res async for res in self.run()]
2023-02-08T06:37:23.3976314Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 201, in <listcomp>
2023-02-08T06:37:23.3976724Z     return [res async for res in self.run()]
2023-02-08T06:37:23.3977144Z   File "/home/runner/work/dffml/dffml/dffml/cli/ml.py", line 91, in run
2023-02-08T06:37:23.3977599Z     self.model, self.sources, update=self.update, keep_record=True
2023-02-08T06:37:23.3978082Z   File "/home/runner/work/dffml/dffml/dffml/high_level/ml.py", line 280, in predict
2023-02-08T06:37:23.3978530Z     sctx = await astack.enter_async_context(records_to_sources(*args))
2023-02-08T06:37:23.3979049Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/contextlib.py", line 570, in enter_async_context
2023-02-08T06:37:23.3979480Z     result = await _cm_type.__aenter__(cm)
2023-02-08T06:37:23.3979956Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/contextlib.py", line 170, in __aenter__
2023-02-08T06:37:23.3980374Z     return await self.gen.__anext__()
2023-02-08T06:37:23.3980819Z   File "/home/runner/work/dffml/dffml/dffml/util/internal.py", line 59, in records_to_sources
2023-02-08T06:37:23.3981211Z     async with sources as sources:
2023-02-08T06:37:23.3981681Z   File "/home/runner/work/dffml/dffml/dffml/util/asynchelper.py", line 75, in __aenter__
2023-02-08T06:37:23.3982113Z     await self.__stack.enter_async_context(item)
2023-02-08T06:37:23.3982595Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/contextlib.py", line 570, in enter_async_context
2023-02-08T06:37:23.3983023Z     result = await _cm_type.__aenter__(cm)
2023-02-08T06:37:23.3983464Z   File "/home/runner/work/dffml/dffml/dffml/util/asynchelper.py", line 75, in __aenter__
2023-02-08T06:37:23.3984172Z     await self.__stack.enter_async_context(item)
2023-02-08T06:37:23.3984809Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/contextlib.py", line 570, in enter_async_context
2023-02-08T06:37:23.3985227Z     result = await _cm_type.__aenter__(cm)
2023-02-08T06:37:23.3985671Z   File "/home/runner/work/dffml/dffml/dffml/source/dfpreprocess.py", line 246, in __aenter__
2023-02-08T06:37:23.3986105Z     self.source = await self.config.source.__aenter__()
2023-02-08T06:37:23.3986578Z   File "/home/runner/work/dffml/dffml/dffml/source/file.py", line 47, in __aenter__
2023-02-08T06:37:23.3986950Z     await self._open()
2023-02-08T06:37:23.3987354Z   File "/home/runner/work/dffml/dffml/dffml/source/file.py", line 87, in _open
2023-02-08T06:37:23.3987729Z     await self.load_fd(fd)
2023-02-08T06:37:23.3988136Z   File "/home/runner/work/dffml/dffml/dffml/source/csv.py", line 195, in load_fd
2023-02-08T06:37:23.3988538Z     async with self._open_csv(fd) as open_file:
2023-02-08T06:37:23.3989018Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/contextlib.py", line 170, in __aenter__
2023-02-08T06:37:23.3989430Z     return await self.gen.__anext__()
2023-02-08T06:37:23.3989858Z   File "/home/runner/work/dffml/dffml/dffml/source/csv.py", line 86, in _open_csv
2023-02-08T06:37:23.3990248Z     await self.read_csv(fd, open_file)
2023-02-08T06:37:23.3990675Z   File "/home/runner/work/dffml/dffml/dffml/source/csv.py", line 121, in read_csv
2023-02-08T06:37:23.3991042Z     row[loadfile]
2023-02-08T06:37:23.3991474Z   File "/home/runner/work/dffml/dffml/dffml/configloader/configloader.py", line 135, in load_file
2023-02-08T06:37:23.3991935Z     self.parsers, self.async_exit_stack, filepath, base_dir=base_dir
2023-02-08T06:37:23.3992423Z   File "/home/runner/work/dffml/dffml/dffml/configloader/configloader.py", line 109, in load_file
2023-02-08T06:37:23.3992855Z     config_path, conf_dict = await _get_config(path)
2023-02-08T06:37:23.3993338Z   File "/home/runner/work/dffml/dffml/dffml/configloader/configloader.py", line 91, in _get_config
2023-02-08T06:37:23.3993788Z     parsers, exit_stack, temp_filepath, base_dir=base_dir
2023-02-08T06:37:23.3994273Z   File "/home/runner/work/dffml/dffml/dffml/configloader/configloader.py", line 76, in load_single_file
2023-02-08T06:37:23.3994753Z     return config_path, await parsers[filetype].loadb(path.read_bytes())
2023-02-08T06:37:23.3995308Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/pathlib.py", line 1214, in read_bytes
2023-02-08T06:37:23.3996101Z     with self.open(mode='rb') as f:
2023-02-08T06:37:23.3996533Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/pathlib.py", line 1208, in open
2023-02-08T06:37:23.3996900Z     opener=self._opener)
2023-02-08T06:37:23.3997404Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/pathlib.py", line 1063, in _opener
2023-02-08T06:37:23.3998015Z     return self._accessor.open(self, flags, mode)
2023-02-08T06:37:23.4005860Z FileNotFoundError: [Errno 2] No such file or directory: 'image1.png'


2023-02-08T06:46:12.3097962Z test_shell (test_quickstart.TestQuickstart) ... ok
2023-02-08T06:46:12.3098487Z 
2023-02-08T06:46:12.3098743Z ======================================================================
2023-02-08T06:46:12.3099174Z ERROR: test_shell (MNIST.test_mnist.TestMNIST)
2023-02-08T06:46:12.3099859Z ----------------------------------------------------------------------
2023-02-08T06:46:12.3100313Z Traceback (most recent call last):
2023-02-08T06:46:12.3100892Z   File "/home/runner/work/dffml/dffml/examples/MNIST/test_mnist.py", line 77, in test_shell
2023-02-08T06:46:12.3101389Z     ["bash", sh_filepath("predict.sh")]
2023-02-08T06:46:12.3101974Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/subprocess.py", line 411, in check_output
2023-02-08T06:46:12.3102445Z     **kwargs).stdout
2023-02-08T06:46:12.3102925Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/subprocess.py", line 512, in run
2023-02-08T06:46:12.3103378Z     output=stdout, stderr=stderr)
2023-02-08T06:46:12.3104073Z subprocess.CalledProcessError: Command '['bash', '/home/runner/work/dffml/dffml/examples/MNIST/predict.sh']' returned non-zero exit status 1.
2023-02-08T06:46:12.3104481Z 
2023-02-08T06:46:12.3104705Z ======================================================================
2023-02-08T06:46:12.3105154Z ERROR: test_shell_pytorch (flower17.test_flower17.TestFLOWER17)
2023-02-08T06:46:12.3105723Z ----------------------------------------------------------------------
2023-02-08T06:46:12.3106380Z Traceback (most recent call last):
2023-02-08T06:46:12.3106904Z   File "/home/runner/work/dffml/dffml/dffml/util/asynctestcase.py", line 114, in run_it
2023-02-08T06:46:12.3107424Z     result = self.loop.run_until_complete(coro(*args, **kwargs))
2023-02-08T06:46:12.3107999Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
2023-02-08T06:46:12.3108478Z     return future.result()
2023-02-08T06:46:12.3109005Z   File "/home/runner/work/dffml/dffml/examples/flower17/test_flower17.py", line 92, in test_shell_pytorch
2023-02-08T06:46:12.3109551Z     records = await CLI.cli(*cmd[1:-1])
2023-02-08T06:46:12.3110036Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 224, in cli
2023-02-08T06:46:12.3110465Z     return await cmd.do_run()
2023-02-08T06:46:12.3110929Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 201, in do_run
2023-02-08T06:46:12.3111399Z     return [res async for res in self.run()]
2023-02-08T06:46:12.3111907Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 201, in <listcomp>
2023-02-08T06:46:12.3112487Z     return [res async for res in self.run()]
2023-02-08T06:46:12.3112951Z   File "/home/runner/work/dffml/dffml/dffml/cli/ml.py", line 91, in run
2023-02-08T06:46:12.3113452Z     self.model, self.sources, update=self.update, keep_record=True
2023-02-08T06:46:12.3113976Z   File "/home/runner/work/dffml/dffml/dffml/high_level/ml.py", line 280, in predict
2023-02-08T06:46:12.3114494Z     sctx = await astack.enter_async_context(records_to_sources(*args))
2023-02-08T06:46:12.3115073Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/contextlib.py", line 570, in enter_async_context
2023-02-08T06:46:12.3115569Z     result = await _cm_type.__aenter__(cm)
2023-02-08T06:46:12.3116088Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/contextlib.py", line 170, in __aenter__
2023-02-08T06:46:12.3116567Z     return await self.gen.__anext__()
2023-02-08T06:46:12.3117073Z   File "/home/runner/work/dffml/dffml/dffml/util/internal.py", line 59, in records_to_sources
2023-02-08T06:46:12.3117540Z     async with sources as sources:
2023-02-08T06:46:12.3118036Z   File "/home/runner/work/dffml/dffml/dffml/util/asynchelper.py", line 75, in __aenter__
2023-02-08T06:46:12.3118524Z     await self.__stack.enter_async_context(item)
2023-02-08T06:46:12.3119056Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/contextlib.py", line 570, in enter_async_context
2023-02-08T06:46:12.3119540Z     result = await _cm_type.__aenter__(cm)
2023-02-08T06:46:12.3120035Z   File "/home/runner/work/dffml/dffml/dffml/util/asynchelper.py", line 75, in __aenter__
2023-02-08T06:46:12.3120507Z     await self.__stack.enter_async_context(item)
2023-02-08T06:46:12.3121018Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/contextlib.py", line 570, in enter_async_context
2023-02-08T06:46:12.3121491Z     result = await _cm_type.__aenter__(cm)
2023-02-08T06:46:12.3121972Z   File "/home/runner/work/dffml/dffml/dffml/source/file.py", line 47, in __aenter__
2023-02-08T06:46:12.3122399Z     await self._open()
2023-02-08T06:46:12.3122844Z   File "/home/runner/work/dffml/dffml/dffml/source/file.py", line 87, in _open
2023-02-08T06:46:12.3123271Z     await self.load_fd(fd)
2023-02-08T06:46:12.3123740Z   File "/home/runner/work/dffml/dffml/dffml/source/csv.py", line 195, in load_fd
2023-02-08T06:46:12.3124202Z     async with self._open_csv(fd) as open_file:
2023-02-08T06:46:12.3124715Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/contextlib.py", line 170, in __aenter__
2023-02-08T06:46:12.3125180Z     return await self.gen.__anext__()
2023-02-08T06:46:12.3125665Z   File "/home/runner/work/dffml/dffml/dffml/source/csv.py", line 86, in _open_csv
2023-02-08T06:46:12.3126112Z     await self.read_csv(fd, open_file)
2023-02-08T06:46:12.3126583Z   File "/home/runner/work/dffml/dffml/dffml/source/csv.py", line 121, in read_csv
2023-02-08T06:46:12.3127012Z     row[loadfile]
2023-02-08T06:46:12.3127586Z   File "/home/runner/work/dffml/dffml/dffml/configloader/configloader.py", line 135, in load_file
2023-02-08T06:46:12.3128121Z     self.parsers, self.async_exit_stack, filepath, base_dir=base_dir
2023-02-08T06:46:12.3128669Z   File "/home/runner/work/dffml/dffml/dffml/configloader/configloader.py", line 109, in load_file
2023-02-08T06:46:12.3129257Z     config_path, conf_dict = await _get_config(path)
2023-02-08T06:46:12.3129787Z   File "/home/runner/work/dffml/dffml/dffml/configloader/configloader.py", line 91, in _get_config
2023-02-08T06:46:12.3130306Z     parsers, exit_stack, temp_filepath, base_dir=base_dir
2023-02-08T06:46:12.3130855Z   File "/home/runner/work/dffml/dffml/dffml/configloader/configloader.py", line 76, in load_single_file
2023-02-08T06:46:12.3131401Z     return config_path, await parsers[filetype].loadb(path.read_bytes())
2023-02-08T06:46:12.3131972Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/pathlib.py", line 1214, in read_bytes
2023-02-08T06:46:12.3132518Z     with self.open(mode='rb') as f:
2023-02-08T06:46:12.3133097Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/pathlib.py", line 1208, in open
2023-02-08T06:46:12.3133543Z     opener=self._opener)
2023-02-08T06:46:12.3134045Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/pathlib.py", line 1063, in _opener
2023-02-08T06:46:12.3134520Z     return self._accessor.open(self, flags, mode)
2023-02-08T06:46:12.3135077Z FileNotFoundError: [Errno 2] No such file or directory: 'daisy.jpg'
2023-02-08T06:46:12.3135581Z 
2023-02-08T06:46:12.3135807Z ======================================================================
2023-02-08T06:46:12.3136257Z ERROR: test_shell_sklearn (flower17.test_flower17.TestFLOWER17)
2023-02-08T06:46:12.3136824Z ----------------------------------------------------------------------
2023-02-08T06:46:12.3137262Z Traceback (most recent call last):
2023-02-08T06:46:12.3137768Z   File "/home/runner/work/dffml/dffml/dffml/util/asynctestcase.py", line 114, in run_it
2023-02-08T06:46:12.3138284Z     result = self.loop.run_until_complete(coro(*args, **kwargs))
2023-02-08T06:46:12.3138858Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
2023-02-08T06:46:12.3139334Z     return future.result()
2023-02-08T06:46:12.3139838Z   File "/home/runner/work/dffml/dffml/examples/flower17/test_flower17.py", line 56, in test_shell_sklearn
2023-02-08T06:46:12.3140301Z     records = await CLI.cli(*cmd)
2023-02-08T06:46:12.3140769Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 224, in cli
2023-02-08T06:46:12.3141206Z     return await cmd.do_run()
2023-02-08T06:46:12.3141665Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 201, in do_run
2023-02-08T06:46:12.3142114Z     return [res async for res in self.run()]
2023-02-08T06:46:12.3142601Z   File "/home/runner/work/dffml/dffml/dffml/util/cli/cmd.py", line 201, in <listcomp>
2023-02-08T06:46:12.3143061Z     return [res async for res in self.run()]
2023-02-08T06:46:12.3143533Z   File "/home/runner/work/dffml/dffml/dffml/cli/ml.py", line 91, in run
2023-02-08T06:46:12.3144024Z     self.model, self.sources, update=self.update, keep_record=True
2023-02-08T06:46:12.3144540Z   File "/home/runner/work/dffml/dffml/dffml/high_level/ml.py", line 288, in predict
2023-02-08T06:46:12.3145006Z     async for record in mctx.predict(sctx):
2023-02-08T06:46:12.3145521Z   File "/home/runner/work/dffml/dffml/model/scikit/dffml_model_scikit/scikit_base.py", line 184, in predict
2023-02-08T06:46:12.3146042Z     async for record in sources.with_features(self.features):
2023-02-08T06:46:12.3146564Z   File "/home/runner/work/dffml/dffml/dffml/source/source.py", line 202, in with_features
2023-02-08T06:46:12.3147036Z     "\n\nLooking for records with the "
2023-02-08T06:46:12.3147563Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/unittest/case.py", line 1112, in assertSetEqual
2023-02-08T06:46:12.3148069Z     self.fail(self._formatMessage(msg, standardMsg))
2023-02-08T06:46:12.3148692Z   File "/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/unittest/case.py", line 693, in fail
2023-02-08T06:46:12.3149210Z     raise self.failureException(msg)
2023-02-08T06:46:12.3149741Z dffml.source.source.NoRecordsWithMatchingFeatures: Items in the first set but not the second:
2023-02-08T06:46:12.3150265Z 'Haralick'
2023-02-08T06:46:12.3150616Z 'Histogram'
2023-02-08T06:46:12.3150981Z 'HuMoments' : 
2023-02-08T06:46:12.3151219Z 
2023-02-08T06:46:12.3151767Z Looking for records with the all of the following features: ['Histogram', 'HuMoments', 'Haralick']. But no records had all of them. Options for features are: set(). Searched 0 records.
2023-02-08T06:46:12.3152207Z 
2023-02-08T06:46:12.3152540Z ----------------------------------------------------------------------
2023-02-08T06:46:12.3152971Z Ran 10 tests in 6626.170s
@johnandersen777 johnandersen777 added the bug Something isn't working label Feb 9, 2023
johnandersen777 pushed a commit that referenced this issue Feb 9, 2023
@pdxjohnny
ci: testing: Disable macos
b730bc8 
Failing due to MNIST?

Related: #1436
@pyther-hub
Copy link
Contributor

sir please elaborate the issue more @pdxjohnny @programmer290399

@johnandersen777
Copy link
Author

@pyther-hub I'm not sure yet what the issue is, some debugging may be required. The following should run one of the failing tests:

$ python -m unittest -v test_mnist.TestMNIST.test_shell

Likely a download URL may have changed

@pyther-hub
Copy link
Contributor

https://github.com/intel/dffml/files/4283897/train-labels-idx1-ubyte.gz" this was the URL link now it is not working, what is the new link or how should I make one of my own?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@johnandersen777 @pyther-hub