Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

checkpoint file causing errors. #21

Closed
kharanshu2 opened this issue Mar 16, 2023 · 1 comment
Closed

checkpoint file causing errors. #21

kharanshu2 opened this issue Mar 16, 2023 · 1 comment

Comments

@kharanshu2
Copy link

Checkpoint file is causing errors while running inference in command line(as given in the inference example): (I have tried it on 2L and 1L models and same error occurs):

  1. When I keep model path and not checkpoint path in ckpt_name the inference command:
    (base) D:\Repos\covid\COVIDNet-CT>python run_covidnet_ct.py infer --model_dir D:\Repos\covid\COVID-Net_CT-1_S --meta_name D:\Repos\covid\COVID-Net_CT-1_S\model.meta --ckpt_name D:\Repos\covid\COVID-Net_CT-1_S --image_file D:\Repos\covid\data\covid1.png
    WARNING:tensorflow:From run_covidnet_ct.py:361: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From run_covidnet_ct.py:361: The name tf.logging.ERROR is deprecated. Please use tf.compat.v1.logging.ERROR instead.

2023-03-16 10:19:18.597935: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Loading meta graph from D:\Repos\covid\COVID-Net_CT-1_S\model.meta
Loading weights from D:\Repos\covid\COVID-Net_CT-1_S
2023-03-16 10:19:19.828731: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open D:\Repos\covid\COVID-Net_CT-1_S: Unknown: NewRandomAccessFile failed to Create/Open: D:\Repos\covid\COVID-Net_CT-1_S : Access is denied.
; Input/output error
2023-03-16 10:19:19.830576: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open D:\Repos\covid\COVID-Net_CT-1_S: Unknown: NewRandomAccessFile failed to Create/Open: D:\Repos\covid\COVID-Net_CT-1_S : Access is denied.
; Input/output error
2023-03-16 10:19:19.830663: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at save_restore_tensor.cc:175 : Data loss: Unable to open table file D:\Repos\covid\COVID-Net_CT-1_S: Unknown: NewRandomAccessFile failed to Create/Open: D:\Repos\covid\COVID-Net_CT-1_S : Access is denied.
; Input/output error
Traceback (most recent call last):
File "C:\Users\KharanshuNaghera\miniconda3\lib\site-packages\tensorflow_core\python\client\session.py", line 1365, in _do_call
return fn(*args)
File "C:\Users\KharanshuNaghera\miniconda3\lib\site-packages\tensorflow_core\python\client\session.py", line 1350, in _run_fn
target_list, run_metadata)
File "C:\Users\KharanshuNaghera\miniconda3\lib\site-packages\tensorflow_core\python\client\session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file D:\Repos\covid\COVID-Net_CT-1_S: Unknown: NewRandomAccessFile failed to Create/Open: D:\Repos\covid\COVID-Net_CT-1_S : Access is denied.
; Input/output error
[[{{node save/RestoreV2}}]]

  1. When I keep checkpoint file in ckpt_name:
    (base) D:\Repos\covid\COVIDNet-CT>python run_covidnet_ct.py infer --model_dir D:\Repos\covid\COVID-Net_CT-1_S --meta_name D:\Repos\covid\COVID-Net_CT-1_S\model.meta --ckpt_name D:\Repos\covid\COVID-Net_CT-1_S\checkpoint --image_file D:\Repos\covid\data\covid1.png
    WARNING:tensorflow:From run_covidnet_ct.py:361: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From run_covidnet_ct.py:361: The name tf.logging.ERROR is deprecated. Please use tf.compat.v1.logging.ERROR instead.

2023-03-16 10:22:38.860381: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Loading meta graph from D:\Repos\covid\COVID-Net_CT-1_S\model.meta
Loading weights from D:\Repos\covid\COVID-Net_CT-1_S\checkpoint
2023-03-16 10:22:40.122130: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open D:\Repos\covid\COVID-Net_CT-1_S\checkpoint: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
2023-03-16 10:22:40.123169: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open D:\Repos\covid\COVID-Net_CT-1_S\checkpoint: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
2023-03-16 10:22:40.123238: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at save_restore_tensor.cc:175 : Data loss: Unable to open table file D:\Repos\covid\COVID-Net_CT-1_S\checkpoint: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
Traceback (most recent call last):
File "C:\Users\KharanshuNaghera\miniconda3\lib\site-packages\tensorflow_core\python\client\session.py", line 1365, in _do_call
return fn(*args)
File "C:\Users\KharanshuNaghera\miniconda3\lib\site-packages\tensorflow_core\python\client\session.py", line 1350, in _run_fn
target_list, run_metadata)
File "C:\Users\KharanshuNaghera\miniconda3\lib\site-packages\tensorflow_core\python\client\session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file D:\Repos\covid\COVID-Net_CT-1_S\checkpoint: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
[[{{node save/RestoreV2}}]]

--Model does work when no path is to ckpt file is provided.

Further more while running the ipynb file provided for visualization during execution of this block:

Create full paths

meta_file = os.path.join(MODEL_DIR, META_NAME)
ckpt = os.path.join(MODEL_DIR, CKPT_NAME)

Load metagraph and create session

graph, sess, saver = load_graph(meta_file)

Load checkpoint

with graph.as_default():
load_ckpt(ckpt, sess, saver)
final_conv, pooled_grads = make_gradcam_graph(graph)

this error occurs:

Loading meta graph from D:/Repos/covid/COVID-Net_CT-1_S/model.meta
Loading weights from D:/Repos/covid/COVID-Net_CT-1_S/checkpoint
INFO:tensorflow:Restoring parameters from D:/Repos/covid/COVID-Net_CT-1_S/checkpoint


DataLossError Traceback (most recent call last)
~\miniconda3\envs\covidnet\lib\site-packages\tensorflow_core\python\client\session.py in _do_call(self, fn, *args)
1364 try:
-> 1365 return fn(*args)
1366 except errors.OpError as e:

~\miniconda3\envs\covidnet\lib\site-packages\tensorflow_core\python\client\session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata)
1349 return self._call_tf_sessionrun(options, feed_dict, fetch_list,
-> 1350 target_list, run_metadata)
1351

~\miniconda3\envs\covidnet\lib\site-packages\tensorflow_core\python\client\session.py in _call_tf_sessionrun(self, options, feed_dict, fetch_list, target_list, run_metadata)
1442 fetch_list, target_list,
-> 1443 run_metadata)
1444

DataLossError: Unable to open table file D:\Repos\covid\COVID-Net_CT-1_S\checkpoint: Data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?
[[{{node save/RestoreV2}}]]

@kharanshu2
Copy link
Author

tensorflow/models#2676 (comment)

this link helped really, apparently you are supposed to load model.data-00000-of-00001 and not checkpoint and use just the name "model" instead of full "model.data-00000-of-00001"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant