Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix TemporaryDirectory error when exporting models on Windows #749

Merged
merged 1 commit into from
Jun 6, 2024
Merged

Fix TemporaryDirectory error when exporting models on Windows #749

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions optimum/intel/openvino/modeling.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,9 @@ def _from_transformers(

save_dir = TemporaryDirectory()
save_dir_path = Path(save_dir.name)
# This attribute is needed to keep one reference on the temporary directory, since garbage collecting
# would end-up removing the directory containing the underlying OpenVINO model
cls._model_save_dir_tempdirectory_instance = save_dir

# If load_in_8bit and quantization_config not specified then ov_config is set to None and will be set by default in convert depending on the model size
if load_in_8bit is None and not quantization_config:
Expand Down
3 changes: 3 additions & 0 deletions optimum/intel/openvino/modeling_base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -366,6 +366,9 @@ def _from_transformers(

save_dir = TemporaryDirectory()
save_dir_path = Path(save_dir.name)
# This attribute is needed to keep one reference on the temporary directory, since garbage collecting
# would end-up removing the directory containing the underlying OpenVINO model
cls._model_save_dir_tempdirectory_instance = save_dir
Comment on lines +369 to +371
Copy link
Collaborator

@echarlaix echarlaix Jun 6, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think this could be moved to OVBaseModel.__init__() as done in :

optimum-intel/optimum/intel/openvino/modeling_diffusion.py

Lines 106 to 115 in 2c79d98

# This attribute is needed to keep one reference on the temporary directory, since garbage collecting
# would end-up removing the directory containing the underlying OpenVINO model
self._model_save_dir_tempdirectory_instance = None
if isinstance(model_save_dir, TemporaryDirectory):
self._model_save_dir_tempdirectory_instance = model_save_dir
self._model_save_dir = Path(model_save_dir.name)
elif isinstance(model_save_dir, str):
self._model_save_dir = Path(model_save_dir)
else:
self._model_save_dir = model_save_dir

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would require changing some more logic; it doesn't work out of the box. Do you want me to do that?

In https://github.com/huggingface/optimum-intel/blob/main/optimum/intel/openvino/modeling_base.py#L368 the TemporaryDirectory is converted to a Path, which is then past to both main_export and _from_pretrained. So OVBaseModel__init__() model_save_dir is not a TemporaryDirectory but a Path. And to prevent the TemporaryDirectory from automatically being deleted, _model_save_dir_tempdirectory_instance must be set to the actual TemporaryDirectory instance.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the update, for now I think we can merge and will see if it can be added in a following PR


# If load_in_8bit and quantization_config not specified then ov_config is set to None and will be set by default in convert depending on the model size
if load_in_8bit is None and not quantization_config:
Expand Down
4 changes: 4 additions & 0 deletions optimum/intel/openvino/modeling_base_seq2seq.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,10 @@ def _from_transformers(
save_dir = TemporaryDirectory()
save_dir_path = Path(save_dir.name)

# This attribute is needed to keep one reference on the temporary directory, since garbage collecting
# would end-up removing the directory containing the underlying OpenVINO model
cls._model_save_dir_tempdirectory_instance = save_dir

if task is None:
task = cls.export_feature
if use_cache:
Expand Down
4 changes: 3 additions & 1 deletion optimum/intel/openvino/modeling_decoder.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,6 @@ def __init__(
quantization_config=quantization_config,
**kwargs,
)

self.is_dynamic = dynamic_shapes
use_cache = kwargs.pop("use_cache", True)
model_has_sinks = model_has_state(self.model)
Expand Down Expand Up @@ -256,6 +255,9 @@ def _from_transformers(

save_dir = TemporaryDirectory()
save_dir_path = Path(save_dir.name)
# This attribute is needed to keep one reference on the temporary directory, since garbage collecting
# would end-up removing the directory containing the underlying OpenVINO model
cls._model_save_dir_tempdirectory_instance = save_dir

if task is None:
task = cls.export_feature
Expand Down
Loading