Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

free tree on AsdfFile.close #1575

Merged
merged 3 commits into from
Aug 7, 2023
Merged

Conversation

braingram
Copy link
Contributor

With these changes, when an AsdfFile object is closed the tree (which already could not be accessed because of the tree property gatekeeping) is set to a blank AsdfObject.

This allows the garbage collector to clean up objects referenced in the tree even though the AsdfFile object does not fall out of scope.

Fixes #1558

@braingram braingram added Downstream CI development No backport required labels Jun 28, 2023
@github-actions github-actions bot modified the milestone: 3.0.0 Jun 28, 2023
@braingram braingram marked this pull request as ready for review June 28, 2023 18:50
@braingram braingram requested a review from a team as a code owner June 28, 2023 18:50
asdf/_tests/_issues/test_1558.py Outdated Show resolved Hide resolved
@@ -449,6 +449,9 @@ def close(self):
self._fd.close()
self._fd = None
self._closed = True
# as we're closing the file, also empty out the
# tree so that references to array data can be released
self._tree = AsdfObject()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about raising a relevant error when the tree of a closed AsdfFile is accessed? I'd prefer a message like "Sorry your AsdfFile is closed" to "KeyError" if I were investigating a bug in my code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A custom exception sounds like a great idea. Currently and OSError is raised so any custom exception we raise could inherit from that exception class for backwards compatibility.

asdf/asdf/asdf.py

Lines 559 to 561 in b43b2c5

if self._closed:
msg = "Cannot access data from closed ASDF file"
raise OSError(msg)

It might make sense to raise the same error type when a block is accessed from a closed file (and any other places where access to a closed file is prevented):

asdf/asdf/block.py

Lines 1290 to 1292 in b43b2c5

if self._fd.is_closed():
msg = "ASDF file has already been closed. Can not get the data."
raise OSError(msg)

Perhaps a ClosedFileAccess exception that inherits from OSerror? That seems reasonably backwards compatible to include in 3.0 (and would work towards cleaning up the rather generic exceptions raised by asdf).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, oops, I didn't notice/remember that we already throw an exception. OSError is plenty enough for me.

Copy link
Contributor

@eslavich eslavich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

With these changes, when an AsdfFile object is closed
the tree (which already could not be accessed because of the
tree property gatekeeping) is set to a blank AsdfObject.

This allows the garbage collector to clean up objects
referenced in the tree even though the AsdfFile object
does not fall out of scope.

Fixes asdf-format#1558
@braingram braingram merged commit 90838f7 into asdf-format:main Aug 7, 2023
28 of 29 checks passed
@braingram braingram deleted the free_tree branch September 8, 2023 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development No backport required Downstream CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

closed AsdfFile instances hold private reference to tree
3 participants