Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Asdftool edit #873

Merged
merged 48 commits into from
Oct 9, 2020
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
71c5583
Adding basic edit command to asdftool.
Aug 14, 2020
420d18d
Updating the edit.py file.
kmacdonald-stsci Aug 31, 2020
b846d5f
Pushing an error trying to expose a function in asdf.py to be used in…
kmacdonald-stsci Sep 1, 2020
8b2c9be
Corrected importation of internal function error in edit.py.
kmacdonald-stsci Sep 1, 2020
38c6223
Correcting name formatting of internal use only functions.
kmacdonald-stsci Sep 1, 2020
dbd99bc
Adding utility functions by refactoring class methods.
kmacdonald-stsci Sep 1, 2020
5a9cc7b
Part of the way through asdf.py refactor for code reuse in edit.py.
kmacdonald-stsci Sep 1, 2020
49d07e0
Completed '-e' option for the asdftool 'edit' subcommand.
kmacdonald-stsci Sep 2, 2020
1bc4ef1
The edit command is almost finished. For a rewritten ASDF file, due …
kmacdonald-stsci Sep 2, 2020
83eeb97
Finished the asdftool edit subcommand for edited YAML text with the s…
kmacdonald-stsci Sep 4, 2020
592bc6e
Completed the 'save' portion of the 'edit' subcommand for asdftool. …
kmacdonald-stsci Sep 8, 2020
df07474
Adding a check to make sure the block checks don't go out of bounds.
kmacdonald-stsci Sep 9, 2020
59aa925
Making formatting and usage changes based on feedback on pull request.
kmacdonald-stsci Sep 10, 2020
0f2a04d
Removing debug tracing.
kmacdonald-stsci Sep 11, 2020
dc6cba3
Removing comment.
kmacdonald-stsci Sep 11, 2020
9fb3a88
Integrating feedback from the pull request.
kmacdonald-stsci Sep 14, 2020
d2a6c7c
Changing edit subcommand based on pull request feedback.
kmacdonald-stsci Sep 17, 2020
7187467
Correcting python formatting based on flake8 report.
kmacdonald-stsci Sep 17, 2020
b193d2b
Changing formatting of files and removing unnecessary comments.
kmacdonald-stsci Sep 17, 2020
dd5672f
Adding function comments.
kmacdonald-stsci Sep 18, 2020
89f8de8
Reverting to original asdf.py file.
kmacdonald-stsci Sep 21, 2020
10dc02f
Reverting commands/__init__.py to original style.
kmacdonald-stsci Sep 21, 2020
55f02b0
Adding basic edit command to asdftool.
Aug 14, 2020
24b276a
Adding pytest for the edit subcommand for asdftool.
kmacdonald-stsci Sep 22, 2020
d8d1898
Merge branch 'asdftool-edit' of github.com:kmacdonald-stsci/asdf into…
kmacdonald-stsci Sep 22, 2020
94e1be8
Adding coverage for a smaller YAML edited file.
kmacdonald-stsci Sep 22, 2020
d0f1aa8
Adding parametrization the edit test to test all supported versions.
kmacdonald-stsci Sep 22, 2020
0fbf98b
Removing hardcoded path used for local testing.
kmacdonald-stsci Sep 22, 2020
5ecb3b1
Refactored the automated tester for the edit subcommand of asdftool.
kmacdonald-stsci Sep 22, 2020
6f0d543
Reformatting the tests and changing some values in the tree.
kmacdonald-stsci Sep 22, 2020
9d2a663
Adding formatting changes and checks using feed back from code review.
kmacdonald-stsci Sep 23, 2020
6639916
Formatting changes based on black and tox.
kmacdonald-stsci Sep 23, 2020
7d77ce5
Adding changes based on pull request feedback.
kmacdonald-stsci Sep 23, 2020
d61258f
Adding file closures that may have been causing problems during testi…
kmacdonald-stsci Sep 23, 2020
a74f839
Updating user messages to properly reflect changes to how edit works.
kmacdonald-stsci Sep 23, 2020
b95f2a7
The Windows tests are still failing, so I added the 'with' idiom when…
kmacdonald-stsci Sep 23, 2020
f38b186
The method 'os.rename' always raises an error on Windows. Switching …
kmacdonald-stsci Sep 23, 2020
4f9bf9d
There are still Travis-CI failures on Windows. Changing the way to c…
kmacdonald-stsci Sep 23, 2020
acb658e
Adding binary read and writes to the test to hopefully fix the test o…
kmacdonald-stsci Sep 24, 2020
5fa8171
Adding formatting change to conform with style standards.
kmacdonald-stsci Sep 24, 2020
0102261
Incorporating PR feedback and correctly implementing padding. The ca…
kmacdonald-stsci Oct 2, 2020
4d1cf9c
Added changes to the edit subcommand based on PR feedback.
kmacdonald-stsci Oct 2, 2020
b6e8ce6
Adding rewrite functionality to the 'save' option, including for the …
kmacdonald-stsci Oct 5, 2020
0ab696a
Changed docstrings to conform with standard formatting. Removed extr…
kmacdonald-stsci Oct 5, 2020
27e2370
Format changes based on 'black' tool.
kmacdonald-stsci Oct 5, 2020
4d1d726
Correcting style for formatted strings and removed extraneous comments.
kmacdonald-stsci Oct 5, 2020
2ebe30b
Removed the use of a global variable and changed the way the search f…
kmacdonald-stsci Oct 7, 2020
5620cac
Updating the edit subcommand to handle trying to edit ASDF file conta…
kmacdonald-stsci Oct 9, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions asdf/asdf.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@

from .tags.core import AsdfObject, Software, HistoryEntry, ExtensionMetadata


def get_asdf_library_info():
"""
Get information about asdf to include in the asdf_library entry
Expand Down Expand Up @@ -740,8 +739,7 @@ def _parse_header_line(cls, line):
try:
version = versioning.AsdfVersion(parts[1].decode('ascii'))
except ValueError:
raise ValueError(
"Unparseable version in ASDF file: {0}".format(parts[1]))
raise ValueError("Unparseable version in ASDF file: {0}".format(parts[1]))

return version

Expand All @@ -763,6 +761,9 @@ def _parse_comment_section(cls, content):

@classmethod
def _find_asdf_version_in_comments(cls, comments):
""" From the initial comments line in an ASDF file, capture the ASDF
kmacdonald-stsci marked this conversation as resolved.
Show resolved Hide resolved
version.
"""
for comment in comments:
parts = comment.split()
if len(parts) == 2 and parts[0] == constants.ASDF_STANDARD_COMMENT:
Expand All @@ -785,12 +786,15 @@ def _open_asdf(cls, self, fd, uri=None, mode='r',
ignore_missing_extensions=False,
**kwargs):
"""Attempt to populate AsdfFile data from file-like object"""

# Make sure arguments aren't contradictory
if strict_extension_check and ignore_missing_extensions:
raise ValueError(
"'strict_extension_check' and 'ignore_missing_extensions' are "
"incompatible options")

# Set local variables
# TODO From here to self._mode = mode, can put in a function
# TODO validate_on_read, legacy_fill_schema_defaults = validate_and_schema(kwargs)
if "validate_on_read" in kwargs:
warnings.warn(
"The 'validate_on_read' argument is deprecated, set "
Expand All @@ -811,16 +815,19 @@ def _open_asdf(cls, self, fd, uri=None, mode='r',
else:
legacy_fill_schema_defaults = get_config().legacy_fill_schema_defaults

# Open the file
self._mode = mode

fd = generic_io.get_file(fd, mode=self._mode, uri=uri)
self._fd = fd
# The filename is currently only used for tracing warning information

# Validate the ASDF header
self._fname = self._fd._uri if self._fd._uri else ''
header_line = fd.read_until(b'\r?\n', 2, "newline", include=True)
self._file_format_version = cls._parse_header_line(header_line)
self.version = self._file_format_version

# Read the optional comments line(s)
comment_section = fd.read_until(
b'(%YAML)|(' + constants.BLOCK_MAGIC + b')', 5,
"start of content", include=False, exception=False)
Expand All @@ -836,6 +843,8 @@ def _open_asdf(cls, self, fd, uri=None, mode='r',
if extensions:
self.extensions = extensions

# Read and validate YAML text.
# It's possible there is no YAML, so the next token could be BLOCK_MAGIC
yaml_token = fd.read(4)
has_blocks = False
tree = None
Expand Down Expand Up @@ -879,6 +888,7 @@ def _open_asdf(cls, self, fd, uri=None, mode='r',

if validate_on_read:
try:
# TODO Validation will take some work to separate from the class.
self._validate(tree, reading=True)
except ValidationError:
self.close()
Expand Down
16 changes: 12 additions & 4 deletions asdf/commands/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,19 @@
from .tags import list_tags
from .extension import find_extensions
from .info import info
from .edit import edit


__all__ = ['implode', 'explode', 'to_yaml', 'defragment', 'diff', 'list_tags',
'find_extensions', 'info']

__all__ = [
'defragment',
'diff',
'edit',
'explode',
'find_extensions',
'implode',
'info'
'list_tags',
'to_yaml',
]

# Extracting ASDF-in-FITS files requires Astropy
if importlib.util.find_spec('astropy'):
Expand Down
Loading