Consider upgrading to jsonschema 3.0

[drdavella]

Currently there's only an alpha release, and it causes some errors that are detected by our unit tests. However, once things become more stable we should consider whether we should upgrade, and if so, what changes will need to be made.

Version 3.0 introduces support for Draft-06. This could potentially allow us to update the ASDF Standard to support Draft-06 as well. However, if we are really serious about providing/supporting multiple implementations of ASDF, we need to look at the lowest common denominator across JSON Schema implementations, which might still be Draft-04.

[perrygreenfield]

Is there any urgency to do this? (e.g., very useful new functionality or backward incompatible changes?) Otherwise, we would tend to take a slow approach to such things.

[drdavella]

No, there's no urgency. I just wanted to record the issue here as a reminder for future reference.

That being said, if it's not difficult to upgrade to 3.0 (once it's stable), then we should, since that's presumably where all bug fixes and improvements will wind up.

[perrygreenfield]

Agreed. We should evaluate it once is it a stable release.

[nden]

I came upon this stackoverflow discussion about using implications and if ... else in jsonschema. May be I missed it but I didn't see which versions of the schema support this. However, this looks very useful to have.

[drdavella]

Yes, it does seem very useful. I believe it is supported as of Draft-07. The jsonschema Python module doesn't yet support that draft (and only supports Draft-06 in alpha releases), but it looks like Draft-07 may be supported relatively soon.

Even after that point, moving to Draft-07 may disadvantage other potential implementations of ASDF since JSON Schema support in other languages may not be as advanced. We'll need to have a discussion about how much that really matters, though.

[pllim]

Somehow 3.0 get picked up by one of my conda install that happened before I installed asdf. This caused conflict and to solve that:

• conda uninstall jsonschema

Then, I tried pip install jsonschema and that picked up 2.x but that did not work either. asdf gave a cryptic message about cannot understand something when I load a file. What finally worked:

• pip uninstall jsonschema
• Install asdf from master and let asdf install jsonschema in the process

I would say this kind of installation hurdle is not very user-friendly right now. I spent 30 minutes trying to make this work. FYI.

[drdavella]

Can you provide the error message? It shouldn't have been necessary to do the pip install jsonschema step since it's explicitly given as a dependency of asdf, but it would be useful to see the error.

[pllim]

"Cannot understand something something 1.3.0"

I tried to reproduce it in a clean env but I cannot now. I swear it was happening last night. @ejeschke can be my witness!

The fact that asdf installs jsonschema if not there is nice, but it wasn't playing nice with jsonschema that is already installed (for one reason or another).

[drdavella]

I can't reproduce it either. I realize it's annoying to have a strict dependency on jsonschema, but that's just the situation we have for the near term. Maybe I can try to address this for 2.3.0, but it also depends on which versions of Python jsonschema-3.0 supports. Also the fact that the latest released version of jsonschema on pypi is 2.6.0 means that I don't feel a strong need to do this immediately.

[drdavella]

Given the error message snippet you reported, it sounds like you were probably trying to read a file that was recently produced by the JWST pipeline. Unfortunately, since they run against a dev version of ASDF, it means that you won't be able to use a stable version to read their files. So installing from master was the right thing to do.

For the record, this issue is unrelated to the version of jsonschema.

[Cadair]

It's worth noting that the jsonschema 3.0.1 package is now up on PyPI and the like.

[drdavella]

There's actually a 3.0 release of jsonschema now so this has become more urgent/relevant.

[pllim]

jsonschema<3 now gives this warning:

DeprecationWarning: Using or importing the ABCs from 'collections' instead of
from 'collections.abc' is deprecated, and in 3.8 it will stop working

It has been fixed in jsonschema>=3. Indirectly, this means that if asdf does not support jsonschema>=3, asdf will not work in Python 3.8.

[pllim]

Might still be a problem, see #684 (comment). This is giving me a headache, because Jupyter is asking for jsonschema>=3.

[jdavies-st]

This is an order-of-installation issue, not a problem in asdf.

Fixed by #684.