PROJJSON Schema redirect / CORS issues

[m-mohr]

Until some time ago the PROJJSON schema could be found directly at:
https://proj.org/schemas/v0.5/projjson.schema.json
It's also still the old $id. The PROJ documentation also still refers to this URL (it's another version number, but the issue is the same):
https://proj.org/en/9.4/specifications/projjson.html
[Note: The page points to 0.6 although it seems 0.7 is available?!]

It seems this has changed and now it redirects to:
https://proj.org/en/latest/schemas/v0.5/projjson.schema.json

We've embedded the old URLs in published, versioned specifications such as for STAC and GeoParquet.
Our browser-based STAC tooling can't load the PROJJSON schemas anymore.
The issue seems to be that the old URL doesn't send CORS headers any more.
We'd need to release a new version of the STAC specification to fix the issue with new schema URLs. We'd like to avoid that.

Could CORS for the old URLs be enabled again, please?
[Also, I'm not sure whether the redirect is intentional?]

[rouault]

https://proj.org/en/9.4/specifications/projjson.html

Ah that's an oversight. Just updated to https://proj.org/schemas/v0.7/projjson.schema.json per 5f725fb (master branch) and 0caf671 (9.4 branch)

Regarding CORS issues, I'm not familiar with that. The thing is that now we use ReadTheDocs versionned docs, and no longer a static (single version) github hosted website. And so we have a global redirect from /{foo} to /en/latest/{foo} in the ReadTheDocs settings. CC'ing @mwtoews if he has some clues on what we could do.

An alternative would be that you and us now reference https://proj.org/en/latest/schemas/v0.7/projjson.schema.json as the official URL (assuming it has the CORS headers?)

[rouault]

wondering if using RTD sub-projects (https://docs.readthedocs.io/en/stable/subprojects.html) couldn't be a solution. If we had a "schemas" sub-project, only fed from master builds, that might perhaps do it

[mwtoews]

One solution is to use the replacements extension that uses a mapping configuration which is used in a few places like here. This would require this one variable PROJJSONVERSION to be configured in conf.py with notes to maintain it for each release.

[rouault]

One solution is to use the replacements extension that uses a mapping configuration which is used in a few places like here. This would require this one variable PROJJSONVERSION to be configured in conf.py with notes to maintain it for each release.

@mwtoews Not sure to follow you. How would that help having https://proj.org/schemas/v0.7/projjson.schema.json to be a CORS friendly URL? What you propose if I've understood would be more to help keeping track of the latest version of the projjson spec, but hopefully it won't change very frequently at that point.

[m-mohr]

An alternative would be that you and us now reference https://proj.org/en/latest/schemas/v0.7/projjson.schema.json as the official URL (assuming it has the CORS headers?)

Unfortunately, we can't update released specification schemas. We can do that in new versions, but STAC 1.0.0 and GeoParquet 1.0.0 will both keep referencing the old version and will fail forever. As such I'd be happy if a solution could be found.

An alternative would be that you and us now reference https://proj.org/en/latest/schemas/v0.7/projjson.schema.json as the official URL (assuming it has the CORS headers?)

Yes, it has.

The other question is, is it really good if the schemas are versioned twice / available at dozens of locations?
-> https://proj.org/en/9.4/schemas/v0.5/projjson.schema.json
-> https://proj.org/en/9.3/schemas/v0.5/projjson.schema.json
-> https://proj.org/en/latest/schemas/v0.5/projjson.schema.json
-> ...

Which shall a user use? Maybe the schemas should not live within the versioned documentation and be hosted separately?

[rouault]

is it really good if the schemas are versioned twice?

Well, that hasn't been specifically designed. That's a consequence of having the versioning of the schemas in the /schemas of the git repo (https://github.com/OSGeo/PROJ/tree/master/schemas), and deploying it when building the documentation of a given branch. As RtD adds versioning of the PROJ branches on top of that, you get those 2 levels of versioning.

Referencing to the /latest/is most certainly preferable, in the hopefully unlikely case there would be a need to patch a released version of the schema, instead of publishing a new one.

But maybe we can find a solution to have back the old URL in a CORS friendly way. We just need to figure out if that's possible.

In brainstorming mode: https://raw.githubusercontent.com/OSGeo/PROJ/master/schemas/v0.7/projjson.schema.json is also a candidate CORS compatible URL, but less "good looking".

[m-mohr]

Thanks, but the new URLs have CORS. The issue is that the old URLs are baked into specs and can't really be changed. As such the only solution can be that the old URLs get CORS support, and/or we remove the rediect for the schemas and/or we move the files back to the original place. The alternative is leaving it broken, but I hope there's a better solution. A new URL variant wouldn't help due to the reason given above.

What's responsible for the redirects? We need to hook into there, I think.

[rouault]

What's responsible for the redirects? We need to hook into there, I think.

This is this RtD mechanism: https://docs.readthedocs.io/en/stable/user-defined-redirects.html#user-defined-redirects
In https://readthedocs.org/dashboard/osgeo-proj/redirects/20195/edit/ we have setup an "exact redirect" from /schemas/* to /en/latest/schemas/:splat using HTTP 302. What is done underneath is somewhere in the RtD sources... Probably somewhere in https://github.com/readthedocs/readthedocs.org/tree/main/readthedocs/redirects

[rouault]

I've filed an issue at RtD at readthedocs/readthedocs.org#11223 . Let's see if we get some response

[m-mohr]

I was about to do the same, thank you @rouault!

[rouault]

still in brainstorming mode... Given that the desired resource is /schemas/v0.7/projjson.schema.json, this could ressemble the RTD versioning scheme used for PROJ content (/{language}/{version}/foo) if we'd use language=schemas and version=v0.7. But that looks super messy and not sure we can invent an arbitrary language name and somehow mix PROJ versions with schema versions.

[jorisvandenbossche]

wondering if using RTD sub-projects (https://docs.readthedocs.io/en/stable/subprojects.html) couldn't be a solution. If we had a "schemas" sub-project, only fed from master builds, that might perhaps do it

Personally, this seems like a nice solution?
That should allow to restore the actual URLs without any redirection, and also keeps the URLs somewhat more independent from the readthedocs versioning scheme if also for future versions of the spec we keep using the old URL structure (you never know if in the future the hosting of the docs is again migrated)

[rouault]

Personally, this seems like a nice solution?

actually, looking at https://docs.readthedocs.io/en/stable/subprojects.html, I think it is a wrong track as URLs are like https://docs.example.com/projects/{subproject_name}/en/latest/, so that wouldn't help

[jorisvandenbossche]

Ah, indeed, all the examples indeed include "projects" as first part of the url after the main domain ..

[rouault]

@m-mohr If readthedocs/readthedocs.org#11223 fixes your issue, we can close this issue, right?

[m-mohr]

Indeed, this works fine now. Thanks a lot!