Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support PROJ >= 6.0.0 #922

Closed
tina-junold opened this issue Mar 14, 2019 · 10 comments
Closed

Support PROJ >= 6.0.0 #922

tina-junold opened this issue Mar 14, 2019 · 10 comments

Comments

@tina-junold
Copy link

Currently the build failed if proj >= 6.0.0 is used. The Flag ACCEPT_USE_OF_DEPRECATED_PROJ_API_H has to be defined.

see: https://github.com/postgis/postgis/pull/354/files#r242068366

@jmckenna
Copy link

I agree that this flag is required now (as PROJ 6 was released on 2019-03-01).

Note that PROJ 7 release next year will drop proj_api.h completely.

@pnorman
Copy link
Collaborator

pnorman commented Mar 18, 2019

Do we use proj directly or only via libosmium?

@jmckenna
Copy link

Ah, it seems through libosmium only.

@jmckenna
Copy link

libosmium's associated ticket: osmcode/libosmium#277

@jmckenna
Copy link

You can close this ticket, but I bet this will be an FAQ.

@pnorman pnorman closed this as completed Mar 18, 2019
@jmckenna
Copy link

jmckenna commented Jan 24, 2020

related PROJ 7.0 discussion happening at osmcode/libosmium#295

@lonvia
Copy link
Collaborator

lonvia commented Jan 27, 2020

Reopening this. We are going to need osmium-independent proj >= 6 support.

@lonvia lonvia reopened this Jan 27, 2020
@kennykb
Copy link

kennykb commented Jan 27, 2020

Thanks for reopening. I was going to grumble that I produce some paper maps projected to UTM zone 18 and therefore maintain a small subset of OSM in PostGIS in that projection, so losing support for everything but plate carree and spherical Mercator would be a problem.

@lonvia
Copy link
Collaborator

lonvia commented Jan 28, 2020

#1066 reorders the code so that it becomes fairly easy to implement a generic projection using the proj6 variant and then have cmake detect which one to use. Just add a reprojection-generic-proj6.cpp similar to the existing reprojection-generic-proj4.cpp implementation.

PRs would be most welcome.

joto added a commit to joto/osm2pgsql that referenced this issue Nov 24, 2020
This commit is adding projection support using the PROJ library from
6.1 onwards. There might be an issue with multithreading, but the old
code most likely already had that issue. This needs to be checked.

The CMake config will prefer using the old API, simply because I don't
know how to check for the new API without it trying to use the new API
with version 5-6.0 which don't have everything we need. The code will
only work from 6.1 onwards.

There is a new CMake cache variable USE_PROJ_LIB which can be set to:
* "4": Use version 4 API. Fail CMake if it is not available.
* "6": Use version 6 API. Fail CMake if it is not available.
* "off": Disable PROJ support (resulting binary will only have WGS84
  (4326) and Web Mercator (3857) support)
* "auto": Use version 4 API if available, otherwise try version 6 API
  or fall back to no PROJ support.

See osm2pgsql-dev#922
joto added a commit to joto/osm2pgsql that referenced this issue Nov 25, 2020
This commit is adding projection support using the PROJ library from
6.1 onwards. There might be an issue with multithreading, but the old
code most likely already had that issue. This needs to be checked.

The CMake config will prefer using the old API, simply because I don't
know how to check for the new API without it trying to use the new API
with version 5-6.0 which don't have everything we need. The code will
only work from 6.1 onwards.

There is a new CMake cache variable USE_PROJ_LIB which can be set to:
* "4": Use version 4 API. Fail CMake if it is not available.
* "6": Use version 6 API. Fail CMake if it is not available.
* "off": Disable PROJ support (resulting binary will only have WGS84
  (4326) and Web Mercator (3857) support)
* "auto": Use version 4 API if available, otherwise try version 6 API
  or fall back to no PROJ support.

See osm2pgsql-dev#922
joto added a commit to joto/osm2pgsql that referenced this issue Nov 25, 2020
This commit is adding projection support using the PROJ library from
6.1 onwards.

The CMake config will prefer using the old API, simply because I don't
know how to check for the new API without it trying to use the new API
with version 5-6.0 which don't have everything we need. The code will
only work from 6.1 onwards.

There is a new CMake cache variable USE_PROJ_LIB which can be set to:
* "4": Use version 4 API. Fail CMake if it is not available.
* "6": Use version 6 API. Fail CMake if it is not available.
* "off": Disable PROJ support (resulting binary will only have WGS84
  (4326) and Web Mercator (3857) support)
* "auto": Use version 4 API if available, otherwise try version 6 API
  or fall back to no PROJ support.

Note that we have an issue with multithreading using the PROJ library,
because we use it potentially from multiple threads. The information I
can find about this seems to indicate that this is only a problem for
error reporting, because of the error handling using a global errno-type
variable. So we can't detect failed transformations reliably. But we
don't care about those anyway in the code we have, so I believe this is
probably not a huge problem. This affects the old and the new code, so
this isn't something new. The API version 6 allows to do the
multithreading correctly using a "context" which this commit does use.
But the context is still shared between threads, because we are starting
the threads after the reprojections have been initialized. So all of
this is technically not correct, but we'll probably get away with it for
the time being. We need larger refactorings in the geometry code anyway
which we want to tackle soon and hopefully can resolve this correctly
then.

See osm2pgsql-dev#922
joto added a commit to joto/osm2pgsql that referenced this issue Nov 25, 2020
This commit is adding projection support using the PROJ library from
6.1 onwards.

The CMake config will prefer using the old API, simply because I don't
know how to check for the new API without it trying to use the new API
with version 5-6.0 which don't have everything we need. The code will
only work from 6.1 onwards.

There is a new CMake cache variable USE_PROJ_LIB which can be set to:
* "4": Use version 4 API. Fail CMake if it is not available.
* "6": Use version 6 API. Fail CMake if it is not available.
* "off": Disable PROJ support (resulting binary will only have WGS84
  (4326) and Web Mercator (3857) support)
* "auto": Use version 4 API if available, otherwise try version 6 API
  or fall back to no PROJ support.

Note that we have an issue with multithreading using the PROJ library,
because we use it potentially from multiple threads. The information I
can find about this seems to indicate that this is only a problem for
error reporting, because of the error handling using a global errno-type
variable. So we can't detect failed transformations reliably. But we
don't care about those anyway in the code we have, so I believe this is
probably not a huge problem. This affects the old and the new code, so
this isn't something new. The API version 6 allows to do the
multithreading correctly using a "context" which this commit does use.
But the context is still shared between threads, because we are starting
the threads after the reprojections have been initialized. So all of
this is technically not correct, but we'll probably get away with it for
the time being. We need larger refactorings in the geometry code anyway
which we want to tackle soon and hopefully can resolve this correctly
then.

See osm2pgsql-dev#922
joto added a commit to joto/osm2pgsql that referenced this issue Nov 26, 2020
This commit is adding projection support using the PROJ library from
6.1 onwards.

The CMake config will prefer using the old API, simply because I don't
know how to check for the new API without it trying to use the new API
with version 5-6.0 which don't have everything we need. The code will
only work from 6.1 onwards.

There is a new CMake cache variable USE_PROJ_LIB which can be set to:
* "4": Use version 4 API. Fail CMake if it is not available.
* "6": Use version 6 API. Fail CMake if it is not available.
* "off": Disable PROJ support (resulting binary will only have WGS84
  (4326) and Web Mercator (3857) support)
* "auto": Use version 4 API if available, otherwise try version 6 API
  or fall back to no PROJ support.

Note that we have an issue with multithreading using the PROJ library,
because we use it potentially from multiple threads. The information I
can find about this seems to indicate that this is only a problem for
error reporting, because of the error handling using a global errno-type
variable. So we can't detect failed transformations reliably. But we
don't care about those anyway in the code we have, so I believe this is
probably not a huge problem. This affects the old and the new code, so
this isn't something new. The API version 6 allows to do the
multithreading correctly using a "context" which this commit does use.
But the context is still shared between threads, because we are starting
the threads after the reprojections have been initialized. So all of
this is technically not correct, but we'll probably get away with it for
the time being. We need larger refactorings in the geometry code anyway
which we want to tackle soon and hopefully can resolve this correctly
then.

See osm2pgsql-dev#922
@jmckenna
Copy link

support for PROJ >6.1 added by @joto in #1331

I think this ticket can be closed.

@joto joto closed this as completed Nov 27, 2020
bob-beck pushed a commit to openbsd/ports that referenced this issue Mar 7, 2021
new website at https://osm2pgsql.org/

osm2pgsql-dev/osm2pgsql#922 was fixed so this
now builds against upcoming devel/proj 8 update.

use MODPOSTGRESQL_TEST_CMD = ${MODCMAKE_TEST_TARGET} to run the tests
using the cmake goop, pleasantly surprised to see that this works !
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants