Port to use pyproj v2

[dopplershift]

We should consider replacing our own compiled Cython extension that links directly to libproj with pyproj v2 (as initially mentioned in #1252). Benefits I see:

• Eliminate some packaging issues with users needing to separately compile and install libproj
• We sidestep the need to update our compiled code to account for API deprecations and changes (Convert to proj v5 API #1140, WIP: Updating to PROJ 6+ proj.h C API #1252)
• Eliminate the needs to maintain our Cython code

Cons:

• Takes work
• Might require upstream modification (I have no idea what our compiled wrappers do that we can't do through the pyproj python interfaces)

Given that both pyproj and PROJ are active now, it would seem to benefit us to offload this functionality upstream.

[snowman2]

Additional benefits:

• remove need for pyepsg
• add support for projections defined by WKT, spatial reference ID, CF grid_mapping, etc..
• would resolve WIP: Add from_proj4 function to create CRS from PROJ.4 definitions #1023
• pyproj has wheels
• Can construct CRS from rasterio.crs.CRS & geopandas CRS (now uses pyproj.CRS)

[snowman2]

Probably need to merge the geodesic classes:

• https://github.com/SciTools/cartopy/blob/71a91ecc778d44b7a99fceca0badaddb321d0b89/lib/cartopy/geodesic/_geodesic.pyx
• https://github.com/pyproj4/pyproj/blob/05b4e440ece63d60b98c2c6cb9d2f7d373095f94/pyproj/_geod.pyx

[snowman2]

I have a start here: master...snowman2:pyprojv2

But there are a lot of test failures. I could definitely use some help on that (or feel free to take what I have and continue working on it).

This is the main one that pops up in a lot of places:

• pyproj.exceptions.ProjError: Error creating Transformer from CRS.: (Internal Proj Error: proj_create_operations: Source and target ellipsoid do not belong to the same celestial body) (Related to: WIP: Updating to PROJ 6+ proj.h C API #1252 (comment))

[atwilso]

@snowman, what's the current state of this work? I'm interested in helping untangle the dependency chain that stands in the way of Cartopy wheels but I don't know whether I should be looking at this or at Shapely first.

[snowman2]

#1644 needs to be resolved before the transition to pyproj can be completed.

[atwilso]

Thank you! That gives me a place to start investigating.

[greglucas]

Note that this path would allow us to avoid the rewrite within Cartopy for PROJ8 that is being undertaken in #1752 and let pyproj handle the PROJ interfacing instead.

One major downside of all of this that I'm seeing is that somehow the tests are taking significantly longer in both situations (~10x slowdown, going from 2 minutes to 20 minutes on CI). Is there something in the new header that is inherently slower?

[snowman2]

Based on what I saw in #1252, cartopy uses the global context. This makes things much faster, but it is not thread-safe.

By default, pyproj creates a context for each object for thread safety reasons. The downside is the performance hit you run into. That said, you can tell pyproj to use the global context ref and see some significant performance improvements.

PROJ 8.1+ will include this change that will significantly speed things up without using the global context.

[snowman2]

This might also impact things: OSGeo/PROJ#2734

[greglucas]

Thanks for the summary, @snowman2! That makes it sound like the slowdown is definitely not a major issue moving forward. I'm still in favor of this solution. Would you be willing to open up a PR with that branch you had before? I'll try and ping others for a review on the celestial body/PlateCarree PR I opened.

[snowman2]

Seems even with the global context it is slower. I wonder if this has anything to do with it: https://pyproj4.github.io/pyproj/stable/advanced_examples.html#optimize-transformations

[snowman2]

Also wondering how the speed can compares with PROJ 8.1 (current master) 🤔