[FEATURE] Log problematic geometry causing JTS exceptions #700
Comments
bdon
changed the title
|
@bdon what do you think of this approach: #703 ? When you add |
|
maybe it's safer to use WKB instead of WKT? Wouldn't writing WKT transform floating-point representation into fixed-precision text; if it's parsed back into floating-point later it might be a different number, which might make robustness problems disappear? Will inspect the logs for different classes of errors now. |
Generally it is the safest to use WKB. However, I've seen lots (well, some) of OverlayNG issues which can be reproduced using WKT. So either way is likely to provide good feedback. cheers @bdon ! |
|
Another class of exception from the same run: It is only these two classes of exception that happen, numbering about 10-20 per run for the 100GB output tileset. |
|
We want a separate out-of-band check on the output tileset to check for self-intersection JTS silently creates - my preference is to use https://github.com/mapbox/wagyu |
|
In the It seems like some heuristic code to eliminate very short line segments might avoid these errors. If these are in the input (which seems unlikely) this could be done by some preprocessing. If they are being created during the union process, I'll need some reproducing data to try and identify a fix. |
|
I wonder if there's a better (ie. faster, more robust) solution than Perhaps |
|
Thanks or the feedback @dr-jts! ConcaveHullOfPolygons sounds promising, I tried it out in place of bufferUnionUnbufer and ran into a couple of issues:
Is there a different sequence of operations you think might help here instead? |
|
@bdon it looks like the same spot in planetiler code will catch that exception as well. I'll print out WKT and base64-encoded WKB just to be safe. |
|
Attached are text files with the full WKT and WKB inputs along with stack traces for the 6 exceptions from last night's build: exception_6.txt |
|
It looks like all these errors are due to the "buffer" result being invalid, due to self-intersecting rings. I'm not sure why this is - probably a bug in the buffer algorithm. Here's an example from one of the error cases: Can you provide the distance used to buffer? That will help in looking into this. A workaround is to check the buffered value for validity, and then run Another possible fix: buffering essentially acts as a union in itself. So possibly in |
|
Thanks @dr-jts! Looks like the buffer distance is 0.5.
Yes that's what I initially saw while implementing this, they performed about the same until it got to merging a lot of tiny densely packed building polygons and would hang for a long time. |
|
I did some more testing on this case ( JTS 1.19 produces the invalid buffer (with mitre join) for the first element as you are seeing, which causes the error in However, JTS 1.20 (unreleased - using the current dev code) produces a correct buffer: This unions successfully:
There have been several changes in the codebase that might have improved the buffer output, but I'm not exactly sure what has solved this particular issue. So - you could try building from the source of |
|
As for the low utlity of |
|
Thanks @dr-jts that's great news! @bdon should be able to force his build to use the latest 1.20.0-SNAPSHOT build to test out if this is fixed. It would be great if we could use
|
|
Attached are all 3 exceptions that arose with JTS 1.20.0-SNAPSHOT. Note: this runs on slightly different data than before (since it's OSM) There is a new class of exception that did not appear in the 6 before: jts_1_20_exception_3.txt |
|
It looks like these errors are all due to the initial buffer producing invalid output. Specifically, there's a bug that causes the inward buffer of holes to "invert" and be output as a separate (overlapping) polygon. The workaround is to check buffer output for validity, and if invalid run I'll log this as a JTS issue. It's probably gnarly to fix though, so I doubt I'll have a fix soon. |
|
Would it be sufficient to only do the fix if an exception gets thrown? |
Yes, that's a good idea. |
|
OK I added these test cases and the GeometryFixer fix in #713 - it seems to resolve all of the issues. One other thing that I notice is that the output of the unbuffer operation is often invalid (~12k times per planet render) and gets fixed by this code in Thanks for the help @dr-jts ! |
Excellent. Good that
Yes, a few examples of unbuffer invalid output would be good. I will then open a JTS issue for this - to be worked on at some future date. It's gnarly... |
|
Here are some geometries that result in invalid output when you do And here are some geometries that result in invalid output when you do |
Is your feature request related to a problem? Please describe.
A couple dozen times per run I'll see JTS exceptions that may be related to robustness in OverlayNG polygon boolops.
Example:
It would be ideal to log these so the exception can be reproduced and reported upstream in the JTS issue tracker.
Describe the solution you'd like
Add a configuration flag like
--log-jts-exceptions. When an exception happens, it serializes the input geometries to the JTS operation to GeoJSON on disk so that the exception can be reproduced later. TBD how to name these output files.The text was updated successfully, but these errors were encountered: