Index only mode #799
Comments
|
I'd say that this is something where a pull request would be welcome. |
|
I'm iffy on this idea, because running the index steps requires a decent amount of knowledge of how far its got and what osm2pgsql does. We'll also have people run it on an import which died in pending ways, which won't error out, but will not produce a correct result. |
|
I'm all for making it easy for newbies but not at the expense of making life harder for advanced user than necessary. There are ways to mitigate misuse: mark the option as expert-only and/or add a info line to the output that tells when it is safe to use it. If pending ways does not error out properly then this something that should be fixed on its own, independently of this issue. |
It properly errors out, but if osm2pgsql terminates unexpectedly (e.g. as above) the database remains. This is true in all phases, and there's no possible way to clean up when osm2pgsql gets OOM killed. In any stage past the very start, all the tables exist, and the SQL that's run at the end will run without error. It just won't produce sensible output. |
|
same problem, my osm2pgsql output: Node stats: total(4235496629), max(5290747187) in 33793s Way stats: total(458618737), max(547543126) in 1300890s Relation stats: total(5442482), max(7820612) in 741417s Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Using built-in tag processing pipeline Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Using built-in tag processing pipeline Killed I want run only index mode and don't wont waiting 24 days again |
|
These timing are too high and you don't use lua filtering. Anyway, it is needed to exclude useless data. When you run with flatnodes, don't spend the memory needed by Postgres, use the default or less. |
|
How hard would it be to add a "please reindex" switch that ignores any *.pbf file argument and only rebuilds the indices? |
|
Yes please, this really looks like an all to natural feature - start over the part of the process that fails most frequently, instead of losing hours or even days of successfully performed steps. |
Please add -- index-only mode to existing --append and --create modes.
I try load osm planet. The data has been uploaded. But at the stage of indexation, there was a сrash. Probably out of memory.
It would be desirable not to load again all the data (a few days), to complete the stage of indexing.
Log:
The text was updated successfully, but these errors were encountered: