Speed up VarInt encoding, remove emitTilesInOrder option [#98] #460

bdon · 2023-01-27T03:53:34Z

This changes the user and YML-facing configuration by removing the emitTilesInOrder option, should we keep that dead option in there for now?

The difference between ordered and unordered MBTiles inserts on a 64 core machine isn't significant:

ordered:

0:53:59 INF - ----------------------------------------
0:53:59 INF -   overall          53m59s cpu:44h37m22s gc:3m39s avg:49.6
0:53:59 INF -   lake_centerlines 3s cpu:10s avg:3.1
0:53:59 INF -     read     2x(29% 0.9s done:2s)
0:53:59 INF -     process 62x(1% 0s wait:2s done:1s)
0:53:59 INF -     write    2x(0% 0s wait:2s done:1s)
0:53:59 INF -   water_polygons   37s cpu:20m36s gc:1s avg:33.6
0:53:59 INF -     read     2x(22% 8s wait:8s done:20s)
0:53:59 INF -     process 62x(49% 18s wait:18s)
0:53:59 INF -     write    2x(63% 23s sys:5s wait:13s)
0:53:59 INF -   natural_earth    20s cpu:34s avg:1.7
0:53:59 INF -     read     2x(34% 7s sys:1s done:13s)
0:53:59 INF -     process 62x(1% 0.1s wait:14s done:6s)
0:53:59 INF -     write    2x(0% 0s wait:15s done:6s)
0:53:59 INF -   osm_pass1        1m54s cpu:1h38m44s gc:7s avg:52
0:53:59 INF -     read     1x(2% 3s sys:1s wait:1m40s done:8s)
0:53:59 INF -     process 63x(77% 1m28s sys:8s block:14s)
0:53:59 INF -   osm_pass2        22m30s cpu:22h2m15s gc:1m24s avg:58.8
0:53:59 INF -     read     1x(0% 4s sys:2s wait:21m44s done:38s)
0:53:59 INF -     process 62x(85% 19m8s sys:2s block:7s wait:23s done:6s)
0:53:59 INF -     write    2x(51% 11m23s sys:2m14s wait:7m30s)
0:53:59 INF -   boundaries       17s cpu:27s avg:1.6
0:53:59 INF -   sort             5m8s cpu:1h15m46s gc:20s avg:14.7
0:53:59 INF -     worker 19x(11% 35s sys:13s wait:2m54s done:13s)
0:53:59 INF -   archive          23m2s cpu:19h18m42s gc:1m46s avg:50.3
0:53:59 INF -     read    2x(20% 4m42s sys:1m32s wait:14m8s done:11s)
0:53:59 INF -     merge   1x(23% 5m21s sys:11s wait:16m11s done:6s)
0:53:59 INF -     encode 62x(74% 17m3s sys:12s wait:4m8s done:6s)
0:53:59 INF -     write   1x(39% 8m55s sys:2m5s wait:12m32s)
0:53:59 INF - ----------------------------------------

unordered:

0:55:20 INF - ----------------------------------------
0:55:20 INF -   overall          55m20s cpu:44h50m9s gc:3m41s avg:48.6
0:55:20 INF -   lake_centerlines 3s cpu:9s avg:2.9
0:55:20 INF -     read     2x(28% 0.9s done:2s)
0:55:20 INF -     process 62x(1% 0s wait:2s done:1s)
0:55:20 INF -     write    2x(0% 0s wait:2s done:1s)
0:55:20 INF -   water_polygons   38s cpu:20m50s gc:1s avg:32.5
0:55:20 INF -     read     2x(22% 8s wait:9s done:21s)
0:55:20 INF -     process 62x(47% 18s sys:1s wait:19s)
0:55:20 INF -     write    2x(64% 25s sys:5s wait:13s)
0:55:20 INF -   natural_earth    20s cpu:34s avg:1.7
0:55:20 INF -     read     2x(35% 7s sys:1s done:13s)
0:55:20 INF -     process 62x(1% 0.1s wait:14s done:6s)
0:55:20 INF -     write    2x(0% 0s wait:14s done:6s)
0:55:20 INF -   osm_pass1        1m49s cpu:1h34m17s gc:7s avg:51.8
0:55:20 INF -     read     1x(3% 3s sys:2s wait:1m32s done:7s)
0:55:20 INF -     process 63x(77% 1m24s sys:6s block:14s)
0:55:20 INF -   osm_pass2        22m22s cpu:21h54m57s gc:1m23s avg:58.8
0:55:20 INF -     read     1x(0% 3s sys:1s wait:21m41s done:38s)
0:55:20 INF -     process 62x(85% 19m4s sys:2s block:7s wait:29s)
0:55:20 INF -     write    2x(50% 11m14s sys:2m8s wait:7m36s)
0:55:20 INF -   boundaries       17s cpu:25s avg:1.5
0:55:20 INF -   sort             5m11s cpu:1h23m53s gc:19s avg:16.2
0:55:20 INF -     worker 19x(11% 35s sys:13s wait:2m59s done:10s)
0:55:20 INF -   archive          24m34s cpu:19h34m57s gc:1m50s avg:47.8
0:55:20 INF -     read     2x(19% 4m44s sys:1m33s wait:15m42s done:27s)
0:55:20 INF -     merge    1x(22% 5m19s sys:11s wait:17m48s done:18s)
0:55:20 INF -     encoder 62x(70% 17m13s sys:12s wait:5m25s done:8s)
0:55:20 INF -     write    1x(45% 11m6s sys:2m12s wait:12m7s)
0:55:20 INF - ----------------------------------------

github-actions · 2023-01-27T04:01:07Z

Base 9ce9d1f This Branch 7597645

0:01:54 DEB [archive] - Tile stats:
0:01:54 DEB [archive] - z0 avg:7.9k max:7.9k
0:01:54 DEB [archive] - z1 avg:4k max:4k
0:01:54 DEB [archive] - z2 avg:9.4k max:9.4k
0:01:54 DEB [archive] - z3 avg:4k max:6.4k
0:01:54 DEB [archive] - z4 avg:1.6k max:4.6k
0:01:54 DEB [archive] - z5 avg:1.4k max:8.2k
0:01:54 DEB [archive] - z6 avg:993 max:22k
0:01:54 DEB [archive] - z7 avg:776 max:59k
0:01:54 DEB [archive] - z8 avg:416 max:126k
0:01:54 DEB [archive] - z9 avg:280 max:296k
0:01:54 DEB [archive] - z10 avg:160 max:254k
0:01:54 DEB [archive] - z11 avg:105 max:136k
0:01:54 DEB [archive] - z12 avg:83 max:106k
0:01:54 DEB [archive] - z13 avg:71 max:109k
0:01:54 DEB [archive] - z14 avg:67 max:284k
0:01:54 DEB [archive] - all avg:70 max:296k
0:01:54 DEB [archive] -  # features: 5,426,116
0:01:54 DEB [archive] -     # tiles: 4,115,452
0:01:54 INF [archive] - Finished in 29s cpu:57s gc:1s avg:2
0:01:54 INF [archive] -   read    1x(2% 0.5s wait:28s)
0:01:54 INF [archive] -   encode  2x(62% 18s)
0:01:54 INF [archive] -   write   1x(13% 4s wait:23s)
0:01:54 INF - Finished in 1m54s cpu:3m20s gc:3s avg:1.8
0:01:54 INF - FINISHED!
0:01:54 INF - 
0:01:54 INF - ----------------------------------------
0:01:54 INF - 	overall          1m54s cpu:3m20s gc:3s avg:1.8
0:01:54 INF - 	lake_centerlines 3s cpu:6s avg:1.9
0:01:54 INF - 	  read     1x(50% 2s done:1s)
0:01:54 INF - 	  process  2x(6% 0.2s wait:2s done:1s)
0:01:54 INF - 	  write    1x(0% 0s wait:2s done:1s)
0:01:54 INF - 	water_polygons   25s cpu:44s gc:2s avg:1.8
0:01:54 INF - 	  read     1x(59% 15s wait:2s)
0:01:54 INF - 	  process  2x(27% 7s wait:12s)
0:01:54 INF - 	  write    1x(2% 0.5s wait:24s)
0:01:54 INF - 	natural_earth    21s cpu:25s avg:1.2
0:01:54 INF - 	  read     1x(63% 13s sys:2s done:6s)
0:01:54 INF - 	  process  2x(6% 1s wait:14s done:6s)
0:01:54 INF - 	  write    1x(0% 0s wait:15s done:6s)
0:01:54 INF - 	osm_pass1        3s cpu:6s avg:1.8
0:01:54 INF - 	  read     1x(1% 0s wait:3s)
0:01:54 INF - 	  parse    1x(69% 2s)
0:01:54 INF - 	  process  1x(42% 1s wait:1s)
0:01:54 INF - 	osm_pass2        30s cpu:59s avg:2
0:01:54 INF - 	  read     1x(0% 0s wait:15s done:14s)
0:01:54 INF - 	  process  2x(73% 22s)
0:01:54 INF - 	  write    1x(1% 0.3s wait:29s)
0:01:54 INF - 	boundaries       0s cpu:0.1s avg:2
0:01:54 INF - 	sort             2s cpu:3s avg:1.3
0:01:54 INF - 	  worker  1x(78% 2s)
0:01:54 INF - 	archive          29s cpu:57s gc:1s avg:2
0:01:54 INF - 	  read    1x(2% 0.5s wait:28s)
0:01:54 INF - 	  encode  2x(62% 18s)
0:01:54 INF - 	  write   1x(13% 4s wait:23s)
0:01:54 INF - ----------------------------------------
0:01:54 INF - 	archive	108MB
0:01:54 INF - 	features	276MB
-rw-r--r-- 1 runner docker 65M Jan 27 12:31 run.jar

0:01:54 DEB [archive] - Tile stats:
0:01:54 DEB [archive] - z0 avg:7.9k max:7.9k
0:01:54 DEB [archive] - z1 avg:4k max:4k
0:01:54 DEB [archive] - z2 avg:9.4k max:9.4k
0:01:54 DEB [archive] - z3 avg:4k max:6.4k
0:01:54 DEB [archive] - z4 avg:1.6k max:4.6k
0:01:54 DEB [archive] - z5 avg:1.4k max:8.2k
0:01:54 DEB [archive] - z6 avg:993 max:22k
0:01:54 DEB [archive] - z7 avg:776 max:59k
0:01:54 DEB [archive] - z8 avg:416 max:126k
0:01:54 DEB [archive] - z9 avg:280 max:296k
0:01:54 DEB [archive] - z10 avg:160 max:254k
0:01:54 DEB [archive] - z11 avg:105 max:136k
0:01:54 DEB [archive] - z12 avg:83 max:106k
0:01:54 DEB [archive] - z13 avg:71 max:109k
0:01:54 DEB [archive] - z14 avg:67 max:284k
0:01:54 DEB [archive] - all avg:70 max:296k
0:01:54 DEB [archive] -  # features: 5,426,116
0:01:54 DEB [archive] -     # tiles: 4,115,452
0:01:54 INF [archive] - Finished in 28s cpu:56s gc:1s avg:2
0:01:54 INF [archive] -   read    1x(2% 0.5s wait:27s)
0:01:54 INF [archive] -   encode  2x(63% 18s wait:1s)
0:01:54 INF [archive] -   write   1x(14% 4s wait:23s)
0:01:54 INF - Finished in 1m54s cpu:3m20s gc:3s avg:1.8
0:01:54 INF - FINISHED!
0:01:54 INF - 
0:01:54 INF - ----------------------------------------
0:01:54 INF - 	overall          1m54s cpu:3m20s gc:3s avg:1.8
0:01:54 INF - 	lake_centerlines 4s cpu:6s avg:1.7
0:01:54 INF - 	  read     1x(43% 2s done:2s)
0:01:54 INF - 	  process  2x(7% 0.2s wait:2s done:2s)
0:01:54 INF - 	  write    1x(0% 0s wait:2s done:2s)
0:01:54 INF - 	water_polygons   25s cpu:45s gc:2s avg:1.8
0:01:54 INF - 	  read     1x(58% 14s wait:3s)
0:01:54 INF - 	  process  2x(28% 7s wait:11s)
0:01:54 INF - 	  write    1x(2% 0.5s wait:24s)
0:01:54 INF - 	natural_earth    21s cpu:24s avg:1.2
0:01:54 INF - 	  read     1x(63% 13s sys:2s done:7s)
0:01:54 INF - 	  process  2x(6% 1s wait:13s done:7s)
0:01:54 INF - 	  write    1x(0% 0s wait:14s done:7s)
0:01:54 INF - 	osm_pass1        3s cpu:6s avg:1.8
0:01:54 INF - 	  read     1x(1% 0s wait:3s)
0:01:54 INF - 	  parse    1x(68% 2s)
0:01:54 INF - 	  process  1x(44% 1s wait:2s)
0:01:54 INF - 	osm_pass2        30s cpu:1m avg:2
0:01:54 INF - 	  read     1x(0% 0s wait:15s done:15s)
0:01:54 INF - 	  process  2x(75% 22s)
0:01:54 INF - 	  write    1x(1% 0.3s wait:30s)
0:01:54 INF - 	boundaries       0s cpu:0.1s avg:1.7
0:01:54 INF - 	sort             2s cpu:2s avg:1.2
0:01:54 INF - 	  worker  1x(73% 2s)
0:01:54 INF - 	archive          28s cpu:56s gc:1s avg:2
0:01:54 INF - 	  read    1x(2% 0.5s wait:27s)
0:01:54 INF - 	  encode  2x(63% 18s wait:1s)
0:01:54 INF - 	  write   1x(14% 4s wait:23s)
0:01:54 INF - ----------------------------------------
0:01:54 INF - 	archive	108MB
0:01:54 INF - 	features	276MB
-rw-r--r-- 1 runner docker 65M Jan 27 12:29 run.jar

https://github.com/onthegomap/planetiler/actions/runs/4024463087

ℹ️ Base Logs 9ce9d1f

0:00:00 DEB - argument: config=null (path to config file)
0:00:00 DEB - argument: area=rhode island (name of the extract to download if osm_url/osm_path not specified (i.e. 'monaco' 'rhode island' 'australia' or 'planet'))
0:00:00 INF - argument: stats=use in-memory stats
0:00:00 DEB - argument: madvise=true (default value for whether to use linux madvise(random) to improve memory-mapped read performance for temporary storage)
0:00:00 DEB - argument: storage=mmap (default storage type for temporary data, one of [ram, mmap, direct])
0:00:00 DEB - argument: threads=2 (num threads)
0:00:00 DEB - argument: write_threads=1 (number of threads to use when writing temp features)
0:00:00 DEB - argument: process_threads=2 (number of threads to use when processing input features)
0:00:00 DEB - argument: bounds=Env[-74.07 : -17.84, 21.34 : 43.55] (bounds)
0:00:00 DEB - argument: polygon=null (a .poly file that limits output to tiles intersecting the shape)
0:00:00 DEB - argument: minzoom=0 (minimum zoom level)
0:00:00 DEB - argument: maxzoom=14 (maximum zoom level up to 15)
0:00:00 DEB - argument: render_maxzoom=14 (maximum rendering zoom level up to 15)
0:00:00 DEB - argument: feature_read_threads=1 (number of threads to use when reading features at tile write time)
0:00:00 DEB - argument: loginterval=10 seconds (time between logs)
0:00:00 DEB - argument: skip_mbtiles_index_creation=false (skip adding index to mbtiles file)
0:00:00 DEB - argument: optimize_db=false (Vacuum analyze mbtiles after writing)
0:00:00 DEB - argument: emit_tiles_in_order=true (emit tiles in index order)
0:00:00 DEB - argument: force=false (overwriting output file and ignore disk/RAM warnings)
0:00:00 DEB - argument: gzip_temp=false (gzip temporary feature storage (uses more CPU, but less disk space))
0:00:00 DEB - argument: mmap_temp=true (use memory-mapped IO for temp feature files)
0:00:00 DEB - argument: sort_max_readers=6 (maximum number of concurrent read threads to use when sorting chunks)
0:00:00 DEB - argument: sort_max_writers=6 (maximum number of concurrent write threads to use when sorting chunks)
0:00:00 DEB - argument: nodemap_type=sparsearray (type of node location map, one of [noop, sortedtable, sparsearray, array])
0:00:00 DEB - argument: nodemap_storage=mmap (storage for node location map, one of [ram, mmap, direct])
0:00:00 DEB - argument: nodemap_madvise=true (use linux madvise(random) for node locations)
0:00:00 DEB - argument: multipolygon_geometry_storage=mmap (storage for multipolygon geometries, one of [ram, mmap, direct])
0:00:00 DEB - argument: multipolygon_geometry_madvise=true (use linux madvise(random) for temporary multipolygon geometry storage)
0:00:00 DEB - argument: http_user_agent=Planetiler downloader (https://github.com/onthegomap/planetiler) (User-Agent header to set when downloading files over HTTP)
0:00:00 DEB - argument: http_timeout=30 seconds (Timeout to use when downloading files over HTTP)
0:00:00 DEB - argument: http_retries=1 (Retries to use when downloading files over HTTP)
0:00:00 DEB - argument: download_chunk_size_mb=100 (Size of file chunks to download in parallel in megabytes)
0:00:00 DEB - argument: download_threads=1 (Number of parallel threads to use when downloading each file)
0:00:00 DEB - argument: min_feature_size_at_max_zoom=0.0625 (Default value for the minimum size in tile pixels of features to emit at the maximum zoom level to allow for overzooming)
0:00:00 DEB - argument: min_feature_size=1.0 (Default value for the minimum size in tile pixels of features to emit below the maximum zoom level)
0:00:00 DEB - argument: simplify_tolerance_at_max_zoom=0.0625 (Default value for the tile pixel tolerance to use when simplifying features at the maximum zoom level to allow for overzooming)
0:00:00 DEB - argument: simplify_tolerance=0.1 (Default value for the tile pixel tolerance to use when simplifying features below the maximum zoom level)
0:00:00 DEB - argument: osm_lazy_reads=true (Read OSM blocks from disk in worker threads)
0:00:00 DEB - argument: compact_db=true (Reduce the DB size by separating and deduping the tile data)
0:00:00 DEB - argument: skip_filled_tiles=false (Skip writing tiles containing only polygon fills to the output)
0:00:00 DEB - argument: tile_warning_size_mb=1.0 (Maximum size in megabytes of a tile to emit a warning about)
0:00:00 DEB - argument: tmpdir=data/tmp (temp directory)
0:00:00 DEB - argument: only_download=false (download source data then exit)
0:00:00 DEB - argument: download=false (download sources)
0:00:00 DEB - argument: temp_nodes=data/tmp/node.db (temp node db location)
0:00:00 DEB - argument: temp_multipolygons=data/tmp/multipolygon.db (temp multipolygon db location)
0:00:00 DEB - argument: temp_features=data/tmp/feature.db (temp feature db location)
0:00:00 DEB - argument: osm_parse_node_bounds=false (parse bounds from OSM nodes instead of header)
0:00:00 DEB - argument: only_fetch_wikidata=false (fetch wikidata translations then quit)
0:00:00 DEB - argument: fetch_wikidata=false (fetch wikidata translations then continue)
0:00:00 DEB - argument: use_wikidata=true (use wikidata translations)
0:00:00 DEB - argument: wikidata_cache=data/sources/wikidata_names.json (wikidata cache file)
0:00:00 DEB - argument: lake_centerlines_path=data/sources/lake_centerline.shp.zip (lake_centerlines shapefile path)
0:00:00 DEB - argument: free_lake_centerlines_after_read=false (delete lake_centerlines input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: water_polygons_path=data/sources/water-polygons-split-3857.zip (water_polygons shapefile path)
0:00:00 DEB - argument: free_water_polygons_after_read=false (delete water_polygons input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: natural_earth_path=data/sources/natural_earth_vector.sqlite.zip (natural_earth sqlite db path)
0:00:00 DEB - argument: free_natural_earth_after_read=false (delete natural_earth input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: osm_path=data/sources/rhode_island.osm.pbf (osm OSM input file path)
0:00:00 DEB - argument: free_osm_after_read=false (delete osm input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: mbtiles=data/out.mbtiles (output tile archive)
0:00:00 DEB - argument: version=false (show version then exit)
0:00:00 INF - Planetiler build git hash: 9ce9d1f5b2eed8668da3084df22906079df6a191
0:00:00 INF - Planetiler build version: 0.6-SNAPSHOT
0:00:00 INF - Planetiler build timestamp: 2023-01-27T12:29:28.371Z
0:00:00 DEB - argument: transliterate=true (attempt to transliterate latin names)
0:00:00 DEB - argument: languages=am,ar,az,be,bg,br,bs,ca,co,cs,cy,da,de,el,en,eo,es,et,eu,fi,fr,fy,ga,gd,he,hi,hr,hu,hy,id,is,it,ja,ja_kana,ja_rm,ja-Latn,ja-Hira,ka,kk,kn,ko,ko-Latn,ku,la,lb,lt,lv,mk,mt,ml,nl,no,oc,pl,pt,rm,ro,ru,sk,sl,sq,sr,sr-Latn,sv,ta,te,th,tr,uk,zh (languages to use)
0:00:00 DEB - argument: only_layers= (Include only certain layers)
0:00:00 DEB - argument: exclude_layers= (Exclude certain layers)
0:00:00 DEB - argument: boundary_country_names=true (boundary layer: add left/right codes of neighboring countries)
0:00:00 DEB - argument: transportation_z13_paths=false (transportation(_name) layer: show all paths on z13)
0:00:00 DEB - argument: building_merge_z13=true (building layer: merge nearby buildings at z13)
0:00:00 DEB - argument: transportation_name_brunnel=false (transportation_name layer: set to false to omit brunnel and help merge long highways)
0:00:00 DEB - argument: transportation_name_size_for_shield=false (transportation_name layer: allow road names on shorter segments (ie. they will have a shield))
0:00:00 DEB - argument: transportation_name_limit_merge=false (transportation_name layer: limit merge so we don't combine different relations to help merge long highways)
0:00:00 DEB - argument: transportation_name_minor_refs=false (transportation_name layer: include name and refs from minor road networks if not present on a way)
0:00:00 DEB - argument: mbtiles_name=OpenMapTiles ('name' attribute for tileset metadata)
0:00:00 DEB - argument: mbtiles_description=A tileset showcasing all layers in OpenMapTiles. https://openmaptiles.org ('description' attribute for tileset metadata)
0:00:00 DEB - argument: mbtiles_attribution=<a href="https://www.openmaptiles.org/" target="_blank">&copy; OpenMapTiles</a> <a href="https://www.openstreetmap.org/copyright" target="_blank">&copy; OpenStreetMap contributors</a> ('attribution' attribute for tileset metadata)
0:00:00 DEB - argument: mbtiles_version=3.14.0 ('version' attribute for tileset metadata)
0:00:00 DEB - argument: mbtiles_type=baselayer ('type' attribute for tileset metadata)
0:00:00 DEB - argument: help=false (show arguments then exit)
0:00:00 INF - Building OpenMapTilesProfile profile into data/out.mbtiles in these phases:
0:00:00 INF -   lake_centerlines: Process features in data/sources/lake_centerline.shp.zip
0:00:00 INF -   water_polygons: Process features in data/sources/water-polygons-split-3857.zip
0:00:00 INF -   natural_earth: Process features in data/sources/natural_earth_vector.sqlite.zip
0:00:00 INF -   osm_pass1: Pre-process OpenStreetMap input (store node locations then relation members)
0:00:00 INF -   osm_pass2: Process OpenStreetMap nodes, ways, then relations
0:00:00 INF -   sort: Sort rendered features by tile ID
0:00:00 INF -   archive: Encode each tile and write to data/out.mbtiles
0:00:00 INF - no wikidata translations found, run with --fetch-wikidata to download
0:00:00 DEB - ✓ 197M storage on / (/dev/root) requested for read phase disk, 35G available
0:00:00 DEB -  - 44M used for temporary node location cache
0:00:00 DEB -  - 6.7M used for temporary multipolygon geometry cache
0:00:00 DEB -  - 145M used for temporary feature storage
0:00:00 DEB - ✓ 218M storage on / (/dev/root) requested for write phase disk, 35G available
0:00:00 DEB -  - 145M used for temporary feature storage
0:00:00 DEB -  - 72M used for archive output
0:00:00 DEB - ✓ 313M JVM heap requested for read phase, 4.2G available
0:00:00 DEB -  - 300M used for sparsearray node location in-memory index
0:00:00 DEB -  - 13M used for temporary profile storage
0:00:00 DEB - ✓ 51M storage on / (/dev/root) requested for read phase, 35G available
0:00:00 DEB -  - 44M used for sparsearray node location cache
0:00:00 DEB -  - 6.7M used for multipolygon way geometries
0:00:00 DEB - ✓ 51M temporary files and 2.9G of free memory for OS to cache them
0:00:00 INF - Using merge sort feature map, chunk size=1431mb max workers=2
0:00:00 INF [lake_centerlines] - 
0:00:00 INF [lake_centerlines] - Starting...
0:00:04 INF [lake_centerlines] -  read: [  59k 100%  28k/s ] write: [    0    0/s ] 0    
    cpus: 1.8 gc:  4% heap: 172M/4.2G direct: 237k postGC: 78M
    ->     (0/3) -> read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:04 INF [lake_centerlines] - Finished in 3s cpu:6s avg:1.9
0:00:04 INF [lake_centerlines] -   read     1x(50% 2s done:1s)
0:00:04 INF [lake_centerlines] -   process  2x(6% 0.2s wait:2s done:1s)
0:00:04 INF [lake_centerlines] -   write    1x(0% 0s wait:2s done:1s)
0:00:04 INF [water_polygons] - 
0:00:04 INF [water_polygons] - Starting...
0:00:14 INF [water_polygons] -  read: [ 2.2k  15%  222/s ] write: [  47k 4.6k/s ] 1.4G 
    cpus: 2 gc: 10% heap: 985M/4.2G direct: 51M postGC: 885M
    ->     (0/3) -> read(58%) ->   (20/1k) -> process(41% 15%) -> (1.3k/53k) -> write( 0%)
0:00:24 INF [water_polygons] -  read: [ 5.4k  38%  320/s ] write: [ 300k  25k/s ] 1.4G 
    cpus: 1.7 gc:  7% heap: 3.4G/4.2G direct: 51M postGC: 1.3G
    ->     (0/3) -> read(67%) ->    (3/1k) -> process(25% 24%) -> (251/53k) -> write( 0%)
0:00:29 INF [water_polygons] -  read: [  14k 100% 1.8k/s ] write: [ 4.3M 806k/s ] 186M 
    cpus: 1.5 gc:  7% heap: 3.1G/4.2G direct: 51M postGC: 1.5G
    ->     (0/3) -> read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( 9%)
0:00:29 INF [water_polygons] - Finished in 25s cpu:44s gc:2s avg:1.8
0:00:29 INF [water_polygons] -   read     1x(59% 15s wait:2s)
0:00:29 INF [water_polygons] -   process  2x(27% 7s wait:12s)
0:00:29 INF [water_polygons] -   write    1x(2% 0.5s wait:24s)
0:00:29 INF [natural_earth] - 
0:00:29 INF [natural_earth] - Starting...
0:00:29 INF [natural_earth] - unzipping /home/runner/work/planetiler/planetiler/data/sources/natural_earth_vector.sqlite.zip to data/tmp/natearth.sqlite
0:00:35 INF [natural_earth] - unzipping /home/runner/work/planetiler/planetiler/data/sources/natural_earth_vector.sqlite.zip to data/tmp/natearth.sqlite
0:00:45 INF [natural_earth] -  read: [  90k  26%   9k/s ] write: [    0    0/s ] 1.6G 
    cpus: 1.2 gc:  0% heap: 2.8G/4.2G direct: 51M postGC: 1.5G
    ->     (0/3) -> read(94%) ->    (1/1k) -> process( 3%  3%) -> (106/53k) -> write( 0%)
0:00:50 INF [natural_earth] -  read: [ 349k 100%  51k/s ] write: [  181   36/s ] 186M 
    cpus: 1.4 gc:  0% heap: 2.2G/4.2G direct: 51M postGC: 1.5G
    ->     (0/3) -> read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:50 INF [natural_earth] - Finished in 21s cpu:25s avg:1.2
0:00:50 INF [natural_earth] -   read     1x(63% 13s sys:2s done:6s)
0:00:50 INF [natural_earth] -   process  2x(6% 1s wait:14s done:6s)
0:00:50 INF [natural_earth] -   write    1x(0% 0s wait:15s done:6s)
0:00:50 INF [osm_pass1] - 
0:00:50 INF [osm_pass1] - Starting...
0:00:52 INF [osm_pass1:process] - Finished nodes: 4,635,548 (2M/s) in 2s cpu:4s avg:1.7
0:00:53 INF [osm_pass1:process] - Finished ways: 339,122 (484k/s) in 0.7s cpu:1s avg:2
0:00:53 INF [osm_pass1:process] - Finished relations: 7,952 (72k/s) in 0.1s cpu:0.2s avg:1.9
0:00:53 INF [osm_pass1] -  nodes: [ 4.6M 1.4M/s ] 440M  ways: [ 339k 108k/s ] rels: [ 7.9k 2.5k/s ] blocks: [  624  200/s ]
    cpus: 1.8 gc:  0% heap: 2.2G/4.2G direct: 51M postGC: 2.1G hppc: 466k
    read( -%) ->     (0/4) -> parse( -%) ->     (0/4) -> process( -%)
0:00:53 DEB [osm_pass1] - Processed 624 blocks:
0:00:53 DEB [osm_pass1] -   nodes: 4,635,548 (2M/s) in 2s cpu:4s avg:1.7
0:00:53 DEB [osm_pass1] -   ways: 339,122 (484k/s) in 0.7s cpu:1s avg:2
0:00:53 DEB [osm_pass1] -   relations: 7,952 (72k/s) in 0.1s cpu:0.2s avg:1.9
0:00:53 INF [osm_pass1] - Finished in 3s cpu:6s avg:1.8
0:00:53 INF [osm_pass1] -   read     1x(1% 0s wait:3s)
0:00:53 INF [osm_pass1] -   parse    1x(69% 2s)
0:00:53 INF [osm_pass1] -   process  1x(42% 1s wait:1s)
0:00:53 INF [osm_pass2] - 
0:00:53 INF [osm_pass2] - Starting...
0:00:55 DEB [osm_pass2:process] - Sorting long long multimap...
0:00:56 INF [osm_pass2:process] - Finished nodes: 4,635,548 (1.9M/s) in 2s cpu:5s avg:2
0:00:56 DEB [osm_pass2:process] - Sorted long long multimap 0s cpu:0s avg:1.6
0:00:56 WAR [osm_pass2:process] - No GB polygon for inferring route network types
0:01:03 INF [osm_pass2] -  nodes: [ 4.6M 100% 463k/s ] 440M  ways: [ 193k  57%  19k/s ] rels: [    0   0%    0/s ] features: [ 4.9M  65k/s ] 1.6G  blocks: [  603  97%   60/s ]
    cpus: 2 gc:  1% heap: 3G/4.2G direct: 51M postGC: 903M relInfo: 426k mpGeoms: 649k 
    read( 0%) ->   (11/13) -> process(64% 63%) ->  (1k/53k) -> write( 2%)
0:01:11 INF [osm_pass2:process] - Finished ways: 339,122 (21k/s) in 16s cpu:31s avg:2
0:01:13 INF [osm_pass2] -  nodes: [ 4.6M 100%    0/s ] 440M  ways: [ 339k 100%  14k/s ] rels: [ 2.1k  27%  216/s ] features: [ 5.3M  41k/s ] 1.6G  blocks: [  623 100%    1/s ]
    cpus: 2 gc:  1% heap: 1.7G/4.2G direct: 51M postGC: 904M relInfo: 426k mpGeoms: 19M  
    read( -%) ->    (0/13) -> process(77% 77%) -> (581/53k) -> write( 1%)
0:01:21 INF [osm_pass2:process] - Finished relations: 7,952 (862/s) in 9s cpu:18s avg:2
0:01:23 INF [osm_pass2] -  nodes: [ 4.6M 100%    0/s ] 440M  ways: [ 339k 100%    0/s ] rels: [ 7.9k 100%  602/s ] features: [ 5.4M 3.3k/s ] 276M  blocks: [  624 100%   <1/s ]
    cpus: 2 gc:  1% heap: 1.5G/4.2G direct: 51M postGC: 896M relInfo: 426k mpGeoms: 19M  
    read( -%) ->    (0/13) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:01:23 DEB [osm_pass2] - Processed 624 blocks:
0:01:23 DEB [osm_pass2] -   nodes: 4,635,548 (1.9M/s) in 2s cpu:5s avg:2
0:01:23 DEB [osm_pass2] -   ways: 339,122 (21k/s) in 16s cpu:31s avg:2
0:01:23 DEB [osm_pass2] -   relations: 7,952 (862/s) in 9s cpu:18s avg:2
0:01:23 INF [osm_pass2] - Finished in 30s cpu:59s avg:2
0:01:23 INF [osm_pass2] -   read     1x(0% 0s wait:15s done:14s)
0:01:23 INF [osm_pass2] -   process  2x(73% 22s)
0:01:23 INF [osm_pass2] -   write    1x(1% 0.3s wait:29s)
0:01:23 INF [boundaries] - 
0:01:23 INF [boundaries] - Starting...
0:01:23 INF [boundaries] - Creating polygons for 1 boundaries
0:01:23 WAR [boundaries] - Unable to form closed polygon for OSM relation 148838 (likely missing edges)
0:01:23 INF [boundaries] - Finished creating 0 country polygons
0:01:23 INF [boundaries] - Finished in 0s cpu:0.1s avg:2
0:01:23 INF - Deleting node.db to make room for output file
0:01:23 INF [sort] - 
0:01:23 INF [sort] - Starting...
0:01:23 INF [sort] - Grouped 8 chunks into 1
0:01:25 INF [sort] -  chunks: [   1 /   1 100% ] 276M 
    cpus: 1.3 gc:  0% heap: 2G/4.2G direct: 51M postGC: 896M
    ->     (0/3) -> worker( -%)
0:01:25 INF [sort] - Finished in 2s cpu:3s avg:1.3
0:01:25 INF [sort] -   worker  1x(78% 2s)
0:01:25 INF - read:0s write:0s sort:0s
0:01:25 INF [archive] - 
0:01:25 INF [archive] - Starting...
0:01:25 DEB [archive:write] - Execute mbtiles: create table metadata (name text, value text);
0:01:25 DEB [archive:write] - Execute mbtiles: create unique index name on metadata (name);
0:01:25 DEB [archive:write] - Execute mbtiles: create table tiles_shallow (
  zoom_level integer,
  tile_column integer,
  tile_row integer,
  tile_data_id integer

  , primary key(zoom_level,tile_column,tile_row)

) without rowid

0:01:25 DEB [archive:write] - Execute mbtiles: create table tiles_data (
  tile_data_id integer primary key,
  tile_data blob
)

0:01:25 DEB [archive:write] - Execute mbtiles: create view tiles AS
select
  tiles_shallow.zoom_level as zoom_level,
  tiles_shallow.tile_column as tile_column,
  tiles_shallow.tile_row as tile_row,
  tiles_data.tile_data as tile_data
from tiles_shallow
join tiles_data on tiles_shallow.tile_data_id = tiles_data.tile_data_id

0:01:25 DEB [archive:write] - Set mbtiles metadata: name=OpenMapTiles
0:01:26 DEB [archive:write] - Set mbtiles metadata: format=pbf
0:01:26 DEB [archive:write] - Set mbtiles metadata: description=A tileset showcasing all layers in OpenMapTiles. https://openmaptiles.org
0:01:26 DEB [archive:write] - Set mbtiles metadata: attribution=<a href="https://www.openmaptiles.org/" target="_blank">&copy; OpenMapTiles</a> <a href="https://www.openstreetmap.org/copyright" target="_blank">&copy; OpenStreetMap contributors</a>
0:01:26 DEB [archive:write] - Set mbtiles metadata: version=3.14.0
0:01:26 DEB [archive:write] - Set mbtiles metadata: type=baselayer
0:01:26 DEB [archive:write] - Set mbtiles metadata: bounds=-74.07,21.34,-17.84,43.55
0:01:26 DEB [archive:write] - Set mbtiles metadata: center=-45.955,32.445,3
0:01:26 DEB [archive:write] - Set mbtiles metadata: minzoom=0
0:01:26 DEB [archive:write] - Set mbtiles metadata: maxzoom=14
0:01:26 DEB [archive:write] - Set mbtiles metadata: json={"vector_layers":[{"id":"aerodrome_label","fields":{"name_int":"String","iata":"String","ele_ft":"Number","name_de":"String","name":"String","icao":"String","name:en":"String","class":"String","ele":"Number","name_en":"String","name:latin":"String"},"minzoom":10,"maxzoom":14},{"id":"aeroway","fields":{"ref":"String","class":"String"},"minzoom":10,"maxzoom":14},{"id":"boundary","fields":{"disputed":"Number","admin_level":"Number","maritime":"Number"},"minzoom":0,"maxzoom":14},{"id":"building","fields":{"colour":"String","render_height":"Number","render_min_height":"Number","hide_3d":"Boolean"},"minzoom":13,"maxzoom":14},{"id":"housenumber","fields":{"housenumber":"String"},"minzoom":14,"maxzoom":14},{"id":"landcover","fields":{"subclass":"String","class":"String","_numpoints":"Number"},"minzoom":7,"maxzoom":14},{"id":"landuse","fields":{"class":"String"},"minzoom":4,"maxzoom":14},{"id":"mountain_peak","fields":{"name_int":"String","customary_ft":"Number","ele_ft":"Number","name_de":"Str... 2302 more characters
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:version=0.6-SNAPSHOT
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:githash=9ce9d1f5b2eed8668da3084df22906079df6a191
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:buildtime=2023-01-27T12:29:28.371Z
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:osm:osmosisreplicationtime=2023-01-26T21:21:14Z
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:osm:osmosisreplicationseq=3592
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:osm:osmosisreplicationurl=http://download.geofabrik.de/north-america/us/rhode-island-updates
0:01:28 INF [archive:write] - Starting z0
0:01:28 INF [archive:write] - Finished z0 in 0s cpu:0s avg:0, now starting z1
0:01:28 INF [archive:write] - Finished z1 in 0s cpu:0s avg:0, now starting z2
0:01:28 INF [archive:write] - Finished z2 in 0s cpu:0s avg:0, now starting z3
0:01:28 INF [archive:write] - Finished z3 in 0s cpu:0s avg:0, now starting z4
0:01:28 INF [archive:write] - Finished z4 in 0s cpu:0s avg:0, now starting z5
0:01:28 INF [archive:write] - Finished z5 in 0s cpu:0s avg:0, now starting z6
0:01:28 INF [archive:write] - Finished z6 in 0s cpu:0s avg:0, now starting z7
0:01:28 INF [archive:write] - Finished z7 in 0.6s cpu:1s avg:2, now starting z8
0:01:30 INF [archive:write] - Finished z8 in 2s cpu:3s avg:2, now starting z9
0:01:33 INF [archive:write] - Finished z9 in 3s cpu:6s avg:2, now starting z10
0:01:34 INF [archive:write] - Finished z10 in 1s cpu:2s avg:2, now starting z11
0:01:35 INF [archive] -  features: [ 192k   4%  18k/s ] 276M  tiles: [  18k 1.8k/s ] 2.5M 
    cpus: 2 gc:  5% heap: 1.7G/4.2G direct: 51M postGC: 1.4G
    read( 1%) -> (214/217) -> encode(60% 62%) -> (215/216) -> write( 1%)
    last tile: 11/617/762 (z11 5%) https://www.openstreetmap.org/#map=11/41.77131/-71.54297
0:01:37 INF [archive:write] - Finished z11 in 3s cpu:5s avg:1.9, now starting z12
0:01:40 INF [archive:write] - Finished z12 in 4s cpu:8s avg:2, now starting z13
0:01:45 INF [archive] -  features: [ 855k  16%  66k/s ] 276M  tiles: [ 291k  27k/s ] 16M  
    cpus: 2 gc:  2% heap: 3.4G/4.2G direct: 51M postGC: 1.5G
    read( 1%) -> (214/217) -> encode(67% 66%) -> (215/216) -> write( 3%)
    last tile: 13/2470/3047 (z13 5%) https://www.openstreetmap.org/#map=13/41.80408/-71.45508
0:01:49 INF [archive:write] - Finished z13 in 8s cpu:17s avg:2, now starting z14
0:01:54 DEB [archive:write] - Shallow tiles written: 4,115,452
0:01:54 DEB [archive:write] - Tile data written: 18,366 (100% omitted)
0:01:54 DEB [archive:write] - Unique tile hashes: 8,447
0:01:54 INF [archive:write] - Finished z14 in 5s cpu:10s avg:1.9
0:01:54 INF [archive] -  features: [ 5.4M 100% 536k/s ] 276M  tiles: [ 4.1M 449k/s ] 108M 
    cpus: 1.9 gc:  3% heap: 2G/4.2G direct: 51M postGC: 1.5G
    read( -%) ->   (0/217) -> encode( -%  -%) ->   (0/216) -> write(40%)
    last tile: 14/7380/5985 (z14 100%) https://www.openstreetmap.org/#map=14/43.56447/-17.84180
0:01:54 DEB [archive] - Tile stats:
0:01:54 DEB [archive] - z0 avg:7.9k max:7.9k
0:01:54 DEB [archive] - z1 avg:4k max:4k
0:01:54 DEB [archive] - z2 avg:9.4k max:9.4k
0:01:54 DEB [archive] - z3 avg:4k max:6.4k
0:01:54 DEB [archive] - z4 avg:1.6k max:4.6k
0:01:54 DEB [archive] - z5 avg:1.4k max:8.2k
0:01:54 DEB [archive] - z6 avg:993 max:22k
0:01:54 DEB [archive] - z7 avg:776 max:59k
0:01:54 DEB [archive] - z8 avg:416 max:126k
0:01:54 DEB [archive] - z9 avg:280 max:296k
0:01:54 DEB [archive] - z10 avg:160 max:254k
0:01:54 DEB [archive] - z11 avg:105 max:136k
0:01:54 DEB [archive] - z12 avg:83 max:106k
0:01:54 DEB [archive] - z13 avg:71 max:109k
0:01:54 DEB [archive] - z14 avg:67 max:284k
0:01:54 DEB [archive] - all avg:70 max:296k
0:01:54 DEB [archive] -  # features: 5,426,116
0:01:54 DEB [archive] -     # tiles: 4,115,452
0:01:54 INF [archive] - Finished in 29s cpu:57s gc:1s avg:2
0:01:54 INF [archive] -   read    1x(2% 0.5s wait:28s)
0:01:54 INF [archive] -   encode  2x(62% 18s)
0:01:54 INF [archive] -   write   1x(13% 4s wait:23s)
0:01:54 INF - Finished in 1m54s cpu:3m20s gc:3s avg:1.8
0:01:54 INF - FINISHED!
0:01:54 INF - 
0:01:54 INF - ----------------------------------------
0:01:54 INF - 	overall          1m54s cpu:3m20s gc:3s avg:1.8
0:01:54 INF - 	lake_centerlines 3s cpu:6s avg:1.9
0:01:54 INF - 	  read     1x(50% 2s done:1s)
0:01:54 INF - 	  process  2x(6% 0.2s wait:2s done:1s)
0:01:54 INF - 	  write    1x(0% 0s wait:2s done:1s)
0:01:54 INF - 	water_polygons   25s cpu:44s gc:2s avg:1.8
0:01:54 INF - 	  read     1x(59% 15s wait:2s)
0:01:54 INF - 	  process  2x(27% 7s wait:12s)
0:01:54 INF - 	  write    1x(2% 0.5s wait:24s)
0:01:54 INF - 	natural_earth    21s cpu:25s avg:1.2
0:01:54 INF - 	  read     1x(63% 13s sys:2s done:6s)
0:01:54 INF - 	  process  2x(6% 1s wait:14s done:6s)
0:01:54 INF - 	  write    1x(0% 0s wait:15s done:6s)
0:01:54 INF - 	osm_pass1        3s cpu:6s avg:1.8
0:01:54 INF - 	  read     1x(1% 0s wait:3s)
0:01:54 INF - 	  parse    1x(69% 2s)
0:01:54 INF - 	  process  1x(42% 1s wait:1s)
0:01:54 INF - 	osm_pass2        30s cpu:59s avg:2
0:01:54 INF - 	  read     1x(0% 0s wait:15s done:14s)
0:01:54 INF - 	  process  2x(73% 22s)
0:01:54 INF - 	  write    1x(1% 0.3s wait:29s)
0:01:54 INF - 	boundaries       0s cpu:0.1s avg:2
0:01:54 INF - 	sort             2s cpu:3s avg:1.3
0:01:54 INF - 	  worker  1x(78% 2s)
0:01:54 INF - 	archive          29s cpu:57s gc:1s avg:2
0:01:54 INF - 	  read    1x(2% 0.5s wait:28s)
0:01:54 INF - 	  encode  2x(62% 18s)
0:01:54 INF - 	  write   1x(13% 4s wait:23s)
0:01:54 INF - ----------------------------------------
0:01:54 INF - 	archive	108MB
0:01:54 INF - 	features	276MB
-rw-r--r-- 1 runner docker 65M Jan 27 12:31 run.jar

ℹ️ This Branch Logs 7597645

0:00:00 DEB - argument: config=null (path to config file)
0:00:00 DEB - argument: area=rhode island (name of the extract to download if osm_url/osm_path not specified (i.e. 'monaco' 'rhode island' 'australia' or 'planet'))
0:00:00 INF - argument: stats=use in-memory stats
0:00:00 DEB - argument: madvise=true (default value for whether to use linux madvise(random) to improve memory-mapped read performance for temporary storage)
0:00:00 DEB - argument: storage=mmap (default storage type for temporary data, one of [ram, mmap, direct])
0:00:00 DEB - argument: threads=2 (num threads)
0:00:00 DEB - argument: write_threads=1 (number of threads to use when writing temp features)
0:00:00 DEB - argument: process_threads=2 (number of threads to use when processing input features)
0:00:00 DEB - argument: bounds=Env[-74.07 : -17.84, 21.34 : 43.55] (bounds)
0:00:00 DEB - argument: polygon=null (a .poly file that limits output to tiles intersecting the shape)
0:00:00 DEB - argument: minzoom=0 (minimum zoom level)
0:00:00 DEB - argument: maxzoom=14 (maximum zoom level up to 15)
0:00:00 DEB - argument: render_maxzoom=14 (maximum rendering zoom level up to 15)
0:00:00 DEB - argument: feature_read_threads=1 (number of threads to use when reading features at tile write time)
0:00:00 DEB - argument: loginterval=10 seconds (time between logs)
0:00:00 DEB - argument: skip_mbtiles_index_creation=false (skip adding index to mbtiles file)
0:00:00 DEB - argument: optimize_db=false (Vacuum analyze mbtiles after writing)
0:00:00 DEB - argument: force=false (overwriting output file and ignore disk/RAM warnings)
0:00:00 DEB - argument: gzip_temp=false (gzip temporary feature storage (uses more CPU, but less disk space))
0:00:00 DEB - argument: mmap_temp=true (use memory-mapped IO for temp feature files)
0:00:00 DEB - argument: sort_max_readers=6 (maximum number of concurrent read threads to use when sorting chunks)
0:00:00 DEB - argument: sort_max_writers=6 (maximum number of concurrent write threads to use when sorting chunks)
0:00:00 DEB - argument: nodemap_type=sparsearray (type of node location map, one of [noop, sortedtable, sparsearray, array])
0:00:00 DEB - argument: nodemap_storage=mmap (storage for node location map, one of [ram, mmap, direct])
0:00:00 DEB - argument: nodemap_madvise=true (use linux madvise(random) for node locations)
0:00:00 DEB - argument: multipolygon_geometry_storage=mmap (storage for multipolygon geometries, one of [ram, mmap, direct])
0:00:00 DEB - argument: multipolygon_geometry_madvise=true (use linux madvise(random) for temporary multipolygon geometry storage)
0:00:00 DEB - argument: http_user_agent=Planetiler downloader (https://github.com/onthegomap/planetiler) (User-Agent header to set when downloading files over HTTP)
0:00:00 DEB - argument: http_timeout=30 seconds (Timeout to use when downloading files over HTTP)
0:00:00 DEB - argument: http_retries=1 (Retries to use when downloading files over HTTP)
0:00:00 DEB - argument: download_chunk_size_mb=100 (Size of file chunks to download in parallel in megabytes)
0:00:00 DEB - argument: download_threads=1 (Number of parallel threads to use when downloading each file)
0:00:00 DEB - argument: min_feature_size_at_max_zoom=0.0625 (Default value for the minimum size in tile pixels of features to emit at the maximum zoom level to allow for overzooming)
0:00:00 DEB - argument: min_feature_size=1.0 (Default value for the minimum size in tile pixels of features to emit below the maximum zoom level)
0:00:00 DEB - argument: simplify_tolerance_at_max_zoom=0.0625 (Default value for the tile pixel tolerance to use when simplifying features at the maximum zoom level to allow for overzooming)
0:00:00 DEB - argument: simplify_tolerance=0.1 (Default value for the tile pixel tolerance to use when simplifying features below the maximum zoom level)
0:00:00 DEB - argument: osm_lazy_reads=true (Read OSM blocks from disk in worker threads)
0:00:00 DEB - argument: compact_db=true (Reduce the DB size by separating and deduping the tile data)
0:00:00 DEB - argument: skip_filled_tiles=false (Skip writing tiles containing only polygon fills to the output)
0:00:00 DEB - argument: tile_warning_size_mb=1.0 (Maximum size in megabytes of a tile to emit a warning about)
0:00:00 DEB - argument: tmpdir=data/tmp (temp directory)
0:00:00 DEB - argument: only_download=false (download source data then exit)
0:00:00 DEB - argument: download=false (download sources)
0:00:00 DEB - argument: temp_nodes=data/tmp/node.db (temp node db location)
0:00:00 DEB - argument: temp_multipolygons=data/tmp/multipolygon.db (temp multipolygon db location)
0:00:00 DEB - argument: temp_features=data/tmp/feature.db (temp feature db location)
0:00:00 DEB - argument: osm_parse_node_bounds=false (parse bounds from OSM nodes instead of header)
0:00:00 DEB - argument: only_fetch_wikidata=false (fetch wikidata translations then quit)
0:00:00 DEB - argument: fetch_wikidata=false (fetch wikidata translations then continue)
0:00:00 DEB - argument: use_wikidata=true (use wikidata translations)
0:00:00 DEB - argument: wikidata_cache=data/sources/wikidata_names.json (wikidata cache file)
0:00:00 DEB - argument: lake_centerlines_path=data/sources/lake_centerline.shp.zip (lake_centerlines shapefile path)
0:00:00 DEB - argument: free_lake_centerlines_after_read=false (delete lake_centerlines input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: water_polygons_path=data/sources/water-polygons-split-3857.zip (water_polygons shapefile path)
0:00:00 DEB - argument: free_water_polygons_after_read=false (delete water_polygons input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: natural_earth_path=data/sources/natural_earth_vector.sqlite.zip (natural_earth sqlite db path)
0:00:00 DEB - argument: free_natural_earth_after_read=false (delete natural_earth input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: osm_path=data/sources/rhode_island.osm.pbf (osm OSM input file path)
0:00:00 DEB - argument: free_osm_after_read=false (delete osm input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: mbtiles=data/out.mbtiles (output tile archive)
0:00:00 DEB - argument: version=false (show version then exit)
0:00:00 INF - Planetiler build git hash: 75976451ed502c3696ccf239a385bc120706f79d
0:00:00 INF - Planetiler build version: 0.6-SNAPSHOT
0:00:00 INF - Planetiler build timestamp: 2023-01-27T12:28:50.252Z
0:00:00 DEB - argument: transliterate=true (attempt to transliterate latin names)
0:00:00 DEB - argument: languages=am,ar,az,be,bg,br,bs,ca,co,cs,cy,da,de,el,en,eo,es,et,eu,fi,fr,fy,ga,gd,he,hi,hr,hu,hy,id,is,it,ja,ja_kana,ja_rm,ja-Latn,ja-Hira,ka,kk,kn,ko,ko-Latn,ku,la,lb,lt,lv,mk,mt,ml,nl,no,oc,pl,pt,rm,ro,ru,sk,sl,sq,sr,sr-Latn,sv,ta,te,th,tr,uk,zh (languages to use)
0:00:00 DEB - argument: only_layers= (Include only certain layers)
0:00:00 DEB - argument: exclude_layers= (Exclude certain layers)
0:00:00 DEB - argument: boundary_country_names=true (boundary layer: add left/right codes of neighboring countries)
0:00:00 DEB - argument: transportation_z13_paths=false (transportation(_name) layer: show all paths on z13)
0:00:00 DEB - argument: building_merge_z13=true (building layer: merge nearby buildings at z13)
0:00:00 DEB - argument: transportation_name_brunnel=false (transportation_name layer: set to false to omit brunnel and help merge long highways)
0:00:00 DEB - argument: transportation_name_size_for_shield=false (transportation_name layer: allow road names on shorter segments (ie. they will have a shield))
0:00:00 DEB - argument: transportation_name_limit_merge=false (transportation_name layer: limit merge so we don't combine different relations to help merge long highways)
0:00:00 DEB - argument: transportation_name_minor_refs=false (transportation_name layer: include name and refs from minor road networks if not present on a way)
0:00:00 DEB - argument: mbtiles_name=OpenMapTiles ('name' attribute for tileset metadata)
0:00:00 DEB - argument: mbtiles_description=A tileset showcasing all layers in OpenMapTiles. https://openmaptiles.org ('description' attribute for tileset metadata)
0:00:00 DEB - argument: mbtiles_attribution=<a href="https://www.openmaptiles.org/" target="_blank">&copy; OpenMapTiles</a> <a href="https://www.openstreetmap.org/copyright" target="_blank">&copy; OpenStreetMap contributors</a> ('attribution' attribute for tileset metadata)
0:00:00 DEB - argument: mbtiles_version=3.14.0 ('version' attribute for tileset metadata)
0:00:00 DEB - argument: mbtiles_type=baselayer ('type' attribute for tileset metadata)
0:00:00 DEB - argument: help=false (show arguments then exit)
0:00:00 INF - Building OpenMapTilesProfile profile into data/out.mbtiles in these phases:
0:00:00 INF -   lake_centerlines: Process features in data/sources/lake_centerline.shp.zip
0:00:00 INF -   water_polygons: Process features in data/sources/water-polygons-split-3857.zip
0:00:00 INF -   natural_earth: Process features in data/sources/natural_earth_vector.sqlite.zip
0:00:00 INF -   osm_pass1: Pre-process OpenStreetMap input (store node locations then relation members)
0:00:00 INF -   osm_pass2: Process OpenStreetMap nodes, ways, then relations
0:00:00 INF -   sort: Sort rendered features by tile ID
0:00:00 INF -   archive: Encode each tile and write to data/out.mbtiles
0:00:00 INF - no wikidata translations found, run with --fetch-wikidata to download
0:00:00 DEB - ✓ 197M storage on / (/dev/root) requested for read phase disk, 35G available
0:00:00 DEB -  - 44M used for temporary node location cache
0:00:00 DEB -  - 6.7M used for temporary multipolygon geometry cache
0:00:00 DEB -  - 145M used for temporary feature storage
0:00:00 DEB - ✓ 218M storage on / (/dev/root) requested for write phase disk, 35G available
0:00:00 DEB -  - 145M used for temporary feature storage
0:00:00 DEB -  - 72M used for archive output
0:00:00 DEB - ✓ 313M JVM heap requested for read phase, 4.2G available
0:00:00 DEB -  - 300M used for sparsearray node location in-memory index
0:00:00 DEB -  - 13M used for temporary profile storage
0:00:00 DEB - ✓ 51M storage on / (/dev/root) requested for read phase, 35G available
0:00:00 DEB -  - 44M used for sparsearray node location cache
0:00:00 DEB -  - 6.7M used for multipolygon way geometries
0:00:00 DEB - ✓ 51M temporary files and 2.9G of free memory for OS to cache them
0:00:00 INF - Using merge sort feature map, chunk size=1431mb max workers=2
0:00:00 INF [lake_centerlines] - 
0:00:00 INF [lake_centerlines] - Starting...
0:00:04 INF [lake_centerlines] -  read: [  59k 100%  31k/s ] write: [    0    0/s ] 0    
    cpus: 1.8 gc:  3% heap: 172M/4.2G direct: 237k postGC: 77M
    ->     (0/3) -> read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:04 INF [lake_centerlines] - Finished in 4s cpu:6s avg:1.7
0:00:04 INF [lake_centerlines] -   read     1x(43% 2s done:2s)
0:00:04 INF [lake_centerlines] -   process  2x(7% 0.2s wait:2s done:2s)
0:00:04 INF [lake_centerlines] -   write    1x(0% 0s wait:2s done:2s)
0:00:04 INF [water_polygons] - 
0:00:04 INF [water_polygons] - Starting...
0:00:14 INF [water_polygons] -  read: [ 2.2k  15%  222/s ] write: [  51k   5k/s ] 1.4G 
    cpus: 2 gc: 10% heap: 1.4G/4.2G direct: 51M postGC: 1G
    ->     (0/3) -> read(55%) ->   (25/1k) -> process(43% 15%) -> (1.6k/53k) -> write( 0%)
0:00:24 INF [water_polygons] -  read: [ 5.5k  39%  334/s ] write: [ 300k  24k/s ] 1.4G 
    cpus: 1.7 gc:  6% heap: 2.8G/4.2G direct: 51M postGC: 1.4G
    ->     (0/3) -> read(69%) ->    (0/1k) -> process(23% 25%) -> (595/53k) -> write( 0%)
0:00:29 INF [water_polygons] -  read: [  14k 100% 1.8k/s ] write: [ 4.3M 826k/s ] 186M 
    cpus: 1.6 gc: 10% heap: 3G/4.2G direct: 51M postGC: 1.6G
    ->     (0/3) -> read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:29 INF [water_polygons] - Finished in 25s cpu:45s gc:2s avg:1.8
0:00:29 INF [water_polygons] -   read     1x(58% 14s wait:3s)
0:00:29 INF [water_polygons] -   process  2x(28% 7s wait:11s)
0:00:29 INF [water_polygons] -   write    1x(2% 0.5s wait:24s)
0:00:29 INF [natural_earth] - 
0:00:29 INF [natural_earth] - Starting...
0:00:29 INF [natural_earth] - unzipping /home/runner/work/planetiler/planetiler/data/sources/natural_earth_vector.sqlite.zip to data/tmp/natearth.sqlite
0:00:36 INF [natural_earth] - unzipping /home/runner/work/planetiler/planetiler/data/sources/natural_earth_vector.sqlite.zip to data/tmp/natearth.sqlite
0:00:46 INF [natural_earth] -  read: [ 130k  37%  13k/s ] write: [    0    0/s ] 1.6G 
    cpus: 1.2 gc:  0% heap: 2.9G/4.2G direct: 51M postGC: 1.6G
    ->     (0/3) -> read(98%) ->  (363/1k) -> process( 5%  5%) -> (108/53k) -> write( 0%)
0:00:50 INF [natural_earth] -  read: [ 349k 100%  62k/s ] write: [  181   51/s ] 186M 
    cpus: 1.7 gc:  0% heap: 2.2G/4.2G direct: 51M postGC: 1.6G
    ->     (0/3) -> read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:50 INF [natural_earth] - Finished in 21s cpu:24s avg:1.2
0:00:50 INF [natural_earth] -   read     1x(63% 13s sys:2s done:7s)
0:00:50 INF [natural_earth] -   process  2x(6% 1s wait:13s done:7s)
0:00:50 INF [natural_earth] -   write    1x(0% 0s wait:14s done:7s)
0:00:50 INF [osm_pass1] - 
0:00:50 INF [osm_pass1] - Starting...
0:00:52 INF [osm_pass1:process] - Finished nodes: 4,635,548 (1.9M/s) in 2s cpu:4s avg:1.7
0:00:53 INF [osm_pass1:process] - Finished ways: 339,122 (433k/s) in 0.8s cpu:2s avg:2
0:00:53 INF [osm_pass1:process] - Finished relations: 7,952 (102k/s) in 0.1s cpu:0.2s avg:2.1
0:00:53 INF [osm_pass1] -  nodes: [ 4.6M 1.3M/s ] 440M  ways: [ 339k 101k/s ] rels: [ 7.9k 2.3k/s ] blocks: [  624  187/s ]
    cpus: 1.8 gc:  1% heap: 936M/4.2G direct: 51M postGC: 878M hppc: 466k
    read( -%) ->     (0/4) -> parse( -%) ->     (0/4) -> process( -%)
0:00:53 DEB [osm_pass1] - Processed 624 blocks:
0:00:53 DEB [osm_pass1] -   nodes: 4,635,548 (1.9M/s) in 2s cpu:4s avg:1.7
0:00:53 DEB [osm_pass1] -   ways: 339,122 (433k/s) in 0.8s cpu:2s avg:2
0:00:53 DEB [osm_pass1] -   relations: 7,952 (102k/s) in 0.1s cpu:0.2s avg:2.1
0:00:53 INF [osm_pass1] - Finished in 3s cpu:6s avg:1.8
0:00:53 INF [osm_pass1] -   read     1x(1% 0s wait:3s)
0:00:53 INF [osm_pass1] -   parse    1x(68% 2s)
0:00:53 INF [osm_pass1] -   process  1x(44% 1s wait:2s)
0:00:53 INF [osm_pass2] - 
0:00:53 INF [osm_pass2] - Starting...
0:00:55 DEB [osm_pass2:process] - Sorting long long multimap...
0:00:55 DEB [osm_pass2:process] - Sorted long long multimap 0s cpu:0s avg:1.4
0:00:55 INF [osm_pass2:process] - Finished nodes: 4,635,548 (2M/s) in 2s cpu:4s avg:2
0:00:55 WAR [osm_pass2:process] - No GB polygon for inferring route network types
0:01:03 INF [osm_pass2] -  nodes: [ 4.6M 100% 463k/s ] 440M  ways: [ 193k  57%  19k/s ] rels: [    0   0%    0/s ] features: [ 4.9M  65k/s ] 1.6G  blocks: [  604  97%   60/s ]
    cpus: 2 gc:  1% heap: 1.1G/4.2G direct: 51M postGC: 892M relInfo: 426k mpGeoms: 649k 
    read( 0%) ->   (11/13) -> process(66% 60%) -> (921/53k) -> write( 2%)
0:01:11 INF [osm_pass2:process] - Finished ways: 339,122 (20k/s) in 16s cpu:32s avg:2
0:01:13 INF [osm_pass2] -  nodes: [ 4.6M 100%    0/s ] 440M  ways: [ 339k 100%  14k/s ] rels: [ 2.1k  27%  215/s ] features: [ 5.3M  41k/s ] 1.6G  blocks: [  623 100%    1/s ]
    cpus: 2 gc:  1% heap: 2.2G/4.2G direct: 51M postGC: 893M relInfo: 426k mpGeoms: 19M  
    read( -%) ->    (0/13) -> process(79% 78%) -> (1.2k/53k) -> write( 1%)
0:01:20 INF [osm_pass2:process] - Finished relations: 7,952 (873/s) in 9s cpu:18s avg:2
0:01:23 INF [osm_pass2] -  nodes: [ 4.6M 100%    0/s ] 440M  ways: [ 339k 100%    0/s ] rels: [ 7.9k 100%  583/s ] features: [ 5.4M 3.3k/s ] 276M  blocks: [  624 100%   <1/s ]
    cpus: 2 gc:  1% heap: 2.2G/4.2G direct: 51M postGC: 892M relInfo: 426k mpGeoms: 19M  
    read( -%) ->    (0/13) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:01:23 DEB [osm_pass2] - Processed 624 blocks:
0:01:23 DEB [osm_pass2] -   nodes: 4,635,548 (2M/s) in 2s cpu:4s avg:2
0:01:23 DEB [osm_pass2] -   ways: 339,122 (20k/s) in 16s cpu:32s avg:2
0:01:23 DEB [osm_pass2] -   relations: 7,952 (873/s) in 9s cpu:18s avg:2
0:01:23 INF [osm_pass2] - Finished in 30s cpu:1m avg:2
0:01:23 INF [osm_pass2] -   read     1x(0% 0s wait:15s done:15s)
0:01:23 INF [osm_pass2] -   process  2x(75% 22s)
0:01:23 INF [osm_pass2] -   write    1x(1% 0.3s wait:30s)
0:01:23 INF [boundaries] - 
0:01:23 INF [boundaries] - Starting...
0:01:23 INF [boundaries] - Creating polygons for 1 boundaries
0:01:23 WAR [boundaries] - Unable to form closed polygon for OSM relation 148838 (likely missing edges)
0:01:23 INF [boundaries] - Finished creating 0 country polygons
0:01:23 INF [boundaries] - Finished in 0s cpu:0.1s avg:1.7
0:01:23 INF - Deleting node.db to make room for output file
0:01:23 INF [sort] - 
0:01:23 INF [sort] - Starting...
0:01:23 INF [sort] - Grouped 8 chunks into 1
0:01:25 INF [sort] -  chunks: [   1 /   1 100% ] 276M 
    cpus: 1.2 gc:  0% heap: 2.7G/4.2G direct: 51M postGC: 892M
    ->     (0/3) -> worker( -%)
0:01:25 INF [sort] - Finished in 2s cpu:2s avg:1.2
0:01:25 INF [sort] -   worker  1x(73% 2s)
0:01:25 INF - read:0s write:0s sort:0s
0:01:25 INF [archive] - 
0:01:25 INF [archive] - Starting...
0:01:26 DEB [archive:write] - Execute mbtiles: create table metadata (name text, value text);
0:01:26 DEB [archive:write] - Execute mbtiles: create unique index name on metadata (name);
0:01:26 DEB [archive:write] - Execute mbtiles: create table tiles_shallow (
  zoom_level integer,
  tile_column integer,
  tile_row integer,
  tile_data_id integer

  , primary key(zoom_level,tile_column,tile_row)

) without rowid

0:01:26 DEB [archive:write] - Execute mbtiles: create table tiles_data (
  tile_data_id integer primary key,
  tile_data blob
)

0:01:26 DEB [archive:write] - Execute mbtiles: create view tiles AS
select
  tiles_shallow.zoom_level as zoom_level,
  tiles_shallow.tile_column as tile_column,
  tiles_shallow.tile_row as tile_row,
  tiles_data.tile_data as tile_data
from tiles_shallow
join tiles_data on tiles_shallow.tile_data_id = tiles_data.tile_data_id

0:01:26 DEB [archive:write] - Set mbtiles metadata: name=OpenMapTiles
0:01:26 DEB [archive:write] - Set mbtiles metadata: format=pbf
0:01:26 DEB [archive:write] - Set mbtiles metadata: description=A tileset showcasing all layers in OpenMapTiles. https://openmaptiles.org
0:01:26 DEB [archive:write] - Set mbtiles metadata: attribution=<a href="https://www.openmaptiles.org/" target="_blank">&copy; OpenMapTiles</a> <a href="https://www.openstreetmap.org/copyright" target="_blank">&copy; OpenStreetMap contributors</a>
0:01:26 DEB [archive:write] - Set mbtiles metadata: version=3.14.0
0:01:26 DEB [archive:write] - Set mbtiles metadata: type=baselayer
0:01:26 DEB [archive:write] - Set mbtiles metadata: bounds=-74.07,21.34,-17.84,43.55
0:01:26 DEB [archive:write] - Set mbtiles metadata: center=-45.955,32.445,3
0:01:26 DEB [archive:write] - Set mbtiles metadata: minzoom=0
0:01:26 DEB [archive:write] - Set mbtiles metadata: maxzoom=14
0:01:26 DEB [archive:write] - Set mbtiles metadata: json={"vector_layers":[{"id":"aerodrome_label","fields":{"name_int":"String","iata":"String","ele_ft":"Number","name_de":"String","name":"String","icao":"String","name:en":"String","class":"String","ele":"Number","name_en":"String","name:latin":"String"},"minzoom":10,"maxzoom":14},{"id":"aeroway","fields":{"ref":"String","class":"String"},"minzoom":10,"maxzoom":14},{"id":"boundary","fields":{"disputed":"Number","admin_level":"Number","maritime":"Number"},"minzoom":0,"maxzoom":14},{"id":"building","fields":{"colour":"String","render_height":"Number","render_min_height":"Number","hide_3d":"Boolean"},"minzoom":13,"maxzoom":14},{"id":"housenumber","fields":{"housenumber":"String"},"minzoom":14,"maxzoom":14},{"id":"landcover","fields":{"subclass":"String","class":"String","_numpoints":"Number"},"minzoom":7,"maxzoom":14},{"id":"landuse","fields":{"class":"String"},"minzoom":4,"maxzoom":14},{"id":"mountain_peak","fields":{"name_int":"String","customary_ft":"Number","ele_ft":"Number","name_de":"Str... 2302 more characters
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:version=0.6-SNAPSHOT
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:githash=75976451ed502c3696ccf239a385bc120706f79d
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:buildtime=2023-01-27T12:28:50.252Z
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:osm:osmosisreplicationtime=2023-01-26T21:21:14Z
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:osm:osmosisreplicationseq=3592
0:01:26 DEB [archive:write] - Set mbtiles metadata: planetiler:osm:osmosisreplicationurl=http://download.geofabrik.de/north-america/us/rhode-island-updates
0:01:28 INF [archive:write] - Starting z0
0:01:28 INF [archive:write] - Finished z0 in 0s cpu:0s avg:0, now starting z1
0:01:28 INF [archive:write] - Finished z1 in 0s cpu:0s avg:0, now starting z2
0:01:28 INF [archive:write] - Finished z2 in 0s cpu:0s avg:0, now starting z3
0:01:28 INF [archive:write] - Finished z3 in 0s cpu:0s avg:110.7, now starting z4
0:01:28 INF [archive:write] - Finished z4 in 0s cpu:0s avg:0, now starting z5
0:01:28 INF [archive:write] - Finished z5 in 0s cpu:0s avg:0, now starting z6
0:01:28 INF [archive:write] - Finished z6 in 0s cpu:0s avg:0, now starting z7
0:01:28 INF [archive:write] - Finished z7 in 0.7s cpu:1s avg:2, now starting z8
0:01:30 INF [archive:write] - Finished z8 in 2s cpu:4s avg:2, now starting z9
0:01:33 INF [archive:write] - Finished z9 in 3s cpu:6s avg:2, now starting z10
0:01:34 INF [archive:write] - Finished z10 in 1s cpu:2s avg:2, now starting z11
0:01:36 INF [archive] -  features: [ 189k   3%  18k/s ] 276M  tiles: [  18k 1.8k/s ] 2.5M 
    cpus: 2 gc:  3% heap: 3.1G/4.2G direct: 51M postGC: 1.4G
    read( 1%) -> (214/217) -> encode(61% 62%) -> (215/216) -> write( 1%)
    last tile: 11/617/762 (z11 5%) https://www.openstreetmap.org/#map=11/41.77131/-71.54297
0:01:37 INF [archive:write] - Finished z11 in 3s cpu:5s avg:2, now starting z12
0:01:40 INF [archive:write] - Finished z12 in 4s cpu:8s avg:2, now starting z13
0:01:46 INF [archive] -  features: [ 860k  16%  67k/s ] 276M  tiles: [ 291k  27k/s ] 14M  
    cpus: 2 gc:  2% heap: 2.6G/4.2G direct: 51M postGC: 1.5G
    read( 1%) -> (213/217) -> encode(69% 67%) -> (215/216) -> write( 3%)
    last tile: 13/2470/3047 (z13 5%) https://www.openstreetmap.org/#map=13/41.80408/-71.45508
0:01:49 INF [archive:write] - Finished z13 in 8s cpu:16s avg:2, now starting z14
0:01:54 DEB [archive:write] - Shallow tiles written: 4,115,452
0:01:54 DEB [archive:write] - Tile data written: 18,468 (100% omitted)
0:01:54 DEB [archive:write] - Unique tile hashes: 8,549
0:01:54 INF [archive:write] - Finished z14 in 5s cpu:9s avg:1.9
0:01:54 INF [archive] -  features: [ 5.4M 100% 568k/s ] 276M  tiles: [ 4.1M 476k/s ] 108M 
    cpus: 1.9 gc:  3% heap: 3.3G/4.2G direct: 51M postGC: 1.5G
    read( -%) ->   (0/217) -> encode( -%  -%) ->   (0/216) -> write( -%)
    last tile: 14/7380/5985 (z14 100%) https://www.openstreetmap.org/#map=14/43.56447/-17.84180
0:01:54 DEB [archive] - Tile stats:
0:01:54 DEB [archive] - z0 avg:7.9k max:7.9k
0:01:54 DEB [archive] - z1 avg:4k max:4k
0:01:54 DEB [archive] - z2 avg:9.4k max:9.4k
0:01:54 DEB [archive] - z3 avg:4k max:6.4k
0:01:54 DEB [archive] - z4 avg:1.6k max:4.6k
0:01:54 DEB [archive] - z5 avg:1.4k max:8.2k
0:01:54 DEB [archive] - z6 avg:993 max:22k
0:01:54 DEB [archive] - z7 avg:776 max:59k
0:01:54 DEB [archive] - z8 avg:416 max:126k
0:01:54 DEB [archive] - z9 avg:280 max:296k
0:01:54 DEB [archive] - z10 avg:160 max:254k
0:01:54 DEB [archive] - z11 avg:105 max:136k
0:01:54 DEB [archive] - z12 avg:83 max:106k
0:01:54 DEB [archive] - z13 avg:71 max:109k
0:01:54 DEB [archive] - z14 avg:67 max:284k
0:01:54 DEB [archive] - all avg:70 max:296k
0:01:54 DEB [archive] -  # features: 5,426,116
0:01:54 DEB [archive] -     # tiles: 4,115,452
0:01:54 INF [archive] - Finished in 28s cpu:56s gc:1s avg:2
0:01:54 INF [archive] -   read    1x(2% 0.5s wait:27s)
0:01:54 INF [archive] -   encode  2x(63% 18s wait:1s)
0:01:54 INF [archive] -   write   1x(14% 4s wait:23s)
0:01:54 INF - Finished in 1m54s cpu:3m20s gc:3s avg:1.8
0:01:54 INF - FINISHED!
0:01:54 INF - 
0:01:54 INF - ----------------------------------------
0:01:54 INF - 	overall          1m54s cpu:3m20s gc:3s avg:1.8
0:01:54 INF - 	lake_centerlines 4s cpu:6s avg:1.7
0:01:54 INF - 	  read     1x(43% 2s done:2s)
0:01:54 INF - 	  process  2x(7% 0.2s wait:2s done:2s)
0:01:54 INF - 	  write    1x(0% 0s wait:2s done:2s)
0:01:54 INF - 	water_polygons   25s cpu:45s gc:2s avg:1.8
0:01:54 INF - 	  read     1x(58% 14s wait:3s)
0:01:54 INF - 	  process  2x(28% 7s wait:11s)
0:01:54 INF - 	  write    1x(2% 0.5s wait:24s)
0:01:54 INF - 	natural_earth    21s cpu:24s avg:1.2
0:01:54 INF - 	  read     1x(63% 13s sys:2s done:7s)
0:01:54 INF - 	  process  2x(6% 1s wait:13s done:7s)
0:01:54 INF - 	  write    1x(0% 0s wait:14s done:7s)
0:01:54 INF - 	osm_pass1        3s cpu:6s avg:1.8
0:01:54 INF - 	  read     1x(1% 0s wait:3s)
0:01:54 INF - 	  parse    1x(68% 2s)
0:01:54 INF - 	  process  1x(44% 1s wait:2s)
0:01:54 INF - 	osm_pass2        30s cpu:1m avg:2
0:01:54 INF - 	  read     1x(0% 0s wait:15s done:15s)
0:01:54 INF - 	  process  2x(75% 22s)
0:01:54 INF - 	  write    1x(1% 0.3s wait:30s)
0:01:54 INF - 	boundaries       0s cpu:0.1s avg:1.7
0:01:54 INF - 	sort             2s cpu:2s avg:1.2
0:01:54 INF - 	  worker  1x(73% 2s)
0:01:54 INF - 	archive          28s cpu:56s gc:1s avg:2
0:01:54 INF - 	  read    1x(2% 0.5s wait:27s)
0:01:54 INF - 	  encode  2x(63% 18s wait:1s)
0:01:54 INF - 	  write   1x(14% 4s wait:23s)
0:01:54 INF - ----------------------------------------
0:01:54 INF - 	archive	108MB
0:01:54 INF - 	features	276MB
-rw-r--r-- 1 runner docker 65M Jan 27 12:29 run.jar

msbarry · 2023-01-27T10:51:18Z

Double check for any more references to emit_tiles_in_order, I see one in custtommap readme and planetiler.schema.json.

msbarry · 2023-01-27T10:53:16Z

planetiler-core/src/main/java/com/onthegomap/planetiler/util/VarInt.java

-   * @param v            the value to encode
-   * @param outputStream the OutputStream to add the encoded value
-   */
-  public static void putVarLong(long v, OutputStream outputStream) throws IOException {


I could see the outputstream version being useful in other contexts, but if the only purpose of it here is for pmtiles then seems fine to only include what it needs - we can always pull the old version out of git history.