Warn when running with insufficient memory or disk space

[msbarry]

Fail fast when trying to run planetiler with insufficient memory or disk space. Will just log a warning instead when you use the --force parameter. Also adds an option that lets you delete an input file with --free-osm-after-read before writing the mbtiles output file to reduce peak disk requirements Fixes #70

[github-actions]

Base 8a6997f

This Branch 8fcc787

0:02:14 DEB [mbtiles] - Tile stats: 0:02:14 DEB [mbtiles] - z0 avg:7.9k max:7.9k 0:02:14 DEB [mbtiles] - z1 avg:4k max:4k 0:02:14 DEB [mbtiles] - z2 avg:9.4k max:9.4k 0:02:14 DEB [mbtiles] - z3 avg:3.9k max:6.4k 0:02:14 DEB [mbtiles] - z4 avg:1.6k max:4.6k 0:02:14 DEB [mbtiles] - z5 avg:1.4k max:8.1k 0:02:14 DEB [mbtiles] - z6 avg:1.4k max:24k 0:02:14 DEB [mbtiles] - z7 avg:896 max:33k 0:02:14 DEB [mbtiles] - z8 avg:365 max:48k 0:02:14 DEB [mbtiles] - z9 avg:296 max:278k 0:02:14 DEB [mbtiles] - z10 avg:164 max:232k 0:02:14 DEB [mbtiles] - z11 avg:107 max:131k 0:02:14 DEB [mbtiles] - z12 avg:85 max:118k 0:02:14 DEB [mbtiles] - z13 avg:72 max:109k 0:02:14 DEB [mbtiles] - z14 avg:68 max:257k 0:02:14 DEB [mbtiles] - all avg:70 max:0 0:02:14 DEB [mbtiles] - # features: 5,276,185 0:02:14 DEB [mbtiles] - # tiles: 4,115,465 0:02:14 INF [mbtiles] - Finished in 31s cpu:56s gc:1s avg:1.8 0:02:14 INF [mbtiles] - read 1x(8% 2s wait:24s) 0:02:14 INF [mbtiles] - encode 2x(44% 14s wait:8s) 0:02:14 INF [mbtiles] - write 1x(36% 11s sys:1s wait:16s) 0:02:14 INF - Finished in 2m14s cpu:3m49s gc:4s avg:1.7 0:02:14 INF - FINISHED! 0:02:14 INF - ---------------------------------------- 0:02:14 INF - overall 2m14s cpu:3m49s gc:4s avg:1.7 0:02:14 INF - lake_centerlines 0.8s cpu:1s avg:1.7 0:02:14 INF - water_polygons 28s cpu:48s gc:2s avg:1.7 0:02:14 INF - read 1x(60% 17s sys:1s wait:3s) 0:02:14 INF - process 2x(28% 8s wait:14s) 0:02:14 INF - write 1x(4% 1s wait:27s) 0:02:14 INF - natural_earth 18s cpu:21s avg:1.2 0:02:14 INF - read 1x(64% 11s sys:1s done:5s) 0:02:14 INF - process 2x(13% 2s wait:12s done:5s) 0:02:14 INF - write 1x(0% 0s wait:13s done:5s) 0:02:14 INF - osm_pass1 4s cpu:8s avg:1.8 0:02:14 INF - read 1x(3% 0.1s wait:4s) 0:02:14 INF - parse 1x(65% 3s) 0:02:14 INF - process 1x(37% 2s wait:2s) 0:02:14 INF - osm_pass2 39s cpu:1m18s avg:2 0:02:14 INF - read 1x(0% 0.1s wait:4s done:35s) 0:02:14 INF - process 2x(76% 30s) 0:02:14 INF - write 1x(1% 0.4s wait:39s) 0:02:14 INF - boundaries 0s cpu:0.1s avg:1.2 0:02:14 INF - sort 4s cpu:5s avg:1.2 0:02:14 INF - worker 2x(43% 2s done:2s) 0:02:14 INF - mbtiles 31s cpu:56s gc:1s avg:1.8 0:02:14 INF - read 1x(8% 2s wait:24s) 0:02:14 INF - encode 2x(44% 14s wait:8s) 0:02:14 INF - write 1x(36% 11s sys:1s wait:16s) 0:02:14 INF - ---------------------------------------- 0:02:14 INF - features 268MB 0:02:14 INF - mbtiles 514MB -rw-r--r-- 1 runner docker 52M Mar 3 12:21 run.jar

0:02:12 DEB [mbtiles] - Tile stats: 0:02:12 DEB [mbtiles] - z0 avg:7.9k max:7.9k 0:02:12 DEB [mbtiles] - z1 avg:4k max:4k 0:02:12 DEB [mbtiles] - z2 avg:9.4k max:9.4k 0:02:12 DEB [mbtiles] - z3 avg:3.9k max:6.4k 0:02:12 DEB [mbtiles] - z4 avg:1.6k max:4.6k 0:02:12 DEB [mbtiles] - z5 avg:1.4k max:8.1k 0:02:12 DEB [mbtiles] - z6 avg:1.4k max:24k 0:02:12 DEB [mbtiles] - z7 avg:896 max:33k 0:02:12 DEB [mbtiles] - z8 avg:365 max:48k 0:02:12 DEB [mbtiles] - z9 avg:295 max:278k 0:02:12 DEB [mbtiles] - z10 avg:164 max:232k 0:02:12 DEB [mbtiles] - z11 avg:107 max:131k 0:02:12 DEB [mbtiles] - z12 avg:85 max:118k 0:02:12 DEB [mbtiles] - z13 avg:72 max:109k 0:02:12 DEB [mbtiles] - z14 avg:68 max:257k 0:02:12 DEB [mbtiles] - all avg:70 max:0 0:02:12 DEB [mbtiles] - # features: 5,276,185 0:02:12 DEB [mbtiles] - # tiles: 4,115,465 0:02:12 INF [mbtiles] - Finished in 31s cpu:57s gc:1s avg:1.8 0:02:12 INF [mbtiles] - read 1x(8% 2s wait:25s) 0:02:12 INF [mbtiles] - encode 2x(45% 14s wait:8s) 0:02:12 INF [mbtiles] - write 1x(36% 11s sys:1s wait:16s) 0:02:12 INF - Finished in 2m12s cpu:3m49s gc:4s avg:1.7 0:02:12 INF - FINISHED! 0:02:12 INF - ---------------------------------------- 0:02:12 INF - overall 2m12s cpu:3m49s gc:4s avg:1.7 0:02:12 INF - lake_centerlines 0.8s cpu:1s avg:1.7 0:02:12 INF - water_polygons 28s cpu:50s gc:2s avg:1.8 0:02:12 INF - read 1x(59% 17s sys:1s wait:3s) 0:02:12 INF - process 2x(29% 8s wait:12s) 0:02:12 INF - write 1x(4% 1s wait:27s) 0:02:12 INF - natural_earth 12s cpu:19s avg:1.5 0:02:12 INF - read 1x(90% 11s sys:1s) 0:02:12 INF - process 2x(19% 2s wait:11s) 0:02:12 INF - write 1x(0% 0s wait:12s) 0:02:12 INF - osm_pass1 4s cpu:7s avg:1.8 0:02:12 INF - read 1x(3% 0.1s wait:3s) 0:02:12 INF - parse 1x(71% 3s) 0:02:12 INF - process 1x(36% 1s wait:2s) 0:02:12 INF - osm_pass2 39s cpu:1m17s avg:2 0:02:12 INF - read 1x(0% 0.1s wait:4s done:35s) 0:02:12 INF - process 2x(75% 30s) 0:02:12 INF - write 1x(1% 0.4s wait:39s) 0:02:12 INF - boundaries 0s cpu:0.1s avg:1.8 0:02:12 INF - sort 5s cpu:6s avg:1.3 0:02:12 INF - worker 2x(44% 2s done:2s) 0:02:12 INF - mbtiles 31s cpu:57s gc:1s avg:1.8 0:02:12 INF - read 1x(8% 2s wait:25s) 0:02:12 INF - encode 2x(45% 14s wait:8s) 0:02:12 INF - write 1x(36% 11s sys:1s wait:16s) 0:02:12 INF - ---------------------------------------- 0:02:12 INF - features 268MB 0:02:12 INF - mbtiles 514MB -rw-r--r-- 1 runner docker 52M Mar 3 12:19 run.jar

https://github.com/onthegomap/planetiler/actions/runs/1927932836

ℹ️ Base Logs 8a6997f

0:00:00 DEB - argument: config=null (path to config file)
0:00:00 DEB - argument: area=rhode island (name of the extract to download if osm_url/osm_path not specified (i.e. 'monaco' 'rhode island' 'australia' or 'planet'))
0:00:00 INF - Using in-memory stats

0:00:00 INF [overall] - Starting...
0:00:00 DEB - argument: bounds=Env[-74.07 : -17.84, 21.34 : 43.55] (bounds)
0:00:00 DEB - argument: threads=2 (num threads)
0:00:00 DEB - argument: loginterval=10 seconds (time between logs)
0:00:00 DEB - argument: minzoom=0 (minimum zoom level)
0:00:00 DEB - argument: maxzoom=14 (maximum zoom level (limit 14))
0:00:00 DEB - argument: defer_mbtiles_index_creation=false (skip adding index to mbtiles file)
0:00:00 DEB - argument: optimize_db=false (optimize mbtiles after writing)
0:00:00 DEB - argument: emit_tiles_in_order=true (emit tiles in index order)
0:00:00 DEB - argument: force=false (force overwriting output file)
0:00:00 DEB - argument: gzip_temp=false (gzip temporary feature storage (uses more CPU, but less disk space))
0:00:00 DEB - argument: sort_max_readers=6 (maximum number of concurrent read threads to use when sorting chunks)
0:00:00 DEB - argument: sort_max_writers=6 (maximum number of concurrent write threads to use when sorting chunks)
0:00:00 DEB - argument: nodemap_type=sortedtable (type of node location map: noop, sortedtable, or sparsearray)
0:00:00 DEB - argument: nodemap_storage=mmap (storage for location map: mmap or ram)
0:00:00 DEB - argument: http_user_agent=Planetiler downloader (https://github.com/onthegomap/planetiler) (User-Agent header to set when downloading files over HTTP)
0:00:00 DEB - argument: http_timeout=30 seconds (Timeout to use when downloading files over HTTP)
0:00:00 DEB - argument: download_chunk_size_mb=100 (Size of file chunks to download in parallel in megabytes)
0:00:00 DEB - argument: download_threads=1 (Number of parallel threads to use when downloading each file)
0:00:00 DEB - argument: min_feature_size_at_max_zoom=0.0625 (Default value for the minimum size in tile pixels of features to emit at the maximum zoom level to allow for overzooming)
0:00:00 DEB - argument: min_feature_size=1.0 (Default value for the minimum size in tile pixels of features to emit below the maximum zoom level)
0:00:00 DEB - argument: simplify_tolerance_at_max_zoom=0.0625 (Default value for the tile pixel tolerance to use when simplifying features at the maximum zoom level to allow for overzooming)
0:00:00 DEB - argument: simplify_tolerance=0.1 (Default value for the tile pixel tolerance to use when simplifying features below the maximum zoom level)
0:00:00 DEB - argument: osm_lazy_reads=false (Read OSM blocks from disk in worker threads)
0:00:00 DEB - argument: tmpdir=data/tmp (temp directory)
0:00:00 DEB - argument: only_download=false (download source data then exit)
0:00:00 DEB - argument: download=false (download sources)
0:00:00 DEB - argument: only_fetch_wikidata=false (fetch wikidata translations then quit)
0:00:00 DEB - argument: fetch_wikidata=false (fetch wikidata translations then continue)
0:00:00 DEB - argument: use_wikidata=true (use wikidata translations)
0:00:00 DEB - argument: wikidata_cache=data/sources/wikidata_names.json (wikidata cache file)
0:00:00 DEB - argument: lake_centerlines_path=data/sources/lake_centerline.shp.zip (lake_centerlines shapefile path)
0:00:00 DEB - argument: water_polygons_path=data/sources/water-polygons-split-3857.zip (water_polygons shapefile path)
0:00:00 DEB - argument: natural_earth_path=data/sources/natural_earth_vector.sqlite.zip (natural_earth sqlite db path)
0:00:00 DEB - argument: osm_path=data/sources/rhode_island.osm.pbf (osm OSM input file path)
0:00:00 DEB - argument: mbtiles=data/out.mbtiles (mbtiles output file)
0:00:00 DEB - argument: transliterate=true (attempt to transliterate latin names)
0:00:00 DEB - argument: languages=am,ar,az,be,bg,br,bs,ca,co,cs,cy,da,de,el,en,eo,es,et,eu,fi,fr,fy,ga,gd,he,hi,hr,hu,hy,id,is,it,ja,ja_kana,ja_rm,ja-Latn,ja-Hira,ka,kk,kn,ko,ko-Latn,ku,la,lb,lt,lv,mk,mt,ml,nl,no,oc,pl,pt,rm,ro,ru,sk,sl,sq,sr,sr-Latn,sv,ta,te,th,tr,uk,zh (languages to use)
0:00:00 DEB - argument: only_layers= (Include only certain layers)
0:00:00 DEB - argument: exclude_layers= (Exclude certain layers)
0:00:00 DEB - argument: boundary_country_names=true (boundary layer: add left/right codes of neighboring countries)
0:00:00 DEB - argument: transportation_z13_paths=false (transportation(_name) layer: show all paths on z13)
0:00:00 DEB - argument: building_merge_z13=true (building layer: merge nearby buildings at z13)
0:00:00 DEB - argument: transportation_name_brunnel=false (transportation_name layer: set to false to omit brunnel and help merge long highways)
0:00:00 DEB - argument: transportation_name_size_for_shield=false (transportation_name layer: allow road names on shorter segments (ie. they will have a shield))
0:00:00 DEB - argument: transportation_name_limit_merge=false (transportation_name layer: limit merge so we don't combine different relations to help merge long highways)
0:00:00 DEB - argument: mbtiles_name=OpenMapTiles ('name' attribute for mbtiles metadata)
0:00:00 DEB - argument: mbtiles_description=A tileset showcasing all layers in OpenMapTiles. https://openmaptiles.org ('description' attribute for mbtiles metadata)
0:00:00 DEB - argument: mbtiles_attribution=<a href="https://www.openmaptiles.org/" target="_blank">&copy; OpenMapTiles</a> <a href="https://www.openstreetmap.org/copyright" target="_blank">&copy; OpenStreetMap contributors</a> ('attribution' attribute for mbtiles metadata)
0:00:00 DEB - argument: mbtiles_version=3.13.0 ('version' attribute for mbtiles metadata)
0:00:00 DEB - argument: mbtiles_type=baselayer ('type' attribute for mbtiles metadata)
0:00:00 DEB - argument: help=false (show arguments then exit)
0:00:00 INF - Building BasemapProfile profile into data/out.mbtiles in these phases:
0:00:00 INF -   lake_centerlines: Process features in data/sources/lake_centerline.shp.zip
0:00:00 INF -   water_polygons: Process features in data/sources/water-polygons-split-3857.zip
0:00:00 INF -   natural_earth: Process features in data/sources/natural_earth_vector.sqlite.zip
0:00:00 INF -   osm_pass1: Pre-process OpenStreetMap input (store node locations then relation members)
0:00:00 INF -   osm_pass2: Process OpenStreetMap nodes, ways, then relations
0:00:00 INF -   sort: Sort rendered features by tile ID
0:00:00 INF -   mbtiles: Encode each tile and write to data/out.mbtiles
0:00:00 INF - error loading /home/runner/work/planetiler/planetiler/data/sources/wikidata_names.json: java.nio.file.NoSuchFileException: data/sources/wikidata_names.json
0:00:01 INF - Using merge sort feature map, chunk size=1000mb workers=2
0:00:02 INF - dataFileCache open start

0:00:02 INF [lake_centerlines] - Starting...
0:00:03 INF [lake_centerlines] -  read: [  14k 100%  20k/s ] write: [    0    0/s ] 0    
    cpus: 1.7 gc:  0% mem: 166M/4.2G postGC: 45M
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:03 INF [lake_centerlines] -  read: [  14k 100%    0/s ] write: [    0    0/s ] 0    
    cpus: 1.8 gc:  0% mem: 166M/4.2G postGC: 45M
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:03 INF [lake_centerlines] - Finished in 0.8s cpu:1s avg:1.7
0:00:03 INF [lake_centerlines] -   read     1x(60% 0.5s)
0:00:03 INF [lake_centerlines] -   process  2x(15% 0.1s)
0:00:03 INF [lake_centerlines] -   write    1x(0% 0s)

0:00:03 INF [water_polygons] - Starting...
0:00:13 INF [water_polygons] -  read: [ 1.9k  13%  190/s ] write: [  47k 4.6k/s ] 2.8M 
    cpus: 1.9 gc: 11% mem: 2G/4.2G postGC: 989M
    read(56%) ->   (10/1k) -> process(37% 24%) -> (905/53k) -> write( 0%)
0:00:23 INF [water_polygons] -  read: [ 4.6k  32%  276/s ] write: [ 256k  20k/s ] 13M  
    cpus: 1.7 gc:  8% mem: 1.8G/4.2G postGC: 1.4G
    read(71%) ->    (6/1k) -> process(33% 16%) -> (509/53k) -> write( 1%)
0:00:31 INF [water_polygons] -  read: [  14k 100% 1.2k/s ] write: [ 4.3M 520k/s ] 186M 
    cpus: 1.5 gc:  4% mem: 3.1G/4.2G postGC: 1.6G
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:31 INF [water_polygons] -  read: [  14k 100%    0/s ] write: [ 4.3M    0/s ] 186M 
    cpus: 0 gc:  0% mem: 3.1G/4.2G postGC: 1.6G
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:31 INF [water_polygons] - Finished in 28s cpu:48s gc:2s avg:1.7
0:00:31 INF [water_polygons] -   read     1x(60% 17s sys:1s wait:3s)
0:00:31 INF [water_polygons] -   process  2x(28% 8s wait:14s)
0:00:31 INF [water_polygons] -   write    1x(4% 1s wait:27s)
0:00:31 INF [natural_earth] - unzipping /home/runner/work/planetiler/planetiler/data/sources/natural_earth_vector.sqlite.zip to data/tmp/natearth.sqlite

0:00:37 INF [natural_earth] - Starting...
0:00:52 INF [natural_earth] -  read: [ 257k  74%  25k/s ] write: [    0    0/s ] 186M 
    cpus: 1.5 gc:  2% mem: 1.9G/4.2G postGC: 1.6G
    read(90%) ->   (67/1k) -> process(18% 16%) -> (132/53k) -> write( 0%)
0:00:54 INF [natural_earth] -  read: [ 349k 100%  37k/s ] write: [  181   73/s ] 186M 
    cpus: 1.7 gc:  0% mem: 2.5G/4.2G postGC: 1.6G
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:54 INF [natural_earth] -  read: [ 349k 100%    0/s ] write: [  181    0/s ] 186M 
    cpus: 2 gc:  0% mem: 2.5G/4.2G postGC: 1.6G
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:54 INF [natural_earth] - Finished in 18s cpu:21s avg:1.2
0:00:54 INF [natural_earth] -   read     1x(64% 11s sys:1s done:5s)
0:00:54 INF [natural_earth] -   process  2x(13% 2s wait:12s done:5s)
0:00:54 INF [natural_earth] -   write    1x(0% 0s wait:13s done:5s)

0:00:55 INF [osm_pass1] - Starting...
0:00:59 INF [osm_pass1] -  nodes: [ 4.5M   1M/s ] 353M  ways: [ 327k  76k/s ] rels: [ 7.4k 1.7k/s ] blocks: [  614  144/s ]
    cpus: 1.8 gc:  1% mem: 791M/4.2G postGC: 776M hppc: 880k
    read( -%) ->     (0/4) -> parse( -%) ->     (0/4) -> process( -%)
0:00:59 DEB [osm_pass1] - processed blocks:614 nodes:4,573,793 ways:327,099 relations:7,400
0:00:59 INF [osm_pass1] - Finished in 4s cpu:8s avg:1.8
0:00:59 INF [osm_pass1] -   read     1x(3% 0.1s wait:4s)
0:00:59 INF [osm_pass1] -   parse    1x(65% 3s)
0:00:59 INF [osm_pass1] -   process  1x(37% 2s wait:2s)

0:00:59 INF [osm_pass2] - Starting...
0:01:04 DEB [osm_pass2:process] - Sorting long long multimap...
0:01:04 DEB [osm_pass2:process] - Sorted long long multimap 0s cpu:0s avg:1.6
0:01:04 WAR [osm_pass2:process] - No GB polygon for inferring route network types
0:01:09 INF [osm_pass2] -  nodes: [ 4.5M 100% 457k/s ] 353M  ways: [  40k  12% 3.9k/s ] rels: [    0   0%    0/s ] features: [ 4.6M  29k/s ] 211M  blocks: [  577  94%   57/s ]
    cpus: 2 gc:  1% mem: 2G/4.2G postGC: 805M hppc: 1.1M
    read( -%) ->  (35/103) -> process(68% 66%) -> (1.7k/53k) -> write( 1%)
0:01:19 INF [osm_pass2] -  nodes: [ 4.5M 100%    0/s ] 353M  ways: [ 216k  66%  17k/s ] rels: [    0   0%    0/s ] features: [   5M  42k/s ] 236M  blocks: [  599  98%    2/s ]
    cpus: 2 gc:  1% mem: 3G/4.2G postGC: 799M hppc: 9.3M
    read( -%) ->  (13/103) -> process(68% 68%) -> (998/53k) -> write( 2%)
0:01:29 INF [osm_pass2] -  nodes: [ 4.5M 100%    0/s ] 353M  ways: [ 327k 100%  11k/s ] rels: [ 2.4k  33%  240/s ] features: [ 5.2M  20k/s ] 256M  blocks: [  613 100%    1/s ]
    cpus: 2 gc:  1% mem: 2.8G/4.2G postGC: 805M hppc:  29M
    read( -%) ->   (0/103) -> process(86% 83%) -> (618/53k) -> write( 1%)
0:01:38 INF [osm_pass2] -  nodes: [ 4.5M 100%    0/s ] 353M  ways: [ 327k 100%    0/s ] rels: [ 7.4k 100%  540/s ] features: [ 5.2M 2.6k/s ] 268M  blocks: [  614 100%   <1/s ]
    cpus: 2 gc:  1% mem: 1.6G/4.2G postGC: 807M hppc:  29M
    read( -%) ->   (0/103) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:01:38 INF [osm_pass2] -  nodes: [ 4.5M 100%    0/s ] 353M  ways: [ 327k 100%    0/s ] rels: [ 7.4k 100%    0/s ] features: [ 5.2M    0/s ] 268M  blocks: [  614 100%    0/s ]
    cpus: 0 gc:  0% mem: 1.6G/4.2G postGC: 807M hppc:  29M
    read( -%) ->   (0/103) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:01:38 DEB [osm_pass2] - processed blocks:614 nodes:4,573,793 ways:327,099 relations:7,400
0:01:38 INF [osm_pass2] - Finished in 39s cpu:1m18s avg:2
0:01:38 INF [osm_pass2] -   read     1x(0% 0.1s wait:4s done:35s)
0:01:38 INF [osm_pass2] -   process  2x(76% 30s)
0:01:38 INF [osm_pass2] -   write    1x(1% 0.4s wait:39s)

0:01:38 INF [boundaries] - Starting...
0:01:38 INF [boundaries] - Creating polygons for 1 boundaries
0:01:38 WAR [boundaries] - Unable to form closed polygon for OSM relation 148838 (likely missing edges)
0:01:38 INF [boundaries] - Finished creating 0 country polygons
0:01:38 INF [boundaries] - Finished in 0s cpu:0.1s avg:1.2
0:01:38 INF - Deleting node.db to make room for mbtiles

0:01:38 INF [sort] - Starting...
0:01:43 INF [sort] -  chunks: [   1 /   1 100% ] 268M 
    cpus: 1.2 gc:  0% mem: 2.1G/4.2G postGC: 807M
    ->     (0/4) -> worker( -%  -%)
0:01:43 INF [sort] -  chunks: [   1 /   1 100% ] 268M 
    cpus: 4.5 gc:  0% mem: 2.1G/4.2G postGC: 807M
    ->     (0/4) -> worker( -%  -%)
0:01:43 INF [sort] - Finished in 4s cpu:5s avg:1.2
0:01:43 INF [sort] -   worker  2x(43% 2s done:2s)
0:01:43 INF - read:1s write:1s sort:1s

0:01:43 INF [mbtiles] - Starting...
0:01:43 DEB [mbtiles:write] - Execute mbtiles: create table metadata (name text, value text);
0:01:43 DEB [mbtiles:write] - Execute mbtiles: create unique index name on metadata (name);
0:01:43 DEB [mbtiles:write] - Execute mbtiles: create table tiles (zoom_level integer, tile_column integer, tile_row, tile_data blob);
0:01:43 DEB [mbtiles:write] - Execute mbtiles: create unique index tile_index on tiles (zoom_level, tile_column, tile_row)
0:01:43 DEB [mbtiles:write] - Set mbtiles metadata: name=OpenMapTiles
0:01:43 DEB [mbtiles:write] - Set mbtiles metadata: format=pbf
0:01:43 DEB [mbtiles:write] - Set mbtiles metadata: description=A tileset showcasing all layers in OpenMapTiles. https://openmaptiles.org
0:01:43 DEB [mbtiles:write] - Set mbtiles metadata: attribution=<a href="https://www.openmaptiles.org/" target="_blank">&copy; OpenMapTiles</a> <a href="https://www.openstreetmap.org/copyright" target="_blank">&copy; OpenStreetMap contributors</a>
0:01:43 DEB [mbtiles:write] - Set mbtiles metadata: version=3.13.0
0:01:43 DEB [mbtiles:write] - Set mbtiles metadata: type=baselayer
0:01:43 DEB [mbtiles:write] - Set mbtiles metadata: bounds=-74.07,21.34,-17.84,43.55
0:01:43 DEB [mbtiles:write] - Set mbtiles metadata: center=-45.955,32.445,3
0:01:43 DEB [mbtiles:write] - Set mbtiles metadata: minzoom=0
0:01:43 DEB [mbtiles:write] - Set mbtiles metadata: maxzoom=14
0:01:44 DEB [mbtiles:write] - Set mbtiles metadata: json={"vector_layers":[{"id":"aerodrome_label","fields":{"name_int":"String","iata":"String","ele_ft":"Number","name_de":"String","name":"String","icao":"String","name:en":"String","class":"String","name_en":"String","name:latin":"String","ele":"Number"},"minzoom":10,"maxzoom":14},{"id":"aeroway","fields":{"ref":"String","class":"String"},"minzoom":10,"maxzoom":14},{"id":"boundary","fields":{"disputed":"Number","admin_level":"Number","maritime":"Number","disputed_name":"String"},"minzoom":0,"maxzoom":14},{"id":"building","fields":{"colour":"String","render_height":"Number","render_min_height":"Number"},"minzoom":13,"maxzoom":14},{"id":"housenumber","fields":{"housenumber":"String"},"minzoom":14,"maxzoom":14},{"id":"landcover","fields":{"subclass":"String","class":"String","_numpoints":"Number"},"minzoom":8,"maxzoom":14},{"id":"landuse","fields":{"class":"String"},"minzoom":4,"maxzoom":14},{"id":"mountain_peak","fields":{"name_int":"String","customary_ft":"Number","ele_ft":"Number","name_de":"String","name":"String","rank":"Number","class":"String","name_en":"String","name:latin":"String","ele":"Number"},"minzoom":7,"maxzoom":14},{"id":"park","fields":{"name_int":"String","name_de":"String","name":"String","name:en":"String","class":"String","name_en":"String","name:latin":"String"},"minzoom":6,"maxzoom":14},{"id":"place","fields":{"name:fy":"String","name_int":"String","capital":"Number","name:uk":"String","name:pl":"String","name:nl":"String","name:be":"String","name:ru":"String","name:ko":"String","name_de":"String","name":"String","rank":"Number","name:en":"String","name:eo":"String","class":"String","name:hu":"String","name:ta":"String","name:zh":"String","name_en":"String","name:latin":"String"},"minzoom":2,"maxzoom":14},{"id":"poi","fields":{"name_int":"String","level":"Number","name:nonlatin":"String","layer":"Number","name_de":"String","name":"String","subclass":"String","indoor":"Number","name:en":"String","class":"String","name:zh":"String","name_en":"String","name:latin":"String"},"minzoom":12,"maxzoom":14},{"id":"transportation","fields":{"access":"String","brunnel":"String","expressway":"Number","surface":"String","bicycle":"String","level":"Number","ramp":"Number","mtb_scale":"String","toll":"Number","oneway":"Number","layer":"Number","network":"String","horse":"String","service":"String","subclass":"String","class":"String","foot":"String"},"minzoom":4,"maxzoom":14},{"id":"transportation_name","fields":{"name_int":"String","name:nonlatin":"String","route_4":"String","route_3":"String","route_2":"String","route_1":"String","layer":"Number","network":"String","ref":"String","name_de":"String","name":"String","subclass":"String","ref_length":"Number","class":"String","name_en":"String","name:latin":"String"},"minzoom":6,"maxzoom":14},{"id":"water","fields":{"intermittent":"Number","class":"String"},"minzoom":0,"maxzoom":14},{"id":"water_name","fields":{"name_int":"String","name:nonlatin":"String","name_de":"String","name":"String","intermittent":"Number","class":"String","name_en":"String","name:latin":"String"},"minzoom":9,"maxzoom":14},{"id":"waterway","fields":{"name_int":"String","brunnel":"String","name_de":"String","_relid":"Number","intermittent":"Number","name":"String","class":"String","name:latin":"String","name_en":"String"},"minzoom":4,"maxzoom":14}]}
0:01:44 INF [mbtiles:write] - Starting z0
0:01:44 INF [mbtiles:write] - Finished z0 in 0s cpu:0s avg:0, now starting z1
0:01:44 INF [mbtiles:write] - Finished z1 in 0s cpu:0s avg:0, now starting z2
0:01:44 INF [mbtiles:write] - Finished z2 in 0s cpu:0s avg:58, now starting z3
0:01:44 INF [mbtiles:write] - Finished z3 in 0s cpu:0s avg:0, now starting z4
0:01:44 INF [mbtiles:write] - Finished z4 in 0s cpu:0s avg:0, now starting z5
0:01:44 INF [mbtiles:write] - Finished z5 in 0s cpu:0s avg:0, now starting z6
0:01:44 INF [mbtiles:write] - Finished z6 in 0s cpu:0s avg:0, now starting z7
0:01:44 INF [mbtiles:write] - Finished z7 in 0.2s cpu:0.4s avg:1.9, now starting z8
0:01:46 INF [mbtiles:write] - Finished z8 in 1s cpu:3s avg:1.9, now starting z9
0:01:46 INF [mbtiles:write] - Finished z9 in 0.8s cpu:2s avg:2, now starting z10
0:01:47 INF [mbtiles:write] - Finished z10 in 0.2s cpu:0.4s avg:1.9, now starting z11
0:01:48 INF [mbtiles:write] - Finished z11 in 0.9s cpu:2s avg:2, now starting z12
0:01:50 INF [mbtiles:write] - Finished z12 in 3s cpu:5s avg:2, now starting z13
0:01:53 INF [mbtiles] -  features: [ 644k  12%  61k/s ] 268M  tiles: [ 290k  29k/s ] 46M  
    cpus: 2 gc:  6% mem: 1.2G/4.2G postGC: 854M
    read( 5%) -> (214/217) -> encode(56% 56%) -> (215/216) -> write( 9%)
    last tile: 13/2469/3048 (z13 4%) https://www.openstreetmap.org/#map=13/41.77131/-71.49902
0:02:03 INF [mbtiles] -  features: [ 1.7M  33% 110k/s ] 268M  tiles: [ 983k  69k/s ] 129M 
    cpus: 1.9 gc:  1% mem: 2.4G/4.2G postGC: 860M
    read( 4%) ->  (21/217) -> encode(56% 53%) -> (183/216) -> write(18%)
    last tile: 13/3610/3126 (z13 93%) https://www.openstreetmap.org/#map=13/39.16414/-21.35742
0:02:04 INF [mbtiles:write] - Finished z13 in 14s cpu:27s avg:2, now starting z14
0:02:13 INF [mbtiles] -  features: [ 5.2M 100% 352k/s ] 268M  tiles: [   4M 306k/s ] 506M 
    cpus: 1.5 gc:  1% mem: 2.9G/4.2G postGC: 916M
    read( -%) ->   (0/217) -> encode( -%  -%) ->  (67/216) -> write(82%)
    last tile: 14/7325/6838 (z14 97%) https://www.openstreetmap.org/#map=14/28.49766/-19.05029
0:02:14 INF [mbtiles:write] - Finished z14 in 10s cpu:15s avg:1.5
0:02:14 INF [mbtiles] -  features: [ 5.2M 100%    0/s ] 268M  tiles: [ 4.1M 355k/s ] 514M 
    cpus: 1.1 gc:  0% mem: 2.9G/4.2G postGC: 916M
    read( -%) ->   (0/217) -> encode( -%  -%) ->   (0/216) -> write(96%)
    last tile: 14/7380/5985 (z14 100%) https://www.openstreetmap.org/#map=14/43.56447/-17.84180
0:02:14 DEB [mbtiles] - Tile stats:
0:02:14 DEB [mbtiles] - z0 avg:7.9k max:7.9k
0:02:14 DEB [mbtiles] - z1 avg:4k max:4k
0:02:14 DEB [mbtiles] - z2 avg:9.4k max:9.4k
0:02:14 DEB [mbtiles] - z3 avg:3.9k max:6.4k
0:02:14 DEB [mbtiles] - z4 avg:1.6k max:4.6k
0:02:14 DEB [mbtiles] - z5 avg:1.4k max:8.1k
0:02:14 DEB [mbtiles] - z6 avg:1.4k max:24k
0:02:14 DEB [mbtiles] - z7 avg:896 max:33k
0:02:14 DEB [mbtiles] - z8 avg:365 max:48k
0:02:14 DEB [mbtiles] - z9 avg:296 max:278k
0:02:14 DEB [mbtiles] - z10 avg:164 max:232k
0:02:14 DEB [mbtiles] - z11 avg:107 max:131k
0:02:14 DEB [mbtiles] - z12 avg:85 max:118k
0:02:14 DEB [mbtiles] - z13 avg:72 max:109k
0:02:14 DEB [mbtiles] - z14 avg:68 max:257k
0:02:14 DEB [mbtiles] - all avg:70 max:0
0:02:14 DEB [mbtiles] -  # features: 5,276,185
0:02:14 DEB [mbtiles] -     # tiles: 4,115,465
0:02:14 INF [mbtiles] - Finished in 31s cpu:56s gc:1s avg:1.8
0:02:14 INF [mbtiles] -   read    1x(8% 2s wait:24s)
0:02:14 INF [mbtiles] -   encode  2x(44% 14s wait:8s)
0:02:14 INF [mbtiles] -   write   1x(36% 11s sys:1s wait:16s)
0:02:14 INF - Finished in 2m14s cpu:3m49s gc:4s avg:1.7
0:02:14 INF - FINISHED!

0:02:14 INF - ----------------------------------------
0:02:14 INF - 	overall          2m14s cpu:3m49s gc:4s avg:1.7
0:02:14 INF - 	lake_centerlines 0.8s cpu:1s avg:1.7
0:02:14 INF - 	water_polygons   28s cpu:48s gc:2s avg:1.7
0:02:14 INF - 	  read     1x(60% 17s sys:1s wait:3s)
0:02:14 INF - 	  process  2x(28% 8s wait:14s)
0:02:14 INF - 	  write    1x(4% 1s wait:27s)
0:02:14 INF - 	natural_earth    18s cpu:21s avg:1.2
0:02:14 INF - 	  read     1x(64% 11s sys:1s done:5s)
0:02:14 INF - 	  process  2x(13% 2s wait:12s done:5s)
0:02:14 INF - 	  write    1x(0% 0s wait:13s done:5s)
0:02:14 INF - 	osm_pass1        4s cpu:8s avg:1.8
0:02:14 INF - 	  read     1x(3% 0.1s wait:4s)
0:02:14 INF - 	  parse    1x(65% 3s)
0:02:14 INF - 	  process  1x(37% 2s wait:2s)
0:02:14 INF - 	osm_pass2        39s cpu:1m18s avg:2
0:02:14 INF - 	  read     1x(0% 0.1s wait:4s done:35s)
0:02:14 INF - 	  process  2x(76% 30s)
0:02:14 INF - 	  write    1x(1% 0.4s wait:39s)
0:02:14 INF - 	boundaries       0s cpu:0.1s avg:1.2
0:02:14 INF - 	sort             4s cpu:5s avg:1.2
0:02:14 INF - 	  worker  2x(43% 2s done:2s)
0:02:14 INF - 	mbtiles          31s cpu:56s gc:1s avg:1.8
0:02:14 INF - 	  read    1x(8% 2s wait:24s)
0:02:14 INF - 	  encode  2x(44% 14s wait:8s)
0:02:14 INF - 	  write   1x(36% 11s sys:1s wait:16s)
0:02:14 INF - ----------------------------------------
0:02:14 INF - 	features	268MB
0:02:14 INF - 	mbtiles	514MB
-rw-r--r-- 1 runner docker 52M Mar  3 12:21 run.jar

ℹ️ This Branch Logs 8fcc787

0:00:00 DEB - argument: config=null (path to config file)
0:00:00 DEB - argument: area=rhode island (name of the extract to download if osm_url/osm_path not specified (i.e. 'monaco' 'rhode island' 'australia' or 'planet'))
0:00:00 INF - Using in-memory stats

0:00:00 INF [overall] - Starting...
0:00:00 DEB - argument: bounds=Env[-74.07 : -17.84, 21.34 : 43.55] (bounds)
0:00:00 DEB - argument: threads=2 (num threads)
0:00:00 DEB - argument: loginterval=10 seconds (time between logs)
0:00:00 DEB - argument: minzoom=0 (minimum zoom level)
0:00:00 DEB - argument: maxzoom=14 (maximum zoom level (limit 14))
0:00:00 DEB - argument: defer_mbtiles_index_creation=false (skip adding index to mbtiles file)
0:00:00 DEB - argument: optimize_db=false (optimize mbtiles after writing)
0:00:00 DEB - argument: emit_tiles_in_order=true (emit tiles in index order)
0:00:00 DEB - argument: force=false (overwriting output file and ignore disk/RAM warnings)
0:00:00 DEB - argument: gzip_temp=false (gzip temporary feature storage (uses more CPU, but less disk space))
0:00:00 DEB - argument: sort_max_readers=6 (maximum number of concurrent read threads to use when sorting chunks)
0:00:00 DEB - argument: sort_max_writers=6 (maximum number of concurrent write threads to use when sorting chunks)
0:00:00 DEB - argument: nodemap_type=sortedtable (type of node location map: noop, sortedtable, or sparsearray)
0:00:00 DEB - argument: nodemap_storage=mmap (storage for location map: mmap or ram)
0:00:00 DEB - argument: http_user_agent=Planetiler downloader (https://github.com/onthegomap/planetiler) (User-Agent header to set when downloading files over HTTP)
0:00:00 DEB - argument: http_timeout=30 seconds (Timeout to use when downloading files over HTTP)
0:00:00 DEB - argument: download_chunk_size_mb=100 (Size of file chunks to download in parallel in megabytes)
0:00:00 DEB - argument: download_threads=1 (Number of parallel threads to use when downloading each file)
0:00:00 DEB - argument: min_feature_size_at_max_zoom=0.0625 (Default value for the minimum size in tile pixels of features to emit at the maximum zoom level to allow for overzooming)
0:00:00 DEB - argument: min_feature_size=1.0 (Default value for the minimum size in tile pixels of features to emit below the maximum zoom level)
0:00:00 DEB - argument: simplify_tolerance_at_max_zoom=0.0625 (Default value for the tile pixel tolerance to use when simplifying features at the maximum zoom level to allow for overzooming)
0:00:00 DEB - argument: simplify_tolerance=0.1 (Default value for the tile pixel tolerance to use when simplifying features below the maximum zoom level)
0:00:00 DEB - argument: osm_lazy_reads=false (Read OSM blocks from disk in worker threads)
0:00:00 DEB - argument: tmpdir=data/tmp (temp directory)
0:00:00 DEB - argument: only_download=false (download source data then exit)
0:00:00 DEB - argument: download=false (download sources)
0:00:00 DEB - argument: only_fetch_wikidata=false (fetch wikidata translations then quit)
0:00:00 DEB - argument: fetch_wikidata=false (fetch wikidata translations then continue)
0:00:00 DEB - argument: use_wikidata=true (use wikidata translations)
0:00:00 DEB - argument: wikidata_cache=data/sources/wikidata_names.json (wikidata cache file)
0:00:00 DEB - argument: lake_centerlines_path=data/sources/lake_centerline.shp.zip (lake_centerlines shapefile path)
0:00:00 DEB - argument: free_lake_centerlines_after_read=false (delete lake_centerlines input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: water_polygons_path=data/sources/water-polygons-split-3857.zip (water_polygons shapefile path)
0:00:00 DEB - argument: free_water_polygons_after_read=false (delete water_polygons input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: natural_earth_path=data/sources/natural_earth_vector.sqlite.zip (natural_earth sqlite db path)
0:00:00 DEB - argument: free_natural_earth_after_read=false (delete natural_earth input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: osm_path=data/sources/rhode_island.osm.pbf (osm OSM input file path)
0:00:00 DEB - argument: free_osm_after_read=false (delete osm input file after reading to make space for output (reduces peak disk usage))
0:00:00 DEB - argument: mbtiles=data/out.mbtiles (mbtiles output file)
0:00:00 DEB - argument: transliterate=true (attempt to transliterate latin names)
0:00:00 DEB - argument: languages=am,ar,az,be,bg,br,bs,ca,co,cs,cy,da,de,el,en,eo,es,et,eu,fi,fr,fy,ga,gd,he,hi,hr,hu,hy,id,is,it,ja,ja_kana,ja_rm,ja-Latn,ja-Hira,ka,kk,kn,ko,ko-Latn,ku,la,lb,lt,lv,mk,mt,ml,nl,no,oc,pl,pt,rm,ro,ru,sk,sl,sq,sr,sr-Latn,sv,ta,te,th,tr,uk,zh (languages to use)
0:00:00 DEB - argument: only_layers= (Include only certain layers)
0:00:00 DEB - argument: exclude_layers= (Exclude certain layers)
0:00:00 DEB - argument: boundary_country_names=true (boundary layer: add left/right codes of neighboring countries)
0:00:00 DEB - argument: transportation_z13_paths=false (transportation(_name) layer: show all paths on z13)
0:00:00 DEB - argument: building_merge_z13=true (building layer: merge nearby buildings at z13)
0:00:00 DEB - argument: transportation_name_brunnel=false (transportation_name layer: set to false to omit brunnel and help merge long highways)
0:00:00 DEB - argument: transportation_name_size_for_shield=false (transportation_name layer: allow road names on shorter segments (ie. they will have a shield))
0:00:00 DEB - argument: transportation_name_limit_merge=false (transportation_name layer: limit merge so we don't combine different relations to help merge long highways)
0:00:00 DEB - argument: mbtiles_name=OpenMapTiles ('name' attribute for mbtiles metadata)
0:00:00 DEB - argument: mbtiles_description=A tileset showcasing all layers in OpenMapTiles. https://openmaptiles.org ('description' attribute for mbtiles metadata)
0:00:00 DEB - argument: mbtiles_attribution=<a href="https://www.openmaptiles.org/" target="_blank">&copy; OpenMapTiles</a> <a href="https://www.openstreetmap.org/copyright" target="_blank">&copy; OpenStreetMap contributors</a> ('attribution' attribute for mbtiles metadata)
0:00:00 DEB - argument: mbtiles_version=3.13.0 ('version' attribute for mbtiles metadata)
0:00:00 DEB - argument: mbtiles_type=baselayer ('type' attribute for mbtiles metadata)
0:00:00 DEB - argument: help=false (show arguments then exit)
0:00:00 INF - Building BasemapProfile profile into data/out.mbtiles in these phases:
0:00:00 INF -   lake_centerlines: Process features in data/sources/lake_centerline.shp.zip
0:00:00 INF -   water_polygons: Process features in data/sources/water-polygons-split-3857.zip
0:00:00 INF -   natural_earth: Process features in data/sources/natural_earth_vector.sqlite.zip
0:00:00 INF -   osm_pass1: Pre-process OpenStreetMap input (store node locations then relation members)
0:00:00 INF -   osm_pass2: Process OpenStreetMap nodes, ways, then relations
0:00:00 INF -   sort: Sort rendered features by tile ID
0:00:00 INF -   mbtiles: Encode each tile and write to data/out.mbtiles
0:00:00 INF - error loading /home/runner/work/planetiler/planetiler/data/sources/wikidata_names.json: java.nio.file.NoSuchFileException: data/sources/wikidata_names.json
0:00:00 INF - Using merge sort feature map, chunk size=1000mb workers=2
0:00:02 INF - dataFileCache open start

0:00:03 INF [lake_centerlines] - Starting...
0:00:03 INF [lake_centerlines] -  read: [  14k 100%  20k/s ] write: [    0    0/s ] 0    
    cpus: 1.7 gc:  0% mem: 166M/4.2G postGC: 46M
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:03 INF [lake_centerlines] -  read: [  14k 100%    0/s ] write: [    0    0/s ] 0    
    cpus: 1.6 gc:  0% mem: 166M/4.2G postGC: 46M
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:03 INF [lake_centerlines] - Finished in 0.8s cpu:1s avg:1.7
0:00:03 INF [lake_centerlines] -   read     1x(68% 0.5s)
0:00:03 INF [lake_centerlines] -   process  2x(16% 0.1s)
0:00:03 INF [lake_centerlines] -   write    1x(0% 0s)

0:00:03 INF [water_polygons] - Starting...
0:00:13 INF [water_polygons] -  read: [ 1.8k  13%  186/s ] write: [  47k 4.6k/s ] 2.8M 
    cpus: 2 gc: 12% mem: 1.8G/4.2G postGC: 1.1G
    read(52%) ->    (0/1k) -> process(20% 44%) -> (690/53k) -> write( 0%)
0:00:23 INF [water_polygons] -  read: [ 4.4k  31%  258/s ] write: [ 233k  18k/s ] 12M  
    cpus: 1.8 gc:  8% mem: 1.8G/4.2G postGC: 1.6G
    read(66%) ->    (0/1k) -> process(31% 23%) -> (1.1k/53k) -> write( 1%)
0:00:32 INF [water_polygons] -  read: [  14k 100% 1.2k/s ] write: [ 4.3M 502k/s ] 186M 
    cpus: 1.5 gc:  5% mem: 2.7G/4.2G postGC: 1.8G
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:32 INF [water_polygons] -  read: [  14k 100%    0/s ] write: [ 4.3M    0/s ] 186M 
    cpus: 0 gc:  0% mem: 2.7G/4.2G postGC: 1.8G
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:32 INF [water_polygons] - Finished in 28s cpu:50s gc:2s avg:1.8
0:00:32 INF [water_polygons] -   read     1x(59% 17s sys:1s wait:3s)
0:00:32 INF [water_polygons] -   process  2x(29% 8s wait:12s)
0:00:32 INF [water_polygons] -   write    1x(4% 1s wait:27s)
0:00:32 INF [natural_earth] - unzipping /home/runner/work/planetiler/planetiler/data/sources/natural_earth_vector.sqlite.zip to data/tmp/natearth.sqlite

0:00:40 INF [natural_earth] - Starting...
0:00:51 INF [natural_earth] -  read: [ 302k  87%  30k/s ] write: [    0    0/s ] 186M 
    cpus: 1.5 gc:  0% mem: 3.4G/4.2G postGC: 1.8G
    read(97%) ->    (0/1k) -> process(21% 19%) -> (132/53k) -> write( 0%)
0:00:53 INF [natural_earth] -  read: [ 349k 100%  26k/s ] write: [  181  101/s ] 186M 
    cpus: 1.5 gc:  1% mem: 2G/4.2G postGC: 1.8G
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:53 INF [natural_earth] -  read: [ 349k 100%    0/s ] write: [  181    0/s ] 186M 
    cpus: 2.8 gc:  0% mem: 2G/4.2G postGC: 1.8G
    read( -%) ->    (0/1k) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:00:53 INF [natural_earth] - Finished in 12s cpu:19s avg:1.5
0:00:53 INF [natural_earth] -   read     1x(90% 11s sys:1s)
0:00:53 INF [natural_earth] -   process  2x(19% 2s wait:11s)
0:00:53 INF [natural_earth] -   write    1x(0% 0s wait:12s)

0:00:53 INF [osm_pass1] - Starting...
0:00:56 INF [osm_pass1] -  nodes: [ 4.5M 1.2M/s ] 353M  ways: [ 327k  90k/s ] rels: [ 7.4k   2k/s ] blocks: [  614  169/s ]
    cpus: 1.8 gc:  1% mem: 832M/4.2G postGC: 807M hppc: 880k
    read( -%) ->     (0/4) -> parse( -%) ->     (0/4) -> process( -%)
0:00:56 DEB [osm_pass1] - processed blocks:614 nodes:4,573,793 ways:327,099 relations:7,400
0:00:56 INF [osm_pass1] - Finished in 4s cpu:7s avg:1.8
0:00:56 INF [osm_pass1] -   read     1x(3% 0.1s wait:3s)
0:00:56 INF [osm_pass1] -   parse    1x(71% 3s)
0:00:56 INF [osm_pass1] -   process  1x(36% 1s wait:2s)

0:00:56 INF [osm_pass2] - Starting...
0:01:01 DEB [osm_pass2:process] - Sorting long long multimap...
0:01:01 DEB [osm_pass2:process] - Sorted long long multimap 0s cpu:0s avg:1.9
0:01:01 WAR [osm_pass2:process] - No GB polygon for inferring route network types
0:01:06 INF [osm_pass2] -  nodes: [ 4.5M 100% 457k/s ] 353M  ways: [  48k  15% 4.7k/s ] rels: [    0   0%    0/s ] features: [ 4.6M  31k/s ] 212M  blocks: [  578  94%   57/s ]
    cpus: 2 gc:  1% mem: 2.3G/4.2G postGC: 830M hppc: 1.1M
    read( -%) ->  (34/103) -> process(66% 68%) -> (1.4k/53k) -> write( 1%)
0:01:16 INF [osm_pass2] -  nodes: [ 4.5M 100%    0/s ] 353M  ways: [ 232k  71%  18k/s ] rels: [    0   0%    0/s ] features: [   5M  41k/s ] 237M  blocks: [  601  98%    2/s ]
    cpus: 2 gc:  1% mem: 1G/4.2G postGC: 834M hppc: 9.3M
    read( -%) ->  (11/103) -> process(70% 70%) -> (599/53k) -> write( 2%)
0:01:26 INF [osm_pass2] -  nodes: [ 4.5M 100%    0/s ] 353M  ways: [ 327k 100% 9.5k/s ] rels: [ 2.7k  37%  269/s ] features: [ 5.2M  19k/s ] 256M  blocks: [  613 100%    1/s ]
    cpus: 1.9 gc:  0% mem: 3.1G/4.2G postGC: 828M hppc:  29M
    read( -%) ->   (0/103) -> process(81% 86%) -> (965/53k) -> write( 1%)
0:01:36 INF [osm_pass2] -  nodes: [ 4.5M 100%    0/s ] 353M  ways: [ 327k 100%    0/s ] rels: [ 7.4k 100%  501/s ] features: [ 5.2M 2.5k/s ] 268M  blocks: [  614 100%   <1/s ]
    cpus: 1.9 gc:  1% mem: 1.6G/4.2G postGC: 830M hppc:  29M
    read( -%) ->   (0/103) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:01:36 INF [osm_pass2] -  nodes: [ 4.5M 100%    0/s ] 353M  ways: [ 327k 100%    0/s ] rels: [ 7.4k 100%    0/s ] features: [ 5.2M    0/s ] 268M  blocks: [  614 100%    0/s ]
    cpus: 2.9 gc:  0% mem: 1.6G/4.2G postGC: 830M hppc:  29M
    read( -%) ->   (0/103) -> process( -%  -%) ->   (0/53k) -> write( -%)
0:01:36 DEB [osm_pass2] - processed blocks:614 nodes:4,573,793 ways:327,099 relations:7,400
0:01:36 INF [osm_pass2] - Finished in 39s cpu:1m17s avg:2
0:01:36 INF [osm_pass2] -   read     1x(0% 0.1s wait:4s done:35s)
0:01:36 INF [osm_pass2] -   process  2x(75% 30s)
0:01:36 INF [osm_pass2] -   write    1x(1% 0.4s wait:39s)

0:01:36 INF [boundaries] - Starting...
0:01:36 INF [boundaries] - Creating polygons for 1 boundaries
0:01:36 WAR [boundaries] - Unable to form closed polygon for OSM relation 148838 (likely missing edges)
0:01:36 INF [boundaries] - Finished creating 0 country polygons
0:01:36 INF [boundaries] - Finished in 0s cpu:0.1s avg:1.8
0:01:36 INF - Deleting node.db to make room for output file

0:01:36 INF [sort] - Starting...
0:01:40 INF [sort] -  chunks: [   1 /   1 100% ] 268M 
    cpus: 1.3 gc:  0% mem: 2.1G/4.2G postGC: 830M
    ->     (0/4) -> worker( -%)
0:01:40 INF [sort] -  chunks: [   1 /   1 100% ] 268M 
    cpus: 0 gc:  0% mem: 2.1G/4.2G postGC: 830M
    ->     (0/4) -> worker( -%)
0:01:40 INF [sort] - Finished in 5s cpu:6s avg:1.3
0:01:40 INF [sort] -   worker  2x(44% 2s done:2s)
0:01:40 INF - read:1s write:1s sort:1s

0:01:41 INF [mbtiles] - Starting...
0:01:41 DEB [mbtiles:write] - Execute mbtiles: create table metadata (name text, value text);
0:01:41 DEB [mbtiles:write] - Execute mbtiles: create unique index name on metadata (name);
0:01:41 DEB [mbtiles:write] - Execute mbtiles: create table tiles (zoom_level integer, tile_column integer, tile_row, tile_data blob);
0:01:41 DEB [mbtiles:write] - Execute mbtiles: create unique index tile_index on tiles (zoom_level, tile_column, tile_row)
0:01:41 DEB [mbtiles:write] - Set mbtiles metadata: name=OpenMapTiles
0:01:41 DEB [mbtiles:write] - Set mbtiles metadata: format=pbf
0:01:41 DEB [mbtiles:write] - Set mbtiles metadata: description=A tileset showcasing all layers in OpenMapTiles. https://openmaptiles.org
0:01:41 DEB [mbtiles:write] - Set mbtiles metadata: attribution=<a href="https://www.openmaptiles.org/" target="_blank">&copy; OpenMapTiles</a> <a href="https://www.openstreetmap.org/copyright" target="_blank">&copy; OpenStreetMap contributors</a>
0:01:41 DEB [mbtiles:write] - Set mbtiles metadata: version=3.13.0
0:01:41 DEB [mbtiles:write] - Set mbtiles metadata: type=baselayer
0:01:41 DEB [mbtiles:write] - Set mbtiles metadata: bounds=-74.07,21.34,-17.84,43.55
0:01:41 DEB [mbtiles:write] - Set mbtiles metadata: center=-45.955,32.445,3
0:01:41 DEB [mbtiles:write] - Set mbtiles metadata: minzoom=0
0:01:41 DEB [mbtiles:write] - Set mbtiles metadata: maxzoom=14
0:01:42 DEB [mbtiles:write] - Set mbtiles metadata: json={"vector_layers":[{"id":"aerodrome_label","fields":{"name_int":"String","iata":"String","ele_ft":"Number","name_de":"String","name":"String","icao":"String","name:en":"String","class":"String","name_en":"String","name:latin":"String","ele":"Number"},"minzoom":10,"maxzoom":14},{"id":"aeroway","fields":{"ref":"String","class":"String"},"minzoom":10,"maxzoom":14},{"id":"boundary","fields":{"disputed":"Number","admin_level":"Number","maritime":"Number","disputed_name":"String"},"minzoom":0,"maxzoom":14},{"id":"building","fields":{"colour":"String","render_height":"Number","render_min_height":"Number"},"minzoom":13,"maxzoom":14},{"id":"housenumber","fields":{"housenumber":"String"},"minzoom":14,"maxzoom":14},{"id":"landcover","fields":{"subclass":"String","class":"String","_numpoints":"Number"},"minzoom":8,"maxzoom":14},{"id":"landuse","fields":{"class":"String"},"minzoom":4,"maxzoom":14},{"id":"mountain_peak","fields":{"name_int":"String","customary_ft":"Number","ele_ft":"Number","name_de":"String","name":"String","rank":"Number","class":"String","name_en":"String","name:latin":"String","ele":"Number"},"minzoom":7,"maxzoom":14},{"id":"park","fields":{"name_int":"String","name_de":"String","name":"String","name:en":"String","class":"String","name_en":"String","name:latin":"String"},"minzoom":6,"maxzoom":14},{"id":"place","fields":{"name:fy":"String","name_int":"String","capital":"Number","name:uk":"String","name:pl":"String","name:nl":"String","name:be":"String","name:ru":"String","name:ko":"String","name_de":"String","name":"String","rank":"Number","name:en":"String","name:eo":"String","class":"String","name:hu":"String","name:ta":"String","name:zh":"String","name_en":"String","name:latin":"String"},"minzoom":2,"maxzoom":14},{"id":"poi","fields":{"name_int":"String","level":"Number","name:nonlatin":"String","layer":"Number","name_de":"String","name":"String","subclass":"String","indoor":"Number","name:en":"String","class":"String","name:zh":"String","name_en":"String","name:latin":"String"},"minzoom":12,"maxzoom":14},{"id":"transportation","fields":{"access":"String","brunnel":"String","expressway":"Number","surface":"String","bicycle":"String","level":"Number","ramp":"Number","mtb_scale":"String","toll":"Number","oneway":"Number","layer":"Number","network":"String","horse":"String","service":"String","subclass":"String","class":"String","foot":"String"},"minzoom":4,"maxzoom":14},{"id":"transportation_name","fields":{"name_int":"String","name:nonlatin":"String","route_4":"String","route_3":"String","route_2":"String","route_1":"String","layer":"Number","network":"String","ref":"String","name_de":"String","name":"String","subclass":"String","ref_length":"Number","class":"String","name_en":"String","name:latin":"String"},"minzoom":6,"maxzoom":14},{"id":"water","fields":{"intermittent":"Number","class":"String"},"minzoom":0,"maxzoom":14},{"id":"water_name","fields":{"name_int":"String","name:nonlatin":"String","name_de":"String","name":"String","intermittent":"Number","class":"String","name_en":"String","name:latin":"String"},"minzoom":9,"maxzoom":14},{"id":"waterway","fields":{"name_int":"String","brunnel":"String","name_de":"String","_relid":"Number","intermittent":"Number","name":"String","class":"String","name:latin":"String","name_en":"String"},"minzoom":4,"maxzoom":14}]}
0:01:42 INF [mbtiles:write] - Starting z0
0:01:42 INF [mbtiles:write] - Finished z0 in 0s cpu:0s avg:0, now starting z1
0:01:42 INF [mbtiles:write] - Finished z1 in 0s cpu:0s avg:0, now starting z2
0:01:42 INF [mbtiles:write] - Finished z2 in 0s cpu:0s avg:0, now starting z3
0:01:42 INF [mbtiles:write] - Finished z3 in 0s cpu:0s avg:0, now starting z4
0:01:42 INF [mbtiles:write] - Finished z4 in 0s cpu:0s avg:0, now starting z5
0:01:42 INF [mbtiles:write] - Finished z5 in 0s cpu:0s avg:0, now starting z6
0:01:42 INF [mbtiles:write] - Finished z6 in 0s cpu:0s avg:0, now starting z7
0:01:42 INF [mbtiles:write] - Finished z7 in 0s cpu:0s avg:0, now starting z8
0:01:43 INF [mbtiles:write] - Finished z8 in 1s cpu:3s avg:2, now starting z9
0:01:44 INF [mbtiles:write] - Finished z9 in 0.7s cpu:1s avg:2, now starting z10
0:01:45 INF [mbtiles:write] - Finished z10 in 0.5s cpu:1s avg:2, now starting z11
0:01:45 INF [mbtiles:write] - Finished z11 in 0.7s cpu:1s avg:2, now starting z12
0:01:48 INF [mbtiles:write] - Finished z12 in 2s cpu:5s avg:2, now starting z13
0:01:51 INF [mbtiles] -  features: [ 644k  12%  61k/s ] 268M  tiles: [ 290k  29k/s ] 46M  
    cpus: 2 gc:  4% mem: 1.9G/4.2G postGC: 866M
    read( 5%) -> (214/217) -> encode(58% 58%) -> (215/216) -> write( 9%)
    last tile: 13/2469/3048 (z13 4%) https://www.openstreetmap.org/#map=13/41.77131/-71.49902
0:02:01 INF [mbtiles] -  features: [ 1.6M  32% 101k/s ] 268M  tiles: [ 862k  57k/s ] 115M 
    cpus: 2 gc:  2% mem: 2.7G/4.2G postGC: 860M
    read( 3%) ->   (0/217) -> encode(53% 58%) -> (215/216) -> write(15%)
    last tile: 13/3412/3000 (z13 78%) https://www.openstreetmap.org/#map=13/43.32518/-30.05859
0:02:02 INF [mbtiles:write] - Finished z13 in 14s cpu:28s avg:2, now starting z14
0:02:11 INF [mbtiles] -  features: [ 5.2M 100% 360k/s ] 268M  tiles: [ 3.9M 306k/s ] 492M 
    cpus: 1.6 gc:  2% mem: 1.3G/4.2G postGC: 850M
    read( -%) ->   (0/217) -> encode(26% 26%) -> (185/216) -> write(82%)
    last tile: 14/7228/7177 (z14 94%) https://www.openstreetmap.org/#map=14/21.75950/-21.18164
0:02:12 INF [mbtiles:write] - Finished z14 in 10s cpu:15s avg:1.5
0:02:12 INF [mbtiles] -  features: [ 5.2M 100%    0/s ] 268M  tiles: [ 4.1M 365k/s ] 514M 
    cpus: 1 gc:  0% mem: 1.3G/4.2G postGC: 850M
    read( -%) ->   (0/217) -> encode( -%  -%) ->   (0/216) -> write( -%)
    last tile: 14/7380/5985 (z14 100%) https://www.openstreetmap.org/#map=14/43.56447/-17.84180
0:02:12 DEB [mbtiles] - Tile stats:
0:02:12 DEB [mbtiles] - z0 avg:7.9k max:7.9k
0:02:12 DEB [mbtiles] - z1 avg:4k max:4k
0:02:12 DEB [mbtiles] - z2 avg:9.4k max:9.4k
0:02:12 DEB [mbtiles] - z3 avg:3.9k max:6.4k
0:02:12 DEB [mbtiles] - z4 avg:1.6k max:4.6k
0:02:12 DEB [mbtiles] - z5 avg:1.4k max:8.1k
0:02:12 DEB [mbtiles] - z6 avg:1.4k max:24k
0:02:12 DEB [mbtiles] - z7 avg:896 max:33k
0:02:12 DEB [mbtiles] - z8 avg:365 max:48k
0:02:12 DEB [mbtiles] - z9 avg:295 max:278k
0:02:12 DEB [mbtiles] - z10 avg:164 max:232k
0:02:12 DEB [mbtiles] - z11 avg:107 max:131k
0:02:12 DEB [mbtiles] - z12 avg:85 max:118k
0:02:12 DEB [mbtiles] - z13 avg:72 max:109k
0:02:12 DEB [mbtiles] - z14 avg:68 max:257k
0:02:12 DEB [mbtiles] - all avg:70 max:0
0:02:12 DEB [mbtiles] -  # features: 5,276,185
0:02:12 DEB [mbtiles] -     # tiles: 4,115,465
0:02:12 INF [mbtiles] - Finished in 31s cpu:57s gc:1s avg:1.8
0:02:12 INF [mbtiles] -   read    1x(8% 2s wait:25s)
0:02:12 INF [mbtiles] -   encode  2x(45% 14s wait:8s)
0:02:12 INF [mbtiles] -   write   1x(36% 11s sys:1s wait:16s)
0:02:12 INF - Finished in 2m12s cpu:3m49s gc:4s avg:1.7
0:02:12 INF - FINISHED!

0:02:12 INF - ----------------------------------------
0:02:12 INF - 	overall          2m12s cpu:3m49s gc:4s avg:1.7
0:02:12 INF - 	lake_centerlines 0.8s cpu:1s avg:1.7
0:02:12 INF - 	water_polygons   28s cpu:50s gc:2s avg:1.8
0:02:12 INF - 	  read     1x(59% 17s sys:1s wait:3s)
0:02:12 INF - 	  process  2x(29% 8s wait:12s)
0:02:12 INF - 	  write    1x(4% 1s wait:27s)
0:02:12 INF - 	natural_earth    12s cpu:19s avg:1.5
0:02:12 INF - 	  read     1x(90% 11s sys:1s)
0:02:12 INF - 	  process  2x(19% 2s wait:11s)
0:02:12 INF - 	  write    1x(0% 0s wait:12s)
0:02:12 INF - 	osm_pass1        4s cpu:7s avg:1.8
0:02:12 INF - 	  read     1x(3% 0.1s wait:3s)
0:02:12 INF - 	  parse    1x(71% 3s)
0:02:12 INF - 	  process  1x(36% 1s wait:2s)
0:02:12 INF - 	osm_pass2        39s cpu:1m17s avg:2
0:02:12 INF - 	  read     1x(0% 0.1s wait:4s done:35s)
0:02:12 INF - 	  process  2x(75% 30s)
0:02:12 INF - 	  write    1x(1% 0.4s wait:39s)
0:02:12 INF - 	boundaries       0s cpu:0.1s avg:1.8
0:02:12 INF - 	sort             5s cpu:6s avg:1.3
0:02:12 INF - 	  worker  2x(44% 2s done:2s)
0:02:12 INF - 	mbtiles          31s cpu:57s gc:1s avg:1.8
0:02:12 INF - 	  read    1x(8% 2s wait:25s)
0:02:12 INF - 	  encode  2x(45% 14s wait:8s)
0:02:12 INF - 	  write   1x(36% 11s sys:1s wait:16s)
0:02:12 INF - ----------------------------------------
0:02:12 INF - 	features	268MB
0:02:12 INF - 	mbtiles	514MB
-rw-r--r-- 1 runner docker 52M Mar  3 12:19 run.jar