Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integer overflow #28

Open
BarryReidPropeller opened this issue Aug 21, 2015 · 7 comments
Open

Integer overflow #28

BarryReidPropeller opened this issue Aug 21, 2015 · 7 comments

Comments

@BarryReidPropeller
Copy link

Hey guys, I'm getting this error when processing some DEM files, any idea what it could be?

0...10...20...30...40...50...60...70...80...90...ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=25536, nSrcYSize=25536
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
@kalmykov
Copy link

I'm also experiencing same problem. This happens only at level 0. Probably this is because my GeoTIFF is > 4 GB ?

@gberaudo
Copy link
Contributor

There is information about dealing with overflow on the main page:

ctb-tile will resample data from the source dataset when generating tilesets for the various zoom levels. This can lead to performance issues and datatype overflows at lower zoom levels (e.g. level 0) when the source dataset is very large. To overcome this the too...

@buma
Copy link

buma commented Dec 6, 2015

This also happened to me but only if Gtiff type was float32 if I converted it to Int16 and used latest master it converted without errors. cbt-export export tiles that look OK but cesium only renders edges.

@mrgloom
Copy link

mrgloom commented Oct 30, 2016

This workaround solves the problem
#37 (comment)

@mrgloom
Copy link

mrgloom commented Nov 24, 2016

Also what part of code produce Integer overflow ? I can't find it inside terrain builder code, is it inside some part of GDAL lib?

@markerikson markerikson mentioned this issue Mar 27, 2018
Closed
@ediebold ediebold mentioned this issue Mar 27, 2018
Open
@mrgloom mrgloom mentioned this issue May 22, 2018
Closed
@mrgloom
Copy link

mrgloom commented Aug 2, 2018 •
edited
Loading

Still have about the same error using fresh master (8dd633c):

In maps/terrain_tiles I have only these levels generated when ctb fails: 12 13 14 15 16 17 18 19

What other debug info I can provide?

ctb-tile -c 28 -m 32000000000 -s 19 -e 0 maps/global_dem.vrt -o maps/terrain_tiles
0...10...20...30...40...50...60...70...80...90...ERROR 1: Integer overflow : nSrcXSize=26975, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=26975, nSrcYSize=32003
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=26975, nSrcYSize=44633
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: Integer overflow : nSrcXSize=26975, nSrcYSize=22703
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=32003
ERROR 1: Integer overflow : nSrcXSize=26975, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=22703
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=32003
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=22703
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=32003
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Error: Error: Could not read heights from rasterError: Could not read heights from raster
Error: Could not read heights from rasterError: Could not read heights from raster
Error: Could not read heights from raster
Could not read heights from raster
Error: Error: Error: Could not read heights from raster
Error: Could not read heights from raster

Error: 
Could not read heights from raster
Error: Could not read heights from raster
Could not read heights from raster
Could not read heights from raster
Error: Could not read heights from raster
Could not read heights from raster
Error: Could not read heights from raster
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=22703
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=32003
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=22703
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=32003
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster
ERROR 1: Integer overflow : nSrcXSize=44633, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
Error: Could not read heights from raster

@mrgloom
Copy link

mrgloom commented Aug 4, 2018

After build with #64 ctb fails with Integer overflow and also with Segmentation fault:

ctb-tile -c 28 -m 32000000000 -s 19 -e 0 maps/global_dem.vrt -o maps/terrain_tiles
0...10...20...30...40...50...60...70...80...90...ERROR 1: Integer overflow : nSrcXSize=26975, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=26975, nSrcYSize=32003
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: Integer overflow : nSrcXSize=26975, nSrcYSize=44633
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
/home/my_user/bin/cesium_web_tiler/run_dem_terrain_tiler.sh: line 36: 24909 Segmentation fault      (core dumped) /home/my_user/bin/cesium_web_tiler/ctb-tile -c $NUMBER_OF_THREADS -m $MEMORY_LIMIT_BYTES -s $LEVELS_OF_DETAIL -e 0 $MAPS/global_dem.vrt -o $TERRAIN_TILE_OUTPUT_DIR

My dataset is pretty large:

chunks/1/dem_1_ctb.tif Size is 19959, 4899
chunks/2/dem_2_ctb.tif Size is 23586, 12074
chunks/3/dem_3_ctb.tif Size is 17645, 9986
chunks/4/dem_4_ctb.tif Size is 17852, 10201
chunks/5/dem_5_ctb.tif Size is 11980, 13273
chunks/6/dem_6_ctb.tif Size is 3936, 15193
chunks/7/dem_7_ctb.tif Size is 21975, 12461
chunks/8/dem_8_ctb.tif Size is 18397, 10634
chunks/9/dem_9_ctb.tif Size is 18992, 11043
chunks/10/dem_10_ctb.tif Size is 18638, 10974
chunks/11/dem_11_ctb.tif Size is 19068, 10738
chunks/12/dem_12_ctb.tif Size is 20782, 11839
chunks/13/dem_13_ctb.tif Size is 22171, 11949
chunks/14/dem_14_ctb.tif Size is 18362, 9974
chunks/15/dem_15_ctb.tif Size is 18707, 10209
chunks/16/dem_16_ctb.tif Size is 18369, 9883
chunks/17/dem_17_ctb.tif Size is 19283, 10605
chunks/18/dem_18_ctb.tif Size is 21028, 7391
chunks/19/dem_19_ctb.tif Size is 26781, 6805
chunks/20/dem_20_ctb.tif Size is 26591, 6870
chunks/21/dem_21_ctb.tif Size is 26338, 6924
chunks/22/dem_22_ctb.tif Size is 26377, 6675
chunks/23/dem_23_ctb.tif Size is 26541, 6901
chunks/24/dem_24_ctb.tif Size is 25901, 6841
chunks/25/dem_25_ctb.tif Size is 24106, 6593
chunks/26/dem_26_ctb.tif Size is 24038, 6470
chunks/27/dem_27_ctb.tif Size is 24681, 6460
chunks/28/dem_28_ctb.tif Size is 23563, 6441
chunks/29/dem_29_ctb.tif Size is 24438, 6528
chunks/30/dem_30_ctb.tif Size is 23882, 4632
chunks/31/dem_31_ctb.tif Size is 25720, 2267
chunks/32/dem_32_ctb.tif Size is 25769, 2346
chunks/33/dem_33_ctb.tif Size is 26187, 2304
chunks/34/dem_34_ctb.tif Size is 25747, 2438
chunks/35/dem_35_ctb.tif Size is 24563, 2666
chunks/36/dem_36_ctb.tif Size is 24310, 2578
chunks/37/dem_37_ctb.tif Size is 25335, 2485
chunks/38/dem_38_ctb.tif Size is 23609, 2598
chunks/39/dem_39_ctb.tif Size is 16040, 7366
chunks/40/dem_40_ctb.tif Size is 18516, 9021

98M	chunks/1/dem_1_ctb.tif
136M	chunks/2/dem_2_ctb.tif
107M	chunks/3/dem_3_ctb.tif
106M	chunks/4/dem_4_ctb.tif
109M	chunks/5/dem_5_ctb.tif
122M	chunks/6/dem_6_ctb.tif
136M	chunks/7/dem_7_ctb.tif
113M	chunks/8/dem_8_ctb.tif
115M	chunks/9/dem_9_ctb.tif
116M	chunks/10/dem_10_ctb.tif
113M	chunks/11/dem_11_ctb.tif
128M	chunks/12/dem_12_ctb.tif
143M	chunks/13/dem_13_ctb.tif
112M	chunks/14/dem_14_ctb.tif
114M	chunks/15/dem_15_ctb.tif
111M	chunks/16/dem_16_ctb.tif
123M	chunks/17/dem_17_ctb.tif
117M	chunks/18/dem_18_ctb.tif
138M	chunks/19/dem_19_ctb.tif
138M	chunks/20/dem_20_ctb.tif
141M	chunks/21/dem_21_ctb.tif
140M	chunks/22/dem_22_ctb.tif
144M	chunks/23/dem_23_ctb.tif
142M	chunks/24/dem_24_ctb.tif
132M	chunks/25/dem_25_ctb.tif
132M	chunks/26/dem_26_ctb.tif
134M	chunks/27/dem_27_ctb.tif
128M	chunks/28/dem_28_ctb.tif
136M	chunks/29/dem_29_ctb.tif
127M	chunks/30/dem_30_ctb.tif
136M	chunks/31/dem_31_ctb.tif
136M	chunks/32/dem_32_ctb.tif
139M	chunks/33/dem_33_ctb.tif
137M	chunks/34/dem_34_ctb.tif
137M	chunks/35/dem_35_ctb.tif
134M	chunks/36/dem_36_ctb.tif
149M	chunks/37/dem_37_ctb.tif
144M	chunks/38/dem_38_ctb.tif
136M	chunks/39/dem_39_ctb.tif
123M	chunks/40/dem_40_ctb.tif

This was referenced Aug 4, 2018
Open
Closed
@markerikson markerikson mentioned this issue Sep 7, 2018
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@kalmykov @buma @mrgloom @gberaudo @BarryReidPropeller