Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix level of detail at high pitch #4750

Open
wants to merge 30 commits into
base: globe
Choose a base branch
from

Conversation

NathanMOlson
Copy link

@NathanMOlson NathanMOlson commented Sep 23, 2024

This fixes #3983 and improves #4703 by calculating the desired zoom level for each tile independently. Each tile zoom is calculated by scaling the center point zoom by the ratio of (3D) distance-to-tile to distance-to-center. For Mercator transform, the old behavior is preserved when there is no terrain and pitch < 60 degrees. For Globe transform, the new behavior applies in all cases.

Before:

before_terrain before

After:

after2_terrain after2

Here's my understanding of the previous implementation:
image
image

And here's a drawing of how it works now (in crude 3d):

image

Here's a description of how it works.

The center point is rendered at the requested zoom level.
the 3D distance from the camera to each candidate tile is calculated and the desired zoom for that tile is the requested zoom scaled by ratio of camera-to-center distance to camera-to-tile distance. If the candidate tile is lower resolution than desired, the tile is split.
Note that rather than use the true 3D distance to each tile (accounting for terrain), it is assumed that all tiles are at the same elevation as the center point. This prevents "resolution explosion" when the camera is very close to a hill, and keeps the resolution variation from map center smooth.

@NathanMOlson NathanMOlson changed the base branch from main to globe September 23, 2024 21:49
@codecov-commenter
Copy link

codecov-commenter commented Sep 23, 2024

Codecov Report

Attention: Patch coverage is 84.53608% with 15 lines in your changes missing coverage. Please review.

Project coverage is 63.01%. Comparing base (d0981a0) to head (4f58b3b).

Files with missing lines Patch % Lines
src/ui/map.ts 50.00% 5 Missing ⚠️
src/geo/projection/globe_transform.ts 60.00% 4 Missing ⚠️
src/geo/projection/mercator_covering_tiles.ts 89.65% 2 Missing and 1 partial ⚠️
src/geo/transform_helper.ts 70.00% 2 Missing and 1 partial ⚠️

❗ There is a different number of reports uploaded between BASE (d0981a0) and HEAD (4f58b3b). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (d0981a0) HEAD (4f58b3b)
9 8
Additional details and impacted files
@@             Coverage Diff             @@
##            globe    #4750       +/-   ##
===========================================
- Coverage   87.98%   63.01%   -24.98%     
===========================================
  Files         265      265               
  Lines       37577    37628       +51     
  Branches     2336     1702      -634     
===========================================
- Hits        33064    23713     -9351     
- Misses       3482    12966     +9484     
+ Partials     1031      949       -82     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@NathanMOlson
Copy link
Author

I could use some help with the failing render and integration tests.

I've looked through the render results and they look good to me, we are loading higher-resolution tiles at high pitch, which is the desired and expected behavior.

Example:

image

The integration tests rely on specific terrain tiles that are kept in /test/integration/assets/tiles; the newly requested tiles are not present, causing test failure.

  1. Who can make the call of whether the new results are acceptable?
  2. Is there an automatic way to change the render tests to pass with the results generated by this branch?
  3. If I need to add new terrain tiles to the repository, how do I go about generating them?

@HarelM
Copy link
Collaborator

HarelM commented Sep 24, 2024

Check out the readme in the integration test folder.
You can use UPDATE flag to update the results.
You can download the html from the CI and copy the image (right click, copy image) and replace the current image.
Regarding terrain tiles, you can take the relevant ones from the demo tiles repo according to what's missing, assuming it's the right position, if not, you can move the center to the relevant place.
Hope that helps.
Let me know if not.

@HarelM
Copy link
Collaborator

HarelM commented Sep 24, 2024

There was a concern about performance when the terrain is using a higher zoom, which will create a higher resolution terrain, which will impact performance.
While I believe this fix is the right approach, how this performance risk mitigate?

@NathanMOlson
Copy link
Author

There was a concern about performance when the terrain is using a higher zoom, which will create a higher resolution terrain, which will impact performance. While I believe this fix is the right approach, how this performance risk mitigate?

@HarelM This is how the performance risk is mitigated: 53f42b6. Terrain sources now observe the deltaZoom property, which is currently hardcoded to 1, meaning terrain tiles load at half resolution.

@HarelM
Copy link
Collaborator

HarelM commented Sep 24, 2024

Overall, this looks like a good solution as it seems very specific to covering tiles code, which I'm happy to see is in specific files and generally contained.
Can you share a bit more about the changes in terms of calculations and design, as it is hard to understand from the changes in the code.
i.e. how was it before vs how it is now, and some drawing to illustrate it if possible.

Thanks for all the works on this! Truly appreciated.

@NathanMOlson
Copy link
Author

Can you share a bit more about the changes in terms of calculations and design, as it is hard to understand from the changes in the code.
i.e. how was it before vs how it is now, and some drawing to illustrate it if possible.

Here's my understanding of the previous implementation:
image
image

And here's a drawing of how it works now (in crude 3d):

image

Here's a description of how it works.

  1. The center point is rendered at the requested zoom level.
  2. the 3D distance from the camera to each candidate tile is calculated and the desired zoom for that tile is the requested zoom scaled by ratio of camera-to-center distance to camera-to-tile distance. If the candidate tile is lower resolution than desired, the tile is split.

Note that rather than use the true 3D distance to each tile (accounting for terrain), it is assumed that all tiles are at the same elevation as the center point. This prevents "resolution explosion" when the camera is very close to a hill, and keeps the resolution variation from map center smooth.

@HarelM
Copy link
Collaborator

HarelM commented Sep 24, 2024

Thanks! Can you update the initial PR comment with these drawing so that people coming to this PR in the future can get a good grasp of the change fairly fast?
Cc: @kubapelc

@HarelM
Copy link
Collaborator

HarelM commented Sep 25, 2024

I've looked at some of the "x" terrain render tests and I'm not sure I understand what I see.
Closer to the camera the tiles are in higher resolution, then at the center the resolution is lower, but then when going further away the resolution increases again, this looked strange to me...

@HarelM
Copy link
Collaborator

HarelM commented Sep 25, 2024

Also, there were a lot of osm tiles added, are all of them really needed? Can't we use the "x"s instead, or less tiles? (It's like 50 or so tiles, I hope it's not for a single scene...)

…hed". I am not sure what the actual problem is, but it seems to have something to do with globeCoveringTiles() returning tiles that don't match the zoom level returned by globeCoveringZoom(). So we don't do that at low zoom levels.
@NathanMOlson
Copy link
Author

I added the number tiles like you requested, and a render test using them.

Here's the effects of this PR. For these images the requested zoom is 16.25 (which gets scaled to 17.25 for theses 256x256 tiles). At pitch =0, all tiles are from z17.

For reference, pitch = 0:
image

Before:
60 degrees
globe60
75 degrees
globe75

After:
60 degrees
expected
75 degrees
actual75

@HarelM
Copy link
Collaborator

HarelM commented Sep 26, 2024

This is a great showcase of which tiles are used in the map! The colors you added makes it even more visible! Great!
As a side note, when running the render test in debug mode you can see the requests printed to the console for fetching tiles, from that you can easily count how many tiles are fetched.
You can run the render test on a single file to see only its results.

In any case, looking at the graph you added, I think the DEM is ok, but the raster is too "heavy", looking at the image at 75 degrees I think 19 tiles are not needed and also tiles after the center needs to change "faster" to lower resolution.

That's my two cents, let me know what you think.

@HarelM
Copy link
Collaborator

HarelM commented Sep 26, 2024

CC: @louwers note that this change might cause a mismatch between native and web as this changes which zoom level tiles are loaded. If this only affect rendering when terrain is on then this shouldn't be a problem at this point as native doesn't support terrain, but it's worth remembering.

@NathanMOlson
Copy link
Author

In any case, looking at the graph you added, I think the DEM is ok, but the raster is too "heavy", looking at the image at 75 degrees I think 19 tiles are not needed and also tiles after the center needs to change "faster" to lower resolution.

In the images above, the center point is on a hill. This is the "worst case" for this change. Here's an example of the opposite, center point in a valley (requested zoom level 13.25, which is 14.25 for 256x256 tiles):
image
In this image, the nearest tiles are rendered at too low resolution, which means this doesn't entirely fix #4703.

The cause of this is that the distance to each tile is calculated assuming the tile is at the same elevation as the center point. This means areas lower than the center point will be rendered with "too high" resolution and areas above the center point will be rendered with "too low" resolution.

Here's what the "no elevation" case looks like:
image
In this case, tile selection is "perfect", meaning all tiles in the image are as close as possible to the x resolution requested at the center point.

To fully resolve #4703, the distance to each tile would be calculated using the elevation of each tile. This would complicate the code significantly, as well as making the requested tiles much "jumpier" in response to camera motion. So I decided not to attempt that in this PR.

I really like the proposed tile selection scheme, it is very understandable (at least to me :)) and based on the distance from camera to each tile. I could change the relationship between distance and tile selection (to reduce resolution and top and bottom of image as you suggest), but that would require an arbitrary fudge factor that would uglify what currently has a beautiful geometric interpretation :)

If I need to change the behavior, can you give me a target for max tiles to load (or ratio to current number of tiles loaded)?

@NathanMOlson
Copy link
Author

@HarelM How's this look? This is what happens if I don't try to maintain constant tile width on screen. Instead I try to maintain constant tile area on screen. It definitely uses less tiles, but I'm worried it won't solve the original problem.

image

@xabbu42
Copy link
Contributor

xabbu42 commented Sep 26, 2024

How does this interact with styling and corresponding data changes depending on the zoom level?

For example we only show small roads or buildings on high zoom levels and also include the data in vector tiles on high zoom levels. So mixing different zoom levels may lead to inconsistencies on the map. This problem already exists now as far as I can tell, but judging from the images I fear this pull requests will increase the mixing of zoomlevels especially close to the camera where the inconsistencies are more noticeable. Is this true?

@AbelVM
Copy link

AbelVM commented Sep 26, 2024

We were talking about a related issue at https://osmus.slack.com/archives/C01G3D28DAB/p1727331286766569 .

Maybe an issue about these concerns should be opened for future reference

@NathanMOlson
Copy link
Author

I fear this pull requests will increase the mixing of zoomlevels especially close to the camera where the inconsistencies are more noticeable. Is this true?

The main effect of this PR is to shift tiles at high pitch angles to higher zoom levels, not to increase the mixing of zoom levels. For example, in the field-of-view-number render test, the zoom levels ranged from 14-17 before this PR and 15-18 after this PR.

@HarelM
Copy link
Collaborator

HarelM commented Sep 26, 2024

Maybe we can introduce some sort of parameter to define the behavior of the tiles? Somewhat similar to the exaggeration parameter...?
Can you create a pitch 60 no terrain tiles with numbers so we could better understand the current behavior?
Looking at the image above, I would expect to see more zoom 13 tiles at the center (three rows instead of just one) making the zoom changes less aggressive, maybe, IDK...
Do you think you can create an html page where one could move a range input to play with it and see how it behaves with different values?
Generally speaking, I would prefer if the slope of the curve of how many tiles are loaded would be less steep.
60-70 tiles max at pitch 85 sounds more reasonable to me, but it's just a hunch...

@HarelM
Copy link
Collaborator

HarelM commented Sep 26, 2024

Can you also better explain the mountain vs valley effect?
For each tile the center altitude is calculated so you could get that "for free" if needed I think, not sure complicating the tile loading algorithm is with it.
Maybe a way to replace the tile loading logic would allow people to finetune this for their needs in some situations... I'm just throwing ideas here.

@NathanMOlson
Copy link
Author

Can you also better explain the mountain vs valley effect?

In this PR's implementation, tile selection is independent of terrain. Here's a picture that illustration the mountain/valley effect:
image

It is not the center elevation we need to fix this. We would need the elevation of each tile, in order to calculate how far from the camera it is so we can figure out what zoom level it is. This seems like it would get messy in a hurry (you can't figure out what tiles to request until you know their elevation, but you don't know the elevation until you have the tiles. What elevation is used for a tile? Max? min? mean? ...)

@NathanMOlson
Copy link
Author

Can you create a pitch 60 no terrain tiles with numbers so we could better understand the current behavior?

The behavior at pitch <=60 with no terrain tiles is preserved from before. In this case, as before, all tiles are loaded at the requested zoom level of the center point.

image

@NathanMOlson
Copy link
Author

Maybe we can introduce some sort of parameter to define the behavior of the tiles? Somewhat similar to the exaggeration parameter...?

Could do. That parameter would have to trickle down to mercatorCoveringTiles from somewhere, where would it originate?

Do you think you can create an html page where one could move a range input to play with it and see how it behaves with different values?

No, I don't know how to do that.

@xabbu42
Copy link
Contributor

xabbu42 commented Sep 26, 2024

I fear this pull requests will increase the mixing of zoomlevels especially close to the camera where the inconsistencies are more noticeable. Is this true?

The main effect of this PR is to shift tiles at high pitch angles to higher zoom levels, not to increase the mixing of zoom levels. For example, in the field-of-view-number render test, the zoom levels ranged from 14-17 before this PR and 15-18 after this PR.

Good to know. Also it is good to know that we can sidestep the issue by restricting pitch to 60 degree.

That said in the before picture without terrain, I see consistent tiles used for the lower 2/3 of the picture where inconsistencies would be very noticeable. the first border between tiles of different zoom levels is quite far in the distance. In the after picture on the other hand the lowest row uses an additional zoom level with possible inconsistencies very close to the camera.

But maybe this is not a problem in the real world if the same zoom for styling is used. after all maps tend to just add data on higher zoom levels and not remove it. so in most cases using zoom 15 tiles with a zoom 14 style will just fetch some features which are not displayed (assuming corresponding minzoom values in the style) and still show a consistent map.

@HarelM
Copy link
Collaborator

HarelM commented Sep 27, 2024

All the configurations options here have something that the user can understand:
https://maplibre.org/maplibre-gl-js/docs/API/type-aliases/MapOptions/
If we decide to add another configuration option we need to make sure it can be understood by the user to avoid trial and error as much as possible.
A different approach to the same configuration could be something like "maxDifferentZoomLevelsInSceneForMaxPitch".
It's a mouth full and maybe we can shorten the parameter name, but a user can define this as 1 for example to make sure all tiles are the same or 10 to create a scene where the closest tile has a very high resolution.

This is just an idea, if you can think or a different/better way to convey the idea to user that can work too.

@NathanMOlson
Copy link
Author

NathanMOlson commented Sep 27, 2024

@HarelM I added a parameter with the dubious name pitchBehavior. You can explore the effect of this parameter here: https://nathanmolson.github.io/lodfix_demo. You can control pitch and pitchBehavior using the ugly controls in the top right corner. :)

The reasonable range for pitchBehavior is [0, 2]. I've set the default value to 1. The integer values have the following interpretations:

0: As the map pitches, maintain constant tile screen x-resolution at the center. (Loads the most tiles)
1: As the map pitches, maintain constant tile screen area at the center.
2: As the map pitches, maintain constant tile screen y-resolution at the center. (Loads the fewest tiles)

Values outside [0,2] can also be applied, but I can't think of any reason to do so.

Of particular interest is the behavior as pitch crosses 60 degrees, with and without terrain. To match previous behavior, "below 60 degrees, no terrain" is a special case that loads all tiles with the same zoom level.

@HarelM
Copy link
Collaborator

HarelM commented Sep 27, 2024

Why is 60 degrees no terrain a special case?
I think that we should aim at making the behavior smoother. Right now when crossing 60 degrees the tiles loading definition changes dramatically.
I think it might be better to find a way to make it smoother.
I also still feel that the changes of tiles zoom towards the horizon is aggressive no matter which parameter value I use.
It might be worth an online discussion somehow as I think thread is becoming longer than expected.

@NathanMOlson
Copy link
Author

Why is 60 degrees no terrain a special case?

"60 degrees no terrain" is a special case because it was already a special case. I don't know why it was already a special case, and I didn't want to change that when I didn't know the reason behind it. I would personally love to eliminate that special case and have a smooth transition across all pitch angles.

It might be worth an online discussion somehow as I think thread is becoming longer than expected.

Sure.

@NathanMOlson NathanMOlson mentioned this pull request Sep 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Raster tiles are much lower detail when terrain is enabled
5 participants