OS.get_window_safe_area returning area off screen

[Tekuzo]

Godot version

3.3.2 Stable

System information

iOS 14, Android 11

Issue description

I am currently working on a mobile game, and one of my testers owns an iPhone 11 and he let me know that my UI is being cut off by the notch on his phone.

I am trying to use the function OS.get_window_safe_area() to adjust my user interface so that my UI is not cut off by the notch. The documentation says that this function returns a rect2, and the rect2 documentation says that it is a X, Y, Width and Height so I use the following code to adjust the rect of a control node that has my UI in it.

var safe_area = get_window_safe_area();
var pos_x = safe_area.position.x;
var pos_y = safe_area.position.y;
var safe_width = safe_area.end.x;
var safe_height = safe_area.end.y;
var safe_position = Vector2(pos_x, pos_y);
var safe_size = Vector2(safe_width, safe_height);
$canvaslayer/control.set_position(safe_position, false);
$canvaslayer/control.set_size(safe_size, false);

When I adjust my control node with the position and width / height of the safe area, this is what my UI looks like

The function is returning a size that is larger than the resolution that my game is configured to run at.

I suspect that I have some sort of configuration problem in the display / window settings, but I have googled to try to resolve this issue, I have posted on forums and discords, and nobody that I have spoken to seems to know how to use this function properly or what is happening with my game.

Steps to reproduce

My game is set up with a resolution of 360 * 640, and I have the test width and test height set to the same dimensions.

The stretch mode is set to viewport, and the stretch aspect is set to expand.

When I run this function on my iPhone SE it returns (0, 0, 750, 1334).

The position of 0,0 I would expect on my personal iPhone (the iPhone SE does not have a notch), the 750, 1334 is the resolution of the phone (which would imply that the full screen width and height is safe), but when I apply this size to the control, it shifts my UI around. Items that are anchored on the right and side, and bottom are moved off screen.

This issue doesn't happen when I run the game on PC / Mac / Linux because it renders in a 360 * 640 window.

Once again, I suspect that this is User Error and not an actual bug. I have asked in forums and discords and I have googled to no avail on how to get this function to work the way I would like it to.

Minimal reproduction project

Sample Project.zip

[Calinou]

Can you reproduce this in landscape orientation (both in the Project Settings and export preset)? I wonder if portrait mode breaks it somehow.

Remember that in 3.3, the iOS device orientation needs to be set both in the Project Settings and the export preset. This will no longer be the case in 3.4 since #48943 was merged.

[Tekuzo]

Good question, I will test, when I get home. I forgot the dongle to my M1 at home.

[Tekuzo]

Orientation inside of project settings is set to Sensor_Portrait
and when I export the project I have "Portrait" and "Portrait Upside Down" selected

When I get home, I will disable the sensor portrait and force the game to be just portrait normal and see if there is a difference

[kleonc]

var safe_area = get_window_safe_area();
var pos_x = safe_area.position.x;
var pos_y = safe_area.position.y;
var safe_width = safe_area.end.x;
var safe_height = safe_area.end.y;
var safe_position = Vector2(pos_x, pos_y);
var safe_size = Vector2(safe_width, safe_height);
$canvaslayer/control.set_position(safe_position, false);
$canvaslayer/control.set_size(safe_size, false);

Rect2 returned by the OS.get_window_safe_area() is supposed to be in the window coordinate space so you'd need to convert it to the proper coordinate space that the given Control belongs to. You can read more about what transformations are being applied and in what order in the docs (although I'd say there are some errors in that linked page, e.g. it seems to incorrectly refer to the "post-stretch viewport coordinates" (which is equivalent to "window coordinates" for the root viewport edit: not true, see my next comment) as "screen coordinates").
So what I think should work is something like this (although I didn't test it at all):

var control = $canvaslayer/control

# Just some explanation:
# control.get_viewport_transform() # transforms from "relative to CanvasLayer" to "relative to Viewport (post-stretch)"
# control.get_global_transform() # transforms from "relative to itself" to "relative to CanvasLayer"
# control.get_transform() # transforms from "relative to itself" to "relative to parent"
# control.get_transform().affine_inverse() # transforms from "relative to parent" to "relative to itself"
# and transformations are applied from right to left, so:

# combined: parent -> itself -> CanvasLayer -> Viewport (post-stretch)
var parent_to_viewport = control.get_viewport_transform() * control.get_global_transform() * control.get_transform().affine_inverse()

var viewport_to_parent = parent_to_viewport.affine_inverse()

var safe_area_relative_to_parent = viewport_to_parent.xform(get_window_safe_area())
control.rect_position = safe_area_relative_to_parent.position
control.rect_size = safe_area_relative_to_parent.size

[Tekuzo]

var safe_area = get_window_safe_area();
var pos_x = safe_area.position.x;
var pos_y = safe_area.position.y;
var safe_width = safe_area.end.x;
var safe_height = safe_area.end.y;
var safe_position = Vector2(pos_x, pos_y);
var safe_size = Vector2(safe_width, safe_height);
$canvaslayer/control.set_position(safe_position, false);
$canvaslayer/control.set_size(safe_size, false);

Rect2 returned by the OS.get_window_safe_area() is supposed to be in the window coordinate space so you'd need to convert it to the proper coordinate space that the given Control belongs to. You can read more about what transformations are being applied and in what order in the docs (although I'd say there are some errors in that linked page, e.g. it seems to incorrectly refer to the "post-stretch viewport coordinates" (which is equivalent to "window coordinates" for the root viewport) as "screen coordinates").
So what I think should work is something like this (although I didn't test it at all):

var control = $canvaslayer/control

# Just some explanation:
# control.get_viewport_transform() # transforms from "relative to CanvasLayer" to "relative to Viewport (post-stretch)"
# control.get_global_transform() # transforms from "relative to itself" to "relative to CanvasLayer"
# control.get_transform() # transforms from "relative to itself" to "relative to parent"
# control.get_transform().affine_inverse() # transforms from "relative to parent" to "relative to itself"
# and transformations are applied from right to left, so:

# combined: parent -> itself -> CanvasLayer -> Viewport (post-stretch)
var parent_to_viewport = control.get_viewport_transform() * control.get_global_transform() * control.get_transform().affine_inverse()

var viewport_to_parent = parent_to_viewport.affine_inverse()

var safe_area_relative_to_parent = viewport_to_parent.xform(get_window_safe_area())
control.rect_position = safe_area_relative_to_parent.position
control.rect_size = safe_area_relative_to_parent.size

So, I implemented your code exactly.

and this is the result that I got when it ran on my iPhone SE.

[kleonc]

@Tekuzo Ok, I've done some testing myself. Seems like "viewport rect" -> "screen rect" part of the transformation isn't incorporated into root viewport's transform (note that I'm refering to 3.x branch, didn't take a look at master) and this transformation is of course affected by the window stretch settings from the project settings (reference: loading settings and updating root rect). The problem is that Viewport::get_attach_to_screen_rect() isn't exposed/binded so I think it's not possible to obtain that transformation in GDScript (it's possible to set it using Viewport.set_attach_to_screen_rect() or VisualServer.viewport_attach_to_screen() though). At least I didn't find a way to do so. 🤔
As a workaround you can manually incorporate it into calculations. This should work (at least for your specific case, works for me on Android):

var window_to_root = Transform2D.IDENTITY.scaled(get_tree().root.size / OS.window_size)
var safe_area_root = window_to_root.xform(OS.get_window_safe_area())

var control = $CanvasLayer/Control

assert(control.get_viewport() == get_tree().root, "Assumption: control is not in a nested Viewport")
var parent_to_root = control.get_viewport_transform() * control.get_global_transform() * control.get_transform().affine_inverse()
var root_to_parent = parent_to_root.affine_inverse()

var safe_area_relative_to_parent = root_to_parent.xform(safe_area_root)
control.rect_position = safe_area_relative_to_parent.position
control.rect_size = safe_area_relative_to_parent.size

[Tekuzo]

Perfect, I will ask my brother in law to test on his iPhone 11 now.

[Calinou]

Isn't there still something we need to fix/improve on our end? For instance, exposing Viewport::get_attach_to_screen_rect() to scripting.

[Tekuzo]

I am not sure if I can answer that question.
Should I reopen the issue?

[Calinou]

Should I reopen the issue?

It seems there's still something to fix/improve, so I'll reopen it.

[naithar]

@Tekuzo you can also use the example code from this PR: #40761 with minor changes (OS.get_window_safe_area and OS.get_window_size instead of DisplayServer.screen_get_usable_rect and DisplayServer.screen_get_size).
That's the base of what I'm using to work with safe area and UI.
I'm also using viewport's size_changed signal to update safe area in case orientation or anything else forces view to resize.

[alexzheng]

That's because the rect returned by get_window_safe_area is in device pixel, not in viewport coordinate space. It can not be used directly for node in gdscript.
I usually convert it by a scale of get_viewport_rect().size.x/OS.window_size.x
That's not convenient.

[Forien]

I was looking everywhere, but I still can't figure this out, especially since Godot 4 changed some of the API.

I found via google the solution by @kleonc but it no longer works and I am too inexperienced with Godot to figure it out myself.

All I want is to set up a margin to push everything out of the notch (or just the UI stuff)

[ESnider]

Hey, so it's a year and a half later. I've searched and it's not clear to me if this is solved or still waiting for a fix. Anyone else working on Godot mobile games, and especially on iOS?