2d frustum culling #3944

Weasy666 · 2022-02-14T13:35:53Z

Objective

Enable frustum culling for 2D, to be more precise, for Mesh2d, Sprite and TextureAtlasSprite.

Solution

Mesh2d is essentially only a regular Mesh with its handle wrapped by Mesh2dHandle, which enables it to be drawn by the 2d pipeline. As such it already has the compute_aabb() function, which i used for frustum culling the same way it is done for the regular Mesh.
Sprite has an image/texture, which provides a size, or a user specified custom_size. I've added ComputedVisibility to the SpriteBundle and use the size to create an Aabb and insert it as a component. The visibility is computed by the already existing check_visibility system.
For TextureAtlasSprite i use the user specified custom_size, or a size computed from the sprite's Rect as specified in the TextureAtlas. I've added ComputedVisibility to the SpriteSheetBundle too and the rest is the same as with what i have done for the Sprite.

Benchmarks

System: Windows 10, AMD Ryzen 5 5600XT, AMD Radeon RX 570

	Bevymark	many_sprites
without culling	~32-33 fps / CPU ~9%	~52 fps / CPU ~19%
with culling	~32-33 fps / CPU ~10%	~60 fps / CPU ~6%

Commands

cargo run --release --example bevymark -- 13 10000
cargo run --release --example many_sprites

Additional stuff

I have run tracy-trace too, here are two captures of many_sprites, one with and one without culling.
tracy_traces.zip

The traces show, that a lot of the additional fps come from the time drop in queue_sprites, from 15.5ms to 43us. That should come from the reduced number of extracted sprites that need to be queued.

alice-i-cecile

The math checks out and the code quality is good. The benchmarks are encouraging too! I really like reducing performance footguns for new users.

However, this needs a lot of tests: this functionality is notoriously fragile. We should test with:

scaled sprites
scaled cameras
orthographic zoomed cameras
rotated cameras and sprites
sprites that are on the border of the screen
sprites that are larger than the camera's view
sprites that are moving onto the screen
sprites that are just spawned

alice-i-cecile · 2022-02-14T17:48:40Z

crates/bevy_sprite/src/render/mod.rs

@@ -178,6 +181,45 @@ impl SpecializedPipeline for SpritePipeline {
    }
 }

+pub fn calculate_bounds(


This needs doc strings: it's a public system.

Ups...that's an oversight, the system doesn't need to be public.

Hm...well maybe not exactly an oversight, because all the other functions in that file are pub too. But i can make it pub(crate) calculate_bounds if you want.

Weasy666 · 2022-02-14T21:53:18Z

However, this needs a lot of tests: this functionality is notoriously fragile. We should test with:

scaled sprites

scaled cameras

orthographic zoomed cameras

rotated cameras and sprites

sprites that are on the border of the screen

sprites that are larger than the camera's view

sprites that are moving onto the screen

sprites that are just spawned

Uff...well...do such tests exist for 3d frustum culling? I am mostly using what is already used for 3d, so it should work equally well for 2d as it does for 3d.

But i have tested all your points

scaled sprites: manually tested with sprite and a scale of Vec3::new(2.0, 1.0, 1.0) and Vec3::new(2.0, 2.0, 1.0)
scaled cameras: manually tested with sprite and the camera transform set to a scale of Vec3::new(0.5, 0.5, 1.0) and Vec3::new(2.0, 2.0, 1.0)
~~orthographic zoomed cameras:~~ that is essentially just a scaled camera.
rotated cameras and sprites: covered by bevymark and many_sprites
sprites that are on the border of the screen: manually tested with mesh2d and sprite_sheet, by moving the camera to values around x = 700.
sprites that are larger than the camera's view: manually tested with sprite and a scale of Vec3::new(10.0, 10.0, 1.0)
sprites that are moving onto the screen: covered by bevymark
sprites that are just spawned: covered by bevymark

alice-i-cecile · 2022-02-14T22:20:12Z

Uff...well...do such tests exist for 3d frustum culling

There aren't, but there should be :p I didn't review that PR though. It should be fine to create these tests for either 2D or 3D.

But i have tested all your points

It's great to hear that these all work, and it makes me much more confident in this PR. But tests aren't just for verifying that the code works at time of creation; they're essential for ensuring that code isn't accidentally broken in the future <3 I want contributors to feel confident when trying to further optimize or refactor this code, so we should add automated tests for this to the code base.

Weasy666 · 2022-02-14T22:38:14Z

Sure, i can understand that and think it is useful. But such tests also don't exist for 3d, so i think that is out of scope for this PR and should probably be its own PR which adds such tests for both cases, 2d and 3d.

superdump

A couple of minor suggestions that are probably inconsequential but worth trying anyway, and please try batch insertion as for many entities it should be faster.

superdump · 2022-03-03T22:20:12Z

crates/bevy_sprite/src/render/mod.rs

+            let size = sprite.custom_size.unwrap_or_else(|| image.size());
+            let aabb = Aabb {
+                center: Vec3::ZERO,
+                half_extents: size.extend(0.0) * 0.5,


Maybe faster?

Suggested change

half_extents: size.extend(0.0) * 0.5,

half_extents: (0.5 * size).extend(0.0),

Will try that. Now that you mention this, i noticed that in some other places divisions are used instead of multiplications, but as far as i know or as far as i was taught, multiplications are preferred because they are (in most cases) faster than divisions. Is this something we want to change? Not in this PR, but in a new one.

superdump · 2022-03-04T00:13:23Z

crates/bevy_sprite/src/render/mod.rs

+                    .unwrap_or_else(|| (rect.min - rect.max).abs());
+                let aabb = Aabb {
+                    center: Vec3::ZERO,
+                    half_extents: size.extend(0.0) * 0.5,


Suggested change

half_extents: size.extend(0.0) * 0.5,

half_extents: (0.5 * size).extend(0.0),

superdump · 2022-03-04T00:14:41Z

crates/bevy_sprite/src/mesh2d/mesh.rs

+    for (entity, mesh_handle) in without_aabb.iter() {
+        if let Some(mesh) = meshes.get(&mesh_handle.0) {
+            if let Some(aabb) = mesh.compute_aabb() {
+                commands.entity(entity).insert(aabb);


Batch insertion may be faster. Please try it.

Moxinilian · 2022-03-05T19:08:39Z

Thank you for this! I tested it in its current state. Unfortunately it broke the rendering of my project. I have made a small reproducing example:

use bevy::prelude::*;

fn main() {
    App::new()
        .add_plugins(DefaultPlugins)
        .add_startup_system(setup)
        .run();
}

const SPRITE_WIDTH: f32 = 32.0;
const SPRITE_HEIGHT: f32 = 32.0;

fn setup(
    mut commands: Commands,
    asset_server: Res<AssetServer>,
    mut texture_atlases: ResMut<Assets<TextureAtlas>>,
) {
    let texture = asset_server.load("texture.png");
    let mut camera = OrthographicCameraBundle::new_2d();

    camera.transform.scale.z = 1000.0;
    commands.spawn_bundle(camera);

    for i in 0..10 {
        for j in 0..10 {
            let (x, y) = (i as f32 * SPRITE_WIDTH, j as f32 * SPRITE_HEIGHT);

            commands.spawn_bundle(SpriteBundle {
                texture: texture.clone(),
                transform: Transform::from_translation(Vec3::new(x, y, -y)),
                ..Default::default()
            });
        }
    }
}

Expected result (from current master and bevy 0.6) [sorry for the hexagonal tile, that's what I had at hand]:

Result with this PR:

Weasy666 · 2022-03-09T14:46:02Z

Sorry for the late reply! Can you reproduce this bug with cubes and OrthographicCameraBundle::new_3d()?

EDIT: Tried it myself. 3D seems to not have the same problem. That will be fun to debug 🙃.

The 3D code i used

use bevy::prelude::*;

fn main() {
    App::new()
        .insert_resource(Msaa { samples: 4 })
        .add_plugins(DefaultPlugins)
        .add_startup_system(setup)
        .run();
}

const CUBE_WIDTH: f32 = 0.11;
const CUBE_HEIGHT: f32 = 0.11;

fn setup(
    mut commands: Commands,
    mut meshes: ResMut<Assets<Mesh>>,
) {
    let mesh = meshes.add(Mesh::from(shape::Cube { size: 0.1 }));
    let mut camera = OrthographicCameraBundle::new_3d();
    camera.transform.scale.z = 1000.0;
    commands.spawn_bundle(camera);

    for i in 0..10 {
        for j in 0..10 {
            let (x, y) = (i as f32 * CUBE_WIDTH, j as f32 * CUBE_HEIGHT);
            commands.spawn_bundle(PbrBundle {
                mesh: mesh.clone(),
                transform: Transform::from_translation(Vec3::new(x, y, -y)),
                ..default()
            });
        }
    }
}

Weasy666 · 2022-03-09T16:07:28Z

Ok...it boils down to how the two orthographic cameras are set up.
2D has the camera at Transform::from_xyz(0.0, 0.0, 1000.0 - 0.1), while the 3D orthographic camera is at Transform::from_xyz(0.0, 0.0, 0.0). That means that 0 on the z-axis is closest to the 2D orthographic camera, so if z is negative in a sprite transform, then that "moves" the sprite essentially behind the camera, and it gets culled. How do other engines behave in such a case? Is this behavior expected/normal for an orthographic camera?

IceSentry · 2022-12-25T19:53:11Z

if z is negative in a sprite transform, then that "moves" the sprite essentially behind the camera

In the context of bevy we use a reverse-z, so I would expect negative z to show since it should be in front of the camera.

Other than that, I would indeed expect sprites behind the camera to be culled, it doesn't really make sense to have sprites behind the camera.

Weasy666 · 2023-04-21T14:22:13Z

Closing in favor of #7885

@Weasy666

#7885) # Objective - Add `Aabb` calculation for `Sprite`, `TextureAtlasSprite` and `Mesh2d`. - Enable frustum culling for 2D entities since frustum culling requires a `Aabb` component in the entity to function. - Improve 2D performance massively when there are many sprites out of view. (ex: `many_sprites`) ## Solution - Derived from @Weasy666's #3944 pull request, which had no activity since multiple months. - Adapted the code to the latest version of Bevy. - Added support for sprites with non-center `Anchor`s to avoid culling prematurely when part of the sprite is still in view or not culling when sprite is already out of view. ### Note - Gives 15.8x performance boosts in some scenarios. (5 fps vs 79 fps with 409600 sprites in `many_sprites`) --------- Co-authored-by: ira <[email protected]>

Weasy666 added 3 commits February 14, 2022 10:01

Add frustum culling for Mesh2d

896dd4b

Add frustum culling for Sprite

370409b

Add frustum culling for TextureAtlasSprite

570d481

github-actions bot added the S-Needs-Triage This issue needs to be labelled label Feb 14, 2022

alice-i-cecile added C-Performance A change motivated by improving speed, memory usage or compile times A-Rendering Drawing game state to the screen and removed S-Needs-Triage This issue needs to be labelled labels Feb 14, 2022

alice-i-cecile requested changes Feb 14, 2022

View reviewed changes

alice-i-cecile approved these changes Feb 14, 2022

View reviewed changes

superdump requested changes Mar 4, 2022

View reviewed changes

james7132 mentioned this pull request Jun 24, 2022

render::view::RenderLayers does not affect SpriteBundle #5089

Closed

mockersf added the S-Adopt-Me The original PR author has no intent to complete this work. Pick me up! label Dec 25, 2022

rparrett mentioned this pull request Jan 31, 2023

[Merged by Bors] - Cleanup many sprites stress tests #7436

Closed

opstic mentioned this pull request Mar 3, 2023

Add Aabb calculation for Sprite, TextureAtlasSprite and Mesh2d #7885

Merged

Weasy666 closed this Apr 21, 2023

opstic mentioned this pull request Jul 13, 2023

Sprites not being rendered with translation.z < 0.0 #9138

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

2d frustum culling #3944

2d frustum culling #3944

Weasy666 commented Feb 14, 2022 •

edited

Loading

alice-i-cecile left a comment •

edited

Loading

alice-i-cecile Feb 14, 2022

Weasy666 Feb 14, 2022

Weasy666 Feb 14, 2022

Weasy666 commented Feb 14, 2022

alice-i-cecile commented Feb 14, 2022

Weasy666 commented Feb 14, 2022

superdump left a comment

superdump Mar 3, 2022

Weasy666 Mar 4, 2022 •

edited

Loading

superdump Mar 4, 2022

superdump Mar 4, 2022

Moxinilian commented Mar 5, 2022 •

edited

Loading

Weasy666 commented Mar 9, 2022 •

edited

Loading

Weasy666 commented Mar 9, 2022

IceSentry commented Dec 25, 2022 •

edited

Loading

Weasy666 commented Apr 21, 2023

	half_extents: size.extend(0.0) * 0.5,
	half_extents: (0.5 * size).extend(0.0),

2d frustum culling #3944

2d frustum culling #3944

Conversation

Weasy666 commented Feb 14, 2022 • edited Loading

Objective

Solution

Benchmarks

Commands

Additional stuff

alice-i-cecile left a comment • edited Loading

Choose a reason for hiding this comment

alice-i-cecile Feb 14, 2022

Choose a reason for hiding this comment

Weasy666 Feb 14, 2022

Choose a reason for hiding this comment

Weasy666 Feb 14, 2022

Choose a reason for hiding this comment

Weasy666 commented Feb 14, 2022

alice-i-cecile commented Feb 14, 2022

Weasy666 commented Feb 14, 2022

superdump left a comment

Choose a reason for hiding this comment

superdump Mar 3, 2022

Choose a reason for hiding this comment

Weasy666 Mar 4, 2022 • edited Loading

Choose a reason for hiding this comment

superdump Mar 4, 2022

Choose a reason for hiding this comment

superdump Mar 4, 2022

Choose a reason for hiding this comment

Moxinilian commented Mar 5, 2022 • edited Loading

Weasy666 commented Mar 9, 2022 • edited Loading

Weasy666 commented Mar 9, 2022

IceSentry commented Dec 25, 2022 • edited Loading

Weasy666 commented Apr 21, 2023

Weasy666 commented Feb 14, 2022 •

edited

Loading

alice-i-cecile left a comment •

edited

Loading

Weasy666 Mar 4, 2022 •

edited

Loading

Moxinilian commented Mar 5, 2022 •

edited

Loading

Weasy666 commented Mar 9, 2022 •

edited

Loading

IceSentry commented Dec 25, 2022 •

edited

Loading