Support recording multiple CommandBuffers in RenderContext (bevyengin…

…e#7248)

# Objective
`RenderContext`, the core abstraction for running the render graph, currently only supports recording one `CommandBuffer` across the entire render graph. This means the entire buffer must be recorded sequentially, usually via the render graph itself. This prevents parallelization and forces users to only encode their commands in the render graph.

## Solution
Allow `RenderContext` to store a `Vec<CommandBuffer>` that it progressively appends to. By default, the context will not have a command encoder, but will create one as soon as either `begin_tracked_render_pass` or the `command_encoder` accesor is first called. `RenderContext::add_command_buffer` allows users to interrupt the current command encoder, flush it to the vec, append a user-provided `CommandBuffer` and reset the command encoder to start a new buffer. Users or the render graph will call `RenderContext::finish` to retrieve the series of buffers for submitting to the queue.

This allows users to encode their own `CommandBuffer`s outside of the render graph, potentially in different threads, and store them in components or resources.

Ideally, in the future, the core pipeline passes can run in `RenderStage::Render` systems and end up saving the completed command buffers to either `Commands` or a field in `RenderPhase`. 

## Alternatives
The alternative is to use to use wgpu's `RenderBundle`s, which can achieve similar results; however it's not universally available (no OpenGL, WebGL, and DX11).

---

## Changelog
Added: `RenderContext::new`
Added: `RenderContext::add_command_buffer`
Added: `RenderContext::finish`
Changed: `RenderContext::render_device` is now private. Use the accessor `RenderContext::render_device()` instead.
Changed: `RenderContext::command_encoder` is now private. Use the accessor `RenderContext::command_encoder()` instead.
Changed: `RenderContext` now supports adding external `CommandBuffer`s for inclusion into the render graphs. These buffers can be encoded outside of the render graph (i.e. in a system).

## Migration Guide
`RenderContext`'s fields are now private. Use the accessors on `RenderContext` instead, and construct it with `RenderContext::new`.

crates/bevy_core_pipeline/src/core_2d/main_pass_2d_node.rs

@@ -101,7 +101,7 @@ impl Node for MainPass2dNode {
             };
 
             render_context
-                .command_encoder
+                .command_encoder()
                 .begin_render_pass(&pass_descriptor);
         }
 

crates/bevy_core_pipeline/src/core_3d/main_pass_3d_node.rs

@@ -205,7 +205,7 @@ impl Node for MainPass3dNode {
             };
 
             render_context
-                .command_encoder
+                .command_encoder()
                 .begin_render_pass(&pass_descriptor);
         }
 

crates/bevy_core_pipeline/src/fxaa/node.rs

@@ -78,7 +78,7 @@ impl Node for FxaaNode {
             Some((id, bind_group)) if source.id() == *id => bind_group,
             cached_bind_group => {
                 let sampler = render_context
-                    .render_device
+                    .render_device()
                     .create_sampler(&SamplerDescriptor {
                         mipmap_filter: FilterMode::Linear,
                         mag_filter: FilterMode::Linear,
@@ -88,7 +88,7 @@ impl Node for FxaaNode {
 
                 let bind_group =
                     render_context
-                        .render_device
+                        .render_device()
                         .create_bind_group(&BindGroupDescriptor {
                             label: None,
                             layout: &fxaa_pipeline.texture_bind_group,
@@ -120,7 +120,7 @@ impl Node for FxaaNode {
         };
 
         let mut render_pass = render_context
-            .command_encoder
+            .command_encoder()
             .begin_render_pass(&pass_descriptor);
 
         render_pass.set_pipeline(pipeline);

crates/bevy_core_pipeline/src/prepass/node.rs

@@ -122,7 +122,7 @@ impl Node for PrepassNode {
 
         if let Some(prepass_depth_texture) = &view_prepass_textures.depth {
             // Copy depth buffer to texture
-            render_context.command_encoder.copy_texture_to_texture(
+            render_context.command_encoder().copy_texture_to_texture(
                 view_depth_texture.texture.as_image_copy(),
                 prepass_depth_texture.texture.as_image_copy(),
                 view_prepass_textures.size,

crates/bevy_core_pipeline/src/tonemapping/node.rs

@@ -72,12 +72,12 @@ impl Node for TonemappingNode {
             Some((id, bind_group)) if source.id() == *id => bind_group,
             cached_bind_group => {
                 let sampler = render_context
-                    .render_device
+                    .render_device()
                     .create_sampler(&SamplerDescriptor::default());
 
                 let bind_group =
                     render_context
-                        .render_device
+                        .render_device()
                         .create_bind_group(&BindGroupDescriptor {
                             label: None,
                             layout: &tonemapping_pipeline.texture_bind_group,
@@ -112,7 +112,7 @@ impl Node for TonemappingNode {
         };
 
         let mut render_pass = render_context
-            .command_encoder
+            .command_encoder()
             .begin_render_pass(&pass_descriptor);
 
         render_pass.set_pipeline(pipeline);

crates/bevy_core_pipeline/src/upscaling/node.rs

@@ -63,12 +63,12 @@ impl Node for UpscalingNode {
             Some((id, bind_group)) if upscaled_texture.id() == *id => bind_group,
             cached_bind_group => {
                 let sampler = render_context
-                    .render_device
+                    .render_device()
                     .create_sampler(&SamplerDescriptor::default());
 
                 let bind_group =
                     render_context
-                        .render_device
+                        .render_device()
                         .create_bind_group(&BindGroupDescriptor {
                             label: None,
                             layout: &upscaling_pipeline.texture_bind_group,
@@ -108,7 +108,7 @@ impl Node for UpscalingNode {
         };
 
         let mut render_pass = render_context
-            .command_encoder
+            .command_encoder()
             .begin_render_pass(&pass_descriptor);
 
         render_pass.set_pipeline(pipeline);

crates/bevy_render/src/camera/camera_driver_node.rs

@@ -98,7 +98,7 @@ impl Node for CameraDriverNode {
             };
 
             render_context
-                .command_encoder
+                .command_encoder()
                 .begin_render_pass(&pass_descriptor);
         }
 

crates/bevy_render/src/renderer/graph_runner.rs

@@ -58,18 +58,12 @@ impl RenderGraphRunner {
         queue: &wgpu::Queue,
         world: &World,
     ) -> Result<(), RenderGraphRunnerError> {
-        let command_encoder =
-            render_device.create_command_encoder(&wgpu::CommandEncoderDescriptor::default());
-        let mut render_context = RenderContext {
-            render_device,
-            command_encoder,
-        };
-
+        let mut render_context = RenderContext::new(render_device);
         Self::run_graph(graph, None, &mut render_context, world, &[])?;
         {
             #[cfg(feature = "trace")]
             let _span = info_span!("submit_graph_commands").entered();
-            queue.submit(vec![render_context.command_encoder.finish()]);
+            queue.submit(render_context.finish());
         }
         Ok(())
     }

crates/bevy_render/src/renderer/mod.rs

@@ -17,7 +17,9 @@ use bevy_ecs::prelude::*;
 use bevy_time::TimeSender;
 use bevy_utils::Instant;
 use std::sync::Arc;
-use wgpu::{Adapter, AdapterInfo, CommandEncoder, Instance, Queue, RequestAdapterOptions};
+use wgpu::{
+    Adapter, AdapterInfo, CommandBuffer, CommandEncoder, Instance, Queue, RequestAdapterOptions,
+};
 
 /// Updates the [`RenderGraph`] with all of its nodes and then runs it to render the entire frame.
 pub fn render_system(world: &mut World) {
@@ -278,20 +280,68 @@ pub async fn initialize_renderer(
 /// The [`RenderDevice`] is used to create render resources and the
 /// the [`CommandEncoder`] is used to record a series of GPU operations.
 pub struct RenderContext {
-    pub render_device: RenderDevice,
-    pub command_encoder: CommandEncoder,
+    render_device: RenderDevice,
+    command_encoder: Option<CommandEncoder>,
+    command_buffers: Vec<CommandBuffer>,
 }
 
 impl RenderContext {
+    /// Creates a new [`RenderContext`] from a [`RenderDevice`].
+    pub fn new(render_device: RenderDevice) -> Self {
+        Self {
+            render_device,
+            command_encoder: None,
+            command_buffers: Vec::new(),
+        }
+    }
+
+    /// Gets the underlying [`RenderDevice`].
+    pub fn render_device(&self) -> &RenderDevice {
+        &self.render_device
+    }
+
+    /// Gets the current [`CommandEncoder`].
+    pub fn command_encoder(&mut self) -> &mut CommandEncoder {
+        self.command_encoder.get_or_insert_with(|| {
+            self.render_device
+                .create_command_encoder(&wgpu::CommandEncoderDescriptor::default())
+        })
+    }
+
     /// Creates a new [`TrackedRenderPass`] for the context,
     /// configured using the provided `descriptor`.
     pub fn begin_tracked_render_pass<'a>(
         &'a mut self,
         descriptor: RenderPassDescriptor<'a, '_>,
     ) -> TrackedRenderPass<'a> {
-        TrackedRenderPass::new(
-            &self.render_device,
-            self.command_encoder.begin_render_pass(&descriptor),
-        )
+        // Cannot use command_encoder() as we need to split the borrow on self
+        let command_encoder = self.command_encoder.get_or_insert_with(|| {
+            self.render_device
+                .create_command_encoder(&wgpu::CommandEncoderDescriptor::default())
+        });
+        let render_pass = command_encoder.begin_render_pass(&descriptor);
+        TrackedRenderPass::new(&self.render_device, render_pass)
+    }
+
+    /// Append a [`CommandBuffer`] to the queue.
+    ///
+    /// If present, this will flush the currently unflushed [`CommandEncoder`]
+    /// into a [`CommandBuffer`] into the queue before append the provided
+    /// buffer.
+    pub fn add_command_buffer(&mut self, command_buffer: CommandBuffer) {
+        self.flush_encoder();
+        self.command_buffers.push(command_buffer);
+    }
+
+    /// Finalizes the queue and returns the queue of [`CommandBuffer`]s.
+    pub fn finish(mut self) -> Vec<CommandBuffer> {
+        self.flush_encoder();
+        self.command_buffers
+    }
+
+    fn flush_encoder(&mut self) {
+        if let Some(encoder) = self.command_encoder.take() {
+            self.command_buffers.push(encoder.finish());
+        }
     }
 }

examples/shader/compute_shader_game_of_life.rs

@@ -216,7 +216,7 @@ impl render_graph::Node for GameOfLifeNode {
         let pipeline = world.resource::<GameOfLifePipeline>();
 
         let mut pass = render_context
-            .command_encoder
+            .command_encoder()
             .begin_compute_pass(&ComputePassDescriptor::default());
 
         pass.set_bind_group(0, texture_bind_group, &[]);