From 024c724fa058f2b64c0083f50245cfebce171c3a Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Thu, 13 Jan 2022 16:22:14 -0500 Subject: [PATCH 001/313] Initial motivation --- rfcs/45-stageless.md | 120 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 120 insertions(+) create mode 100644 rfcs/45-stageless.md diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md new file mode 100644 index 00000000..fcf92114 --- /dev/null +++ b/rfcs/45-stageless.md @@ -0,0 +1,120 @@ +# Feature Name: `stageless` + +## Summary + +The existing stage boundaries result in a large number of complications: preventing us from operating across stages. +Moreover, several related features (run criteria, states and fixed timesteps) are rather fragile and overly complex. +This tangled mess needs to be holistically refactored: simplifying the scheduling model down into a single `Schedule`, moving towards an intent-based configuration style, taking full advantage of labels and leveraging exclusive systems to handle complex logic. + +## Motivation + +The fundamental challenges with the current stage-driven scheduling abstraction are [numerous](https://github.com/bevyengine/bevy/discussions/2801). +Of particular note: + +- Dependencies do not work across stages. +- States do not work across stages. +- Plugins can add new, standalone stages. +- Run criteria (including states!) cannot be composed. +- The stack-driven state model is overly elaborate and does not enable enough use cases to warrant its complexity. +- Fixed timestep run criteria do not behave in a fashion that supports. +- The architecture for turn-based games and other even slightly unusual patterns is not obvious. +- We cannot easily implement a builder-style strategy for configuring systems. + +Unfortunately, all of these problems are deeply interwoven. Despite our best efforts, fixing them incrementally at either the design or implementation stage is impossible, as it results in myopic architecture choices and terrible technical debt. + +## User-facing explanation + +This explanation is, in effect, user-facing documentation for the new design. +In addition to a few new concepts, it throws out much of the current system scheduling design that you may be familiar with. +The following elements are radically reworked: + +- schedules (flattened) +- run criteria (can no longer loop, are now systems) +- states (simplified, no longer run-criteria powered) +- fixed time steps (no longer a run criteria) +- exclusive systems (no longer special-cased) +- stages (yeet) + +### Scheduling overview and intent-based configuration + +Systems in Bevy are stored in a `Schedule`: a collection of **configured systems**. +Each frame, the `App`'s `runner` function will run the schedule, +causing the **scheduler** to run the systems in parallel using a strategy that respects all of the configured constraints (or panic, if it is unsatisfiable). + +In the beginning, each `Schedule` is entirely unordered: systems will be selected in an arbitrary order and run if and only if all of the data that it must access is free. +Just like with standard borrow checking, multiple systems can read from the same data at once, but writing to the data requires an exclusive lock. + +While this is a good, simple strategy for maximizing parallelism, it comes with some serious drawbacks for reproducibility and correctness. +The relative execution order of systems tends to matter! +In simple projects and toy examples, the solution to this looks simple: let the user specify a single global ordering and use this to determine the order in the case of conflicts! + +Unfortunately, this strategy falls apart at scale, particularly when attempting to integrate third-party dependencies (in the form of plugins), who have complex, interacting requirements that must be respected. +While users may be able to find a single working strategy through trial-and-error (and a robust test suite!), this strategy becomes ossified: impossible to safely change to incorporate new features or optimize performance. + +The fundamental problem is a mismatch between *how* the schedule is configured and *why* it is configured that way. +Information about "physics must run after input but before rendering" is implicitly encoded into the order of our systems, and changes which violate these constraints will silently fail in far-reaching ways without giving any indication as to what the user did wrong. + +TODO: intent-based configuration. + +## Implementation strategy + +This is the technical portion of the RFC. +Try to capture the broad implementation strategy, +and then focus in on the tricky details so that: + +- Its interaction with other features is clear. +- It is reasonably clear how the feature would be implemented. +- Corner cases are dissected by example. + +When necessary, this section should return to the examples given in the previous section and explain the implementation details that make them work. + +When writing this section be mindful of the following [repo guidelines](https://github.com/bevyengine/rfcs): + +- **RFCs should be scoped:** Try to avoid creating RFCs for huge design spaces that span many features. Try to pick a specific feature slice and describe it in as much detail as possible. Feel free to create multiple RFCs if you need multiple features. +- **RFCs should avoid ambiguity:** Two developers implementing the same RFC should come up with nearly identical implementations. +- **RFCs should be "implementable":** Merged RFCs should only depend on features from other merged RFCs and existing Bevy features. It is ok to create multiple dependent RFCs, but they should either be merged at the same time or have a clear merge order that ensures the "implementable" rule is respected. + +## Drawbacks + +Why should we *not* do this? + +## Rationale and alternatives + +- Why is this design the best in the space of possible designs? +- What other designs have been considered and what is the rationale for not choosing them? +- What objections immediately spring to mind? How have you addressed them? +- What is the impact of not doing this? +- Why is this important to implement as a feature of Bevy itself, rather than an ecosystem crate? + +## \[Optional\] Prior art + +Discuss prior art, both the good and the bad, in relation to this proposal. +This can include: + +- Does this feature exist in other libraries and what experiences have their community had? +- Papers: Are there any published papers or great posts that discuss this? + +This section is intended to encourage you as an author to think about the lessons from other tools and provide readers of your RFC with a fuller picture. + +Note that while precedent set by other engines is some motivation, it does not on its own motivate an RFC. + +## Unresolved questions + +- What parts of the design do you expect to resolve through the RFC process before this gets merged? +- What parts of the design do you expect to resolve through the implementation of this feature before the feature PR is merged? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +## \[Optional\] Future possibilities + +Think about what the natural extension and evolution of your proposal would +be and how it would affect Bevy as a whole in a holistic way. +Try to use this section as a tool to more fully consider other possible +interactions with the engine in your proposal. + +This is also a good place to "dump ideas", if they are out of scope for the +RFC you are writing but otherwise related. + +Note that having something written down in the future-possibilities section +is not a reason to accept the current or a future RFC; such notes should be +in the section on motivation or rationale in this or subsequent RFCs. +If a feature or change has no direct value on its own, expand your RFC to include the first valuable feature that would build on it. From 7c8776f682dba0f4596ec1b4703e9f5c0b40381b Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Thu, 13 Jan 2022 18:59:02 -0500 Subject: [PATCH 002/313] Basics of system configuration --- rfcs/45-stageless.md | 130 +++++++++++++++++++++++++++++++++++++------ 1 file changed, 114 insertions(+), 16 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index fcf92114..baeb764b 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -33,16 +33,20 @@ The following elements are radically reworked: - states (simplified, no longer run-criteria powered) - fixed time steps (no longer a run criteria) - exclusive systems (no longer special-cased) +- command processing (now performed in a `flush_commands` exclusive system) +- labels (can now be directly configured) - stages (yeet) +- system sets (yeet) ### Scheduling overview and intent-based configuration Systems in Bevy are stored in a `Schedule`: a collection of **configured systems**. Each frame, the `App`'s `runner` function will run the schedule, -causing the **scheduler** to run the systems in parallel using a strategy that respects all of the configured constraints (or panic, if it is unsatisfiable). +causing the **scheduler** to run the systems in parallel using a strategy that respects all of the configured constraints (or panic, if it is **unsatisfiable**). In the beginning, each `Schedule` is entirely unordered: systems will be selected in an arbitrary order and run if and only if all of the data that it must access is free. Just like with standard borrow checking, multiple systems can read from the same data at once, but writing to the data requires an exclusive lock. +Systems which cannot be run in parallel are said to be **incompatible**. While this is a good, simple strategy for maximizing parallelism, it comes with some serious drawbacks for reproducibility and correctness. The relative execution order of systems tends to matter! @@ -52,9 +56,101 @@ Unfortunately, this strategy falls apart at scale, particularly when attempting While users may be able to find a single working strategy through trial-and-error (and a robust test suite!), this strategy becomes ossified: impossible to safely change to incorporate new features or optimize performance. The fundamental problem is a mismatch between *how* the schedule is configured and *why* it is configured that way. -Information about "physics must run after input but before rendering" is implicitly encoded into the order of our systems, and changes which violate these constraints will silently fail in far-reaching ways without giving any indication as to what the user did wrong. +The fact that "physics must run after input but before rendering" is implicitly encoded into the order of our systems, and changes which violate these constraints will silently fail in far-reaching ways without giving any indication as to what the user did wrong. -TODO: intent-based configuration. +The solution is **intent-based configuration**: record the exact, minimal set of constraints needed for a scheduling strategy to be valid, and then let the scheduler figure it out! +As the number of systems in your app grows, the number of possible scheduling strategies grows exponentially, and the set of *valid* scheduling strategies grows almost as quickly. +However, each system will continue to work hand-in-hand with a small number of closely-related systems, interlocking in a straightforward and comprehensible way. + +By configuring our systems with straightforward, local rules, we can allow the scheduler to pick one of the possible valid paths, optimizing as it sees fit without breaking our logic. +Just as importantly, we are not over-constraining our ordering, allowing subtle ordering bugs to surface quickly and then be stamped out with a well-considered rule. + +### Introduction to system configuration + +When a function is added to a schedule, it is turned into a `RawSystem` and combined with a `SystemConfig` struct to create a `ConfiguredSystem`. + +A system may be configured in the following ways: + +- it may have any number of **labels** (which implement the `SystemLabel` trait) + - labels are used by other systems to determine ordering constraints + - e.g. `.add_system(gravity.label(GameLogic::Physics))` + - labels themselves may be configured, allowing you to define high-level structure in your `App` in a single place + - e.g `app.configure_label(GameLogic::Physics, SystemConfig::new().label(GameLogic::Physics).after(GameLogic::Input))` +- it may have ordering constraints, causing it to run before or after other systems + - there are several variations on this, see the next section for details + - e.g. `.add_system(player_controls.before(GameLogic::Physics))` +- it may have one or more **run criteria** attached + - a system is only executed if all of its run criteria returns `true` + - **states** are a special, more complex pattern that use run criteria to determine whether or not a system should run in a current state + - e.g. `.add_system(physics.run_if(GameLogic::Physics))` + +Defining system configuration in a `SystemConfig` struct, can be useful to reuse, compose and quickly apply complex configuration strategies, before applying these strategies to labels or individual systems: + +```rust +#[derive(SystemLabel)] +enum PhysicsLabel { + Forces, + CollisionDetection, + CollisionHandling, +} + +impl Plugin for PhysicsPlugin{ + + fn build(app: &mut App){ + // Import all of the `PhysicsLabel` variants for brevity + use PhysicsLabel::*; + + // Within each frame, physics logic needs to occur after input handling, but before rendering + let mut common_physics_config = SystemConfig::new().after(CoreLabel::InputHandling).before(CoreLabel::Rendering); + + app + // We can reuse this shared configuration on each of our labels + .add_label(Forces.configure(common_physics_config).before(CollisionDetection)) + .add_label(CollisionDetection.configure(common_physics_config).before(CollisionHandling)) + .add_label(CollisionHandling.configure(common_physics_config)) + // And then apply that config to each of the systems that have this label + .add_system(gravity.label(Forces)) + // These systems have a linear chain of ordering dependencies between them + // Systems earlier in the chain must run before those later in the chain + .add_system_chain([broad_pass, narrow_pass].label(CollisionDetection)) + .add_system(compute_forces.label(CollisionHandling)) + } +} +``` + +### System ordering + +The most important way in which we can configure systems is by specifying **when** they run. +The ordering of systems is always defined relative to other systems: either directly or by checking the systems that belong to a label. + +There are two basic forms of system ordering constraints: + +1. **Strict ordering constraints:** `strictly_before` and `strictly_after` + 1. A system cannot be scheduled until "strictly before" systems must have been completed this frame. + 2. Simple and explicit. + 3. Can cause unneccessary blocking, particularly when systems are configured at a high-level. +2. **If-needed ordering constraints:** `.before` and `.after` + 1. A system cannot be scheduled until any "before" systems that it is incompatible with have completed this frame. + 2. In 99% of cases, this is the desired behavior. Unless you are using interior mutability, systems that are compatible will always be **commutative**: their ordering doesn't matter. + +Applying an ordering constraint to or from a label causes a ordering constraint to be created between all individual members of that label. + +In addition to the `.before` and `.after` methods, you can use **system chains** to create very simple linear dependencies between the succesive members of an array of systems. + +When discussing system ordering, it is particularly important to call out the `flush_commands` system. +This **exclusive system** (meaning, it can modify the entire `World` in arbitrary ways and cannot be run in parallel with other systems) collects all created commands and applies them to the `World`. + +Commands (commonly used to spawn and despawn entities or add and remove components) will not take effect until this system is run, so be sure that you run a copy of the `flush_commands` system before relying on the result of a command! + +TODO: create more sugar for this, then make an example! + +### Run criteria + +### States + +### Complex control flow + +#### Fixed time steps ## Implementation strategy @@ -76,27 +172,29 @@ When writing this section be mindful of the following [repo guidelines](https:// ## Drawbacks -Why should we *not* do this? +1. This will be a massively breaking change to every single Bevy user. +2. It will be harder to immediately understand the global structure of Bevy apps. ## Rationale and alternatives -- Why is this design the best in the space of possible designs? -- What other designs have been considered and what is the rationale for not choosing them? -- What objections immediately spring to mind? How have you addressed them? -- What is the impact of not doing this? -- Why is this important to implement as a feature of Bevy itself, rather than an ecosystem crate? +### Why can't you label labels? + +If each label is associated with a `SystemConfig`, and part of this struct is which labels are applied, why can't we label labels themselves? -## \[Optional\] Prior art +In short: this way lies madness. +While at first it may seem appealing to be able create nested hierarchies of labels, this ends up recreating many of the problems of a stage-based design's global perspective. +If labels can themselves cause other labels to be applied, it becomes dramatically harder to track how systems are configured, and why they are configured in that way. -Discuss prior art, both the good and the bad, in relation to this proposal. -This can include: +This is a problem for users, as it makes code much harder to read, reason about and maintain. +It is also a problem for our tools, forcing a recursive architecture when generating schedules and limiting the ability for developer tools to communicate both the how and why of a system's configuration. -- Does this feature exist in other libraries and what experiences have their community had? -- Papers: Are there any published papers or great posts that discuss this? +### Why is if-needed the correct default ordering strategy? -This section is intended to encourage you as an author to think about the lessons from other tools and provide readers of your RFC with a fuller picture. +Strict ordering is simple and explicit, and will never result in strange logic errors. +On the other hand, it *will* result in pointless and surprising blocking behavior, possibly leading to unsatisfiable schedules. -Note that while precedent set by other engines is some motivation, it does not on its own motivate an RFC. +If-needed ordering is the corret stategy in virtually cases: in Bevy, interior mutability at the component or resource level is rare, almost never needed and results in other serious and subtle bugs. +As we move towards specifying system ordering dependencies at scale, it is critical to avoid spuriously breaking users schedules, and silent, pointless performance hits are never good. ## Unresolved questions From cc8aa27c626a9442e52b92a2c2310cd52b3c10cf Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Thu, 13 Jan 2022 20:44:35 -0500 Subject: [PATCH 003/313] Run criteria design --- rfcs/45-stageless.md | 56 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 52 insertions(+), 4 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index baeb764b..20820940 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -120,7 +120,7 @@ impl Plugin for PhysicsPlugin{ ### System ordering -The most important way in which we can configure systems is by specifying **when** they run. +The most important way we can configure systems is by telling the scheduler *when* they should be run. The ordering of systems is always defined relative to other systems: either directly or by checking the systems that belong to a label. There are two basic forms of system ordering constraints: @@ -146,6 +146,47 @@ TODO: create more sugar for this, then make an example! ### Run criteria +While ordering constraints determine *when* a system will run, **run criteria** will determine *if* it will run at all. +A run criteria is a special kind of system, which can read data from the `World` and returns a boolean value. +If its output is `true`, the system it is attached to will run during this pass of the `Schedule`; +if it is `false`, the system will be skipped. +Systems that are skipped are considered completed for the purposes of ordering constraints. + +Let's examine a few ways we can specify run criteria: + +```rust +// This function can be used as a run criteria system, +// because it only reads from the `World` and returns `bool` +fn construction_timer_finished(timer: Res) -> bool { + timer.finished() +} + +fn main(){ + App::new() + .add_plugins(DefaultPlugins) + // We can add functions as run criteria + .add_system(update_construction_progress.run_if(construction_timer_finished)) + // We can use closures for simple one-off run criteria, + // which automatically fetch the appropriate data from the `World` + .add_system(spawn_more_enemies.run_if(|difficulty: Res| difficulty >= 9000)) + // The `run_if_resource_is` method is convenient syntactic sugar that generates a run criteria + // for when you want to check the value of a resource (commonly an enum) + .add_system(gravity.run_if_resource_equals(Gravity::Enabled)) + // Run criteria can be attached to labels: a copy of the run criteria will be applied to each system with that label + .configure_label(GameLabel::Physics, |label: impl SystemLabel| {label.run_if_resource_equals(Paused(false))} ) + .run() +} +``` + +When multiple run criteria are attached to the same system, the system will run if and only if all of those run criteria return true. + +Run criteria are evaluated "just before" the system that is attached to is run. +The state that they read from will always be "valid" when the system that they control is being run: this is important to avoid strange bugs caused by race conditions and non-deterministic system ordering. +It is impossible, for example, for a game to be paused *after* the run criteria checked if the game was paused but before the system that relied on the game not being paused completes. +In order to understand exactly what this means, we need to understand atomic ordering constraints. + +### Atomic ordering constraints + ### States ### Complex control flow @@ -196,11 +237,18 @@ On the other hand, it *will* result in pointless and surprising blocking behavio If-needed ordering is the corret stategy in virtually cases: in Bevy, interior mutability at the component or resource level is rare, almost never needed and results in other serious and subtle bugs. As we move towards specifying system ordering dependencies at scale, it is critical to avoid spuriously breaking users schedules, and silent, pointless performance hits are never good. +### Why can't we just use system chaining for run criteria? + +There are two reasons why this doesn't work: + +1. The system we're applying a run criteria does not have an input type. +2. System chaining does not work if the chained systems are incompatible. This is far too limiting. + ## Unresolved questions -- What parts of the design do you expect to resolve through the RFC process before this gets merged? -- What parts of the design do you expect to resolve through the implementation of this feature before the feature PR is merged? -- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? +- Should we allow users to compose run criteria in more complex ways? + - How does this work for run criteria that are not locally defined? +- How do we ergonomically allow users to specify that commands must be flushed between two systems? ## \[Optional\] Future possibilities From 2e4890680175ea1e66458eecc0bbafcdb401ac98 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 00:56:19 -0500 Subject: [PATCH 004/313] Atomic ordering constraints --- rfcs/45-stageless.md | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 20820940..dea0812c 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -151,6 +151,7 @@ A run criteria is a special kind of system, which can read data from the `World` If its output is `true`, the system it is attached to will run during this pass of the `Schedule`; if it is `false`, the system will be skipped. Systems that are skipped are considered completed for the purposes of ordering constraints. +If for some reason run criteria are themselves skipped, they are considered to have returned `true`. Let's examine a few ways we can specify run criteria: @@ -178,7 +179,7 @@ fn main(){ } ``` -When multiple run criteria are attached to the same system, the system will run if and only if all of those run criteria return true. +**When multiple run criteria are attached to the same system, the system will run if and only if all of those run criteria return true.** Run criteria are evaluated "just before" the system that is attached to is run. The state that they read from will always be "valid" when the system that they control is being run: this is important to avoid strange bugs caused by race conditions and non-deterministic system ordering. @@ -187,6 +188,26 @@ In order to understand exactly what this means, we need to understand atomic ord ### Atomic ordering constraints +**Atomic ordering constraints** are a form of ordering constraints that ensure that two systems are run directly after each other. +At least, as far as "directly after" is a meaningful concept in a parallel, nonlinear ordering. +Like the atom, they are indivisible: nothing can get between them! + +To be fully precise, if we have system `A` that runs "atomically before" system `B` (`.add_system(a.atomically_before(b))`): + +- `A` is strictly before `B` +- the data locks (in terms of read and write accesses to the `World`) of system `A` will not be released until system `B` has completed +- the data locks from system `A` are ignored for the purposes of checking if system `B` can run + - after all, they're not actualy locked, merely reserved + - competing read-locks caused by other systems will still need to complete before a data store can be mutated; the effect is to "ignore the locks of `A`", not "skip the check completely" + +In addition to their use in run criteria, atomic ordering constraints are extremely useful for forcing tight interlocking behavior between systems. +They allow us to ensure that the state read from and written to the earlier system cannot be invalidated. +This functionality is extremely useful when updating indexes (used to look up components by value): +allowing you to ensure that the index is still valid when the system using the index uses it. + +Of course, atomic ordering constraints should be used thoughtfully. +As the strictest of the three types of system ordering dependency, they can easily result in unsatisfiable schedules if applied to large groups of systems at once. + ### States ### Complex control flow @@ -244,6 +265,11 @@ There are two reasons why this doesn't work: 1. The system we're applying a run criteria does not have an input type. 2. System chaining does not work if the chained systems are incompatible. This is far too limiting. +### Why aren't run criteria cached? + +In practice, almost all run criteria are extremely simple, and fast to check. +Verifying that the cache is still valid will require access to the data anyways, and involve more overhead than simple computations on one or two resources. + ## Unresolved questions - Should we allow users to compose run criteria in more complex ways? From 78b81c4056a6617d73be08a8f262ae1f51f6f3ce Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 01:24:35 -0500 Subject: [PATCH 005/313] States design --- rfcs/45-stageless.md | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index dea0812c..5407a852 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -210,6 +210,39 @@ As the strictest of the three types of system ordering dependency, they can easi ### States +**States** are one particularly common and powerful form of run criteria, allowing you to toggle systems on-and-off based on the value of a given resource and smoothly handle transitions between states by running cleanup and initialization systems. +Typically, these are used for relatively complex, far-reaching facets of the game like pausing, loading assets or entering into menus. + +The current value of each state is stored in a resource, which must derive the `State` trait. +These are typically (but not necessarily) enums, where each distinct state is represented as an enum variant. + +Each state is associated with three sets of systems: + +1. **Update systems:** these systems run each frame if and only if the value of the state resource matches the provided value. + 1. `app.add_system(my_system.in_state(GameState::Playing))` +2. **On-enter systems:** these systems run once when the specified state is entered. + 1. `app.add_system(my_system.on_enter(GameState::Playing))` +3. **On-exit systems:** these systems run once when the specified stated is exited. + 1. `app.add_system(my_system.on_enter(GameState::Playing))` + +Update systems are by far the simplest: they're simply powered by run criteria. +`.in_state` is precisely equivalent to `run_if_resource_is`, except with an additional trait bound that the resource must implement `State`. + +On-enter and on-exit systems are stored in dedicated schedules, two per state, within the `App`. +These schedules can be configured in all of the ordinary ways, but, as they live in different schedules, ordering cannot be defined relative to systems in the main schedule. + +Due to their disruptive and far-reaching effects, state transitions do not occur immediately. +Instead, they are deferred (like commands), until the next `state_transition::` exclusive system runs. +This system first runs the `on_exit` schedule of the previous state on the world, then runs the `on_enter` schedule of the new state on the world. +Once that is complete, the exclusive system ends and control flow resumes as normal. + +When states are added using `App::add_state::(initial_state)`, one `state_transition` system is added to the app, with the `GeneratedLabel::StateTransition` label. +You can configure when and if this system is scheduled by configuring this label, and you can add additional copies of this system to your schedule where you see fit. + +Apps can have multiple orthogonal states representing independent facets of your game: these operate fully independently. +States can also be defined as a nested enum: these work as you may expect, with each leaf node representing a distinct group of systems. +If you wish to share behavior among siblings, add the systems repeatedly to each sibling, typically by saving a schedule and then using `Shedule::merge` to combine that into the specialized schedule of choice. + ### Complex control flow #### Fixed time steps From 4df1d5a3ea26481bb9001b06486ba4972c8ca0f6 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 01:34:01 -0500 Subject: [PATCH 006/313] Notes on plugin configurability --- rfcs/45-stageless.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 5407a852..46f60ebd 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -134,6 +134,8 @@ There are two basic forms of system ordering constraints: 2. In 99% of cases, this is the desired behavior. Unless you are using interior mutability, systems that are compatible will always be **commutative**: their ordering doesn't matter. Applying an ordering constraint to or from a label causes a ordering constraint to be created between all individual members of that label. +If an ordering is defined relative to a non-existent system or label, it will have no effect, emitting a warning. +This relatively gentle failure mode is important to ensure that plugins can order their systems with relatively strong assumptions that the default system labels exist, but continue to (mostly) work if those systems or labels are not present. In addition to the `.before` and `.after` methods, you can use **system chains** to create very simple linear dependencies between the succesive members of an array of systems. @@ -272,6 +274,18 @@ When writing this section be mindful of the following [repo guidelines](https:// ## Rationale and alternatives +### Doesn't label-configuration allow you to configure systems added by third-party plugins? + +Yes. This is, in fact, a large part of the point. + +The complete inability to configure plugins is the source of [significant user pain](https://github.com/bevyengine/bevy/issues/2160). +Plugin authors cannot know all of the needs of the end user's app, and so some degree of control must be handed back to the end users. + +This control is carefully limited though: systems can only be configured if they are given a public label, and all of the systems under a common label are configured as a group. +In addition, configuration defined by a private label (or directly on systems) cannot be altered, extended or removed by the app: it cannot be named, and so cannot be altered. + +These limitation allows plugin authors to carefully design a public API, ensure critical invariants hold, and make serious internal changes without breaking their dependencies (as long as the public labels and their meanings are relatively stable). + ### Why can't you label labels? If each label is associated with a `SystemConfig`, and part of this struct is which labels are applied, why can't we label labels themselves? From dcfd7c5d83e7bd8c90f97ef3d751ca1f22bdbd30 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 01:36:04 -0500 Subject: [PATCH 007/313] Note on name-space conflict --- rfcs/45-stageless.md | 1 + 1 file changed, 1 insertion(+) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 46f60ebd..f0fd4160 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -322,6 +322,7 @@ Verifying that the cache is still valid will require access to the data anyways, - Should we allow users to compose run criteria in more complex ways? - How does this work for run criteria that are not locally defined? - How do we ergonomically allow users to specify that commands must be flushed between two systems? +- If we want to support a simple `add_system_chain` API as a precursor to a system graph API, what do we rename "system chaining" to? ## \[Optional\] Future possibilities From d75eed8fb2bd6d2880ff39e591c5d46bb8b2681a Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 01:58:32 -0500 Subject: [PATCH 008/313] Flush commands / state ordering constraints --- rfcs/45-stageless.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index f0fd4160..1493fe4b 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -144,7 +144,9 @@ This **exclusive system** (meaning, it can modify the entire `World` in arbitrar Commands (commonly used to spawn and despawn entities or add and remove components) will not take effect until this system is run, so be sure that you run a copy of the `flush_commands` system before relying on the result of a command! -TODO: create more sugar for this, then make an example! +This pattern is so common that a special form of ordering constraint exists for it: **command-flushed ordering constraints**. +If system `A` is `before_and_flush` system `B`, the schedule will be unsatisfiable unless there is an intervening `flush_commands` system. +Note that **this does not insert new copies of a `flush_commands` system**: instead, it is merely used to verify that your schedule has been set up correctly according the specified constraint. ### Run criteria @@ -234,16 +236,18 @@ On-enter and on-exit systems are stored in dedicated schedules, two per state, w These schedules can be configured in all of the ordinary ways, but, as they live in different schedules, ordering cannot be defined relative to systems in the main schedule. Due to their disruptive and far-reaching effects, state transitions do not occur immediately. -Instead, they are deferred (like commands), until the next `state_transition::` exclusive system runs. +Instead, they are deferred (like commands), until the next `flush_state` exclusive system runs. This system first runs the `on_exit` schedule of the previous state on the world, then runs the `on_enter` schedule of the new state on the world. Once that is complete, the exclusive system ends and control flow resumes as normal. -When states are added using `App::add_state::(initial_state)`, one `state_transition` system is added to the app, with the `GeneratedLabel::StateTransition` label. +When states are added using `App::add_state::(initial_state)`, one `flush_state` system is added to the app, with the `GeneratedLabel::StateTransition` label. You can configure when and if this system is scheduled by configuring this label, and you can add additional copies of this system to your schedule where you see fit. +Just like with commands, **state-flushed ordering constraints** can be used to verify that state transitions have run at the appropriate time. +If system `A` is `before_and_flush_state::` system `B`, the schedule will be unsatisfiable unless there is an intervening `flush_state` system. Apps can have multiple orthogonal states representing independent facets of your game: these operate fully independently. States can also be defined as a nested enum: these work as you may expect, with each leaf node representing a distinct group of systems. -If you wish to share behavior among siblings, add the systems repeatedly to each sibling, typically by saving a schedule and then using `Shedule::merge` to combine that into the specialized schedule of choice. +If you wish to share behavior among siblings, add the systems repeatedly to each sibling, typically by saving a schedule and then using `Schedule::merge` to combine that into the specialized schedule of choice. ### Complex control flow @@ -321,7 +325,6 @@ Verifying that the cache is still valid will require access to the data anyways, - Should we allow users to compose run criteria in more complex ways? - How does this work for run criteria that are not locally defined? -- How do we ergonomically allow users to specify that commands must be flushed between two systems? - If we want to support a simple `add_system_chain` API as a precursor to a system graph API, what do we rename "system chaining" to? ## \[Optional\] Future possibilities From d6e87cbc9fe3a7d4e6235fd0a61fdbabf180f004 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 02:22:02 -0500 Subject: [PATCH 009/313] Notes on storing schedules --- rfcs/45-stageless.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 1493fe4b..3c4f6fa1 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -321,6 +321,13 @@ There are two reasons why this doesn't work: In practice, almost all run criteria are extremely simple, and fast to check. Verifying that the cache is still valid will require access to the data anyways, and involve more overhead than simple computations on one or two resources. +### Why do we want to store multiple schedules in the `App`? + +We could store these in a resource in the `World`. +Unfortunately, this seriously impacts the ergonomics of running schedules in exclusive systems due to borrow checker woes. + +Storing the schedules in the `App` alleviates this, as exclusive systems are now just ordinary systems: `&mut World` is compatible with `&mut Schedules`! + ## Unresolved questions - Should we allow users to compose run criteria in more complex ways? From 061f842da3598d660e137ece7dceb5b82fb5e1b6 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 02:22:29 -0500 Subject: [PATCH 010/313] Complex control flows --- rfcs/45-stageless.md | 50 ++++++++++++++++++++++---------------------- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 3c4f6fa1..2fed3509 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -251,25 +251,36 @@ If you wish to share behavior among siblings, add the systems repeatedly to each ### Complex control flow -#### Fixed time steps +Occasionally, you may find yourself yearning for more complex system control flow than "every system runs once in a loop". +When that happens: **reach for an exclusive and run a schedule in it.** -## Implementation strategy +The `App` can store multiple `Schedules` in a `HashMap, Schedule>`. +This is used for the enter and exit schedules of states, but can also be used to store mutate and access additional schedules. +You can even mutate the schedule that you are in, although you cannot mutate systems that are currently running (leading to a runtime panic), as that would be UB. + +Within an exclusive system, you can freely use `Schedule::run(&mut world)`, applying each of the systems in that schedule a single time to the world of the exclusive system. +However, because you're in an ordinary Rust function you're free to use whatever logic and control flow you desire: branch and loop in whatever convoluted fashion you need! + +This can be helpful when: + +- you want to run multiple steps of a set of game logic during a single frame (as you might see in complex turn-based games) +- you need to branch to handle a +- you need to integrate a complex external service, like scripting or a web server -This is the technical portion of the RFC. -Try to capture the broad implementation strategy, -and then focus in on the tricky details so that: +If you need absolute control over the entire schedule, consider having a single root-level exclusive system. -- Its interaction with other features is clear. -- It is reasonably clear how the feature would be implemented. -- Corner cases are dissected by example. +#### Fixed time steps + +Running a set of systems (typically physics) according to a fixed time step is a particularly critical case of this complex control flow. +These systems should run a fixed number of times for each wall-clock second elapsed. +The number of times these systems should be run varies (it may be 0, 1 or more each frame, depending on the time that the frame took to complete). -When necessary, this section should return to the examples given in the previous section and explain the implementation details that make them work. +Simply adding run criteria is inadequate: run criteria can only cause our systems to run a single time, or not run at all. +By moving this logic into its own schedule within an exclusive system, we can loop until the accumulated time has been spent, running the schedule repeatedly and ensuring that our physics always uses the same elapsed delta-time and stays synced with the wall-clock, even in the face of serious stutters. -When writing this section be mindful of the following [repo guidelines](https://github.com/bevyengine/rfcs): +## Implementation strategy -- **RFCs should be scoped:** Try to avoid creating RFCs for huge design spaces that span many features. Try to pick a specific feature slice and describe it in as much detail as possible. Feel free to create multiple RFCs if you need multiple features. -- **RFCs should avoid ambiguity:** Two developers implementing the same RFC should come up with nearly identical implementations. -- **RFCs should be "implementable":** Merged RFCs should only depend on features from other merged RFCs and existing Bevy features. It is ok to create multiple dependent RFCs, but they should either be merged at the same time or have a clear merge order that ensures the "implementable" rule is respected. +TODO: WRITE. ## Drawbacks @@ -336,15 +347,4 @@ Storing the schedules in the `App` alleviates this, as exclusive systems are now ## \[Optional\] Future possibilities -Think about what the natural extension and evolution of your proposal would -be and how it would affect Bevy as a whole in a holistic way. -Try to use this section as a tool to more fully consider other possible -interactions with the engine in your proposal. - -This is also a good place to "dump ideas", if they are out of scope for the -RFC you are writing but otherwise related. - -Note that having something written down in the future-possibilities section -is not a reason to accept the current or a future RFC; such notes should be -in the section on motivation or rationale in this or subsequent RFCs. -If a feature or change has no direct value on its own, expand your RFC to include the first valuable feature that would build on it. +TODO: WRITE. From 4b4eab01c9a0a91903b03714f9bb5d182ef047a2 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 02:34:45 -0500 Subject: [PATCH 011/313] Future possibilities --- rfcs/45-stageless.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 2fed3509..432aa6f6 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -341,10 +341,17 @@ Storing the schedules in the `App` alleviates this, as exclusive systems are now ## Unresolved questions -- Should we allow users to compose run criteria in more complex ways? - - How does this work for run criteria that are not locally defined? - If we want to support a simple `add_system_chain` API as a precursor to a system graph API, what do we rename "system chaining" to? -## \[Optional\] Future possibilities +## Future possibilities -TODO: WRITE. +Despite the large scope of this RFC, it leaves quite a bit of interesting follow-up work to be done: + +1. System-builder syntax (see [RFC #31](https://github.com/bevyengine/rfcs/pull/31)). This will likely be done as part of the rewrite. +2. Opt-in automatic inference of command and state synchronizing systems (see discussion in [RFC #34](https://github.com/bevyengine/rfcs/pull/34)). +3. First-class indexes, built using atomic ordering constraints (and likely automatic inference). +4. Multiple worlds (see [RFC #16](https://github.com/bevyengine/rfcs/pull/43), [RFC #43](https://github.com/bevyengine/rfcs/pull/43)), as a natural extension of the way that apps can store multiple schedules. +5. Opt-in stack-based states (likely in an external crate). +6. More complex strategies for run criteria composition. + 1. This would be very useful, but is a large design that can largely be considered independently of this work. + 2. How does this work for run criteria that are not locally defined? From 6939c0cb6c37cc8fa0e91535e46350d1703a8d9d Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 02:42:57 -0500 Subject: [PATCH 012/313] Bring the idea of multiple schedules to the forefront --- rfcs/45-stageless.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 432aa6f6..7082af0e 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -65,6 +65,16 @@ However, each system will continue to work hand-in-hand with a small number of c By configuring our systems with straightforward, local rules, we can allow the scheduler to pick one of the possible valid paths, optimizing as it sees fit without breaking our logic. Just as importantly, we are not over-constraining our ordering, allowing subtle ordering bugs to surface quickly and then be stamped out with a well-considered rule. +### The `App` stores multiple schedules + +The `App` can store multiple `Schedules` in a `HashMap, Schedule>` storage. +This is used for the enter and exit schedules of states, but can also be used to store, mutate and access additional schedules. +You can even mutate the schedule that you are in, although you cannot mutate systems that are currently running (leading to a runtime panic), as that would be UB. + +You can access the schedules stored in the app using the `&Schedules` or `&mut Schedules` system parameters. +Unsurprisingly, these never conflict with entities or resources in the `World`, as they are stored one level higher. +The main schedule can be accessed using the `MainSchedule` unit struct as a schedule label. + ### Introduction to system configuration When a function is added to a schedule, it is turned into a `RawSystem` and combined with a `SystemConfig` struct to create a `ConfiguredSystem`. @@ -232,7 +242,7 @@ Each state is associated with three sets of systems: Update systems are by far the simplest: they're simply powered by run criteria. `.in_state` is precisely equivalent to `run_if_resource_is`, except with an additional trait bound that the resource must implement `State`. -On-enter and on-exit systems are stored in dedicated schedules, two per state, within the `App`. +On-enter and on-exit systems are stored in dedicated schedules, two per state, within the `App's` `Schedules`. These schedules can be configured in all of the ordinary ways, but, as they live in different schedules, ordering cannot be defined relative to systems in the main schedule. Due to their disruptive and far-reaching effects, state transitions do not occur immediately. @@ -254,11 +264,7 @@ If you wish to share behavior among siblings, add the systems repeatedly to each Occasionally, you may find yourself yearning for more complex system control flow than "every system runs once in a loop". When that happens: **reach for an exclusive and run a schedule in it.** -The `App` can store multiple `Schedules` in a `HashMap, Schedule>`. -This is used for the enter and exit schedules of states, but can also be used to store mutate and access additional schedules. -You can even mutate the schedule that you are in, although you cannot mutate systems that are currently running (leading to a runtime panic), as that would be UB. - -Within an exclusive system, you can freely use `Schedule::run(&mut world)`, applying each of the systems in that schedule a single time to the world of the exclusive system. +Within an exclusive system, you can freely fetch the desired schedule from the `App` with the `&Schedules` (or `&mut Schedules`) system parameter and use `Schedule::run(&mut world)`, applying each of the systems in that schedule a single time to the world of the exclusive system. However, because you're in an ordinary Rust function you're free to use whatever logic and control flow you desire: branch and loop in whatever convoluted fashion you need! This can be helpful when: From f545f771849262d0b4b8bc6f5b05ba3b7245b877 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 02:52:33 -0500 Subject: [PATCH 013/313] Notes on handling of startup systems --- rfcs/45-stageless.md | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 7082af0e..c1440a22 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -26,17 +26,17 @@ Unfortunately, all of these problems are deeply interwoven. Despite our best eff This explanation is, in effect, user-facing documentation for the new design. In addition to a few new concepts, it throws out much of the current system scheduling design that you may be familiar with. -The following elements are radically reworked: +The following elements are substantially reworked: - schedules (flattened) - run criteria (can no longer loop, are now systems) -- states (simplified, no longer run-criteria powered) +- states (simplified, no longer purely run-criteria powered) - fixed time steps (no longer a run criteria) - exclusive systems (no longer special-cased) - command processing (now performed in a `flush_commands` exclusive system) - labels (can now be directly configured) -- stages (yeet) - system sets (yeet) +- stages (yeet) ### Scheduling overview and intent-based configuration @@ -71,9 +71,20 @@ The `App` can store multiple `Schedules` in a `HashMap, S This is used for the enter and exit schedules of states, but can also be used to store, mutate and access additional schedules. You can even mutate the schedule that you are in, although you cannot mutate systems that are currently running (leading to a runtime panic), as that would be UB. +The main and startup schedules can be accessed using the `CoreSchedule::Main` and `CoreSchedule::Startup` labels respectively. +By default systems are added to the main schedule. +You can control this by adding the `.to_schedule(MySchedule::Variant)` system descriptor to your system. + You can access the schedules stored in the app using the `&Schedules` or `&mut Schedules` system parameters. Unsurprisingly, these never conflict with entities or resources in the `World`, as they are stored one level higher. -The main schedule can be accessed using the `MainSchedule` unit struct as a schedule label. + +#### Startup systems + +Startup systems are stored in their own schedule, with the `CoreSchedule::Startup` label. +When using the runner added by `MinimalPlugins` and `DefaultPlugins`, this schedule will run exactly once on app startup. + +You can add startup systems with the `.add_startup_system(my_system)` method on `App`, which is simply sugar for `.add_system(my_system.to_schedule(CoreSchedule::Startup))`. +Startup systems can be run again by running this schedule from within an exclusive system. ### Introduction to system configuration From 3b4ae3311b4f132b6fd815b9074cba04b747ac4b Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 02:59:12 -0500 Subject: [PATCH 014/313] More future work --- rfcs/45-stageless.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index c1440a22..02d635b7 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -358,7 +358,8 @@ Storing the schedules in the `App` alleviates this, as exclusive systems are now ## Unresolved questions -- If we want to support a simple `add_system_chain` API as a precursor to a system graph API, what do we rename "system chaining" to? +- If we want to support a simple `add_system_chain` API as a precursor to a system-graph-specification API, what do we rename "system chaining" to? + - System welding? System fusing? System handling? ## Future possibilities @@ -372,3 +373,5 @@ Despite the large scope of this RFC, it leaves quite a bit of interesting follow 6. More complex strategies for run criteria composition. 1. This would be very useful, but is a large design that can largely be considered independently of this work. 2. How does this work for run criteria that are not locally defined? +7. A more cohesive look at plugin definition and configuration strategies. +8. A graph-based system ordering API for dense, complex dependencies. From 74d1609f84e40d279cbc0bbe19d98670304f8994 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 03:13:39 -0500 Subject: [PATCH 015/313] Use system sets for mass labeling --- rfcs/45-stageless.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 02d635b7..43369d58 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -35,7 +35,7 @@ The following elements are substantially reworked: - exclusive systems (no longer special-cased) - command processing (now performed in a `flush_commands` exclusive system) - labels (can now be directly configured) -- system sets (yeet) +- system sets (now just used to mass-apply a label to a group of systems) - stages (yeet) ### Scheduling overview and intent-based configuration @@ -139,6 +139,10 @@ impl Plugin for PhysicsPlugin{ } ``` +If you just want to run your game logic systems in the middle of your stage, after input is processed but before rendering occurs, add the `CoreLabel::GameLogic` to them. + +You can apply the same label(s) to many systems at once using **system sets** with the `App::add_system_set(systems: impl SystemIterator, labels: impl SystemLabelIterator)` method. + ### System ordering The most important way we can configure systems is by telling the scheduler *when* they should be run. From 2038425a471f41dec951a228b15aeb62728e966a Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 03:13:53 -0500 Subject: [PATCH 016/313] More consideration of drawbacks --- rfcs/45-stageless.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 43369d58..f762504e 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -287,6 +287,7 @@ This can be helpful when: - you want to run multiple steps of a set of game logic during a single frame (as you might see in complex turn-based games) - you need to branch to handle a - you need to integrate a complex external service, like scripting or a web server +- you want to switch the executor used for some portion of your systems If you need absolute control over the entire schedule, consider having a single root-level exclusive system. @@ -306,7 +307,12 @@ TODO: WRITE. ## Drawbacks 1. This will be a massively breaking change to every single Bevy user. + 1. Any non-trivial control over system ordering will need to be completely reworked. + 2. Users will typically need to think a bit harder about exactly when they want their gameplay systems to run. In most cases, they should just add the `CoreLabel::GameLogic` label to them. 2. It will be harder to immediately understand the global structure of Bevy apps. + 1. Powerful system debugging and visualization tools become even more important. +3. States will no longer have stacks. This could break some users, but they should be able to write their own replacement easily enough externally. +4. It will become harder to reason about exactly when command flushing occurs. ## Rationale and alternatives From 1fb22d75ceb0565747cc99a7756b816ac65de4d4 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 11:27:11 -0500 Subject: [PATCH 017/313] Basic implementation strategy --- rfcs/45-stageless.md | 77 ++++++++++++++++++++++++++++++++++++++------ 1 file changed, 68 insertions(+), 9 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index f762504e..f9944cde 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -302,7 +302,55 @@ By moving this logic into its own schedule within an exclusive system, we can lo ## Implementation strategy -TODO: WRITE. +Let's take a look at what implementing this would take: + +1. Completely rip out: + 1. `Stage` + 2. `Schedule` + 3. `SystemDescriptor` + 4. `IntoExclusiveSystem` +2. Build out a basic schedule abstraction: + 1. Create a `ScheduleLabel` trait + 2. Store multiple schedules in an `App` using a dictionary approach + 3. Support adding systems to the schedule + 1. These should be initialized by definition: see [PR #2777](https://github.com/bevyengine/bevy/pull/2777) + 4. Support adding systems to specific schedules and create startup system API + 5. Special-cased data access control for "the entire world" and `Schedules` to support exclusive systems +3. Rebuild core command processing machinery + 1. Begin with trivial event-style commands + 2. Explore how to regain parallelism +4. Add system configuration + 1. Create `SystemConfig` type + 2. Systems labels store a `SystemConfig` + 3. Allow systems to be labelled + 4. Store `ConfiguredSystems` in the schedule + 5. Add `.add_system_set` method + 6. Use [system builder syntax]((https://github.com/bevyengine/rfcs/pull/31), rather than adding more complex methods to `App` +5. Add basic ordering constraints: basic data structures and configuration methods + 1. Begin with strict ordering constraints: simplest and most fundamental + 2. Add if-needed ordering constraints +6. Add atomic ordering constraints +7. Add run criteria + 1. Create `IntoRunCriteriaSystem` machinery + 2. Store the run criteria of each `ConfiguredSystem` in the schedule + 3. Add atomic ordering constraints + 4. Check the values of the run-criteria of systems before deciding whether it can be run + 5. Add methods to configure run criteria +8. Add states + 1. Create `State` trait + 2. Implement on-update states as sugar for simple run criteria + 3. Create on-enter and on-exit schedules + 4. Create sugar for adding systems to these schedules +9. Add new examples + 1. Complex control flow with supplementary schedules + 2. Fixed time-step pattern +10. Port the entire engine to the new paradigm + 1. We almost certainly need to port the improved ambiguity checker over to make this reliable + +Given the massive scope, that sounds relatively straightforward! +However, doing so will break approximately the entire engine, and tests will not pass again until step 10. + +Lets explore some of the more critical and challenging details. ## Drawbacks @@ -370,18 +418,29 @@ Storing the schedules in the `App` alleviates this, as exclusive systems are now - If we want to support a simple `add_system_chain` API as a precursor to a system-graph-specification API, what do we rename "system chaining" to? - System welding? System fusing? System handling? +- What is the best way to handle the migration process from an organizational perspective? + - Get an RFC approved, and then merge a massive PR developed on a off-org branch? + - Straightforward, but very hard to review + - Risks divergence and merge conflicts + - Use an official branch with delegated authority? + - Easier to review, requires delegation, ultimately creates a large PR that needs to be reviewed + - Risks divergence and merge conflicts + - Develop on main as `bevy_ecs2`? + - Some annoying machinery to set up + - Requires more delegation and trust to ECS team + - Avoids divergence and merge conflicts + - Clutters the main branch ## Future possibilities Despite the large scope of this RFC, it leaves quite a bit of interesting follow-up work to be done: -1. System-builder syntax (see [RFC #31](https://github.com/bevyengine/rfcs/pull/31)). This will likely be done as part of the rewrite. -2. Opt-in automatic inference of command and state synchronizing systems (see discussion in [RFC #34](https://github.com/bevyengine/rfcs/pull/34)). -3. First-class indexes, built using atomic ordering constraints (and likely automatic inference). -4. Multiple worlds (see [RFC #16](https://github.com/bevyengine/rfcs/pull/43), [RFC #43](https://github.com/bevyengine/rfcs/pull/43)), as a natural extension of the way that apps can store multiple schedules. -5. Opt-in stack-based states (likely in an external crate). -6. More complex strategies for run criteria composition. +1. Opt-in automatic inference of command and state synchronizing systems (see discussion in [RFC #34](https://github.com/bevyengine/rfcs/pull/34)). +2. First-class indexes, built using atomic ordering constraints (and likely automatic inference). +3. Multiple worlds (see [RFC #16](https://github.com/bevyengine/rfcs/pull/43), [RFC #43](https://github.com/bevyengine/rfcs/pull/43)), as a natural extension of the way that apps can store multiple schedules. +4. Opt-in stack-based states (likely in an external crate). +5. More complex strategies for run criteria composition. 1. This would be very useful, but is a large design that can largely be considered independently of this work. 2. How does this work for run criteria that are not locally defined? -7. A more cohesive look at plugin definition and configuration strategies. -8. A graph-based system ordering API for dense, complex dependencies. +6. A more cohesive look at plugin definition and configuration strategies. +7. A graph-based system ordering API for dense, complex dependencies. From e5f560df5519ec20b5d97819148f1870d5f122d0 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 11:51:08 -0500 Subject: [PATCH 018/313] System config strategy --- rfcs/45-stageless.md | 61 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index f9944cde..581c7fca 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -352,6 +352,65 @@ However, doing so will break approximately the entire engine, and tests will not Lets explore some of the more critical and challenging details. +### Storing and configuring systems + +Schedules store systems in a configured, initialized form. + +```rust +struct Schedule{ + // Each schedule is associated with exactly one `World`, as systems must be initiliazed on it + world_id: WorldId + // We need to be able to quickly look up specific systems and iterate over them + // `SystemId` should be a unique identifier, atomically generated on system insertion + systems: HashMap, + // Label configuration must be stored at the schedule level, + // rather than attached to specific label instances + labels: HashMap, SystemConfig>, +} +``` + +Configured systems are composed of a raw system and its configuration. + +```rust +struct ConfiguredSystem{ + raw_system: RawSystem, + config: SystemConfig, +} +``` + +Raw systems store metadata, cached state, `Local` system parameters and the function pointer to the actual function to be executed: + +```rust +struct RawSystem { + function: F, + // Contains all metadata and state + meta: SystemMeta + last_change_tick: u32, +} +``` + +System configuration stores ordering dependencies, run criteria and labels. +It can be created raw, or stored in association with either system labels or systems. + +```rust +enum OrderingTarget { + Label(Box), + System(SystemId), +} + +struct SystemConfig { + strict_ordering_before: Vec, + if_needed_ordering_before: Vec, + atomic_ordering_before: Vec, + strict_ordering_after: Vec, + if_needed_ordering_after: Vec, + atomic_ordering_after: Vec, + run_criteria: Vec, + // This vector is always empty when this struct is used for labels + labels: Vec, +} +``` + ## Drawbacks 1. This will be a massively breaking change to every single Bevy user. @@ -361,6 +420,8 @@ Lets explore some of the more critical and challenging details. 1. Powerful system debugging and visualization tools become even more important. 3. States will no longer have stacks. This could break some users, but they should be able to write their own replacement easily enough externally. 4. It will become harder to reason about exactly when command flushing occurs. +5. Ambiguity sets are not directly accounted for in the new design. + 1. They need more thought and are rarely used; we can toss them back on later. ## Rationale and alternatives From bb4300235660a8bbd66da04ad75326b5bc499093 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 12:23:36 -0500 Subject: [PATCH 019/313] Flushed ordering constraints implementation strategy --- rfcs/45-stageless.md | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 581c7fca..9d3d273a 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -366,6 +366,9 @@ struct Schedule{ // Label configuration must be stored at the schedule level, // rather than attached to specific label instances labels: HashMap, SystemConfig>, + // Used to quickly look up system ids by the type ids of their underlying function + // when checking atomic ordering constraints + system_function_type_map: HashMap>, } ``` @@ -375,6 +378,12 @@ Configured systems are composed of a raw system and its configuration. struct ConfiguredSystem{ raw_system: RawSystem, config: SystemConfig, + // All of the ordering constraints from `SystemConfig` + // are converted into individual systems. + // Only the systems that this system are after are stored here. + // This is what is actually checked by the scheduler to see if this system can be run. + // All if-needed orderings are either converted to strict orderings at this step or removed. + cached_ordering: CachedOrdering, } ``` @@ -405,12 +414,33 @@ struct SystemConfig { strict_ordering_after: Vec, if_needed_ordering_after: Vec, atomic_ordering_after: Vec, + // The `TypeId` here is the type id of the flushing system + flushed_ordering_before: Vec<(TypeId, OrderingTarget)>, + flushed_ordering_after: Vec<(TypeId, OrderingTarget)>, run_criteria: Vec, // This vector is always empty when this struct is used for labels labels: Vec, } ``` +### Flushed ordering constraints + +Being able to assert that commands are processed (or state is flushed) between two systems is a critical requirement for being able to specify logically-meaningful ordering of systems without a global view of the schedule. + +However, the exact logic involved in doing this is quite complex. +We need to: + +1. **When validating the schedule:** ensure that a flushing system of the appropriate type *could* occur between the pair of systems. +2. **When executing the schedule:** ensure that a flushing system of the appropriate type *does* occur between the pair of systems. + +For this RFC, let's begin with the very simplest strategy: for each pair of systems with a flushed ordering constraint, ensure that a flushing system of the appropriate type is strictly after the first system, and strictly before the second. + +We can check if the flushing system is of the appropriate type by checking the type id of the function pointer stored in the `RawSystem`. +This is cached in the `system_function_type_map` field of the `Schedule` for efficient lookup, yielding the relevant `SystemIds`. + +With the ids of the relevant systems in hand, we can check each constraint: for each pair of systems, we must check that at least one of the relevant flushing systems is between them. +To do so, we need to check both direct *and* transitive strict ordering dependencies, using a depth-first search to walk backwards from the later system using the `CachedOrdering` until a flushing system is reached, and then walking back further from that flushing system until the originating system is reached. + ## Drawbacks 1. This will be a massively breaking change to every single Bevy user. @@ -496,7 +526,7 @@ Storing the schedules in the `App` alleviates this, as exclusive systems are now Despite the large scope of this RFC, it leaves quite a bit of interesting follow-up work to be done: -1. Opt-in automatic inference of command and state synchronizing systems (see discussion in [RFC #34](https://github.com/bevyengine/rfcs/pull/34)). +1. Opt-in automatic insertion of flushing systems for command and state (see discussion in [RFC #34](https://github.com/bevyengine/rfcs/pull/34)). 2. First-class indexes, built using atomic ordering constraints (and likely automatic inference). 3. Multiple worlds (see [RFC #16](https://github.com/bevyengine/rfcs/pull/43), [RFC #43](https://github.com/bevyengine/rfcs/pull/43)), as a natural extension of the way that apps can store multiple schedules. 4. Opt-in stack-based states (likely in an external crate). From f87a7f07da1428f7257c43fc674a4e83a528a49d Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 12:26:38 -0500 Subject: [PATCH 020/313] Fix broken URL --- rfcs/45-stageless.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 9d3d273a..b4e505b0 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -325,7 +325,7 @@ Let's take a look at what implementing this would take: 3. Allow systems to be labelled 4. Store `ConfiguredSystems` in the schedule 5. Add `.add_system_set` method - 6. Use [system builder syntax]((https://github.com/bevyengine/rfcs/pull/31), rather than adding more complex methods to `App` + 6. Use [system builder syntax](https://github.com/bevyengine/rfcs/pull/31), rather than adding more complex methods to `App` 5. Add basic ordering constraints: basic data structures and configuration methods 1. Begin with strict ordering constraints: simplest and most fundamental 2. Add if-needed ordering constraints From 8ee38b7b17cdfe7d4736a987516a6648ba556baa Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 13:25:35 -0500 Subject: [PATCH 021/313] Fail more gracefully on UB schedule modification --- rfcs/45-stageless.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index b4e505b0..b1bdbcb4 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -69,7 +69,7 @@ Just as importantly, we are not over-constraining our ordering, allowing subtle The `App` can store multiple `Schedules` in a `HashMap, Schedule>` storage. This is used for the enter and exit schedules of states, but can also be used to store, mutate and access additional schedules. -You can even mutate the schedule that you are in, although you cannot mutate systems that are currently running (leading to a runtime panic), as that would be UB. +You can even mutate the schedule that you are in, although you cannot mutate systems that are currently running (failing with an error), as that would be UB. The main and startup schedules can be accessed using the `CoreSchedule::Main` and `CoreSchedule::Startup` labels respectively. By default systems are added to the main schedule. From b0201c3ab3c1676508c26f08b04db737e324a6db Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 15:30:52 -0500 Subject: [PATCH 022/313] Minor cleanups from review --- rfcs/45-stageless.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index b1bdbcb4..63ac7746 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -16,7 +16,7 @@ Of particular note: - Plugins can add new, standalone stages. - Run criteria (including states!) cannot be composed. - The stack-driven state model is overly elaborate and does not enable enough use cases to warrant its complexity. -- Fixed timestep run criteria do not behave in a fashion that supports. +- Fixed timestep run criteria do not correctly handle the logic needed for robust game physics. - The architecture for turn-based games and other even slightly unusual patterns is not obvious. - We cannot easily implement a builder-style strategy for configuring systems. @@ -139,7 +139,7 @@ impl Plugin for PhysicsPlugin{ } ``` -If you just want to run your game logic systems in the middle of your stage, after input is processed but before rendering occurs, add the `CoreLabel::GameLogic` to them. +If you just want to run your game logic systems in the middle of your schedule, after input is processed but before rendering occurs, add the `CoreLabel::GameLogic` to them. You can apply the same label(s) to many systems at once using **system sets** with the `App::add_system_set(systems: impl SystemIterator, labels: impl SystemLabelIterator)` method. @@ -285,7 +285,7 @@ However, because you're in an ordinary Rust function you're free to use whatever This can be helpful when: - you want to run multiple steps of a set of game logic during a single frame (as you might see in complex turn-based games) -- you need to branch to handle a +- you need to branch the entire schedule to handle a particularly complex bit of game logic - you need to integrate a complex external service, like scripting or a web server - you want to switch the executor used for some portion of your systems From f5fa4652a85a2e993866a9975fd4ba00a7275ce7 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 15:36:26 -0500 Subject: [PATCH 023/313] Significantly shortened section on intent-based configuration --- rfcs/45-stageless.md | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 63ac7746..3b16c2d5 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -48,17 +48,7 @@ In the beginning, each `Schedule` is entirely unordered: systems will be selecte Just like with standard borrow checking, multiple systems can read from the same data at once, but writing to the data requires an exclusive lock. Systems which cannot be run in parallel are said to be **incompatible**. -While this is a good, simple strategy for maximizing parallelism, it comes with some serious drawbacks for reproducibility and correctness. -The relative execution order of systems tends to matter! -In simple projects and toy examples, the solution to this looks simple: let the user specify a single global ordering and use this to determine the order in the case of conflicts! - -Unfortunately, this strategy falls apart at scale, particularly when attempting to integrate third-party dependencies (in the form of plugins), who have complex, interacting requirements that must be respected. -While users may be able to find a single working strategy through trial-and-error (and a robust test suite!), this strategy becomes ossified: impossible to safely change to incorporate new features or optimize performance. - -The fundamental problem is a mismatch between *how* the schedule is configured and *why* it is configured that way. -The fact that "physics must run after input but before rendering" is implicitly encoded into the order of our systems, and changes which violate these constraints will silently fail in far-reaching ways without giving any indication as to what the user did wrong. - -The solution is **intent-based configuration**: record the exact, minimal set of constraints needed for a scheduling strategy to be valid, and then let the scheduler figure it out! +Bevy controls the ordering of its schedule(s) The solution via **intent-based configuration**: record the exact, minimal set of constraints needed for a scheduling strategy to be valid, and then let the scheduler figure it out! As the number of systems in your app grows, the number of possible scheduling strategies grows exponentially, and the set of *valid* scheduling strategies grows almost as quickly. However, each system will continue to work hand-in-hand with a small number of closely-related systems, interlocking in a straightforward and comprehensible way. From c8b719b380ca3f2489e5c7baa3876dd616ea2f97 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 15:55:17 -0500 Subject: [PATCH 024/313] Clarified and improved label configuration syntax Thanks @inodentry! --- rfcs/45-stageless.md | 56 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 43 insertions(+), 13 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 3b16c2d5..a3bea581 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -95,11 +95,20 @@ A system may be configured in the following ways: - **states** are a special, more complex pattern that use run criteria to determine whether or not a system should run in a current state - e.g. `.add_system(physics.run_if(GameLogic::Physics))` -Defining system configuration in a `SystemConfig` struct, can be useful to reuse, compose and quickly apply complex configuration strategies, before applying these strategies to labels or individual systems: +System configuration can be stored in a `SystemConfig` struct. This can be useful to reuse, compose and quickly apply complex configuration strategies, before applying these strategies to labels or individual systems. + +If you just want to run your game logic systems in the middle of your schedule, after input is processed but before rendering occurs, add the premade `CoreLabel::GameLogic` to them. + +### Label configuration + +Each system label has its own associated `SystemConfig`, stored in the corresponding `Schedule`. +This configuration is applied in an additive fashion to each system with that label. + +You can define labels in a standalone fashion, configuring them at the time of creation: ```rust #[derive(SystemLabel)] -enum PhysicsLabel { +enum Physics { Forces, CollisionDetection, CollisionHandling, @@ -108,28 +117,49 @@ enum PhysicsLabel { impl Plugin for PhysicsPlugin{ fn build(app: &mut App){ - // Import all of the `PhysicsLabel` variants for brevity - use PhysicsLabel::*; - // Within each frame, physics logic needs to occur after input handling, but before rendering let mut common_physics_config = SystemConfig::new().after(CoreLabel::InputHandling).before(CoreLabel::Rendering); app // We can reuse this shared configuration on each of our labels - .add_label(Forces.configure(common_physics_config).before(CollisionDetection)) - .add_label(CollisionDetection.configure(common_physics_config).before(CollisionHandling)) - .add_label(CollisionHandling.configure(common_physics_config)) + .add_label(Forces.configure(common_physics_config).before(Physics::CollisionDetection)) + .add_label(Physics::CollisionDetection.configure(common_physics_config).before(Physics::CollisionHandling)) + .add_label(Physics::CollisionHandling.configure(common_physics_config)) // And then apply that config to each of the systems that have this label .add_system(gravity.label(Forces)) // These systems have a linear chain of ordering dependencies between them // Systems earlier in the chain must run before those later in the chain - .add_system_chain([broad_pass, narrow_pass].label(CollisionDetection)) - .add_system(compute_forces.label(CollisionHandling)) + .add_system_chain([broad_pass, narrow_pass].label(Physics::CollisionDetection)) + // System sets apply a set of labels to a collection of systems + // and are helpful for reducing boilerplate + .add_system_set([compute_forces, collision_damage], Physics::CollisionHandling); + } +} +``` + +Or, you can apply labels to your systems, and then configure them later: + +```rust +impl Plugin for PhysicsPlugin{ + + // This code has the exact same effect as the snipped above + fn build(app: &mut App){ + let mut common_physics_config = SystemConfig::new().after(CoreLabel::InputHandling).before(CoreLabel::Rendering); + + app + .add_system(gravity.label(Forces)) + .add_system_chain([broad_pass, narrow_pass].label(Physics::CollisionDetection)) + .add_system_set([compute_forces, collision_damage], Physics::CollisionHandling) + // This adds the new configuration constraints to the labels in an additive fashion + .configure_label(Physics::Forces.configure(common_physics_config).before(Physics::CollisionDetection)) + .configure_label(Physics::CollisionDetection.configure(common_physics_config).before(Physics::CollisionHandling)) + .configure_label(Physics::CollisionHandling.configure(common_physics_config)); } } ``` -If you just want to run your game logic systems in the middle of your schedule, after input is processed but before rendering occurs, add the `CoreLabel::GameLogic` to them. +For the most part, this is a matter of code organization and style: defining the properties of your labels up-front helps keep your code clean and your configuration centralized. +On the other hand, configuring labels later allows you to modify the configuration of labels added by external plugins, customizing them to fit your game's needs. You can apply the same label(s) to many systems at once using **system sets** with the `App::add_system_set(systems: impl SystemIterator, labels: impl SystemLabelIterator)` method. @@ -193,8 +223,8 @@ fn main(){ // for when you want to check the value of a resource (commonly an enum) .add_system(gravity.run_if_resource_equals(Gravity::Enabled)) // Run criteria can be attached to labels: a copy of the run criteria will be applied to each system with that label - .configure_label(GameLabel::Physics, |label: impl SystemLabel| {label.run_if_resource_equals(Paused(false))} ) - .run() + .configure_label(GameLabel::Physics.run_if_resource_equals(Paused(false))) + .run(); } ``` From 04584378e911ff5801451b5025bcb76255631287 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 16:05:39 -0500 Subject: [PATCH 025/313] Added comment about using atomic ordering to split large systems Credit to @inodentry for the idea --- rfcs/45-stageless.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index a3bea581..0f990178 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -257,6 +257,10 @@ allowing you to ensure that the index is still valid when the system using the i Of course, atomic ordering constraints should be used thoughtfully. As the strictest of the three types of system ordering dependency, they can easily result in unsatisfiable schedules if applied to large groups of systems at once. +On the other hand, atomic ordering constraints can be helpful when attempting to split large complex systems into multiple parts. +By guaranteeing that the initial state of your complex system cannot be altered before the later parts of the system are complete, +you can safely parallelize the rest of the work into seperate systems, improving both performance and maintainability. + ### States **States** are one particularly common and powerful form of run criteria, allowing you to toggle systems on-and-off based on the value of a given resource and smoothly handle transitions between states by running cleanup and initialization systems. From 6dcb8127e123ec608027a5315d59ed710f544d00 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 16:10:08 -0500 Subject: [PATCH 026/313] Expanded on drawbacks of removing state-stack --- rfcs/45-stageless.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 0f990178..153cfadb 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -472,7 +472,10 @@ To do so, we need to check both direct *and* transitive strict ordering dependen 2. Users will typically need to think a bit harder about exactly when they want their gameplay systems to run. In most cases, they should just add the `CoreLabel::GameLogic` label to them. 2. It will be harder to immediately understand the global structure of Bevy apps. 1. Powerful system debugging and visualization tools become even more important. -3. States will no longer have stacks. This could break some users, but they should be able to write their own replacement easily enough externally. +3. State transitions are no longer queued up in a stack. + 1. Arbitrary chains of state transitions can no longer be processed in the same frame, due to the lack of a state stack abstraction. + 2. This also removes "in-stack" and related system groups / logic. + 3. This can be easily added later, or third-party plugins can create their own abstraction. 4. It will become harder to reason about exactly when command flushing occurs. 5. Ambiguity sets are not directly accounted for in the new design. 1. They need more thought and are rarely used; we can toss them back on later. From 7823a0636c1b38f629094c30cbbb4e3865b9baac Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 16:41:52 -0500 Subject: [PATCH 027/313] Future work: systems as entities --- rfcs/45-stageless.md | 1 + 1 file changed, 1 insertion(+) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 153cfadb..f65313c3 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -562,3 +562,4 @@ Despite the large scope of this RFC, it leaves quite a bit of interesting follow 2. How does this work for run criteria that are not locally defined? 6. A more cohesive look at plugin definition and configuration strategies. 7. A graph-based system ordering API for dense, complex dependencies. +8. Store systems in the `World` as entities. From 7506f941807cf8fd5da7b3d2901284ccd3f54d05 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 16:45:50 -0500 Subject: [PATCH 028/313] Typo fix Co-authored-by: Daniel McNab <36049421+DJMcNab@users.noreply.github.com> --- rfcs/45-stageless.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index f65313c3..75cc74c7 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -91,7 +91,7 @@ A system may be configured in the following ways: - there are several variations on this, see the next section for details - e.g. `.add_system(player_controls.before(GameLogic::Physics))` - it may have one or more **run criteria** attached - - a system is only executed if all of its run criteria returns `true` + - a system is only executed if all of its run criteria return `true` - **states** are a special, more complex pattern that use run criteria to determine whether or not a system should run in a current state - e.g. `.add_system(physics.run_if(GameLogic::Physics))` From bbfc8ddbf9fac525ff62062dc6fd82bd1114453d Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 16:47:06 -0500 Subject: [PATCH 029/313] Important fix Co-authored-by: Daniel McNab <36049421+DJMcNab@users.noreply.github.com> --- rfcs/45-stageless.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 75cc74c7..9b3e10c5 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -36,7 +36,7 @@ The following elements are substantially reworked: - command processing (now performed in a `flush_commands` exclusive system) - labels (can now be directly configured) - system sets (now just used to mass-apply a label to a group of systems) -- stages (yeet) +- stages (yoten) ### Scheduling overview and intent-based configuration From 0f3200401ce723a8a0510adc791fbf539e3d64a9 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 16:49:26 -0500 Subject: [PATCH 030/313] Remove suggestion to repeatedly run startup systems Totally possible, but almost always misguided --- rfcs/45-stageless.md | 1 - 1 file changed, 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 9b3e10c5..67e7d3b2 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -74,7 +74,6 @@ Startup systems are stored in their own schedule, with the `CoreSchedule::Startu When using the runner added by `MinimalPlugins` and `DefaultPlugins`, this schedule will run exactly once on app startup. You can add startup systems with the `.add_startup_system(my_system)` method on `App`, which is simply sugar for `.add_system(my_system.to_schedule(CoreSchedule::Startup))`. -Startup systems can be run again by running this schedule from within an exclusive system. ### Introduction to system configuration From 2584ade7125b38f98cf5d8ff1c37e959944cce6e Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 16:52:30 -0500 Subject: [PATCH 031/313] Down with _system! --- rfcs/45-stageless.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 67e7d3b2..ed5df3ed 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -73,7 +73,7 @@ Unsurprisingly, these never conflict with entities or resources in the `World`, Startup systems are stored in their own schedule, with the `CoreSchedule::Startup` label. When using the runner added by `MinimalPlugins` and `DefaultPlugins`, this schedule will run exactly once on app startup. -You can add startup systems with the `.add_startup_system(my_system)` method on `App`, which is simply sugar for `.add_system(my_system.to_schedule(CoreSchedule::Startup))`. +You can add startup systems with the `.add_startup_system(on_startup)` method on `App`, which is simply sugar for `.add_system(on_startup.to_schedule(CoreSchedule::Startup))`. ### Introduction to system configuration @@ -271,11 +271,11 @@ These are typically (but not necessarily) enums, where each distinct state is re Each state is associated with three sets of systems: 1. **Update systems:** these systems run each frame if and only if the value of the state resource matches the provided value. - 1. `app.add_system(my_system.in_state(GameState::Playing))` + 1. `app.add_system(apply_damage.in_state(GameState::Playing))` 2. **On-enter systems:** these systems run once when the specified state is entered. - 1. `app.add_system(my_system.on_enter(GameState::Playing))` + 1. `app.add_system(generate_map.on_enter(GameState::Playing))` 3. **On-exit systems:** these systems run once when the specified stated is exited. - 1. `app.add_system(my_system.on_enter(GameState::Playing))` + 1. `app.add_system(autosave.on_exit(GameState::Playing))` Update systems are by far the simplest: they're simply powered by run criteria. `.in_state` is precisely equivalent to `run_if_resource_is`, except with an additional trait bound that the resource must implement `State`. From 1752c162f276b2c613e8052b1654dd54c0584d92 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 16:53:49 -0500 Subject: [PATCH 032/313] Condense .add_label and .configure_label Credit to @DjMcNab --- rfcs/45-stageless.md | 30 +++--------------------------- 1 file changed, 3 insertions(+), 27 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index ed5df3ed..6b37a86a 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -121,9 +121,9 @@ impl Plugin for PhysicsPlugin{ app // We can reuse this shared configuration on each of our labels - .add_label(Forces.configure(common_physics_config).before(Physics::CollisionDetection)) - .add_label(Physics::CollisionDetection.configure(common_physics_config).before(Physics::CollisionHandling)) - .add_label(Physics::CollisionHandling.configure(common_physics_config)) + .configure_label(Forces.configure(common_physics_config).before(Physics::CollisionDetection)) + .configure_label(Physics::CollisionDetection.configure(common_physics_config).before(Physics::CollisionHandling)) + .configure_label(Physics::CollisionHandling.configure(common_physics_config)) // And then apply that config to each of the systems that have this label .add_system(gravity.label(Forces)) // These systems have a linear chain of ordering dependencies between them @@ -136,30 +136,6 @@ impl Plugin for PhysicsPlugin{ } ``` -Or, you can apply labels to your systems, and then configure them later: - -```rust -impl Plugin for PhysicsPlugin{ - - // This code has the exact same effect as the snipped above - fn build(app: &mut App){ - let mut common_physics_config = SystemConfig::new().after(CoreLabel::InputHandling).before(CoreLabel::Rendering); - - app - .add_system(gravity.label(Forces)) - .add_system_chain([broad_pass, narrow_pass].label(Physics::CollisionDetection)) - .add_system_set([compute_forces, collision_damage], Physics::CollisionHandling) - // This adds the new configuration constraints to the labels in an additive fashion - .configure_label(Physics::Forces.configure(common_physics_config).before(Physics::CollisionDetection)) - .configure_label(Physics::CollisionDetection.configure(common_physics_config).before(Physics::CollisionHandling)) - .configure_label(Physics::CollisionHandling.configure(common_physics_config)); - } -} -``` - -For the most part, this is a matter of code organization and style: defining the properties of your labels up-front helps keep your code clean and your configuration centralized. -On the other hand, configuring labels later allows you to modify the configuration of labels added by external plugins, customizing them to fit your game's needs. - You can apply the same label(s) to many systems at once using **system sets** with the `App::add_system_set(systems: impl SystemIterator, labels: impl SystemLabelIterator)` method. ### System ordering From 8b84f305764c142a094e3de7526b2ec81768a1ae Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 16:55:28 -0500 Subject: [PATCH 033/313] Example typo fixes --- rfcs/45-stageless.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 6b37a86a..3bc1fb26 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -121,11 +121,11 @@ impl Plugin for PhysicsPlugin{ app // We can reuse this shared configuration on each of our labels - .configure_label(Forces.configure(common_physics_config).before(Physics::CollisionDetection)) + .configure_label(Physics::Forces.configure(common_physics_config).before(Physics::CollisionDetection)) .configure_label(Physics::CollisionDetection.configure(common_physics_config).before(Physics::CollisionHandling)) .configure_label(Physics::CollisionHandling.configure(common_physics_config)) // And then apply that config to each of the systems that have this label - .add_system(gravity.label(Forces)) + .add_system(gravity.label(Physics::Forces)) // These systems have a linear chain of ordering dependencies between them // Systems earlier in the chain must run before those later in the chain .add_system_chain([broad_pass, narrow_pass].label(Physics::CollisionDetection)) From 753e6c5bb537df6b82c2e993469167b88fdaa386 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:01:39 -0500 Subject: [PATCH 034/313] Unsatisfiable explanation --- rfcs/45-stageless.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 3bc1fb26..95948701 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -42,7 +42,8 @@ The following elements are substantially reworked: Systems in Bevy are stored in a `Schedule`: a collection of **configured systems**. Each frame, the `App`'s `runner` function will run the schedule, -causing the **scheduler** to run the systems in parallel using a strategy that respects all of the configured constraints (or panic, if it is **unsatisfiable**). +causing the **scheduler** to run the systems in parallel using a strategy that respects all of the configured constraints. +If these constraints cannot be met (for example, a system may want to run both before and after another system), the schedule is said to be **unsatisfiable**, and the scheduler will panic. In the beginning, each `Schedule` is entirely unordered: systems will be selected in an arbitrary order and run if and only if all of the data that it must access is free. Just like with standard borrow checking, multiple systems can read from the same data at once, but writing to the data requires an exclusive lock. From 2f2d869798b7f130c19d0a64cccc8c57e58503ce Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:06:53 -0500 Subject: [PATCH 035/313] Cannot directly mutate current schedule --- rfcs/45-stageless.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 95948701..48fe3afa 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -60,7 +60,7 @@ Just as importantly, we are not over-constraining our ordering, allowing subtle The `App` can store multiple `Schedules` in a `HashMap, Schedule>` storage. This is used for the enter and exit schedules of states, but can also be used to store, mutate and access additional schedules. -You can even mutate the schedule that you are in, although you cannot mutate systems that are currently running (failing with an error), as that would be UB. +For safety reasons, you cannot mutate schedules that are currently being run: instead, you can defer their modification until the end of the main loop using `ScheduleCommands`. The main and startup schedules can be accessed using the `CoreSchedule::Main` and `CoreSchedule::Startup` labels respectively. By default systems are added to the main schedule. @@ -369,6 +369,8 @@ struct Schedule{ // Used to quickly look up system ids by the type ids of their underlying function // when checking atomic ordering constraints system_function_type_map: HashMap>, + // Used to check if mutation is allowed + currently_running: bool, } ``` From b1527b5aff54c9a539873358caac48ec109ff451 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:09:27 -0500 Subject: [PATCH 036/313] Fixed broken sentence --- rfcs/45-stageless.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 48fe3afa..60192873 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -49,7 +49,7 @@ In the beginning, each `Schedule` is entirely unordered: systems will be selecte Just like with standard borrow checking, multiple systems can read from the same data at once, but writing to the data requires an exclusive lock. Systems which cannot be run in parallel are said to be **incompatible**. -Bevy controls the ordering of its schedule(s) The solution via **intent-based configuration**: record the exact, minimal set of constraints needed for a scheduling strategy to be valid, and then let the scheduler figure it out! +In Bevy, schedules are ordered via **intent-based configuration**: record the exact, minimal set of constraints needed for a scheduling strategy to be valid, and then let the scheduler figure it out! As the number of systems in your app grows, the number of possible scheduling strategies grows exponentially, and the set of *valid* scheduling strategies grows almost as quickly. However, each system will continue to work hand-in-hand with a small number of closely-related systems, interlocking in a straightforward and comprehensible way. From e5e7b6a6ad56498ead2b458b85f15383eea39a09 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:10:31 -0500 Subject: [PATCH 037/313] CoreSchedule -> DefaultSchedule --- rfcs/45-stageless.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 60192873..e018774f 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -62,7 +62,7 @@ The `App` can store multiple `Schedules` in a `HashMap, S This is used for the enter and exit schedules of states, but can also be used to store, mutate and access additional schedules. For safety reasons, you cannot mutate schedules that are currently being run: instead, you can defer their modification until the end of the main loop using `ScheduleCommands`. -The main and startup schedules can be accessed using the `CoreSchedule::Main` and `CoreSchedule::Startup` labels respectively. +The main and startup schedules can be accessed using the `DefaultSchedule::Main` and `DefaultSchedule::Startup` labels respectively. By default systems are added to the main schedule. You can control this by adding the `.to_schedule(MySchedule::Variant)` system descriptor to your system. @@ -71,10 +71,10 @@ Unsurprisingly, these never conflict with entities or resources in the `World`, #### Startup systems -Startup systems are stored in their own schedule, with the `CoreSchedule::Startup` label. +Startup systems are stored in their own schedule, with the `DefaultSchedule::Startup` label. When using the runner added by `MinimalPlugins` and `DefaultPlugins`, this schedule will run exactly once on app startup. -You can add startup systems with the `.add_startup_system(on_startup)` method on `App`, which is simply sugar for `.add_system(on_startup.to_schedule(CoreSchedule::Startup))`. +You can add startup systems with the `.add_startup_system(on_startup)` method on `App`, which is simply sugar for `.add_system(on_startup.to_schedule(DefaultSchedule::Startup))`. ### Introduction to system configuration From 26c2e55ccd01c616366760e37c6854337c20c362 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:12:48 -0500 Subject: [PATCH 038/313] Clarity fix on ordering config --- rfcs/45-stageless.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index e018774f..a5f2376c 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -88,8 +88,8 @@ A system may be configured in the following ways: - labels themselves may be configured, allowing you to define high-level structure in your `App` in a single place - e.g `app.configure_label(GameLogic::Physics, SystemConfig::new().label(GameLogic::Physics).after(GameLogic::Input))` - it may have ordering constraints, causing it to run before or after other systems - - there are several variations on this, see the next section for details - - e.g. `.add_system(player_controls.before(GameLogic::Physics))` + - there are several subtly different types of ordering constraints, see the next section for details + - e.g. `.add_system(player_controls.before(GameLabels::Physics).after(CoreLabels::Input))` - it may have one or more **run criteria** attached - a system is only executed if all of its run criteria return `true` - **states** are a special, more complex pattern that use run criteria to determine whether or not a system should run in a current state From 04701dbffccbc27333e25bf93df2c393db772860 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:20:26 -0500 Subject: [PATCH 039/313] Explain why we can't use a Vec / usize strategy for SystemId --- rfcs/45-stageless.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index a5f2376c..5220c040 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -361,7 +361,9 @@ struct Schedule{ // Each schedule is associated with exactly one `World`, as systems must be initiliazed on it world_id: WorldId // We need to be able to quickly look up specific systems and iterate over them - // `SystemId` should be a unique identifier, atomically generated on system insertion + // `SystemId` should be a unique opaque identifier, generated on system insertion + // We cannot use a Vec + a `usize` index as the identifier, + // as the identifier would not be stable when systems were removed, risking serious bugs systems: HashMap, // Label configuration must be stored at the schedule level, // rather than attached to specific label instances From a5f850828d944418b51a71273de80711b40f26cd Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:21:58 -0500 Subject: [PATCH 040/313] Fix label configuration syntax Oops, thanks @DjMcNab --- rfcs/45-stageless.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 5220c040..9b21f36d 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -86,7 +86,7 @@ A system may be configured in the following ways: - labels are used by other systems to determine ordering constraints - e.g. `.add_system(gravity.label(GameLogic::Physics))` - labels themselves may be configured, allowing you to define high-level structure in your `App` in a single place - - e.g `app.configure_label(GameLogic::Physics, SystemConfig::new().label(GameLogic::Physics).after(GameLogic::Input))` + - e.g `app.configure_label(GameLogic::Physics.after(GameLogic::Input))` - it may have ordering constraints, causing it to run before or after other systems - there are several subtly different types of ordering constraints, see the next section for details - e.g. `.add_system(player_controls.before(GameLabels::Physics).after(CoreLabels::Input))` From 6c3ac4708714af1b0573df9320adf38955d86bb5 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:53:59 -0500 Subject: [PATCH 041/313] Rename 0.6 system chaining -> system handling rename --- rfcs/45-stageless.md | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 9b21f36d..50356157 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -129,6 +129,8 @@ impl Plugin for PhysicsPlugin{ .add_system(gravity.label(Physics::Forces)) // These systems have a linear chain of ordering dependencies between them // Systems earlier in the chain must run before those later in the chain + // Other systems can run in between these systems; + // use `add_atomic_system_chain` if this is not desired .add_system_chain([broad_pass, narrow_pass].label(Physics::CollisionDetection)) // System sets apply a set of labels to a collection of systems // and are helpful for reducing boilerplate @@ -159,6 +161,7 @@ If an ordering is defined relative to a non-existent system or label, it will ha This relatively gentle failure mode is important to ensure that plugins can order their systems with relatively strong assumptions that the default system labels exist, but continue to (mostly) work if those systems or labels are not present. In addition to the `.before` and `.after` methods, you can use **system chains** to create very simple linear dependencies between the succesive members of an array of systems. +(Note to readers: this is not the same as "system chaining" in Bevy 0.6 and earlier: that concept has been renamed to "system handling".) When discussing system ordering, it is particularly important to call out the `flush_commands` system. This **exclusive system** (meaning, it can modify the entire `World` in arbitrary ways and cannot be run in parallel with other systems) collects all created commands and applies them to the `World`. @@ -341,10 +344,14 @@ Let's take a look at what implementing this would take: 2. Implement on-update states as sugar for simple run criteria 3. Create on-enter and on-exit schedules 4. Create sugar for adding systems to these schedules -9. Add new examples - 1. Complex control flow with supplementary schedules - 2. Fixed time-step pattern -10. Port the entire engine to the new paradigm +9. Rename "system chaining" to "system handling" + 1. Usage was very confusing for new users + 2. Almost exclusively used for error handling + 3. New concept of "systems with a linear graph of ordering constraints between them" is naturally described as a chain +10. Add new examples + 1. Complex control flow with supplementary schedules + 2. Fixed time-step pattern +11. Port the entire engine to the new paradigm 1. We almost certainly need to port the improved ambiguity checker over to make this reliable Given the massive scope, that sounds relatively straightforward! @@ -493,12 +500,12 @@ On the other hand, it *will* result in pointless and surprising blocking behavio If-needed ordering is the corret stategy in virtually cases: in Bevy, interior mutability at the component or resource level is rare, almost never needed and results in other serious and subtle bugs. As we move towards specifying system ordering dependencies at scale, it is critical to avoid spuriously breaking users schedules, and silent, pointless performance hits are never good. -### Why can't we just use system chaining for run criteria? +### Why can't we just use system handling (previously system chaining) for run criteria? There are two reasons why this doesn't work: 1. The system we're applying a run criteria does not have an input type. -2. System chaining does not work if the chained systems are incompatible. This is far too limiting. +2. System handling does not work if the `SystemParam` of the original and handling systems are incompatible. This is far too limiting. ### Why aren't run criteria cached? @@ -514,8 +521,6 @@ Storing the schedules in the `App` alleviates this, as exclusive systems are now ## Unresolved questions -- If we want to support a simple `add_system_chain` API as a precursor to a system-graph-specification API, what do we rename "system chaining" to? - - System welding? System fusing? System handling? - What is the best way to handle the migration process from an organizational perspective? - Get an RFC approved, and then merge a massive PR developed on a off-org branch? - Straightforward, but very hard to review From 3ca210698912f358e7d0257c6aebc23d3fbb5077 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:57:28 -0500 Subject: [PATCH 042/313] Hide system building implementation details Thanks @Nilirad --- rfcs/45-stageless.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 50356157..efb572f6 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -78,8 +78,6 @@ You can add startup systems with the `.add_startup_system(on_startup)` method on ### Introduction to system configuration -When a function is added to a schedule, it is turned into a `RawSystem` and combined with a `SystemConfig` struct to create a `ConfiguredSystem`. - A system may be configured in the following ways: - it may have any number of **labels** (which implement the `SystemLabel` trait) From 57972bc8a8ba646687cc4c9c09735d3e26797c85 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:58:21 -0500 Subject: [PATCH 043/313] GameLogic label -> AppLogic --- rfcs/45-stageless.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index efb572f6..b2b46f3c 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -95,7 +95,7 @@ A system may be configured in the following ways: System configuration can be stored in a `SystemConfig` struct. This can be useful to reuse, compose and quickly apply complex configuration strategies, before applying these strategies to labels or individual systems. -If you just want to run your game logic systems in the middle of your schedule, after input is processed but before rendering occurs, add the premade `CoreLabel::GameLogic` to them. +If you just want to run your game logic systems in the middle of your schedule, after input is processed but before rendering occurs, add the premade `CoreLabel::AppLogic` to them. ### Label configuration @@ -454,7 +454,7 @@ To do so, we need to check both direct *and* transitive strict ordering dependen 1. This will be a massively breaking change to every single Bevy user. 1. Any non-trivial control over system ordering will need to be completely reworked. - 2. Users will typically need to think a bit harder about exactly when they want their gameplay systems to run. In most cases, they should just add the `CoreLabel::GameLogic` label to them. + 2. Users will typically need to think a bit harder about exactly when they want their gameplay systems to run. In most cases, they should just add the `CoreLabel::AppLogic` label to them. 2. It will be harder to immediately understand the global structure of Bevy apps. 1. Powerful system debugging and visualization tools become even more important. 3. State transitions are no longer queued up in a stack. From 0a1d3bdc87e739cdd908da257bcc6a2e4dc87c42 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:59:23 -0500 Subject: [PATCH 044/313] Better wording Co-authored-by: Federico Rinaldi --- rfcs/45-stageless.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index b2b46f3c..3e547290 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -152,7 +152,7 @@ There are two basic forms of system ordering constraints: 3. Can cause unneccessary blocking, particularly when systems are configured at a high-level. 2. **If-needed ordering constraints:** `.before` and `.after` 1. A system cannot be scheduled until any "before" systems that it is incompatible with have completed this frame. - 2. In 99% of cases, this is the desired behavior. Unless you are using interior mutability, systems that are compatible will always be **commutative**: their ordering doesn't matter. + 2. In the vast majority of cases, this is the desired behavior. Unless you are using interior mutability, systems that are compatible will always be **commutative**: their ordering doesn't matter. Applying an ordering constraint to or from a label causes a ordering constraint to be created between all individual members of that label. If an ordering is defined relative to a non-existent system or label, it will have no effect, emitting a warning. From a4e9f55997dc81840a55de42ef6991c3e9fa3a15 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 17:59:43 -0500 Subject: [PATCH 045/313] Clarity nit Co-authored-by: Federico Rinaldi --- rfcs/45-stageless.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 3e547290..49dba491 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -172,7 +172,7 @@ Note that **this does not insert new copies of a `flush_commands` system**: inst ### Run criteria -While ordering constraints determine *when* a system will run, **run criteria** will determine *if* it will run at all. +While ordering constraints determine *when* a system should run, **run criteria** will determine *if* it will run at all. A run criteria is a special kind of system, which can read data from the `World` and returns a boolean value. If its output is `true`, the system it is attached to will run during this pass of the `Schedule`; if it is `false`, the system will be skipped. From 6e00812658d128605378b009cdba86b1e61dda45 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 18:01:34 -0500 Subject: [PATCH 046/313] Typo fix Co-authored-by: Federico Rinaldi --- rfcs/45-stageless.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 49dba491..b3dff428 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -252,7 +252,7 @@ Each state is associated with three sets of systems: 1. `app.add_system(apply_damage.in_state(GameState::Playing))` 2. **On-enter systems:** these systems run once when the specified state is entered. 1. `app.add_system(generate_map.on_enter(GameState::Playing))` -3. **On-exit systems:** these systems run once when the specified stated is exited. +3. **On-exit systems:** these systems run once when the specified state is exited. 1. `app.add_system(autosave.on_exit(GameState::Playing))` Update systems are by far the simplest: they're simply powered by run criteria. From eff83c48af12e72df7b14b2d5aaf7fec790909d1 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 18:09:40 -0500 Subject: [PATCH 047/313] Run criteria can read but not write from the world --- rfcs/45-stageless.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index b3dff428..31c6d3ca 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -172,8 +172,8 @@ Note that **this does not insert new copies of a `flush_commands` system**: inst ### Run criteria -While ordering constraints determine *when* a system should run, **run criteria** will determine *if* it will run at all. -A run criteria is a special kind of system, which can read data from the `World` and returns a boolean value. +While ordering constraints determine *when* a system will run, **run criteria** will determine *if* it will run at all. +A run criteria is a special kind of system, which can read (but not write) data from the `World` and returns a boolean value. If its output is `true`, the system it is attached to will run during this pass of the `Schedule`; if it is `false`, the system will be skipped. Systems that are skipped are considered completed for the purposes of ordering constraints. From 032080dad66b539b1ee339fbdb017b1b3034f85b Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 18:12:05 -0500 Subject: [PATCH 048/313] Standardize on `run_if_resource_equals` --- rfcs/45-stageless.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 31c6d3ca..a02314cd 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -196,7 +196,7 @@ fn main(){ // We can use closures for simple one-off run criteria, // which automatically fetch the appropriate data from the `World` .add_system(spawn_more_enemies.run_if(|difficulty: Res| difficulty >= 9000)) - // The `run_if_resource_is` method is convenient syntactic sugar that generates a run criteria + // The `run_if_resource_equals` method is convenient syntactic sugar that generates a run criteria // for when you want to check the value of a resource (commonly an enum) .add_system(gravity.run_if_resource_equals(Gravity::Enabled)) // Run criteria can be attached to labels: a copy of the run criteria will be applied to each system with that label @@ -256,7 +256,7 @@ Each state is associated with three sets of systems: 1. `app.add_system(autosave.on_exit(GameState::Playing))` Update systems are by far the simplest: they're simply powered by run criteria. -`.in_state` is precisely equivalent to `run_if_resource_is`, except with an additional trait bound that the resource must implement `State`. +`.in_state` is precisely equivalent to `run_if_resource_equals`, except with an additional trait bound that the resource must implement `State`. On-enter and on-exit systems are stored in dedicated schedules, two per state, within the `App's` `Schedules`. These schedules can be configured in all of the ordinary ways, but, as they live in different schedules, ordering cannot be defined relative to systems in the main schedule. From 95dd56d81ea3a0fc822090aa00868bfdb2b10000 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 18:20:44 -0500 Subject: [PATCH 049/313] Prefer "iteration of the schedule" to "frame" --- rfcs/45-stageless.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index a02314cd..3ea035e6 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -41,9 +41,10 @@ The following elements are substantially reworked: ### Scheduling overview and intent-based configuration Systems in Bevy are stored in a `Schedule`: a collection of **configured systems**. -Each frame, the `App`'s `runner` function will run the schedule, +In each iteration of the schedule (typically corresponding to a single frame), the `App`'s `runner` function will run the schedule, causing the **scheduler** to run the systems in parallel using a strategy that respects all of the configured constraints. If these constraints cannot be met (for example, a system may want to run both before and after another system), the schedule is said to be **unsatisfiable**, and the scheduler will panic. +Schedules can be nested and branched from within **exclusive systems** (which have mutable access to the entire `World`), allowing you to encode arbitrarily complex control flow using ordinary Rust constructs. In the beginning, each `Schedule` is entirely unordered: systems will be selected in an arbitrary order and run if and only if all of the data that it must access is free. Just like with standard borrow checking, multiple systems can read from the same data at once, but writing to the data requires an exclusive lock. @@ -147,11 +148,11 @@ The ordering of systems is always defined relative to other systems: either dire There are two basic forms of system ordering constraints: 1. **Strict ordering constraints:** `strictly_before` and `strictly_after` - 1. A system cannot be scheduled until "strictly before" systems must have been completed this frame. + 1. A system cannot be scheduled until "strictly before" systems must have been completed during this iteration of the schedule. 2. Simple and explicit. 3. Can cause unneccessary blocking, particularly when systems are configured at a high-level. 2. **If-needed ordering constraints:** `.before` and `.after` - 1. A system cannot be scheduled until any "before" systems that it is incompatible with have completed this frame. + 1. A system cannot be scheduled until any "before" systems that it is incompatible with have completed during this iteration of the schedule. 2. In the vast majority of cases, this is the desired behavior. Unless you are using interior mutability, systems that are compatible will always be **commutative**: their ordering doesn't matter. Applying an ordering constraint to or from a label causes a ordering constraint to be created between all individual members of that label. @@ -248,7 +249,7 @@ These are typically (but not necessarily) enums, where each distinct state is re Each state is associated with three sets of systems: -1. **Update systems:** these systems run each frame if and only if the value of the state resource matches the provided value. +1. **Update systems:** these systems run each schedule iteration if and only if the value of the state resource matches the provided value. 1. `app.add_system(apply_damage.in_state(GameState::Playing))` 2. **On-enter systems:** these systems run once when the specified state is entered. 1. `app.add_system(generate_map.on_enter(GameState::Playing))` @@ -458,7 +459,7 @@ To do so, we need to check both direct *and* transitive strict ordering dependen 2. It will be harder to immediately understand the global structure of Bevy apps. 1. Powerful system debugging and visualization tools become even more important. 3. State transitions are no longer queued up in a stack. - 1. Arbitrary chains of state transitions can no longer be processed in the same frame, due to the lack of a state stack abstraction. + 1. Arbitrary chains of state transitions are no longer be processed in the same schedule pass, due to the lack of a state stack abstraction. 2. This also removes "in-stack" and related system groups / logic. 3. This can be easily added later, or third-party plugins can create their own abstraction. 4. It will become harder to reason about exactly when command flushing occurs. From dd05245f3de7ad28f05f02a64543b026d329b6ed Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 18:22:54 -0500 Subject: [PATCH 050/313] Clarify state stack looping drawback --- rfcs/45-stageless.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 3ea035e6..58b80f59 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -459,9 +459,10 @@ To do so, we need to check both direct *and* transitive strict ordering dependen 2. It will be harder to immediately understand the global structure of Bevy apps. 1. Powerful system debugging and visualization tools become even more important. 3. State transitions are no longer queued up in a stack. - 1. Arbitrary chains of state transitions are no longer be processed in the same schedule pass, due to the lack of a state stack abstraction. - 2. This also removes "in-stack" and related system groups / logic. - 3. This can be easily added later, or third-party plugins can create their own abstraction. + 1. By default, arbitrarily long chains of state transitions are no longer be processed in the same schedule pass. + 2. However, users can add as many copies of the `flush_state` system as they would like, and loop it within an exclusive system. + 3. This also removes "in-stack" and related system groups / logic. + 4. This can be easily added later, or third-party plugins can create their own abstraction. 4. It will become harder to reason about exactly when command flushing occurs. 5. Ambiguity sets are not directly accounted for in the new design. 1. They need more thought and are rarely used; we can toss them back on later. From c678b2ac430ca23c9d16e392fed281f949fee255 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 18:38:14 -0500 Subject: [PATCH 051/313] More clarity on satisfiability challenges with atomic run criteria --- rfcs/45-stageless.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 58b80f59..9498a526 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -206,10 +206,15 @@ fn main(){ } ``` -**When multiple run criteria are attached to the same system, the system will run if and only if all of those run criteria return true.** +There are a few important subtleties to bear in mind when working with run criteria: -Run criteria are evaluated "just before" the system that is attached to is run. -The state that they read from will always be "valid" when the system that they control is being run: this is important to avoid strange bugs caused by race conditions and non-deterministic system ordering. +- when multiple run criteria are attached to the same system, the system will run if and only if all of those run criteria return true +- run criteria are evaluated "just before" the system that is attached to is run +- if a run criteria is attached to a label, a run criteria system will be generated for each system that has that label + - this is essential to ensure that run criteria are checking fresh state without creating very difficult to satify ordering constraints + - if you need to ensure that all systems behave the same way during a single pass of the schedule or avoid expensive recomputation, precompute the value and store the result in a resource, then read from that in your run criteria instead + +The state that run criteria read from will always be "valid" when the system that they control is being run: this is important to avoid strange bugs caused by race conditions and non-deterministic system ordering. It is impossible, for example, for a game to be paused *after* the run criteria checked if the game was paused but before the system that relied on the game not being paused completes. In order to understand exactly what this means, we need to understand atomic ordering constraints. @@ -234,6 +239,11 @@ allowing you to ensure that the index is still valid when the system using the i Of course, atomic ordering constraints should be used thoughtfully. As the strictest of the three types of system ordering dependency, they can easily result in unsatisfiable schedules if applied to large groups of systems at once. +For example, consider the simple case, where we have an atomic ordering constraint from system A to system B. +If for any reason commands must be flushed between systems A and system B, we cannot satisfy the schedule. +The reason for this is quite simple: the lock on data from system A will not be released when it completes, but we cannot run exclusive systems unless all locks on the `World's` data are clear. +As we add more of these atomic constraints (particularly if they share a single ancestor), the risk of these problems grows. +In effect, atomically-connected systems look "almost like" a single system from the scheduler's perspective. On the other hand, atomic ordering constraints can be helpful when attempting to split large complex systems into multiple parts. By guaranteeing that the initial state of your complex system cannot be altered before the later parts of the system are complete, From 47ce0bb201f4b84a977c21b421a94419d9128390 Mon Sep 17 00:00:00 2001 From: Alice Cecile Date: Fri, 14 Jan 2022 18:43:19 -0500 Subject: [PATCH 052/313] Tick the timer --- rfcs/45-stageless.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/rfcs/45-stageless.md b/rfcs/45-stageless.md index 9498a526..0c738cdc 100644 --- a/rfcs/45-stageless.md +++ b/rfcs/45-stageless.md @@ -189,11 +189,17 @@ fn construction_timer_finished(timer: Res) -> bool { timer.finished() } +// Timers need to be ticked! +fn tick_construction_timer(timer: ResMut, time: Res