Converting Vulkan target to support executable-create2.

[benvanik]

This produces a new flatbuffer that supports multiple shared modules per HAL executable, reorganizes per-export information to be per-export, and swaps HAL pipeline layouts with internal Vulkan pipeline layouts that are constructed from metadata in the flatbuffer.

This is a startup time regression for as long as we are not linking modules because descriptor set and pipeline layout reuse now happens per executable and not per VM context. The fix is to link modules together. I am not sure why that was never done :/

Progress on #18154.

[benvanik]

(lit test failures are expected during the transition)

[ScottTodd]

Interesting - did you run those speed tests / benchmarks yourself, or did other people report that somewhere? (I'll buy the point about error messages, but startup time is pretty important for us, so would prefer to have high confidence that what we're doing is optimal)

[benvanik]

Yeah, drivers are not required and don't multithread pipeline creation internally (as a general rule Vulkan drivers don't maintain their own thread pools in user mode). The only savings is on the call into the driver which is tiny compared to actual creation and shader JIT that usually happens.

From the Vulkan guide:

For compiling pipelines, vkCreateShaderModule and vkCreateGraphicsPipeline are both allowed to be called from multiple threads at once. A common approach for multithreaded shader compilation is to have a background thread dedicated to it, with it constantly looking into a parallel queue to receive compilation requests, and putting the compiled pipelines into another queue that then the main renderthread will connect to the simulation. This is very important to do if you want to have an engine that doesn’t have a lot of hitching. Compiling shader pipelines can take a very long time, so if you have to compile pipelines at runtime outside of a load screen, then you need to implement such a multithreaded async compile scheme for your game to work well.