[RFC] Streaming vs File I/O Perfomance

[thescientist13]

Type of Change

• New Feature Request
• Documentation / Website
• Improvement / Suggestion
• Bug
• Other (please clarify below)

Summary

Currently the building of the project is done by generating a SPA from the user's project directory, and then serializing that with a browser. The RFC here is to facilitate all that in memory and reduce the amount of filesystem I/O, and see if that can be fed directly to the serialize phase. (essentially cut out the scaffolding phase, somehow).

Details

1. One thought is to just use webpack-dev-server instead of local-web-server but in production "mode" so that we can drop the extra dependency and keep everything in memory.

Ideally this would help improve the build for larger projects.

[thescientist13]

Oh man... this could be exactly what we need for this project
https://www.npmjs.com/package/memory-fs

Even better if webpack adds support for it!
webpack/webpack#9227

Having everything in memory means scaffolding and configuration could all be passed around as data and included into the graph. This could really make working on #17 (and doing things like <meta> a lot a lot easier / cleaner! 🤞

Then we could really have the functional core / imperative shell (and we could unit test all our tasks!! 💪 )
https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell

cc: @hutchgrant

[thescientist13]

I wonder, if having everything in memory, means things like Build Time Data Injection and External Data Sources could also be done this way? 🤔

Could save a lot of headache, and allow patching / updating of the graph to be done dynamically, amongst other things. Basically, we would be able to treat the state of Greenwood as an AST!? 🤩

Also, found this GitHub issue, which could be somewhat related, around having a virtual entry point?
webpack/webpack#6437

[thescientist13]

I think in light of no bundle development with #355 and that webpack is out of the picture, I don't think there's a lot of need for this, though there are some ideas for trying to improve build performance, but that will need to happen a bit later, like after 1.0 and a new issue would get made for that instead.