Optimizing mounting to a serverside rendered HTML tree (DOM hydration)

[dineshselvantdm-zz]

Right now Mithril recalculates the VDOM while mounting over a server-side rendered HTML. Can it be optimized?

[pygy]

It is called DOM hydration, and it would need explicit support.

I was looking into it last autumn before being side tracked by the router.

[dead-claudia]

I also looked into it some in 0.2.x, and it's non-trivial to do in general. You literally have to build the initial tree differently, and it will inevitably run into complications with m.trust. You also have the secondary concern of whether the tree is even correct compared to the vnodes.

[ChrisGitIt]

Hi everybody,

i currently worked on a "hydration" feature for mithril. RIght now its properly working for me. There are just minor changes to mithril.js to make it work (mainly its passing my hydration function to "rendering" mithril methods. So for example
m.route(document.body, '/', json_routes)
... becomes ...
m.route(document.body, '/', json_routes, hydration_func)

I just experienced some smaller issues that i think are "unresolvable". For example, if your "create" a several following text nodes, it will ran into a hydration problem.

For example:
m('div', [ $val1 ? 'OK': '', $val2 ? 'OK': '', ])
=> theses renderings are problematic.

I thinks this is primarily the same issue as with using m.trust ... but as i write this ... i just realize that while having control about the server side rendering (https://github.com/StephanHoyer/mithril-isomorphic-example), it might be possible to enclose all subsequently following text nodes with a tag and this way having the possibility to fully rehydrate the dom on the client side.

Is anybody interested in my early alpha "rehydration" approach? My approach is a single function, currently 8 KB unzipped, commented, beautified. I might publish it, so my approach itself might be verified and extended.

Greats,

Chris

[dead-claudia]

@ChrisGitIt

I just experienced some smaller issues that i think are "unresolvable". For example, if your "create" a several following text nodes, it will ran into a hydration problem.

Try either joining Mithril text nodes where possible or just diffing the nodes without the backing components - that will resolve the diff.

Note that Mithril is missing a hook that would otherwise allow third-party integration with this: the ability to mount a root + component without rendering first. @pygy @tivac Would you all support such an addition, something like m.mount(root, component, {hydrated: true})?

[ChrisGitIt]

Hi isiahmeadows and thanks for your feedback!

Joining text nodes is very difficult. A component can return a text node, arrays can return text nodes ... to make my hydration function work, it needs two runs. The first run would be the thing that you described ({hydrated: true}), the second would be to actually assign the vnode.dom ... this actually was my first approach, and somehow i started all over ... i think i saw the opportunity to do it in one run and keep my code size small.

I would much prefer my solution right now despite the fact that two text nodes can occur ... i just have to be a little carefull writing my components and m() markup.

I think the {hydrated: true} will result in a much bigger mithril code base ... i mean like 1 or 2 more KB's, not much actually, but everybody who does not want hydration has to load 2 more KBs.

Passing a vnode accepting function as third argument is a perfect fit. It requires marginal change to mithril code base (just three functions need adjustment about the third parameter) and if you need hydration, you load and pass in the hydration function ...

Am i on the wrong track? Feedback appreciate!

[dead-claudia]

@ChrisGitIt

Joining text nodes is very difficult. A component can return a text node, arrays can return text nodes ... to make my hydration function work, it needs two runs. The first run would be the thing that you described ({hydrated: true}), the second would be to actually assign the vnode.dom ... this actually was my first approach, and somehow i started all over ... i think i saw the opportunity to do it in one run and keep my code size small.

Note the alternate (diffing the text nodes) would be much easier to do, at the cost of recreating text slices.

I think the {hydrated: true} will result in a much bigger mithril code base ... i mean like 1 or 2 more KB's, not much actually, but everybody who does not want hydration has to load 2 more KBs.

To clarify, that option would literally be just to skip this line, to enable third-party support (we've done similar in other areas to help mithril-node-render and a few others out).

[ChrisGitIt]

Hi isiahmeadows and thanks for your feedback!

I thought about your infos and i think it will unnecessarily add code to the mithril base.

Right now, it looks like all render/render.js
-> createText
-> createComponent
etc. have to be rewritten to appreciate {hydrate:true}

My current soultion just needs marginal changes to mithril:
/render/render.js
function render(dom, vnodes) { ... if (dom.vnodes == null) dom.textContent = ""
becomes

function render(dom, vnodes, hydrate) { ... hydrate(dom, vnodes, hooks, null);

/api/mount.js
=> replace two
redrawService.render(root, *)

with
redrawService.render(root, *, hydrate)

About the diffing text nodes: I'm currently experimenting with this. I think its possible to to properly hydrate text nodes. Its not actually joining the text nodes but to create extra text nodes and remove just the one that needs to be hydrated ... this might lead to a flicker ... i'm not totally convinced with this solution... there also might be a solution by using streams ... well, haven't dived much into them.

Any feedback appreciate!

Greats,

Chris

[leeoniya]

what reservations do you guys have against joining adjacent textnodes? i have not seen any issues with doing so, and it makes hydration pretty trivial [1].

[1] https://github.com/leeoniya/domvm/blob/2.x-dev/src/view/addons/attach.js

[ChrisGitIt]

Hi leeoniya!

And also thanks for your feedback! I just had a quick look at your provided trivial solution.

But i think the problem remains ... i think i have to recapitulate for myself:

If there is a call like m('div, ['A','B=',m(my_component)])) and
var my_component = { view: function() { return Math.random() } }
is given, the rendered root DOM Element would look like this=>

AB=0.23

When dom hydration takes place, m() call from before gives me a vnode like this
{ children: ['A', 'B=', my_component] tag: 'div' }
and my root DOM Element "Node View(!!)" would look like this
{ nodeType: 1 nodeName: 'div' children: [{nodeType: 3, nodeValue: 'AB=0.23'}] }
SOOOO ... i'm not sure if this leads to anywhere ...

When i hydrate through the VNODE, the vnode children will ALL be applied to the same DOM text node ... AND ... when there is a redraw, only the my_component would change ... and the change will directly applied to the vnode.dom attached dom node ... and this dom node is the text node with the content "AB=0.23" SOOOOO --- yeah, i think my thought were right --- > the updated vnode.dom.content would be 0.83 and not the expected AB=0.83.

OK, i'm done. I think this is a very simplified problem description, but the problems that arise while doing something like this are very tricky to track down and messes up the whole redraw cycle.

So my current work in progress solution is to create new text element WHILE hydrating, remove the suspicious "joined" DOM text node and append three dom text nodes childs to the DOM div node.

Any feedback appreciate! I'm also not sure if the problem might be easily resolved within the mithril core. Also not sure if the problem described is actually a problem. During developing my hydration function, it looks like this is a problem ... haven't tested it lately ...this might ruin several days of thinking and testing about resolving the problem ... but @isiahmeadows might have had the same problems ... so i'm confident, this IS a problem ;-)

Greats,

Chris

[pygy]

Alternatively, while hydrating, if you encounter several consecutive text vnodes, you can turn them into a fragment of textNodes and replace the textNode in the DOM with it.

Empty text vnodes between elements will also need special care.

Edit: Specifically:

m('p', '')
m('p', '', m('p'))
m('p', m('p'), '')
m('p', m('p'), '', m('p'))
m('p', m('p'), '', '', m('p'))

[dead-claudia]

@leeoniya I'm accounting for the case of fragments. Here's a contrived example of where joining text nodes would end up not matching Mithril's internal model, and Mithril doesn't currently have a layer between text nodes and its model of their slices.

const A = {
    oncreate({dom}) { /* vnode.dom is the text node's dom */ },
    view() { return ["foo"] },
}

const B = {
    view() { return [m(A), "bar"] },
}

[leeoniya]

you may be interested in @thysultan's feedback starting here:

domvm/domvm#101 (comment)

i don't know if the latest dio still works this way.

[thysultan]

It still does Hydrate.js#L54, though this only works if you have an internal data structure to represent text nodes, Element.js#L202.

[dead-claudia]

Mithril does have an internal data structure, but it'd require significant rewriting to avoid it.

I'm experimenting independently with a way of using slices to update text nodes and fragments while keeping a separate model for the actual DOM tree, so that the rendered internal model only sees fully normalized text nodes and an optimal tree. (It'd also make for easier hydration.)

(As for the status of this experiment, it's still local and closed-source until I actually get something functional and tested.)

[jmooradi]

Has any progress been made on this recently or anyway I can assist the active development?

[barneycarroll]

Been processing the technical proposal in the background - sorry for delayed response. How do we reconcile component identities?

[dead-claudia]

@barneycarroll I suggest we just don't worry about it. Let's assume the tree is correct, and if the DOM differs, we throw an error instead of trying to patch it. That way, the developer finds out much quicker that something went wrong in the SSR translation.

[gilbert]

This is a minor point, but I think an incorrect tree should throw a warning instead of an error, and redraw as normal to overwrite the DOM. That way the page still functions, even if there's a flash of white on load.

[barneycarroll]

@isiahmeadows gotcha. The initial front end draw will be the implicit reconciliation. Divergence (render state X on server, state Y on JS initialisation) is not going to happen.

[dead-claudia]

@gilbert If you've got it set up for SSR, you have the framework for ensuring the data is consistent. Divergence means either a bug in your pipeline or your client just got MitM'd. The first case is generally pretty easy to fix: it's almost always just a content delivery problem, not a problem with your client-side app, and fixing it is sometimes as easy as running a single command. The second case is, of course, nothing you can do about, but is obviously something your user should be aware of.

Honestly, most issues this would provide are things that would force users to do things the right way in the first place.

---

I would be open to offering an escape hatch for tolerating outdated information in attributes and/or text differing only in content or existence, patching those up as necessary by just clearing (if text) and updating the value, but I would still prefer the element structure to throw an error on mismatch, and I'd rather keep the escape hatch down to only what's expected to change. The API for tolerating changes to attributes or text could be as simple as this magic property: nohydrate: ["array", "of", "attributes"] or nohydrate: "single-attribute".

• On elements, any nohydrate attribute/property would be implicitly ignored.
• On elements and text vnodes, the special "textContent" property would result in the text content being ignored. This requires you to use m("#", {nohydrate: "textContent"}, value) if it's truly dynamic.
• While rendering, nohydrate attributes on native elements are simply ignored.

[leeoniya]

Divergence means either a bug in your pipeline or your client just got MitM'd.

the main case where i've run into divergence has been with third-party scripts creating dom elements. sometimes browser plugins do this also (e.g. password managers). this situation becomes more of a nuisance if the entire document is SSRd from the root <html>, but less so if your app root is an element inside <body>.

EDIT: err, i guess this isn't SSR divergence but DOM divergence post-hydration (assuming the hydration is done prior to third party script exec), so affects future redraws.

[dead-claudia]

@leeoniya Most of the time, that divergence is either in text nodes or attributes. As for utilities screwing with the DOM after initial redraw, we've had a recent issue come up with Font Awesome's SVG insanity - they changed the DOM underneath us and installed a mutation observer to continue mucking with it, but offered no hook to allow us to re-sync our own model. So in that case, it's just not playing well with virtual DOM frameworks.

Font Awesome does have an API that would let us hook into their IR and render it how we want it, though, and I'm slapping together a gist to port their react-fontawesome (which uses SVGs internally) to Mithril.

[gilbert]

I understand it's supposed to be the end dev that sets everything up, but what's the benefit of throwing an error and breaking the app instead of a warning hiccup behavior?

[dead-claudia]

@gilbert Two reasons:

1. It's much simpler to implement - the hydration function doesn't need to concern itself with diffing or DOM building, and it can work completely independently from m.render.
2. It encourages best practice of tying the two versions together and automating the process, so it's less likely the problem gets ignored. It's opinionated, I know, but Mithril doesn't really stray away from injecting opinions on obvious ticking time bombs.

[dead-claudia]

There's a third reason, too: what if it's just missing one vnode in an inner component, like this?

var hasData = false
var Data = {view: function () {
	return [
		m("div.data", ...),
	]
}}

var Header = {view: function () {
	return [
		m("nav.stuff", ...),
		m("div.divider"),
		hasData ? m(Data) : null,
		m("div.page", currentPage),
	]
}}

var Home = {view: function () {
	return [
		m(Header),
		m(Body, ...),
		m(Footer),
	]
}}

// In your initialization
hasData = true
m.hydrate(root, m(Home))

<!-- Pretend the extra whitespace isn't here. -->
<!-- Original tree -->
<div id="root">
	<nav class="stuff">...</nav>
	<div class="divider"></div>
	<div class="page">Current Page</div>
	...
</div>

<!-- Rendered tree -->
<div id="root">
	<nav class="stuff">...</nav>
	<div class="divider"></div>
	<div class="data">...</div>
	<div class="page">Current Page</div>
	...
</div>

Specifically, how should it consume <div class="page">? The obvious error correction would be that it should infer a hole and insert a <div class="data"> before it, but this makes it much harder to iterate thanks to the need for lookahead, and Mithril lacks the static analysis needed to know this generically. The naïve, purely iterative error-correcting method would say it should change the class to data, replace the text content with the appropriate children, then insert <div class="page">, resulting in a lot of unnecessary DOM mutation. In fact, this iterative method is basically a full diff + patch pass.

There's also the question of how to handle keyed fragments whose entries are in the wrong order. The obvious solution is to ignore the keys and just update them appropriately, but then you're just doing a giant unkeyed diff/patch that's doing a lot of unnecessary DOM mutation. If you want to avoid mutation, then you're turning an already complex problem into an even more complex statistical problem by trying to infer the keys for each of the DOM nodes before then diffing them. (And by that point, you might as well just do the unkeyed diff/patch.)

---

Or, TL;DR: Requiring consistency is easy. Fixing mistakes is not. What's obvious in error correction isn't easy, and what's easy in error correction is often wrong. Forcing the developer to follow best practices is a lot easier than trying to figure out how to correct their mistakes, and Mithril isn't in the business of magic here.

[gilbert]

Sorry I wasn't clear. What I'm suggesting is: upon an inconsistency, abandon the hydration process altogether and rerender from the top level node, completely from scratch. I'm not suggesting anything fancy or magical, which is why it was a minor point in my mind.

[dead-claudia]

@gilbert You could always catch the error and manually remount. 😉

[ACXgit]

Detecting the inconsistency would be very useful to spot a misbehavior in the isomorphic process (from my perspective, any difference between the two rendered DOMs would mean that something has gone wrong). If throwing an error is the cleanest approach, go for it 👍

[ChrisGitIt]

@isiahmeadows: Can i catch a hydration error on components level? With the "onmatch" Lifecycle? Otherwise, i'm not sure if a complete redraw would be a good idea. Maybe just a "parent" redraw would be a good idea...

[dead-claudia]

@ChrisGitIt I'd instead call onremove on every successfully mounted component, clear the incomplete tree, and then throw the error. You could detect hydration error in this case by checking in onremove that no other lifecycle method (except oninit) was called.