FC-1431: fix/unresolve nodes #195
Conversation
| @@ -145,7 +145,8 @@ | |||
|
|
|||
| (defn child-entry | |||
| [{:keys [first] :as node}] | |||
| [first node]) | |||
| (let [child-node (unresolve node)] | |||
There was a problem hiding this comment.
I'm going to try to summarize my understanding of this to make sure I've got it correct:
child-entry takes a node and returns a tuple of the the first flake it addresses and the node itself. This change unresolves the node itself so it won't carry around the :children or :flakes references.
Is it to reduce the size of the node we need to pass around?
There was a problem hiding this comment.
Yes. This code is mainly used when the database is re-indexed. In that case, the entire index tree is loaded into memory, node by node, and rebalanced. Without this change, the result of the indexing process would hold the all the data from an index in memory, thus eliminating the gains from processing the index node by node instead of all at once. This change instead keeps a pointer to the top level child nodes on disk, and those child nodes keep pointers to their children, and so on. The whole benefit of this change is memory efficiency for the indexing process.
There was a problem hiding this comment.
Nice! Thanks for the explanation, that makes perfect sense.
This patch adds an index api function to remove the cached
:flakes/:childrenattributes from resolved nodes to save memory when we're finished processing these nodes.