Issues and feature requests for Loom integration

[fsaad]

• Enable diagnostics and checkpointing.
  • Kernel counts
  • History of column partition and state CRP.
  • History of logscore.
• Enable timed analysis (not sure whether this is possible by using the Loom API, probably need to interrupt the process, but that leave the metadata in an unusable state).

[fritzo]

Do you plan on integrating the loom.query.QeryServer interface?

[fsaad]

@fritzo Thanks for the question. In the current setup, cgpm invokes Loom only to obtain posterior CrossCat model structures (row and column partitions). Querying the model happens by extracting the model structures and creating a cgpm.crosscat.State object (each "state" roughly represents 1 loom "sample" or chain), and then using the infrastructure in cgpm for sampling and density assessment.

Since cgpm is largely unoptimized python, we definitely would plan on integrating loom.query.QueryServer to bypass the performance bottlenecks of cgpm.crosscat when querying larger datasets. I still have not considered whether it makes more sense to integrate the Loom query server into cgpm, which is mostly intended for flexible querying of shallow networks of both composite crosscat and non-crosscat models, or to bypass cgpm and integrate it by implementing the bayeslite.metamodel interface directly (which would result in more efficient querying, at the expense of reduced modeling and querying flexibility).

These decisions might be on our roadmap, and I will certainly reach out for your feedback if we have any questions or design issues with using/integrating Loom.

[fsaad]

Hi @fritzo, following up on this thread, an intern at the probcomp group wrote a metamodel in bayeslite which bypasses cgpm and uses Loom directly. The LoomMetamodel class is living here:

https://github.com/probcomp/bayeslite/blob/20170707-truell20-loom-integration/src/metamodels/loom_metamodel.py

We still haven't merged the branch since it needs some work and clean up on our side.