Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query caching for offline work #550

Closed
dimitri-yatsenko opened this issue Dec 17, 2018 · 0 comments
Closed

Query caching for offline work #550

dimitri-yatsenko opened this issue Dec 17, 2018 · 0 comments
Assignees
Labels
features needs-discussion Issues requiring further development review and verify impact.
Milestone

Comments

@dimitri-yatsenko
Copy link
Member

dimitri-yatsenko commented Dec 17, 2018

To simplify off-line work, I propose a feature for caching fetch results to enable repeated execution of the code without accessing the database. It would look something like

with dj.caching(refresh=True):  # change to False after first execution
    for m in tuning.Maps & 'animal_id = 17797':
        plot_ori_map(m['ori_map'])

or

dj.set_caching(True, refresh=True):
for m in tuning.Maps & 'animal_id = 17797':
    plot_ori_map(m['ori_map'])

Caching cannot overlap with transactions and thus cannot be used inside make methods. Caching is ideal for exploratory and debugging purposes, not for pipeline work when data integrity is paramount. For example, working on figures or developing a new algorithm would be good use scenarios for that.

The caching filepath would be set in dj.config['query_cache'].

I need this feature for my own paper.

@dimitri-yatsenko dimitri-yatsenko self-assigned this Dec 17, 2018
@dimitri-yatsenko dimitri-yatsenko changed the title Fetch capture for offline work Query caching for offline work Dec 17, 2018
@dimitri-yatsenko dimitri-yatsenko added this to the Release 0.13 milestone Dec 17, 2018
@eywalker eywalker removed this from the Release 0.13 milestone Aug 1, 2019
@eywalker eywalker added the needs-discussion Issues requiring further development review and verify impact. label Apr 14, 2020
@ixcat ixcat added this to the Concept milestone Apr 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
features needs-discussion Issues requiring further development review and verify impact.
Projects
None yet
Development

No branches or pull requests

4 participants