Added .options method for simplified option setting #2306

philippjfr · 2018-02-05T16:24:01Z

Adds a .options method to all objects making it simpler to set options in a flattened way. Two formats are supported:

img.options(width=600, cmap='viridis')

and the more explicit:

img.options({'Image': dict(width=600, cmap='viridis')})

which allows usage on complex nested objects.

Addresses Simpler option setting #222, Explicitly specifying backend for options #803, Add clone option to opts method #2287
Add unit tests
Add docstrings
Add section in getting started and user guides
Move logic into separate utility
Add DynamicMap.options

philippjfr · 2018-02-10T15:37:41Z

I'm very happy with this PR now, the .options method is really convenient and because it has to actually resolve the difference between different kinds of options its also much stricter than other approaches, which should result in a better user experience.

The main question I have now where to document it (I want a section in both the getting started and user guide) and what our guidance on using it is. Should we use it throughout the docs? @jbednar @jlstevens

philippjfr · 2018-02-10T17:15:53Z

Ready to review.

jbednar · 2018-02-10T22:50:04Z

I think we should use this approach consistently throughout the docs, certainly in all places currently using .opts, but also replacing magics in many cases.

jlstevens · 2018-02-13T20:39:21Z

holoviews/util/__init__.py

@@ -105,6 +105,75 @@ def __call__(self, *args, **params):
            return StoreOptions.set_options(obj, options)


+    @classmethod
+    def expand_options(cls, options, backend=None):


If there are any well-defined units of code in this rather long method, I would like them pulled out as private helper classmethods. The warning generation bit can be split out this way I think and there might be other sensible units to pull out.

jlstevens · 2018-02-13T20:42:31Z

holoviews/core/options.py

@@ -1326,13 +1326,13 @@ def tree_to_dict(cls, tree):
        return specs

    @classmethod
-    def propagate_ids(cls, obj, match_id, new_id, applied_keys):
+    def propagate_ids(cls, obj, match_id, new_id, applied_keys, backend=None):


Glad to see this generalized to support the backend argument.

jlstevens · 2018-02-13T20:54:38Z

holoviews/core/dimension.py

+                self.map(lambda x: setattr(x, 'id', None))
+        elif clone:
+            obj = self.map(lambda x: x.clone(id=x.id))
+        StoreOptions.set_options(obj, options, backend=backend, **kwargs)


I think obj=self is really confusing here as it is not accessed until the line:

StoreOptions.set_options(obj, options, backend=backend, **kwargs)

Which can now be expressed in a way I find much more intuitive, removing the need for obj=self entirely:

StoreOptions.set_options(obj if clone else self, options, backend=backend, **kwargs)

I'll also note that although support for the clone argument is reasonable, I would have preferred to see it in a separate PR.

I think obj=self is really confusing here as it is not accessed until the line:

Not quite sure what you mean, it's overridden in the conditional and used twice. I'd have to do this instead:

StoreOptions.set_options(obj if clone else self, options, backend=backend, **kwargs) return obj if clone else self

jlstevens · 2018-02-13T20:59:02Z

I haven't looked at the changes to the docs (i.e notebooks) just yet but I have looked at the code and made a few comments. It mostly looks fine and I am happy the bulk of the new functionality is implemented as a classmethod on hv.opts.

jlstevens · 2018-02-13T21:01:27Z

holoviews/util/__init__.py

+                    if backend is None:
+                        # Check option is invalid for all backends
+                        found = []
+                        for lb in [b for b in loaded_backends if b != 'backend']:


We spotted this bug together - it should be backend (a variable) not 'backend' a string. Is there a unit test that would have caught this bug? The tests did go green....

Just went through this, it's not tested because this is just an optimization to ensure it doesn't check the same backend twice.

philippjfr · 2018-02-18T17:21:57Z

Ready to review again.

jlstevens · 2018-02-19T12:03:24Z

Happy to review once the merge conflicts are resolved. I doubt I'll have many new comments to add once that is done.

jlstevens · 2018-02-19T12:11:18Z

holoviews/util/__init__.py

+        """
+        Generates an error message for an invalid option suggesting
+        similar options through fuzzy matching.
+        """


I should have raised this earlier. Isn't all this logic what OptionError is for? The logic is certainly closely related so maybe OptionError should be enhanced or extended somehow...

I did look into that but I can be much more eager about the validation here because the user can declare an explicit backend and because it has to resolve each option into the appropriate group. This also means I can supply a more specific error message. I think we should look at unifying the validation, especially once the magic allows declaring an explicit backend.

What I would like to know is why the OptionError handling is so horrendously complex. Why does it have to record errors? It seems like that code would be hugely simplified by a utility that checks whether an option is invalid across all backends, so you can eagerly raise as soon as you encounter an unsupported option rather than processing all the options and checking validity at the end and having 3 methods to record options, a context manager to control the option validation, a bunch of state on the OptsMagic etc.

Separately I'm still confused in what cases the existing code actually validates anything, it seems like .opts still has zero validation, since this raises no error:

hv.Curve([]).opts(plot=dict(some_nonexistent_option=3))

What I would like to know is why the OptionError handling is so horrendously complex. Why does it have to record errors? It seems like that code would be hugely simplified by a utility that checks whether an option is invalid across all backends, so you can eagerly raise as soon as you encounter an unsupported option rather than processing all the options and checking validity at the end.

The main reason it is this way is because the errors should be derived from the allowed_keywords specified in the Options. It is true that these allowed keywords are set from the plot and style option declarations which is what your proposed utility would rely on directly. I'm certainly not opposed to simplifying the code with a more direct check but the current design allows more flexibility by having a level of indirection: plot and style option declarations -> allowed keywords on options and errors-> validation.

As I agree we rarely use the extra flexibility that this design affords us, I would be happy to see a PR that simplifies things to plot and style option declarations -> validation. My worry is that such a PR might have to touch a lot more than you might expect (e.g how the errors are processed and printed by the %opts magic(s)).

Right, I definitely want to tackle and simplify that code because I'm having a really tough time following it and I think I'll much prefer an approach that eagerly errors when it hits an unsupported option. But as you say it touches a lot of hairy code so I think I'd like to do that separately.

Currently I'm leaning toward merging this PR and getting a dev release out and then to follow up with a PR to try to unify the option validation. What do you think?

I'll much prefer an approach that eagerly errors when it hits an unsupported option.

That is exactly how it worked when we only ever had to worry about a single backend at a time. The complexity was introduced when we decided to merge the allowed keywords across backends which was implemented in such a manner as to extend the old system (e.g where the magics used the OptionError message to generate a pretty printed HTML suggestion).

Currently I'm leaning toward merging this PR and getting a dev release out and then to follow up with a PR to try to unify the option validation. What do you think?

Makes sense. I don't think of it as 'unifying' the option validation though (it is already unified, which is what makes it ugly). That PR would be more of a refactor/redesign by validating without looking at the allowed keywords for Options.

I suppose one thing the current design lets you do is to declare options for elements/plots that don't exist, using allowed_keywords to do validation. The current design keeps the option system decoupled from elements/plots with allow_keywords acting as the link between them (set somewhere on Store iirc). This (in theory) could let you use the option system in a completely different project/library.

In practice, this decoupling doesn't seem to do much useful (unless you can imagine using the options system for something without needing a corresponding element/plot). In other words, I think there is a sound design underlying the way it is right now but 1) it doesn't add much of value in practice 2) it is becomes complicated once you start taking the union between backends. For this reason, I would be happy to see things simplified, even if it means coupling the options system closer to the elements and plot classes.

Don't think we necessarily need to couple the two closer together (although I haven't considered all the implications). I just feel like it comes down to the way you iterate over the backends. Currently it seems like looping over the backends is the outer loop, which means you always iterate over all backends and can't raise until you've processed them all. The approach I've taken is to validate the current (or explicitly selected) backend first, and if an option is invalid for that backend it falls back to checking whether the option is valid for other backends. This means that as soon as an invalid option is encountered it can raise an error (or warning) rather than having to process everything first and only asserting whether an option was invalid at the end.

My only question then is about the OptsMagic, is there any reason why that can't raise eagerly or does it somehow have to catch errors and reraise them in a special way?

does it somehow have to catch errors and reraise them in a special way?

That is exactly what it does.

Okay, I can still do that easily enough.

Though I will note it does it using the validation_error_message method which might mean there is indeed one single place where your approach could fit in nicely. Of course, there may still be other consequences I haven't thought about...e.g how will normal error validation work if you work with option trees directly?

When you work with an OptionTree directly the allowed_keywords are well defined and no cross-backend validation is needed so that should be fine. I'll open a PR branching off this one to see what it could look like.

jlstevens · 2018-02-19T14:10:17Z

Other than our ongoing discussion (which will become another PR), I am happy to merge once all the tests are green (though I think another PR that is about to be merged will touch the test data).

philippjfr · 2018-02-19T14:15:29Z

Making good progress on getting the validation unified so +1 on merging asap. This PR shouldn't touch test data so I'll try to get it passing without waiting on the other PR.

philippjfr · 2018-02-19T14:39:26Z

Also finding a number of bugs in the existing option validation, so refactoring this is definitely a good idea. How do you feel about dropping the skip_invalid option? I feel like it should either warn or error, although I'm happy to be persuaded otherwise until we have a way of specifying an explicit backend in the opts magic.

jlstevens · 2018-02-19T14:41:16Z

How do you feel about dropping the skip_invalid option?

Fine by me. If I remember correctly, that was the first approach we had when we started supporting multiple backends. I do think it is less relevant now.

philippjfr · 2018-02-19T23:22:57Z

Ready to merge now, refactoring will happen in #2354.

jlstevens · 2018-02-20T01:19:33Z

As the tests are passing, I'll go ahead and merge.

github-actions · 2024-10-25T05:27:43Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

philippjfr added type: feature A major new feature tag: component: options status: WIP labels Feb 5, 2018

philippjfr force-pushed the simplified_option_setting branch from cf5eae6 to 77c94c2 Compare February 5, 2018 16:39

philippjfr removed the status: WIP label Feb 10, 2018

philippjfr requested a review from jlstevens February 13, 2018 18:43

jlstevens reviewed Feb 13, 2018

View reviewed changes

philippjfr force-pushed the simplified_option_setting branch 3 times, most recently from 046107b to 10d7b96 Compare February 14, 2018 23:35

philippjfr force-pushed the simplified_option_setting branch from 10d7b96 to ea7df5b Compare February 19, 2018 12:09

jlstevens reviewed Feb 19, 2018

View reviewed changes

philippjfr mentioned this pull request Feb 19, 2018

Option error refactor #2354

Closed

2 tasks

philippjfr added 4 commits February 19, 2018 15:22

Added .options method for simplified option setting

1784e03

Implemented DynamicMap.options method

e41afeb

Made expand options a classmethod on hv.util.opts

54fddb5

Improved .options validation and docstrings

f297bce

philippjfr added 10 commits February 19, 2018 15:22

Reorganized test code

c11bc86

Correctly handle setting options with explicit backend

ffb9233

Added support for using opts and options without cloning

cbcb230

Improve error handling for options method

1e75f22

Added unit tests for Dimensioned.options method

6586096

Ensured pyplot uses agg backend

9d77601

Added sections about options method to guides

a767b7f

Added clone argument to DynamicMap.options

fb4629c

Refactored expand_options

f297383

Fixed flake issues

a99d4a1

philippjfr force-pushed the simplified_option_setting branch from ea7df5b to a99d4a1 Compare February 19, 2018 15:22

jlstevens merged commit 76c7cc8 into master Feb 20, 2018

philippjfr mentioned this pull request Feb 20, 2018

Explicitly specifying backend for options #803

Closed

philippjfr added this to the v1.10 milestone Feb 20, 2018

philippjfr deleted the simplified_option_setting branch February 20, 2018 03:35

philippjfr added a commit that referenced this pull request Mar 1, 2018

Added .options method for simplified option setting (#2306)

36c8bc8

github-actions bot locked as resolved and limited conversation to collaborators Oct 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added .options method for simplified option setting #2306

Added .options method for simplified option setting #2306

philippjfr commented Feb 5, 2018 •

edited

Loading

philippjfr commented Feb 10, 2018

philippjfr commented Feb 10, 2018

jbednar commented Feb 10, 2018

jlstevens Feb 13, 2018

jlstevens Feb 13, 2018

jlstevens Feb 13, 2018 •

edited

Loading

jlstevens Feb 13, 2018

philippjfr Feb 13, 2018 •

edited

Loading

jlstevens commented Feb 13, 2018 •

edited

Loading

jlstevens Feb 13, 2018 •

edited

Loading

philippjfr Feb 13, 2018

philippjfr commented Feb 18, 2018

jlstevens commented Feb 19, 2018

jlstevens Feb 19, 2018

philippjfr Feb 19, 2018 •

edited

Loading

jlstevens Feb 19, 2018 •

edited

Loading

philippjfr Feb 19, 2018 •

edited

Loading

jlstevens Feb 19, 2018 •

edited

Loading

philippjfr Feb 19, 2018 •

edited

Loading

jlstevens Feb 19, 2018

philippjfr Feb 19, 2018

jlstevens Feb 19, 2018

philippjfr Feb 19, 2018

jlstevens commented Feb 19, 2018

philippjfr commented Feb 19, 2018

philippjfr commented Feb 19, 2018

jlstevens commented Feb 19, 2018

philippjfr commented Feb 19, 2018

jlstevens commented Feb 20, 2018

github-actions bot commented Oct 25, 2024

Added .options method for simplified option setting #2306

Added .options method for simplified option setting #2306

Conversation

philippjfr commented Feb 5, 2018 • edited Loading

philippjfr commented Feb 10, 2018

philippjfr commented Feb 10, 2018

jbednar commented Feb 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jlstevens Feb 13, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

philippjfr Feb 13, 2018 • edited Loading

Choose a reason for hiding this comment

jlstevens commented Feb 13, 2018 • edited Loading

jlstevens Feb 13, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

philippjfr commented Feb 18, 2018

jlstevens commented Feb 19, 2018

Choose a reason for hiding this comment

philippjfr Feb 19, 2018 • edited Loading

Choose a reason for hiding this comment

jlstevens Feb 19, 2018 • edited Loading

Choose a reason for hiding this comment

philippjfr Feb 19, 2018 • edited Loading

Choose a reason for hiding this comment

jlstevens Feb 19, 2018 • edited Loading

Choose a reason for hiding this comment

philippjfr Feb 19, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jlstevens commented Feb 19, 2018

philippjfr commented Feb 19, 2018

philippjfr commented Feb 19, 2018

jlstevens commented Feb 19, 2018

philippjfr commented Feb 19, 2018

jlstevens commented Feb 20, 2018

github-actions bot commented Oct 25, 2024

philippjfr commented Feb 5, 2018 •

edited

Loading

jlstevens Feb 13, 2018 •

edited

Loading

philippjfr Feb 13, 2018 •

edited

Loading

jlstevens commented Feb 13, 2018 •

edited

Loading

jlstevens Feb 13, 2018 •

edited

Loading

philippjfr Feb 19, 2018 •

edited

Loading

jlstevens Feb 19, 2018 •

edited

Loading

philippjfr Feb 19, 2018 •

edited

Loading

jlstevens Feb 19, 2018 •

edited

Loading

philippjfr Feb 19, 2018 •

edited

Loading