Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Defined hv.extension and hv.renderer utilities #1517

Merged
merged 10 commits into from
Jun 5, 2017
Merged

Defined hv.extension and hv.renderer utilities #1517

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
47135c4
Defined hv.renderer utility as baseclass of notebook_extension
jlstevens Jun 5, 2017
766a40e
Alias hv.renderer to notebook_extension when available
jlstevens Jun 5, 2017
d6fbde3
OutputSettings.initialize can now be called safely multiple times
jlstevens Jun 5, 2017
fa2151c
Reintroduced logic to add SVG exporter to the notebook archive
jlstevens Jun 5, 2017
0eab145
Setting hv.archive to NotebookArchive in notebook_extension
jlstevens Jun 5, 2017
cfba05f
Maded load_hvjs into a notebook_extension classmethod
jlstevens Jun 5, 2017
673da72
Renamed hv.renderer to hv.extension
jlstevens Jun 5, 2017
d8dc640
Added a hv.renderer utility to access renderers easily
jlstevens Jun 5, 2017
651c2f1
Made the renderer, output and opts utilities available at the top level
jlstevens Jun 5, 2017
b8986e6
Fixed output magic tab completion
jlstevens Jun 5, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion holoviews/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
from .element import * # noqa (API import)
from .element import __all__ as elements_list
from . import util # noqa (API import)
from .util import renderer # noqa (API import)

# Surpress warnings generated by NumPy in matplotlib
# Expected to be fixed in next matplotlib release
Expand All @@ -39,10 +40,11 @@
try:
import IPython # noqa (API import)
from .ipython import notebook_extension
renderer = notebook_extension
except ImportError as e:
class notebook_extension(param.ParameterizedFunction):
def __call__(self, *args, **opts):
raise Exception("IPython notebook not available")
raise Exception("IPython notebook not available use hv.renderer utility instead.")


# A single holoviews.rc file may be executed if found.
Expand Down
127 changes: 41 additions & 86 deletions holoviews/ipython/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,13 @@
from ..core.options import Store
from ..element.comparison import ComparisonTestCase
from ..interface.collector import Collector
from ..util.settings import list_formats, list_backends
from ..util import renderer
from ..plotting.renderer import Renderer
from .magics import load_magics
from .display_hooks import display # noqa (API import)
from .display_hooks import set_display_hooks
from .widgets import RunProgress

try:
if version_info[0] >= 4:
import nbformat # noqa (ensures availability)
else:
from IPython import nbformat # noqa (ensures availability)
from .archive import notebook_archive
holoviews.archive = notebook_archive
except ImportError:
pass

Collector.interval_hook = RunProgress
AttrTree._disabled_prefixes = ['_repr_','_ipython_canary_method_should_not_exist']
Expand Down Expand Up @@ -82,29 +73,10 @@ def line_magic(self, *args, **kwargs):
self.ip.run_line_magic(*args, **kwargs)


def load_hvjs(logo=False, JS=True, message='HoloViewsJS successfully loaded.'):
class notebook_extension(renderer):
"""
Displays javascript and CSS to initialize HoloViews widgets.
"""
import jinja2
# Evaluate load_notebook.html template with widgetjs code
if JS:
widgetjs, widgetcss = Renderer.html_assets(extras=False, backends=[])
else:
widgetjs, widgetcss = '', ''
templateLoader = jinja2.FileSystemLoader(os.path.dirname(os.path.abspath(__file__)))
jinjaEnv = jinja2.Environment(loader=templateLoader)
template = jinjaEnv.get_template('load_notebook.html')
display(HTML(template.render({'widgetjs': widgetjs,
'widgetcss': widgetcss,
'logo': logo,
'message':message})))


class notebook_extension(param.ParameterizedFunction):
"""
Parameterized function to initialize notebook resources
and register magics.
Notebook specific extension to hv.renderer that offers options for
controlling the notebook environment.
"""

css = param.String(default='', doc="Optional CSS rule set to apply to the notebook.")
Expand All @@ -129,64 +101,30 @@ class notebook_extension(param.ParameterizedFunction):

_loaded = False

# Mapping between backend name and module name
_backends = {'matplotlib': 'mpl',
'bokeh': 'bokeh',
'plotly': 'plotly'}

def __call__(self, *args, **params):
# Get requested backends
imports = [(arg, self._backends[arg]) for arg in args
if arg in self._backends]
for p, val in sorted(params.items()):
if p in self._backends:
imports.append((p, self._backends[p]))
if not imports:
args = ['matplotlib']
imports = [('matplotlib', 'mpl')]

args = list(args)
selected_backend = None
for backend, imp in imports:
try:
__import__('holoviews.plotting.%s' % imp)
if selected_backend is None:
selected_backend = backend
except Exception as e:
if backend in args:
args.pop(args.index(backend))
if backend in params:
params.pop(backend)
if isinstance(e, ImportError):
self.warning("HoloViews %s backend could not be imported, "
"ensure %s is installed." % (backend, backend))
else:
self.warning("Holoviews %s backend could not be imported, "
"it raised the following exception: %s('%s')" %
(backend, type(e).__name__, e))
finally:
if backend == 'matplotlib' and not notebook_extension._loaded:
if 'matplotlib' in Store.renderers:
svg_exporter = Store.renderers['matplotlib'].instance(holomap=None,
fig='svg')
holoviews.archive.exporters = [svg_exporter] +\
holoviews.archive.exporters

Store.output_settings.allowed['backend'] = list_backends()
Store.output_settings.allowed['fig'] = list_formats('fig', backend)
Store.output_settings.allowed['holomap'] = list_formats('holomap', backend)

if selected_backend is None:
raise ImportError('None of the backends could be imported')

super(notebook_extension, self).__call__(*args, **params)
# Abort if IPython not found
try:
ip = params.pop('ip', None) or get_ipython() # noqa (get_ipython)
except:
# Set current backend (usually has to wait until OutputSettings loaded)
Store.current_backend = selected_backend
return

# Notebook archive relies on display hooks being set to work.
try:
if version_info[0] >= 4:
import nbformat # noqa (ensures availability)
else:
from IPython import nbformat # noqa (ensures availability)
from .archive import notebook_archive
holoviews.archive = notebook_archive
except ImportError:
pass

# Not quite right, should be set when switching backends
if 'matplotlib' in Store.renderers and not notebook_extension._loaded:
svg_exporter = Store.renderers['matplotlib'].instance(holomap=None,fig='svg')
holoviews.archive.exporters = [svg_exporter] + holoviews.archive.exporters

p = param.ParamOverrides(self, params)
resources = self._get_resources(args, params)

Expand All @@ -200,10 +138,9 @@ def __call__(self, *args, **params):
if notebook_extension._loaded == False:
param_ext.load_ipython_extension(ip, verbose=False)
load_magics(ip)
Store.output_settings.initialize([backend for backend, _ in imports])
Store.output_settings.initialize(list(Store.renderers.keys()))
set_display_hooks(ip)
notebook_extension._loaded = True
Store.current_backend = selected_backend

css = ''
if p.width is not None:
Expand All @@ -221,7 +158,7 @@ def __call__(self, *args, **params):
Store.renderers[r].load_nb(inline=p.inline)

# Create a message for the logo (if shown)
load_hvjs(logo=p.logo, JS=('holoviews' in resources), message='')
self.load_hvjs(logo=p.logo, JS=('holoviews' in resources), message='')



Expand Down Expand Up @@ -254,6 +191,24 @@ def _get_resources(self, args, params):
resources = ['holoviews'] + resources
return resources

@classmethod
def load_hvjs(cls, logo=False, JS=True, message='HoloViewsJS successfully loaded.'):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@philippjfr Is this ok? Anything else using this (i.e users)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What am I meant to check here? Looks like you just moved it slightly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering if anything else might have been using it. Why wasn't it a classmethod to begin with?

"""
Displays javascript and CSS to initialize HoloViews widgets.
"""
import jinja2
# Evaluate load_notebook.html template with widgetjs code
if JS:
widgetjs, widgetcss = Renderer.html_assets(extras=False, backends=[])
else:
widgetjs, widgetcss = '', ''
templateLoader = jinja2.FileSystemLoader(os.path.dirname(os.path.abspath(__file__)))
jinjaEnv = jinja2.Environment(loader=templateLoader)
template = jinjaEnv.get_template('load_notebook.html')
display(HTML(template.render({'widgetjs': widgetjs,
'widgetcss': widgetcss,
'logo': logo,
'message':message})))

@param.parameterized.bothmethod
def tab_completion_docstring(self_or_cls):
Expand Down
54 changes: 52 additions & 2 deletions holoviews/util/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@
from ..core.spaces import Callable
from ..core import util
from ..streams import Stream
from .settings import OutputSettings

from .settings import OutputSettings, list_formats, list_backends

Store.output_settings = OutputSettings

Expand Down Expand Up @@ -39,6 +38,57 @@ def output(line=None, obj=None, **options):

output.__doc__ = Store.output_settings._generate_docstring()


class renderer(param.ParameterizedFunction):
"""
Helper utility used to load rendering backends.
"""

# Mapping between backend name and module name
_backends = {'matplotlib': 'mpl',
'bokeh': 'bokeh',
'plotly': 'plotly'}

def __call__(self, *args, **params):
# Get requested backends
imports = [(arg, self._backends[arg]) for arg in args
if arg in self._backends]
for p, val in sorted(params.items()):
if p in self._backends:
imports.append((p, self._backends[p]))
if not imports:
args = ['matplotlib']
imports = [('matplotlib', 'mpl')]

args = list(args)
selected_backend = None
for backend, imp in imports:
try:
__import__('holoviews.plotting.%s' % imp)
if selected_backend is None:
selected_backend = backend
except Exception as e:
if backend in args:
args.pop(args.index(backend))
if backend in params:
params.pop(backend)
if isinstance(e, ImportError):
self.warning("HoloViews %s backend could not be imported, "
"ensure %s is installed." % (backend, backend))
else:
self.warning("Holoviews %s backend could not be imported, "
"it raised the following exception: %s('%s')" %
(backend, type(e).__name__, e))
finally:
Store.output_settings.allowed['backend'] = list_backends()
Store.output_settings.allowed['fig'] = list_formats('fig', backend)
Store.output_settings.allowed['holomap'] = list_formats('holomap', backend)

if selected_backend is None:
raise ImportError('None of the backends could be imported')
Store.current_backend = selected_backend


class Dynamic(param.ParameterizedFunction):
"""
Dynamically applies a callable to the Elements in any HoloViews
Expand Down
2 changes: 1 addition & 1 deletion holoviews/util/settings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,7 @@ def update_options(cls, options, items):
@classmethod
def initialize(cls, backend_list):
cls.backend_list = backend_list
backend = cls.options.get('backend', Store.current_backend)
backend = Store.current_backend
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@philippjfr I don't quite know why I had to make this change. I'll try to summarize:

  • In the magic tests, %unload_ext is called which meant that initialize was called multiple times because that unsets _loaded on the notebook extension. If the options dictionary had a value, this meant state was bleeding across tests and breaking.
  • I can't tell what changed: I believe initialize would have been called multiple times in the tests before this PR too.

That said, why isn't this an appropriate fix? When you initialize, it should reset the state even if initialize was called before. Why use a value from cls.options when the purpose here is to reset it to the defaults (line below?).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't be sure tbh, this code was always fairly brittle and I had to reorganize it a few times so methods may just have been badly named. I'd strongly suggest you make a notebook and then switch back and forth between backends with output a few times, rerun the notebook_extension, set fig formats then switch backends and so on. I don't think tests are currently a sufficient indication that things are still working.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I can do that to check. That said, those switches shouldn't be going through initialize each time anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd strongly suggest you make a notebook and then switch back and forth between backends with output a few times, rerun the notebook_extension, set fig formats then switch backends and so on.

I've just done that (included rerunning notebook_extension) and it seems to be working correctly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay happy to merge once test pass.

if backend in Store.renderers:
cls.options = dict({k: cls.defaults[k] for k in cls.remembered})
cls.set_backend(backend)
Expand Down