Implement Image interface #994

philippjfr · 2016-11-30T00:05:08Z

This PR is a WIP in implementing an interface to allow Image Elements to subclass Dataset and use the other interfaces, replacing GridImage. There are a few things to be settled so I'm hoping to discuss these in this PR.

Our Image types need bounds. When you pass in an array the bounds are required, but when using the xarray and grid interfaces, the bounds should be computed from the data. Currently I deduce the bounds by asserting that all samples must be evenly spaced with a small tolerance (10e-9). Eventually we'll have QuadMesh for non-equal spacing, but that will require some more work. Does this sound reasonable?
Dataset does not have an interface to access multiple value arrays at once. This makes it difficult to access RGB and HSV data. We can either accept the overhead of stacking and unstacking these arrays and provide a convenient property on RGB/HSV elements to do this for us, or come up with a general API on the interfaces to get the stacked arrays. There was also the suggestion that other interfaces could support holding stacked value dimension arrays instead of holding them as unstacked arrays per value dimension but that could happen in a later PR.

jlstevens · 2016-12-05T13:29:04Z

Just to summarize a discussion I had with Philipp on gitter regarding the first question:

Our Image types need bounds. When you pass in an array the bounds are required, but when using the xarray and grid interfaces, the bounds should be computed from the data. Currently I deduce the bounds by asserting that all samples must be evenly spaced with a small tolerance (10e-9). Eventually we'll have QuadMesh for non-equal spacing, but that will require some more work. Does this sound reasonable?

We agreed that this is a reasonable approach as long as the tolerance is about avoiding machine precision issues only and not some fuzzy ad hoc tolerance to be tweaked by the user. For instance, instead of 10e-9 a suitable value should be determined for the system, e.g using the sys.float_info.

jlstevens · 2016-12-05T13:41:13Z

Again summarizing our discussion on gitter:

Dataset does not have an interface to access multiple value arrays at once. This makes it difficult to access RGB and HSV data. We can either accept the overhead of stacking and unstacking these arrays and provide a convenient property on RGB/HSV elements to do this for us, or come up with a general API on the interfaces to get the stacked arrays. There was also the suggestion that other interfaces could support holding stacked value dimension arrays instead of holding them as unstacked arrays per value dimension but that could happen in a later PR.

I agree that a future PR supporting multiple vdims in the data API would be very welcome but for now, we can extend dimension_values on RGB elements only. Specifically, we can adapt RGB to make RGB.dimension_values(['R', 'G', 'B']) work in this PR and in a later PR we can support stacked output from dimension_values (for lists of vdims only).

philippjfr · 2017-01-22T15:29:30Z

Now based off #1075 and now slowly reaching API compatibility. Still missing a sample implementation otherwise largely complete. Should also add a lot more unit tests for grid interfaces, which will also cover the new ImageInterface.

Currently I'm also still a bit unclear what to do about Raster itself. It could probably be unified with Image by simply adding integer indices in the constructor, but that would also make Raster inherit from SheetCoordinateSystem.

philippjfr · 2017-01-22T17:11:31Z

Raster is now unified with the image interface. Now I'll have to refactor QuadMesh so it still works independently. Hopefully we'll manage to write interfaces to handle 1D and 2D binned data, e.g. for Histogram and QuadMesh, as part of 1.8.

philippjfr · 2017-03-04T13:51:43Z

tests/testellipsis.py


    def test_raster_ellipsis_slice_value_missing(self):
        data = np.random.rand(10,10)
        try:
            hv.Raster(data)[...,'Non-existent']
        except Exception as e:
-            if str(e) != repr("'z' is the only selectable value dimension"):
+            print(e)


Don't forget to remove.

philippjfr · 2017-04-08T02:31:01Z

Ready for final review and merging once test pass.

philippjfr · 2017-04-08T04:15:28Z

Nevermind still need to make a few changes.

philippjfr · 2017-04-08T13:45:18Z

Ready to merge again, some issues reindexing gridded datasets cropped up but are fixed now.

jlstevens · 2017-04-08T14:37:45Z

holoviews/core/util.py

+    """
+    Computes a bounding range from a number of evenly spaced samples.
+    Will raise an error if samples are not evenly spaced within
+    tolerance.


I would just mention which tolerance - namely as reported by sys.float_info

No longer raises an error, so yeah will fix it.

jlstevens · 2017-04-08T14:39:00Z

holoviews/element/util.py

@@ -52,6 +43,54 @@ def compute_edges(edges):
    return np.concatenate([edges, [edges[-1]+width]])


+def compute_slice_bounds(slices, scs, shape):


Nice to have this code split out as a utility.

I'm not sure I would recommend adding tests for this utility right now as I do want to merge ASAP. At least it is much easier to test now!

jlstevens · 2017-04-08T14:43:26Z

Looks good. I made one comment about a docstring - other than that, I'll merge when you tell me it is ready (and the tests are green).

philippjfr · 2017-04-08T14:44:06Z

Already pushed the updated docstring, just waiting on tests now.

jlstevens · 2017-04-08T15:24:13Z

All tests are green! Merging!

philippjfr added tag: API tag: component: data status: WIP labels Nov 30, 2016

philippjfr added this to the v1.7.0 milestone Nov 30, 2016

philippjfr mentioned this pull request Dec 10, 2016

bounds values set wrongly after calling .select on Image object #702

Closed

philippjfr mentioned this pull request Jan 4, 2017

Tuples as a convenient way to specify slices #220

Closed

philippjfr force-pushed the image_interface branch from 3af6cdd to 30b9a12 Compare January 21, 2017 20:25

philippjfr force-pushed the image_interface branch from 128a304 to dd9c36a Compare January 23, 2017 01:06

This was referenced Feb 5, 2017

Raster clims masked array #1020

Closed

Toggle between value dimensions on HeatMap #1114

Closed

philippjfr mentioned this pull request Mar 1, 2017

%%opts doesn't seem to have any effect when using a Dataset (matplotlib backend) #1166

Closed

philippjfr force-pushed the image_interface branch from dd9c36a to 9407a51 Compare March 2, 2017 04:25

philippjfr commented Mar 4, 2017

View reviewed changes

philippjfr force-pushed the image_interface branch from 1cd4a2f to c215b2f Compare March 4, 2017 14:01

philippjfr modified the milestones: v2.0, v1.7.0 Mar 15, 2017

philippjfr self-assigned this Mar 15, 2017

philippjfr force-pushed the image_interface branch 7 times, most recently from 7736826 to fa4e944 Compare March 27, 2017 16:47

philippjfr added 7 commits April 8, 2017 02:22

Added RGB interface tests

7cfb650

Simplified ImageInterface.select

98cf70a

Cleanup on grid interfaces

1a9b840

Added gridded option to GridInterface.shape

1641960

Cleaned up Image Element

70543a2

Cleaned up plotting code

c824933

Removed strict enforcement of regular sampling on Image

a7a6b23

philippjfr force-pushed the image_interface branch from e39ea4f to a7a6b23 Compare April 8, 2017 01:22

philippjfr added 3 commits April 8, 2017 02:33

Readded closest argument to sample method

b4fa77c

Updated dimension comparison unit tests

3be08ed

Round image densities to machine precision

7a4becd

philippjfr added 2 commits April 8, 2017 13:36

Fixed tuple/list comparison error messages

070c290

Implemented gridded Dataset reindexing

8e31fff

philippjfr force-pushed the image_interface branch from 25dcba9 to 8e31fff Compare April 8, 2017 13:00

Cleanup, comments and docstrings for interfaces

29a1e17

philippjfr force-pushed the image_interface branch from f0f6b0b to 29a1e17 Compare April 8, 2017 13:17

philippjfr added 2 commits April 8, 2017 15:31

Simplified bound_range utility

fa57964

Improved validation in data conversions

105be67

jlstevens reviewed Apr 8, 2017

View reviewed changes

Cleaned up bound_range utility docstring

ba70f8a

jlstevens merged commit 6a1b38e into master Apr 8, 2017

philippjfr mentioned this pull request Apr 8, 2017

Rename Image to ArrayImage and GridImage to Image #1208

Closed

philippjfr deleted the image_interface branch April 11, 2017 12:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Image interface #994

Implement Image interface #994

philippjfr commented Nov 30, 2016 •

edited

Loading

jlstevens commented Dec 5, 2016

jlstevens commented Dec 5, 2016

philippjfr commented Jan 22, 2017 •

edited

Loading

philippjfr commented Jan 22, 2017

philippjfr Mar 4, 2017

philippjfr commented Apr 8, 2017

philippjfr commented Apr 8, 2017

philippjfr commented Apr 8, 2017

jlstevens Apr 8, 2017

philippjfr Apr 8, 2017

jlstevens Apr 8, 2017

jlstevens Apr 8, 2017

jlstevens commented Apr 8, 2017

philippjfr commented Apr 8, 2017

jlstevens commented Apr 8, 2017

		@@ -52,6 +43,54 @@ def compute_edges(edges):
		return np.concatenate([edges, [edges[-1]+width]])


		def compute_slice_bounds(slices, scs, shape):

Implement Image interface #994

Implement Image interface #994

Conversation

philippjfr commented Nov 30, 2016 • edited Loading

jlstevens commented Dec 5, 2016

jlstevens commented Dec 5, 2016

philippjfr commented Jan 22, 2017 • edited Loading

philippjfr commented Jan 22, 2017

philippjfr Mar 4, 2017

Choose a reason for hiding this comment

philippjfr commented Apr 8, 2017

philippjfr commented Apr 8, 2017

philippjfr commented Apr 8, 2017

jlstevens Apr 8, 2017

Choose a reason for hiding this comment

philippjfr Apr 8, 2017

Choose a reason for hiding this comment

jlstevens Apr 8, 2017

Choose a reason for hiding this comment

jlstevens Apr 8, 2017

Choose a reason for hiding this comment

jlstevens commented Apr 8, 2017

philippjfr commented Apr 8, 2017

jlstevens commented Apr 8, 2017

philippjfr commented Nov 30, 2016 •

edited

Loading

philippjfr commented Jan 22, 2017 •

edited

Loading