bucket: NotImplementedError: Could not find signature for execute_node: <Bucket, Series, tuple, bool, bool, bool>

[MarcSkovMadsen]

What happened?

I'm trying to see if I can get the support for Ibis working in hvPlot and HoloViews. Right now for histograms.

Running some code I see NotImplementedError: Could not find signature for execute_node: <Bucket, Series, tuple, bool, bool, bool>.

I've reduced it to this small example

import ibis
import pandas as pd

df = pd.DataFrame({
    "y": [1,2,3,4,5]
})
con = ibis.pandas.connect({"df": df})
table = con.table("df")
bins=[1.0,3.0,5.0]
expr=table
expr=expr.y
expr.bucket(bins).execute()

What version of ibis are you using?

3.2.0

What backend(s) are you using, if any?

Pandas.

My problem started with duckDB though. But the example above was just simpler to provide.

Relevant log output

$ python script3.py
Traceback (most recent call last):
  File "C:\repos\private\hvplot\.venv\lib\site-packages\multipledispatch\dispatcher.py", line 269, in __call__
    func = self._cache[types]
KeyError: (<class 'ibis.expr.operations.histograms.Bucket'>, <class 'pandas.core.series.Series'>, <class 'tuple'>, <class 'bool'>, <class 'bool'>, <class 'bool'>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\repos\private\hvplot\script3.py", line 12, in <module>
    expr.bucket(bins).execute()
  File "C:\repos\private\hvplot\.venv\lib\site-packages\ibis\expr\types\core.py", line 291, in execute
    return self._find_backend(use_default=True).execute(
  File "C:\repos\private\hvplot\.venv\lib\site-packages\ibis\backends\pandas\__init__.py", line 216, in execute
    return execute_and_reset(query, params=params, **kwargs)
  File "C:\repos\private\hvplot\.venv\lib\site-packages\ibis\backends\pandas\core.py", line 491, in execute_and_reset
    result = execute(
  File "C:\repos\private\hvplot\.venv\lib\site-packages\multipledispatch\dispatcher.py", line 278, in __call__
    return func(*args, **kwargs)
  File "C:\repos\private\hvplot\.venv\lib\site-packages\ibis\backends\pandas\trace.py", line 137, in traced_func
    return func(*args, **kwargs)
  File "C:\repos\private\hvplot\.venv\lib\site-packages\ibis\backends\pandas\core.py", line 436, in main_execute
    return execute_with_scope(
  File "C:\repos\private\hvplot\.venv\lib\site-packages\ibis\backends\pandas\core.py", line 222, in execute_with_scope
    result = execute_until_in_scope(
  File "C:\repos\private\hvplot\.venv\lib\site-packages\ibis\backends\pandas\trace.py", line 137, in traced_func
    return func(*args, **kwargs)
  File "C:\repos\private\hvplot\.venv\lib\site-packages\ibis\backends\pandas\core.py", line 363, in execute_until_in_scope
    result = execute_node(
  File "C:\repos\private\hvplot\.venv\lib\site-packages\multipledispatch\dispatcher.py", line 273, in __call__
    raise NotImplementedError(
NotImplementedError: Could not find signature for execute_node: <Bucket, Series, tuple, bool, bool, bool>

Code of Conduct

• I agree to follow this project's Code of Conduct

[cpcloud]

@MarcSkovMadsen Thanks for reporting the issue.

This operation currently isn't implemented for the pandas backend.

Is it a requirement for you to have this work for the pandas backend or was it just a thing you came across while working through the holoviews support?

Ideally, we wouldn't add operations for completeness's sake but if you're using this or planning to, we can add it.

[MarcSkovMadsen]

Hi @cpcloud

I am new to the ibis universe. Ideally I am looking for some way to calculate a histogram that works for any ibis backend. Our users would expect to be able to use any ibis backend to create an ibis histogram if we announce ibis to be supported.

As reference I am also having the problem here #4940.

[MarcSkovMadsen]

If I try to use the code proposed in #4940 it also fails with the error message NotImplementedError: Could not find signature for execute_node: <Bucket, Series, tuple, bool, bool, bool>.

import ibis
import pandas as pd

df = pd.DataFrame({
    "y": [1,2,3,4,5]
})
con = ibis.pandas.connect({"df": df})
table = con.table("df")
expr=table
expr=expr.y

def to_histogram(expr):
    bins=[1.0,3.0,5.0]
    df = expr.to_projection()
    df.mutate(bucket=df.y.bucket(bins)).bucket.value_counts().sort_by('bucket').execute()

to_histogram(expr)

[gforsyth]

Hey @MarcSkovMadsen -- the workaround in #4940 is specific to the (probable) bug in DuckDB -- it won't help with the lack of implementation in the pandas backend.

I understand wanting to offer users the ability to choose any backend so that whatever data the user brings, they can make use of holoviews.

In the case of pandas dataframes, you aren't limited to the pandas backend because there is an option to create a memtable, which loads an in-memory dataframe into duckdb.

[ins] In [1]: import pandas as pd

[ins] In [2]: df = pd.DataFrame(
         ...:     {
         ...:         "value": [1, 2, 3, 4, 5],
         ...:         "group": ["a", "b", "b", "b", "a"],
         ...:     }
         ...: )

[ins] In [3]: import ibis

[ins] In [4]: con = ibis.memtable(df)

[ins] In [5]: con
Out[5]: 
PandasInMemoryTable
  data:
    DataFrameProxy:
         value group
      0      1     a
      1      2     b
      2      3     b
      3      4     b
      4      5     a

[MarcSkovMadsen]

Thanks so much for proposing solutions. The challenge is that the user is not providing a Pandas dataframe to hvplot/ HoloViews. he is just providing data where data is an ibis Table. Right now we are testing for these three backends

Now I will add

to the list of fixtures. There are so many other combinations that I should add because the user might be using anything backend that ibis supports.

[gforsyth]

Ok, so given that, I think that you can probably "just" offer duckdb and sqlite to start.
A user might have data in an existing sqlite db, or an existing duckdb file, but there isn't a pandas representation on disk, and any stored data they might want to load (like a csv or a parquet dataset) can be loaded in duckdb instead.

Short of the situation where you have existing data in a specific on-disk format, I don't think there's any benefit to using a local backend that isn't duckdb -- it's far and away the most performant (of the three above) and there are fast paths from DuckDB to arrow.

[cpcloud]

Closing this out for now. We would happily accept a PR for the pandas backend to implement this!