cannot import name 'AggsProxy' from elasticsearch_dsl.search

[selimt]

When running on Python 3.9 with the latest elasticsearch_dsl release (8.13.0), we are getting the following traceback:

from django_elasticsearch_dsl_drf.filter_backends import (  # noqa: F401
  File "/usr/local/lib/python3.9/site-packages/django_elasticsearch_dsl_drf/filter_backends/__init__.py", line 29, in <module>
    from .suggester import (
  File "/usr/local/lib/python3.9/site-packages/django_elasticsearch_dsl_drf/filter_backends/suggester/__init__.py", line 5, in <module>
    from .functional import FunctionalSuggesterFilterBackend
  File "/usr/local/lib/python3.9/site-packages/django_elasticsearch_dsl_drf/filter_backends/suggester/functional.py", line 70, in <module>
    from elasticsearch_dsl.search import AggsProxy
ImportError: cannot import name 'AggsProxy' from 'elasticsearch_dsl.search' (/usr/local/lib/python3.9/site-packages/elasticsearch_dsl/search.py)

Pinning elasticsearch_dsl to 8.12.0 gets around the problem.

Thanks
-Selim

[1One0]

change from elasticsearch_dsl.search import AggsProxy tofrom elasticsearch_dsl.search_base import AggsProxy, it may work

[selimt]

Yes but this import is in the django-elasticsearch-dsl-drf code. Are you suggesting I test and submit a PR ?

[barseghyanartur]

Folks, I'm very like to use this package again for a project soon. I'll make sure to solve all the issues then.

[miguelgrinberg]

@barseghyanartur The issue here is that your EmptySearch class relies on some internal elements of elasticsearch-dsl-py. Our library went through a significant restructure to add support for asyncio recently, and while we were very careful to not break the public interfaces, that caused a lot of internal elements to move or change.

The easy solution would be for you to import this AggsProxy class from its new location, but this is not a great solution, because we limit our backwards compatibility support to publicly available interfaces. For example, we have made, and will continue to make, changes to the Search class and some of these may break your EmtpySearch class again.

If we were to implement our own version of EmptySearch as a public class, would you be willing to drop your version and use ours? I think this would eliminate any risk of new breakages in the future. Thanks!

[barseghyanartur]

@miguelgrinberg:

Thanks for the suggestion. Please, show me what you have.

Alternatively, if that's something that needs to be easily swappable, I can think of ways to make it swappable.

[miguelgrinberg]

@barseghyanartur when you have a moment have a look at the EmptySearch class that I just have added:

from elasticsearch_dsl import EmptySearch

This is not in a released version yet so you'll need to install the main branch of elasticsearch-dsl-py to get it. This should be a more robust version of yours that is less likely to break in the future.

[miguelgrinberg]

@barseghyanartur we have just released 8.13.1, including our version of EmptySearch. I believe replacing your version with ours will address this issue. Please let me know if you need any additional support.

[barseghyanartur]

@miguelgrinberg:

Thanks. Question. Regarding the compatibility, is elasticsearch-dsl-py 7.x branch still supported? Or is it no longer?

Simply switching to 8.x would mean compatibility issues. I could however, make this EmptySearch quite compatible with whatever you have in your branch. Or really make it swappable. Perhaps, that's the best option.

[miguelgrinberg]

@barseghyanartur The 7.x branch is still supported. What I suggest that you do is to try to import EmptySearch from elasticsearch-dsl and only if that fails use your own version. In your utils.py module you could do something like this:

try:
    from elasticsearch_dsl import EmptySearch
except ImportError:
    class EmptySearch:
        # your own version of this class

I think this would allow you to support all the 7.x and 8.x versions. Would this work for you?

[selimt]

@barseghyanartur just checking to see where we are with this ticket. We rather not pin the elasticsearch-dsl version.

Thanks.

[selimt]

@barseghyanartur , if @priyanshu69code 's change looks good to you , can we apply that change here as well ?

Thanks.

[metalshanked]

@barseghyanartur , if @priyanshu69code 's change looks good to you , can we apply that change here as well ?

Thanks.

Hi, @selimt, @barseghyanartur and @priyanshu69code

There is one more place where this import needs to be updated:
Can we please update this in both locations? It would help so many users Thanks!

Below is the link -->
https://github.com/priyanshu69code/django-elasticsearch-dsl-drf/blob/c9d466951070557d30eeb0412e4b636888c9f991/src/django_elasticsearch_dsl_drf/filter_backends/suggester/functional.py#L70

[priyanshu69code]

@metalshanked

Ok sure But I think I have updated the imports at both the place. But let me confirm that.

[selimt]

@miguelgrinberg I feel this project is abandonware since I haven't heard from @barseghyanartur in months (@barseghyanartur please correct me if I am wrong).

For this issue is there a way to update elasticsearch-dsl-py to be backward compatible so that we can use the latest version of elasticsearch-dsl-py without resorting to a fork?

[miguelgrinberg]

No, unfortunately we cannot block the progress of our client because this project's maintainer does not fix this issue. We try very hard to not break the public interfaces exposed by our client. This situation was caused by this library's decision to import private parts of the client that are not intended to be used externally.

Even if we were willing to somehow revert changes we've made, that would not help you for too long, because the package will eventually also break as other dependencies change, not just ours. At some point you will not be able to upgrade Django or Python because newer versions will not be compatible. And you can't ask Django or the PSF to stop updating their products to prevent old unmaintained code from breaking.

[selimt]

That makes total sense.

Thank you for your consideration. We'll look for another solution.
-Selim