Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3fs dependency is more than 3 years old. #337

Closed
MatthiasRoels opened this issue Sep 14, 2023 · 2 comments
Closed

s3fs dependency is more than 3 years old. #337

MatthiasRoels opened this issue Sep 14, 2023 · 2 comments
Labels
Community Issue/PR opened by the open-source community

Comments

@MatthiasRoels
Copy link
Contributor

Description

Not really a bug, but when I was running a kedro pipeline for a demo project, I noticed that I got a warning from fsspec:

UserWarning: Your installed version of    warnings.py:109
                             s3fs is very old and known to cause                                                                                      
                             severe performance issues, see also https://github.com/dask/dask/issues/10276                                            
                                                                                                                                                      
                             To fix, you should specify a lower version bound on s3fs, or                                                             
                             update the current installation.                                   

Steps to Reproduce

Run kedro-iris starter project with kedro-datasets installed

Your Environment

Include as many relevant details about the environment in which you experienced the bug:

  • Kedro version used (pip show kedro or kedro -V): v0.18.13
  • Kedro plugin and kedro plugin version used (pip show kedro-airflow): kedro-datasets v1.7.0
  • Python version used (python -V): 3.10
  • Operating system and version: Debian
  • fsspec: 2023.9.0
  • s3fs: 0.4.2 (March 2020), current version: 2023.9.0
@astrojuanlu
Copy link
Member

Thanks for raising this @MatthiasRoels. We slept a bit on it but there's a draft PR #167

Do you feel like taking over and opening one yourself? We'll be happy to review and merge.

@astrojuanlu astrojuanlu added the Community Issue/PR opened by the open-source community label Sep 14, 2023
@astrojuanlu astrojuanlu linked a pull request Sep 14, 2023 that will close this issue
4 tasks
@merelcht
Copy link
Member

This was completed in #463

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Community Issue/PR opened by the open-source community
Projects
None yet
3 participants