feat(datasets): Add kwargs for huggingface.HFDataset #580

eromerobilbomatica · 2024-02-28T12:34:16Z

Description

Issue: The kedro_datasets.huggingface.HFDataset class have to accept parameters optionals in call of the function #574

Development notes

Checklist

Opened this PR as a 'Draft Pull Request' if it is work-in-progress
Updated the documentation to reflect the code changes
Added a description of this change in the relevant RELEASE.md file
Added tests to cover my changes

Signed-off-by: Eduardo <[email protected]>

datajoely · 2024-02-28T13:33:13Z

thanks for raising this @eromerobilbomatica !

Signed-off-by: eduardo <[email protected]>

noklam · 2024-02-29T12:26:16Z

@eromerobilbomatica I see a lot of newlines get added in files, is it possible to not include these changes in the PR? Ideally this PR should touch only kedor-datasets but not other plugins.

p.s. I also updated the title

eromerobilbomatica · 2024-02-29T14:32:01Z

That is @noklam, my PR only is connected kedor-datasets. The rest of the chances have been created when I run make plugin=kedro-datasets lint. Thanks for fixing the title.

astrojuanlu · 2024-03-07T08:56:04Z

@eromerobilbomatica There is a legitimate problem, see the failed checks:

kedro.io.core.DatasetError: Failed while loading data from data set HFDataset(dataset_name=openai_humaneval, dataset_tags=['task_categories:text2text-generation', 'annotations_creators:expert-generated', 'language_creators:expert-generated', 'multilinguality:monolingual', 'size_categories:n<1K', 'source_datasets:original', 'language:en', 'license:mit', 'code-generation', 'croissant', 'arxiv:2107.03374', 'region:us']).
'HFDataset' object has no attribute '_pipeline_kwargs'

…y accessing self._dataset_kwargs instead of self._pipeline_kwargs.

eromerobilbomatica · 2024-03-07T09:41:28Z

@astrojuanlu just fixed it, sorry! things of copy and paste.

…bilbomatica/kedro-plugins into huggingface_dataset_load

Signed-off-by: eduardo <[email protected]>

merelcht

Thanks for your contribution @eromerobilbomatica ! ⭐ Can you add the change to the release notes together with your name as contributor (you can see examples in old release notes).

I add myself as as contributor. Signed-off-by: Eduardo <[email protected]>

Add note about the fixed bug. Signed-off-by: Eduardo <[email protected]>

kedro-datasets/RELEASE.md

Fixed my comments and its position in the section. Signed-off-by: Eduardo <[email protected]>

Changed my name to community contributions in the "Upcoming Release" section Signed-off-by: Eduardo <[email protected]>

astrojuanlu · 2024-03-12T13:46:43Z

Woah, it's trying to install Dask 0.8.2, released 2016... https://pypi.org/project/dask/0.8.2/ something very weird going on

merelcht · 2024-03-12T13:56:06Z

Woah, it's trying to install Dask 0.8.2, released 2016... https://pypi.org/project/dask/0.8.2/ something very weird going on

Dask just released a new version: https://pypi.org/project/dask/2024.3.0/#history, pinning it seems to at least fix the installation step but there's other stuff wrong as well: #604

Signed-off-by: Merel Theisen <[email protected]>

astrojuanlu · 2024-03-15T16:32:25Z

astrojuanlu · 2024-03-15T16:33:02Z

@eromerobilbomatica now the yaml.org site disappeared and our links broke, we're waiting a few hours to see if the problem fixes itself

* issue kedro-org#574 * Chance to kedro-org#574 issue Signed-off-by: Eduardo <[email protected]> * lint tool implementation fixes Signed-off-by: eduardo <[email protected]> * lint tool implementation fixes Signed-off-by: eduardo <[email protected]> * lint tool implementation fixes for the second time Signed-off-by: eduardo <[email protected]> * Fixed attribute name typo in HFDataset's _load() method, now correctly accessing self._dataset_kwargs instead of self._pipeline_kwargs. * corrections after applying the `make plugin=kedro-datasets lint` Signed-off-by: eduardo <[email protected]> * Update RELEASE.md I add myself as as contributor. Signed-off-by: Eduardo <[email protected]> * Update RELEASE.md Add note about the fixed bug. Signed-off-by: Eduardo <[email protected]> * Update RELEASE.md Fixed my comments and its position in the section. Signed-off-by: Eduardo <[email protected]> * Update RELEASE.md Changed my name to community contributions in the "Upcoming Release" section Signed-off-by: Eduardo <[email protected]> --------- Signed-off-by: Eduardo <[email protected]> Signed-off-by: eduardo <[email protected]> Signed-off-by: Eduardo <[email protected]> Signed-off-by: Merel Theisen <[email protected]> Co-authored-by: eduardo <[email protected]> Co-authored-by: Nok Lam Chan <[email protected]> Co-authored-by: Juan Luis Cano Rodríguez <[email protected]> Co-authored-by: Merel Theisen <[email protected]> Signed-off-by: tgoelles <[email protected]>

issue kedro-org#574

f6d7061

eromerobilbomatica changed the title ~~Chance to #574 issue~~ fix: Chance to #574 issue Feb 28, 2024

Chance to kedro-org#574 issue

38ce402

Signed-off-by: Eduardo <[email protected]>

eromerobilbomatica force-pushed the huggingface_dataset_load branch from 993444a to 38ce402 Compare February 28, 2024 12:59

eduardorl84 and others added 4 commits February 29, 2024 11:27

lint tool implementation fixes

ab2007c

Signed-off-by: eduardo <[email protected]>

Merge branch 'main' into huggingface_dataset_load

6dcb8a3

lint tool implementation fixes

49bafcf

Signed-off-by: eduardo <[email protected]>

lint tool implementation fixes for the second time

cbf6ebb

Signed-off-by: eduardo <[email protected]>

noklam changed the title ~~fix: Chance to #574 issue~~ feat(datasets): Add kwargs for huggingface.HFDataset Feb 29, 2024

noklam self-requested a review February 29, 2024 12:26

noklam and others added 3 commits March 4, 2024 11:26

Merge branch 'main' into huggingface_dataset_load

f1c10b0

Merge branch 'main' into huggingface_dataset_load

e2d8aac

Merge branch 'main' into huggingface_dataset_load

c9b95aa

Fixed attribute name typo in HFDataset's _load() method, now correctl…

e229f02

…y accessing self._dataset_kwargs instead of self._pipeline_kwargs.

eromerobilbomatica and others added 2 commits March 7, 2024 10:48

Merge branch 'huggingface_dataset_load' of https://github.com/eromero…

93630f5

…bilbomatica/kedro-plugins into huggingface_dataset_load

corrections after applying the make plugin=kedro-datasets lint

d886abe

Signed-off-by: eduardo <[email protected]>

This comment was marked as outdated.

Sign in to view

astrojuanlu approved these changes Mar 7, 2024

View reviewed changes

astrojuanlu requested review from merelcht and lrcouto March 7, 2024 10:55

eromerobilbomatica requested a review from astrojuanlu March 7, 2024 11:59

merelcht approved these changes Mar 11, 2024

View reviewed changes

merelcht linked an issue Mar 11, 2024 that may be closed by this pull request

The kedro_datasets.huggingface.HFDataset class have to accept parameters optionals in call of the function #574

Closed

eromerobilbomatica added 2 commits March 12, 2024 11:47

Update RELEASE.md

a51b615

I add myself as as contributor. Signed-off-by: Eduardo <[email protected]>

Update RELEASE.md

b15000f

Add note about the fixed bug. Signed-off-by: Eduardo <[email protected]>

ankatiyar reviewed Mar 12, 2024

View reviewed changes

kedro-datasets/RELEASE.md Outdated Show resolved Hide resolved

eromerobilbomatica added 2 commits March 12, 2024 14:06

Update RELEASE.md

e535d10

Fixed my comments and its position in the section. Signed-off-by: Eduardo <[email protected]>

Update RELEASE.md

5cd924d

Changed my name to community contributions in the "Upcoming Release" section Signed-off-by: Eduardo <[email protected]>

merelcht mentioned this pull request Mar 12, 2024

build(datasets): Pin dask test requirement as latest introduced installation issues #604

Closed

4 tasks

noklam approved these changes Mar 12, 2024

View reviewed changes

astrojuanlu mentioned this pull request Mar 12, 2024

Decide on definitions of regular and experimental contributions #583

Closed

Merge branch 'main' into huggingface_dataset_load

461e189

Signed-off-by: Merel Theisen <[email protected]>

astrojuanlu enabled auto-merge (squash) March 16, 2024 21:00

astrojuanlu merged commit 2cbb71e into kedro-org:main Mar 16, 2024
47 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(datasets): Add kwargs for huggingface.HFDataset #580

feat(datasets): Add kwargs for huggingface.HFDataset #580

eromerobilbomatica commented Feb 28, 2024

datajoely commented Feb 28, 2024

noklam commented Feb 29, 2024

eromerobilbomatica commented Feb 29, 2024

astrojuanlu commented Mar 7, 2024

eromerobilbomatica commented Mar 7, 2024

This comment was marked as outdated.

merelcht left a comment

astrojuanlu commented Mar 12, 2024

merelcht commented Mar 12, 2024

astrojuanlu commented Mar 15, 2024

astrojuanlu commented Mar 15, 2024

feat(datasets): Add kwargs for huggingface.HFDataset #580

feat(datasets): Add kwargs for huggingface.HFDataset #580

Conversation

eromerobilbomatica commented Feb 28, 2024

Description

Development notes

Checklist

datajoely commented Feb 28, 2024

noklam commented Feb 29, 2024

eromerobilbomatica commented Feb 29, 2024

astrojuanlu commented Mar 7, 2024

eromerobilbomatica commented Mar 7, 2024

This comment was marked as outdated.

merelcht left a comment

Choose a reason for hiding this comment

astrojuanlu commented Mar 12, 2024

merelcht commented Mar 12, 2024

astrojuanlu commented Mar 15, 2024

astrojuanlu commented Mar 15, 2024