Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ray-data] Add alias parameters to the aggregate function, and add quantile fn #34358

Merged
merged 7 commits into from
Apr 19, 2023
Merged

Conversation

yiwei00000
Copy link
Contributor

Hi:
I added the alias parameter in the aggregate function and returned the alias. At the same time, I added aggregate function, quantile .

The following is an example of usage :

from ray.data.aggregate import Max, Mean, Min, Quantile, Std
import ray

ls = [
{"A": i, "B": i ** 2}
for i in range(10)]
ds = ray.data.from_items(ls)

ops = [Max("A", alias_name="max_a"), Min("A", alias_name="min_a"), Mean("A", alias_name="mean_a"),
Std("A", alias_name="std_a"),Quantile("A", percent=0.5, alias_name="quantile_a")]
print(dict(ds.aggregate(*ops)))

rs:
{'max_a': 9, 'min_a': 0, 'mean_a': 4.5, 'std_a': 3.0276503540974917, 'quantile_a': 4.5}

python/ray/data/aggregate.py Outdated Show resolved Hide resolved
python/ray/data/aggregate.py Outdated Show resolved Hide resolved
Copy link
Contributor

@ericl ericl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API +1 from me, cc @c21 @scottjlee

Copy link
Contributor

@scottjlee scottjlee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a couple small nits. Thanks!

python/ray/data/aggregate.py Outdated Show resolved Hide resolved
python/ray/data/aggregate.py Outdated Show resolved Hide resolved
python/ray/data/aggregate.py Outdated Show resolved Hide resolved
@yiwei00000
Copy link
Contributor Author

Hi,is there any other issues? I will cooperate and correct it in a timely manner. @scottjlee @ericl @scv119

@ericl
Copy link
Contributor

ericl commented Apr 19, 2023

Lgtm. I think there's one unresolved comment from Scott above still.

@ericl
Copy link
Contributor

ericl commented Apr 19, 2023

Looks like there's also a merge conflict with master

@yiwei00000 yiwei00000 closed this Apr 19, 2023
@yiwei00000 yiwei00000 reopened this Apr 19, 2023
@yiwei00000
Copy link
Contributor Author

Looks like there's also a merge conflict with master

Yes, I don't know if there is a file name conflict. I merged the upstream/master and the conflict was resolved. @ericl

@ericl ericl merged commit 3411e09 into ray-project:master Apr 19, 2023
@ericl
Copy link
Contributor

ericl commented Apr 19, 2023

Merged, thanks!

@yiwei00000
Copy link
Contributor Author

Merged, thanks!

Thank you very much. I still have some areas to add or modify in the future, including APIs or new features.

elliottower pushed a commit to elliottower/ray that referenced this pull request Apr 22, 2023
ProjectsByJackHe pushed a commit to ProjectsByJackHe/ray that referenced this pull request May 4, 2023
architkulkarni pushed a commit to architkulkarni/ray that referenced this pull request May 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants