You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
dataset: Dataset = ray.data.read_images(paths="", size=(224, 224), parallelism=10)
start_time = time.time()
for data in dataset.iter_batches(batch_size=16, batch_format="numpy"):
pass
print('process time is ', time.time() - start_time)
Issue Severity
High: It blocks me from completing my task.
The text was updated successfully, but these errors were encountered:
yanxiaod123
added
bug
Something that is supposed to be working; but isn't
triage
Needs triage (eg: priority, bug/not-bug, and owning component)
labels
Apr 19, 2023
c21
added
P1
Issue that should be fixed within a few weeks
data
Ray Data-related issues
and removed
triage
Needs triage (eg: priority, bug/not-bug, and owning component)
labels
Apr 21, 2023
anyscalesam
added
P2
Important issue, but not time-critical
and removed
P1
Issue that should be fixed within a few weeks
labels
Nov 2, 2023
What happened + What you expected to happen
I use the following code test, i find apply if branch https://github.com/ray-project/ray/blob/master/python/ray/data/_internal/arrow_ops/transform_pyarrow.py#L283,if i have many chunks,how can i to do?because i search #34352 indicate many chunks due to slow.
Versions / Dependencies
ray==3.0.0.dev0
Reproduction script
dataset: Dataset = ray.data.read_images(paths="", size=(224, 224), parallelism=10)
start_time = time.time()
for data in dataset.iter_batches(batch_size=16, batch_format="numpy"):
pass
print('process time is ', time.time() - start_time)
Issue Severity
High: It blocks me from completing my task.
The text was updated successfully, but these errors were encountered: