Skip to content

Commit

Permalink
[Datasets] [Docs] Add seealso to map-related methods (ray-project#3…
Browse files Browse the repository at this point in the history
…0579)

This PR adds seealso notes to help users distinguish between map, flat_map, and map_batches.

Signed-off-by: Balaji Veeramani <[email protected]>
Signed-off-by: Edward Oakes <[email protected]>
  • Loading branch information
bveeramani authored and edoakes committed Mar 22, 2023
1 parent a8ca410 commit 82004b3
Showing 1 changed file with 40 additions and 0 deletions.
40 changes: 40 additions & 0 deletions python/ray/data/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,20 @@ def map(
must be used.
ray_remote_args: Additional resource requirements to request from
ray (e.g., num_gpus=1 to request GPUs for the map tasks).
.. seealso::
:meth:`~Dataset.flat_map`:
Call this method to create new records from existing ones. Unlike
:meth:`~Dataset.map`, a function passed to :meth:`~Dataset.flat_map`
can return multiple records.
:meth:`~Dataset.flat_map` isn't recommended because it's slow; call
:meth:`~Dataset.map_batches` instead.
:meth:`~Dataset.map_batches`
Call this method to transform batches of data. It's faster and more
flexible than :meth:`~Dataset.map` and :meth:`~Dataset.flat_map`.
"""
if isinstance(fn, CallableClass) and (
compute is None
Expand Down Expand Up @@ -528,6 +542,20 @@ def map_batches(
:meth:`~Dataset.default_batch_format`
Call this function to determine the default batch type.
:meth:`~Dataset.flat_map`:
Call this method to create new records from existing ones. Unlike
:meth:`~Dataset.map`, a function passed to :meth:`~Dataset.flat_map`
can return multiple records.
:meth:`~Dataset.flat_map` isn't recommended because it's slow; call
:meth:`~Dataset.map_batches` instead.
:meth:`~Dataset.map`
Call this method to transform one record at time.
This method isn't recommended because it's slow; call
:meth:`~Dataset.map_batches` instead.
""" # noqa: E501

if batch_format == "native":
Expand Down Expand Up @@ -793,6 +821,18 @@ def flat_map(
must be used.
ray_remote_args: Additional resource requirements to request from
ray (e.g., num_gpus=1 to request GPUs for the map tasks).
.. seealso::
:meth:`~Dataset.map_batches`
Call this method to transform batches of data. It's faster and more
flexible than :meth:`~Dataset.map` and :meth:`~Dataset.flat_map`.
:meth:`~Dataset.map`
Call this method to transform one record at time.
This method isn't recommended because it's slow; call
:meth:`~Dataset.map_batches` instead.
"""
if isinstance(fn, CallableClass) and (
compute is None
Expand Down

0 comments on commit 82004b3

Please sign in to comment.