Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data] Truncate progress bar description #46801

Merged
merged 5 commits into from
Jul 30, 2024
Merged

[Data] Truncate progress bar description #46801

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5816372
truncate progress bar description
scottjlee Jul 25, 2024
670bb30
add context var
scottjlee Jul 26, 2024
dc0095a
include first/last op
scottjlee Jul 29, 2024
a963e89
handle last op
scottjlee Jul 29, 2024
02a2b30
comments
scottjlee Jul 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion python/ray/data/_internal/progress_bar.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,22 @@ class ProgressBar:
because no tasks have finished yet), doesn't display the full
progress bar. Still displays basic progress stats from tqdm."""

# If the name/description of the progress bar exceeds this length,
# it will be truncated.
MAX_NAME_LENGTH = 100

def __init__(
self,
name: str,
total: Optional[int],
unit: str,
position: int = 0,
enabled: Optional[bool] = None,
display_full_name: bool = False,
):
self._desc = name
# If True, disables name trunctating.
self._display_full_name = display_full_name
self._desc = self._truncate_name(name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would this be set to True by the user? Is the expectation that they would not want that or should we have something in data context or execution options to allow for this configuration?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yeah good point, i will need to expose it from DataContext

self._progress = 0
# Prepend a space to the unit for better formatting.
if unit[0] != " ":
Expand Down Expand Up @@ -83,6 +90,11 @@ def __init__(
needs_warning = False
self._bar = None

def _truncate_name(self, name: str) -> str:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a warn-once that the name is getting truncated and that the behavior can be disabled with DEFAULT_ENABLE_PROGRESS_BAR_NAME_TRUNCATION? Not sure if users will know how to disable it otherwise

if not self._display_full_name and len(name) > self.MAX_NAME_LENGTH:
return name[: self.MAX_NAME_LENGTH - 3] + "..."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: (Don't need to do this, just throwing it out as an idea) I'm wondering if we should include some of the text at the end. So, for example, instead of:

Map(spam)->Map(ham)->...

We could do something like

Map(spam)->...->Map(ham)

Might make it clearer which operator it ends on.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah sounds good. now the updated output looks like:

✔️  Dataset execution finished in 32.55 seconds: 100%|█████████████████████████████████████████████████████████████████████████| 20/20 [00:32<00:00,  1.63s/ bundle]]
- ReadCSV->SplitBlocks(20): 0 active, 0 queued, [cpu: 0.0, objects: 0.0B]: : 20 bundle [00:32,  1.63s/ bundle]]name): 3 active, 17 queued, [cpu: 3.0, objects: 768.0
- MapBatches(f_with_really_long_name)->MapBatches(f_with_really_long_name)->...->MapBatches(f_with_really_long_name): 0 active, 0 queued, [cpu: 0.0, objects: 840.0B

return name

def block_until_complete(self, remaining: List[ObjectRef]) -> None:
t = threading.current_thread()
while remaining:
Expand Down Expand Up @@ -117,6 +129,7 @@ def fetch_until_complete(self, refs: List[ObjectRef]) -> List[Any]:
return [ref_to_result[ref] for ref in refs]

def set_description(self, name: str) -> None:
name = self._truncate_name(name)
if self._bar and name != self._desc:
self._desc = name
self._bar.set_description(self._desc)
Expand Down