Fix benchmark script #32635
Fix benchmark script #32635
Conversation
| @@ -101,7 +101,7 @@ def summarize(run_dir, metrics, expand_metrics=False): | |||
| # post-processing of report: show a few selected/important metric | |||
| for metric in metrics: | |||
| keys = metric.split(".") | |||
| value = report | |||
| value = report.to_dict() | |||
There was a problem hiding this comment.
v0.2.0 doesn't need to_dict, but 0.4.0 needs this.
There was a problem hiding this comment.
v0.2.0 of what package?
There was a problem hiding this comment.
sorry, it's optimum-benchmark
There was a problem hiding this comment.
The issue is specifically from this __post_init__ in v0.2.0 that was incomplete ; only casts the config dict to the appropriate dataclass:
https://github.com/huggingface/optimum-benchmark/blob/v0.2.0/optimum_benchmark/base.py#L25-L29
which was fixed/completed in:
https://github.com/huggingface/optimum-benchmark/blob/v0.3.0/optimum_benchmark/benchmark/base.py#L24-L33
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Thanks for fixing @ydshieh!
Change looks OK - but I'm guess it's not backwards compatible if the user has v0.2.0? Is optimum-benchmark version pinned anywhere in the library?
|
Thanks. I update the min. version. |
* fix * >= 0.3.0 --------- Co-authored-by: ydshieh <[email protected]>
* fix * >= 0.3.0 --------- Co-authored-by: ydshieh <[email protected]>
What does this PR do?