Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RLlib] Do not deepcopy input dict for efficiency and consistency with similar methods. #15709

Merged
merged 1 commit into from
Jul 22, 2021

Conversation

duburcqa
Copy link
Contributor

@duburcqa duburcqa commented May 9, 2021

Why are these changes needed?

flatten_dict method used primarily for tensorboard logging is performing deepcopy of the input dict instead of shadow copy. It seems to be unnecessary, but maybe I missed a few edge cases. It may have a significant impact in performance on real world applications (~20%).

Related issue number

Closes #15699

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

Copy link
Contributor

@richardliaw richardliaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Can you add a quick test for this?

Thanks a bunch for submitting this PR.

@richardliaw
Copy link
Contributor

BTW, be sure to cc me on your PRs in the future, as it is hard to keep track of this.

@richardliaw richardliaw self-assigned this May 14, 2021
@richardliaw richardliaw merged commit 362f7b7 into ray-project:master Jul 22, 2021
@duburcqa
Copy link
Contributor Author

Sorry for not writing the unit test, I was very busy recently (writing my PhD thesis !) Glad to see it was merged anyway :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Tune] Significant impact on performance of unnecessary copy TBXLogger.on_result (~20%)
2 participants