Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: add section on basic performance benchmark #5635

Merged

Conversation

sphuber
Copy link
Contributor

@sphuber sphuber commented Sep 8, 2022

The section provides a basic script to benchmark the performance of an AiiDA installation by launching a number of ArithmeticAddCalculation jobs. The script by default automatically sets up the required Code and localhost Computer. All created nodes are automatically deleted from the database at the end.

The documentation gives instructions on how to run the script and provides example output of a run on a typical work station including completion times for runs with variable number of daemon workers. This should give users an idea of the performance of their installation.

@sphuber sphuber requested a review from ltalirz September 8, 2022 13:42
@sphuber sphuber force-pushed the fix/docs-base-performance-test branch 3 times, most recently from 529cad7 to a1d3640 Compare September 8, 2022 14:43
Copy link
Member

@ltalirz ltalirz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks a lot @sphuber !

I've added a few comments on the docs (haven't yet looked at the code much; I assume that will be fine)

docs/source/howto/installation.rst Outdated Show resolved Hide resolved
docs/source/howto/installation.rst Outdated Show resolved Hide resolved
docs/source/howto/installation.rst Outdated Show resolved Hide resolved
docs/source/howto/installation.rst Outdated Show resolved Hide resolved
docs/source/howto/installation.rst Outdated Show resolved Hide resolved
docs/source/howto/installation.rst Outdated Show resolved Hide resolved
@sphuber sphuber force-pushed the fix/docs-base-performance-test branch from e9e9fd1 to a4629f0 Compare September 12, 2022 11:25
@sphuber
Copy link
Contributor Author

sphuber commented Sep 12, 2022

@ltalirz I have addressed the comments and added a section to topics on performance. Even though the latter isn't really part of the focus of this PR, I already added some text as to not have it completely empty. But maybe should not go down the rabbithole too much for this PR on that part.

@ltalirz
Copy link
Member

ltalirz commented Oct 27, 2022

I added one further commit in #5723

If that seems fine, please cherry-pick it here and close that PR

@sphuber sphuber force-pushed the fix/docs-base-performance-test branch 2 times, most recently from ff9f66c to f39d4b8 Compare October 27, 2022 12:39
@sphuber
Copy link
Contributor Author

sphuber commented Oct 27, 2022

If that seems fine, please cherry-pick it here and close that PR

Thanks, all fine. I cherry-picked the commit here. Just corrected a mistake in the script that was probably added by accident and some minor corrections in the docs (broken label).

@ltalirz
Copy link
Member

ltalirz commented Oct 27, 2022

Thanks for the fixes!

I see I still introduced some remaining issues?

<string>:1: (WARNING/2) Inline interpreted text or phrase reference start-string without end-string.
/home/docs/checkouts/readthedocs.org/user_builds/aiida-core/checkouts/5635/docs/source/howto/installation.rst:341: WARNING: Inline literal start-string without end-string.
/home/docs/checkouts/readthedocs.org/user_builds/aiida-core/checkouts/5635/docs/source/topics/performance.rst:53: WARNING: download file not readable: /home/docs/checkouts/readthedocs.org/user_builds/aiida-core/checkouts/5635/docs/source/topics/include/scripts/performance_benchmark_base.py

The section provides a basic script to benchmark the performance of an
AiiDA installation by launching a number of `ArithmeticAddCalculation`
jobs. The script by default automatically sets up the required `Code`
and localhost `Computer`. All created nodes are automatically deleted
from the database at the end.

The documentation gives instructions on how to run the script and
provides example output of a run on a typical work station including
completion times for runs with variable number of daemon workers. This
should give users an idea of the performance of their installation.
@sphuber sphuber force-pushed the fix/docs-base-performance-test branch from f39d4b8 to 12d3721 Compare October 27, 2022 13:05
@sphuber
Copy link
Contributor Author

sphuber commented Oct 27, 2022

/home/docs/checkouts/readthedocs.org/user_builds/aiida-core/checkouts/5635/docs/source/topics/performance.rst:53: WARNING: download file not readable: /home/docs/checkouts/readthedocs.org/user_builds/aiida-core/checkouts/5635/docs/source/topics/include/scripts/performance_benchmark_base.py

This was just an incorrect link (it is relative, but we moved it to another subfolder).

/home/docs/checkouts/readthedocs.org/user_builds/aiida-core/checkouts/5635/docs/source/howto/installation.rst:341: WARNING: Inline literal start-string without end-string.

This was actually a nasty one. The error messages of Sphinx are often not very useful. The problem was the following

``ArithmeticAddCalculation``s

Apparently, you cannot have this s straight after the closing literal block. What made it even more confusing is that this was on line 342 whereas the error message states 341 😅 . Anyway, should be fixed now.

Copy link
Member

@ltalirz ltalirz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks a lot @sphuber !

The performance of AiiDA depends on many factors:

* the hardware that AiiDA is running on
* the environment in which AiiDA is installed
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean by that specifically?

until this section is fleshed out, you could also just leave it out

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had originally added this but already removed it when cherry-picking your commit, where you suggested to remove it. Think you were looking at an outdated diff.

What I mean with the environment is the runtime for AiiDA, i.e. is it installed with Conda, in virtual-env, or system wide, or maybe running in a container, etc. It is not quite clear if and how much of an impact it could have, but I thought it would be good to have the list complete. But now I think it is fine to just omit it for the time being.

@sphuber sphuber merged commit 07c1ba9 into aiidateam:support/1.6.x Oct 27, 2022
@sphuber sphuber deleted the fix/docs-base-performance-test branch October 27, 2022 13:28
sphuber added a commit to sphuber/aiida-core that referenced this pull request Oct 27, 2022
The section provides a basic script to benchmark the performance of an
AiiDA installation by launching a number of `ArithmeticAddCalculation`
jobs. The script by default automatically sets up the required `Code`
and localhost `Computer`. All created nodes are automatically deleted
from the database at the end.

The documentation gives instructions on how to run the script and
provides example output of a run on a typical work station including
completion times for runs with variable number of daemon workers. This
should give users an idea of the performance of their installation.

Cherry-pick: 07c1ba9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants