Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOCS-#2518: add asv usage topic #2549

Merged
merged 3 commits into from
Dec 17, 2020
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions docs/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,58 @@ subset of the test suite. In order to run a specific test run:

The entire test suite is automatically run for each pull request.

Performance measurement
-----------------------

We use Asv_ tool for performance tracking of various Modin functionality.

Here are some scenarios in which Asv can be used:

* It is necessary to check the impact of the new patch on the performance of a certain set of operations:

.. code-block:: bash

asv continuous -f 1.05 src/master HEAD -b TimeGroupBy --launch-method=spawn

* It is necessary to check presence of errors inside of benchmarks after making changes or writing new ones:

.. code-block:: bash

asv run --quick --show-stderr --python=same --launch-method=spawn

* You just need to run the entire test suite to get the current time numbers:

.. code-block:: bash

asv run --launch-method=spawn

* It is necessary to check the range of commits for performance degradation:

.. code-block:: bash

asv run [start_hash]..[end_hash] --launch-method=spawn
asv publish
asv preview

For more consistent results, you may need to use the following parameters:

* ``-a sample_time=1``
* ``-a warmup_time=1``
* ``-a processes=4``

dchigarev marked this conversation as resolved.
Show resolved Hide resolved
Some details about using Modin on Ray with Asv:

* ``--launch-method=forkserver`` is not working
* Each set of parameters for each test is launched in its own process, which brings
a large overhead, since for each process redis server and other necessary binaries
from ray initialization are started and destroyed.

Some details for maintenance:

* ``modin/asv_bench/asv.conf.json`` contains the modin dependencies, with which testing takes place.
We need to keep them up to date according to the dependencies in ``setup.py``.


Building documentation
----------------------

Expand Down Expand Up @@ -204,4 +256,5 @@ More docs on this coming soon...
.. _flake8: http://flake8.pycqa.org/en/latest/
.. _Github Actions: https://github.com/features/actions
.. _testing:
.. _Asv: https://github.com/airspeed-velocity/asv#airspeed-velocity
.. _developer mailing list: https://groups.google.com/forum/#!forum/modin-dev