Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement python -m filprofiler. #84

Merged
merged 2 commits into from
Oct 7, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .changelog/82.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
It's now possible to run Fil by doing `python -m filprofiler` in addition to running it as `fil-profile`.
1 change: 1 addition & 0 deletions .venv
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
venv
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,12 @@ $ fil-profile run yourscript.py --input-file=yourfile

And it will generate a report.

As of version 0.11, you can also run it like this:

```
$ python -m filprofiler run yourscript.py --input-file=yourfile

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@itamarst thanks you very much for the change, a small question can the default be without the run word?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sadly no, because then it'll be hard to add additional commands. For example, I want things like fil-profile run-again, which runs the last command again and then shows you the difference in memory usage.

```

### <a name="oom">Debugging out-of-memory crashes</a>

First, run `free` to figure out how much memory is available—in this case about 6.3GB—and then set a corresponding limit on virtual memory with `ulimit`:
Expand Down
9 changes: 9 additions & 0 deletions filprofiler/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
"""
Allow run the CLI as `python -m filprofiler`.
"""


if __name__ == "__main__":
from ._script import stage_1

stage_1()
7 changes: 6 additions & 1 deletion filprofiler/_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,12 @@

$ fil-profile run -m yourpackage --your-arg=2

For more info visit https://pythonspeed.com/products/filmemoryprofiler/
You can also run the profiler this way:

$ python -m filprofiler run yourprogram.py

For more info, including documentation on Jupyter usage,
visit https://pythonspeed.com/products/filmemoryprofiler/
"""


Expand Down
37 changes: 37 additions & 0 deletions tests/test_endtoend.py
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,38 @@ def test_minus_m():
)


def test_minus_m_minus_m():
"""
`python -m filprofiler -m package` runs the package.
"""
dir = Path("python-benchmarks")
script = (dir / "malloc.py").absolute()
output_dir = Path(mkdtemp())
check_call(
[
sys.executable,
"-m",
"filprofiler",
"-o",
str(output_dir),
"run",
"-m",
"malloc",
"--size",
"50",
],
cwd=dir,
)
allocations = get_allocations(output_dir)
stripped_allocations = {k[3:]: v for (k, v) in allocations.items()}
script = str(script)
path = ((script, "<module>", 32), (script, "main", 28))

assert match(stripped_allocations, {path: big}, as_mb) == pytest.approx(
50 + 10, 0.1
)


def test_ld_preload_disabled_for_subprocesses():
"""
LD_PRELOAD is reset so subprocesses don't get the malloc() preload.
Expand Down Expand Up @@ -254,9 +286,14 @@ def test_no_args():
"""
no_args = run(["fil-profile"], stdout=PIPE, stderr=PIPE)
with_help = run(["fil-profile", "--help"], stdout=PIPE, stderr=PIPE)
no_args_minus_m = run(
[sys.executable, "-m", "filprofiler"], stdout=PIPE, stderr=PIPE
)
assert no_args.returncode == with_help.returncode
assert no_args.stdout == with_help.stdout
assert no_args.stderr == with_help.stderr
assert no_args_minus_m.stdout == with_help.stdout
assert no_args_minus_m.stderr == with_help.stderr


def test_fortran():
Expand Down