Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Benchmark] New benchmarking tool with gnuplot(2) #168

Merged
merged 6 commits into from
Feb 20, 2018

Conversation

shibatch
Copy link
Owner

@shibatch shibatch commented Feb 6, 2018

This is a rebased branch of #98.

This patch replaces the old benchmarking tool with a new tool with gnuplot.

With this new tool, the graph showing execution time can be automatically drawn.
It is now easy to see the difference between different library versions or compilers.
This tool is used to draw the graphs on sleef.org web site.

This patch implements the tool described in #9.

A makefile is used In this patch, and there are a few reasons.

  • It needs to compile the tools for SVML. These tools are independent of the other part, and thus it does not make sense to build these tools using the cmake during a build of library.
  • This makefile is used as a script for measurement. It is handy since ProcData.java has to be also compiled.

@@ -5,6 +5,7 @@ clean :
+"$(MAKE)" --directory=./gencoef clean
+"$(MAKE)" --directory=./libm clean
+"$(MAKE)" --directory=./libm-tester clean
+"$(MAKE)" --directory=./libm-benchmarks clean
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove, we don't care about makefiles.

@@ -0,0 +1,6 @@
.PHONY: clean
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove, we don't care about makefiles.

@@ -0,0 +1,110 @@
BENCHSLEEF := $(shell command -v benchsleef 2> /dev/null)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove, no makefiles.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you even read the comment at the top of this page?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I still think we shouldn't have makefiles down inside the tree of src.

If you really want to keep the makefiles, then I think you should move all this benchmarks and relative scripts at the same level of src, in a folder like svml-benchmarking, and keep it as a sort of independent project inside SLEEF.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally, the whole directory including all the C codes were like an independent project.
I think that is easier to understand.
Why do you think C codes have to be compiled by cmake?

@@ -0,0 +1,153 @@
// Copyright Naoki Shibata 2010 - 2017.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a benchmark, why is it in the plot folder? The plot-scripts folder should just contain the scripts that are used to make the plots.

@@ -0,0 +1,137 @@
// Copyright Naoki Shibata 2010 - 2017.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another benchmark that should sit one folder up?

@@ -0,0 +1,140 @@
// Copyright Naoki Shibata 2010 - 2017.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, benchmark code, not script for plotting.

@@ -0,0 +1,137 @@
// Copyright Naoki Shibata 2010 - 2017.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, benchmark, not script.


if [ -f counter.txt ]
then
counter=`cat counter.txt`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does counter need any particular syntax?

@@ -0,0 +1,17 @@
#!/bin/sh
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not clear how to use this script. Who is invoking the java tool?

@fpetrogalli
Copy link
Collaborator

fpetrogalli commented Feb 19, 2018

Originally, the whole directory including all the C codes were like an independent project.
I think that is easier to understand.

Agree. Please move it at the same level of src.

Why do you think C codes have to be compiled by cmake?

I don't think it anymore. Please remove any cmake configuration related to these benchmark.

@shibatch shibatch merged commit b4dc315 into master Feb 20, 2018
@shibatch shibatch deleted the New_benchmarking_tool_with_gnuplot2 branch February 20, 2018 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants