Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create blank directories for papers missing build information #49

Open
shriram opened this issue Mar 23, 2014 · 3 comments
Open

create blank directories for papers missing build information #49

shriram opened this issue Mar 23, 2014 · 3 comments
Assignees

Comments

@shriram
Copy link
Owner

shriram commented Mar 23, 2014

No description provided.

@shriram shriram self-assigned this Mar 23, 2014
@camild
Copy link
Collaborator

camild commented Mar 23, 2014

Good idea: a paper may be misclassified (e.g., mistakenly claimed as theoretical) and have an artifact to check.

@samth
Copy link
Contributor

samth commented Mar 24, 2014

Indeed, this one is an example: http://reproducibility.cs.arizona.edu/data/tissec15_YavuzNR12_data.txt

As I wrote on G+:

It's marked as "theoretical" in the database, with this comment. "VERIFY:COMMENT[string] Since proofs exist for the effectiveness of the system, I'm not sure either if it is necessary to ask for implementaitons."

But there's also: "PI:COMMENT_CC[string] They measure performance. So, we need access to their code. Proof of correctness means nothing, by the way!"

If you look at the paper, it has performance numbers for an implementation. For the overall numbers, though, it still is counted as "theoretical".

I couldn't find the implementation with about 10 minutes of looking.

@hilmar-hoffmann
Copy link

I second this, too, but I thought

We have focused on a particular, large, and important class: the ones that they have put into "builds" + "build fails".
(shriram posting to #50)

??
Anyway, havng a reprensentation for each paper was acutally what I had originally in mind with #50, so no complaints here.
I wasn't aware that you already had posted #49 covering this a quarter of an hour earlier :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants