Document reputation label #347
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ErikSchierboom
force-pushed
the
add-rep-label
branch
from
1f17c61
to
e337ba0
Compare
ErikSchierboom
commented
Apr 22, 2022
using/product/reputation.md
Outdated
Comment on lines
70
to
74
| | `x:rep/tiny` | 3 (~5 mins) | <ul><li>Fixing a single typo or link</li><li>Removing a blank line or adding a line break</li><li>Changing/adding a single code comment</li></ul> | | ||
| | `x:rep/small` | 5 (~10 mins) | <ul><li>Fixing a single test case, task or example</li><li>Fixing multiple typos or links in a single file</li><li>Clarifying content by adding a few lines to a file</li></ul> | | ||
| | `x:rep/medium` | 12 (~30 mins) | <ul><li>Syncing an exercise with problem-specifications (incl. edits)</li><li>Adding one or more test cases from scratch</li><li>Improving multiple files in an exercise</li><li>Adding mentor notes for an exercise from scratch</li><li>Fixing a small bug in a test runner/analyzer/representer</li><li>Adding analyzer comments for a single exericse</li></ul> | | ||
| | `x:rep/large` | 30 (~2 hrs) | <ul><li>Adding a new concept or practice exercise</li><li>Adding new concept documentation</li><li>Substantial re-writing of an existing concept or exercise</li><li>Adding new CI scripts or other automation</li></ul> | | ||
| | `x:rep/massive` | 100 (~5 hrs) | <ul><li>Creating a test-runner, analyzer, representer or generator from scratch</li><li>Major refactors to those tools</li><li>Creating major documentation from scratch (e.g. contribution or testing guides)</li></ul> | |
There was a problem hiding this comment.
I've slightly rounded the estimated time to make it easier to remember.
kotp
reviewed
Apr 22, 2022
|
|
||
| The examples above can serve as rough orientation when to apply which label but maintainers are free to use their own judgement. | ||
|
|
||
| - The estimated number of time spent should be interpreted as the average time a _maintainer_ would spend on doing the PR. | ||
| - If more than one label is specified, the label with the highest reputation value determines the awarded reputation. | ||
| - If a pull request is still open, no reputation is awarded (yet). | ||
| - If a pull request is closed _without_ merging, no reputation is awarded. |
There was a problem hiding this comment.
This should be "exceptionable". There can be value in the discussion of code that does not come in, and a historical record of why. So reputation should be able to be given even if a PR is not accepted.
There was a problem hiding this comment.
We have others ways of rewarding reputation for those cases.
There was a problem hiding this comment.
But we clarify with the "(yet)" when it makes little sense to aware while a PR is still open, which is probably something that would be expected, yet, do not clarify that the no reputation for closed pull requests is not absolutely no reputation when warranted?
|
Note: update the issue section to only mention rep for large/massive issues. |
|
This is not blocking this particular PR but I wanted to leave a note about two issues I see with the added time values in the reputation table:
This points to the same root cause imo. The current rep sizes and how they are used overvalue small contributions / undervalue large contributions (despite the fact that we usually need more of the large ones). If you wanted to get the most rep out out of 2h you would do a lot of typo fixes, small grammar improvements etc. Because in reality to create something with a large label like a new concept from scratch on a non-trivial topic gives rep for 2h but takes more like 4h, on complex topics (e.g. JavaScript classes and prototypes), I spent more like 10h to create good content and that does not even include the exercise part that was also in the same PR. My guess is other maintainers have similar stories but I might be wrong. |
|
Yeah, getting these numbers right is very difficult. Maybe we should examine the existing PRs that are large/massive to see what they implemented? Large{gh search prs --limit 200 --label 'x:size/large' --json 'author,repository,title,url' --merged --jq '.[] | .repository.nameWithOwner + " | " + .title + " | " + .author.login + " | " + .url'; gh search prs --limit 200 --label 'x:rep/large' --json 'author,repository,title,url' --merged --jq '.[] | .repository.nameWithOwner + " | " + .title + " | " + .author.login + " | " + .url' } | sort -t\| -k1 | awk 'BEGIN{print "repo | title | author | url"; print "-- | -- | -- | --"}1'
Massive{gh search prs --limit 200 --label 'x:size/massive' --json 'author,repository,title,url' --merged --jq '.[] | .repository.nameWithOwner + " | " + .title + " | " + .author.login + " | " + .url'; gh search prs --limit 200 --label 'x:rep/massive' --json 'author,repository,title,url' --merged --jq '.[] | .repository.nameWithOwner + " | " + .title + " | " + .author.login + " | " + .url' } | sort -t\| -k1 | awk 'BEGIN{print "repo | title | author | url"; print "-- | -- | -- | --"}1'
|
|
I've merged this PR as I want to get the docs updated. We should however continue discussing what the proper reputation values should be. |
|
Could you maybe create a new issue (not sure where exactly) and paste your table (and suggested time values?) in there? Then we can ask some other people to give input as well. I think looking at those examples is a good idea but I don't think this PR is the right place for the discussion. |
This was referenced Jul 14, 2022
|
Issue opened: exercism/exercism#6440 |
|
(just here today thanks for thinking I'm a 100x dev) I agree with your assessment and I'll try to provide some hour estimates in the linked issue. I spend more than 5 hours on many of the massive prs and that's... Even with being a NX dev. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
x:replabel