"Weighting" system for Google Code-In

[tech4GT]

I suggest after a discussion with @SidharthBansal to have a Bounty system on issues throughout GCI to track the student performance.
We can have these bounties as labels on the github issues we will raise during GCI and the bounty shall vary on how big or complex the issue is.
I think these slabs should be enough to cover issues of all complexities.
50
100
150
200
250
300
350
400
450
500

So we can create labels like bounty:50 and so on
This will help us keep it fair so that someone working on a very complex issue is compensated properly. @publiclab/reviewers please give your suggestions on this.
This is in concert with #3168 and #3276

[welcome]

Thanks for opening your first issue here! Please follow the issue template to help us help you 👍🎉😄

[SidharthBansal]

Varun the bot has errors. You are not a newbie. lol.
Thanks for opening the issue.

[tech4GT]

Oh it's fine this is my first issue on plots😄

[SidharthBansal]

@tech4GT can you please divide these bounties in some categories. So that it will be easy for GCI mentors to tag them up

[SidharthBansal]

Like writing a page of documentation will have x bounties, making a button ui will have y bounties, implementing that button will have higher bounties and all that. So that we will have a fair system

[tech4GT]

Oh sure we can do that once @jywarren and others approve of this system. Is that fine?

[SidharthBansal]

Also, I will say that instead of ten we can have fewer levels say 3/4 levels of bounty. It will be quite easy to pick from 3/4 bounty levels then from 10 bounty levels.
Like we can have 100 250 and 500

[SidharthBansal]

yeah other reviews are really important

[tech4GT]

Yeah we can work out the details once everyone on the team is OK with this.

[ebarry]

Wow interesting. Can someone explain what bounties mean?

I would like to relate this concept to the concept that is currently in my head, which is --- Broadly in Public Lab (as in the environmental research part), we try to reward all contributions equally, to overturn some of the exclusivity that builds up in domains of expertise.

Thank you for any explanations you can add!!!! <3!

[jywarren]

Yes, I've been thinking hard on this one for a few days, and my main worry would be that some people might consider certain types of work to be "worth more" in a way that might not be fair, even by mistake or oversight. It's a tough thing to figure out what is worth more or less or what's easier or harder for people.

Definitely there are some tasks that just involve lots and lots of lines of code to write. But just because these are easier to quantify as "harder" doesn't mean that we can ignore the kinds of tasks that might take a lot of invisible work. So while I love the goal of this (fairness), I'm cautious!

[tech4GT]

Hi @ebarry
Actually bounties are like points that you get on completion of a task. So the bigger the task the more the points(bounty). The idea is to create labels on GitHub issues which will clearly tell the contributors how much points they will get on solving that issue. We can then take our descision based on the points earned rather than number of issues solved. This should be more fair to people who would be solving relatively complex tasks.

[tech4GT]

@jywarren you are right, we would have to be very careful while setting the bounties. This should not be a problem if bounties are set on a project by mentors who are very well versed with it, what say?

[SidharthBansal]

@jywarren your concern is right. I thought about this thing. For me may be technology t1 is tough and technology t2 is easy but for other guy it may be opposite. So, I agree that hard and easy are relative terms. Just trying to make the competition transparent to students

…

On Mon, Sep 10, 2018, 8:57 PM Jeffrey Warren ***@***.***> wrote: Yes, I've been thinking hard on this one for a few days, and my main worry would be that some people might consider certain types of work to be "worth more" in a way that might not be fair, even by mistake or oversight. It's a tough thing to figure out what is worth more or less or what's easier or harder for people. Definitely there are some tasks that just involve lots and lots of lines of code to write. But just because these are easier to quantify as "harder" doesn't mean that we can ignore the kinds of tasks that might take a lot of invisible work. So while I love the goal of this (fairness), I'm cautious! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3284 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AUACQ6BTNYvTZ1iS-AcQGe8yc_WyTIJTks5uZoTkgaJpZM4WWL-7> .

[SidharthBansal]

I will say that we can have 2 categories or three. Like very basic category in which 50 points (/bounties/score) is given to candidate. 250 who does a fairly big task. Usually people make many small one liner edits at many places and the no of commits for them is higher than those who did a big patch of 500 lines. So, the judging criterion should be fair. Like front end images can be 50, implementing a feature with test is 250. There will be a significant difference in the points which will help us to decide the winner

…

On Mon, Sep 10, 2018, 9:01 PM Varun Gupta ***@***.***> wrote: @jywarren <https://github.com/jywarren> you are right, we would have to be very careful while setting the bounties. This should not be a problem if bounties are set on a project by mentors who are very well versed with it, what say? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3284 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AUACQyhd-Ah5DaFspWIWdhXn-81Kd5gGks5uZoXigaJpZM4WWL-7> .

[SidharthBansal]

My main focus is how will we decide the winner. This decision should be transparent and fair as per the spirit of competition. We can do this either by bounty system or by simply lines of code or by number of PRS merged+ number of issues opened. On Mon, Sep 10, 2018, 9:05 PM Sidharth Bansal <[email protected]> wrote:

…

I will say that we can have 2 categories or three. Like very basic category in which 50 points (/bounties/score) is given to candidate. 250 who does a fairly big task. Usually people make many small one liner edits at many places and the no of commits for them is higher than those who did a big patch of 500 lines. So, the judging criterion should be fair. Like front end images can be 50, implementing a feature with test is 250. There will be a significant difference in the points which will help us to decide the winner On Mon, Sep 10, 2018, 9:01 PM Varun Gupta ***@***.***> wrote: > @jywarren <https://github.com/jywarren> you are right, we would have to > be very careful while setting the bounties. This should not be a problem if > bounties are set on a project by mentors who are very well versed with it, > what say? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#3284 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AUACQyhd-Ah5DaFspWIWdhXn-81Kd5gGks5uZoXigaJpZM4WWL-7> > . >

[jywarren]

I will say that we can have 2 categories or three.

I like this to keep the labels more minimal -- and to think about the kinds of work that makes it a "big" one -- imagine they do one of this list, two of the list, or three of:

1. code change (<10 lines)
2. code change (10+ lines)
3. back-and-forth discussion with community for input
4. install and take a screenshot of the feature
5. write a test
6. write documentation

This not as a formal rule but as a rough guide... what do you think? Any others?

If they do 1, it's "small" -- if they do 3, it's "medium" and if they do more than five, it's "large"?

if students help each other out, or are extra friendly, maybe we can assign extra points using a label like teamwork?

[SidharthBansal]

So this means we will have an excel list type of spreadsheet where we will assign points to folks?

[SidharthBansal]

We also need to think of a feasible way to evaluate. If we will assign points for community bonding then this would be really tough for us to keep a check.

[SidharthBansal]

install and take a screenshot of the feature

Sorry to say but we should not break the issue in such a small manner that mentor's work will become exponential and code which is broken into chunks are too small to have independent significance.
I will suggest that we can expect regular prs as we currently get from new comers. Its better to teach them what a PR should include. How to talk on issues and PRs?
We can expect them to respond to the issue. Then open a pr. Submit their patch (code/test/documentation) with necessary screenshots.

[jywarren]

Sure, i think this is reasonable. Just trying to come up with a quick way to evaluate "task size"

[SidharthBansal]

I think one way is to implement highly modular code and to count the number of PRs merged Or Lines of Code and see overall how the human is. This approach is easy as we can see all collaborators contribution here

So we dont need any book keeping. This will also reduce significant mentors work and is easy to evaluate. Also transparent to the folks.
Its strange that after long discussion we jump to the initial solution which we had earlier.

[jywarren]

I am going to make small medium large tags to help us if we need. This can be a hybrid - consider both lines of code, but also when needed, use these tags to help recognize kinds of work that aren't reflected in the lines of code. Maybe the student did a lot of research outside, or talked a lot with community to investigate a problem. Maybe they are making FTOs or something else which doesn't result in new lines. In these cases, we can use the "weighting" tags to help guide evaluation. 👍

[jywarren]

Closing as I've updated the main description on the task list issue!