-
Notifications
You must be signed in to change notification settings - Fork 62
How "fair" is the current Trust metric system: Analyzing its effectiveness #752
Comments
The most straightforward change to the existing system is to tweak the parameters for the expected number of masters, journeyers, and apprentices; for example on May 9: #375 (comment) As motivation for this or other changes, please give two or three specific examples where the current system is working poorly. (We have another 5 days to address them before the 201805 pay period voting closes, of course.)
They are already able to do so, no? If not, please give an example. p.s. Where exactly did those numbers come from? When you say "80% of rewards" do you mean by dollar amount? Or by counting issue / worker combinations? I get:
select has_rating, count(*), sum(reward_usd)
from (
select pay.issue_num, pay.worker, reward_usd
, case when a.rating is null then 0 else 1 end has_rating
from reward pay
left join authorities a on pay.worker = a.login
where pay.pay_period = '2018-05-01'
) detail
group by has_rating; I can't reproduce the 109. I can find: p.p.s. what is the discussion label for? Doesn't every issue involve discussion? And the title of the issue says it is only about analyzing the effectiveness of the trust metric, but the bounty-contract label suggests it's a change to the process, and you note some proposed changes, and the measure of completion is that the trust metric is amended. I guess we'll see where we end up... And why the quotes around "fair"? You're using the word in its usual sense, no? |
@dckc I think you need to update your records, Here are the total rewards so far: Members with trust ratings have a total rewards of $14,911 while members without trust rating has a total of $4,858 see here. This is what I mean by "about 80% of rewards". @TrenchFloat, you voted 5% for Makys on #672 but I don't see her input, maybe you might want to explain this, also, @jimscarver voted 5% for @ICA3DaR5 ON #681 where there was no input from @ICA3DaR5 and 25% for @nonnykul who is the main actor on the project. How do we co-relate this? Project #662 has been completed but no rewards has been awarded to it since the votes of the participants carries no weight, The Principles and Values WG working document states:
Also, I would like all voters on #672 to kindly explain the basis for their rewards votes, there were a lot of participation in that issue but I can see only a few rewards. PS- All figures were given as at 20180604 9:00am GMT+1 |
Nice observation @David405 I think the current method of voting is quite biased and elitist. To make the voting process easier, I suggest that each label lead should vote for issues in their label and people voted for should also raise concerns if they feel their vote is insufficient or bias. This would create a more transparent system that is open to scrutiny. Another method that could be possible is that each issue when voted for should submit a statement of the amount of work done and everyone's capacity who participated in the issue to the trust metric members . |
From the ongoing work in #471 by @jimscarver @barneycinnamon, it was stated here that RChain as a coop subscribes to the International Co-op Alliance’s set of Principles
From the Cooperative Principles, two stated principles includes Democratic Member Control and Member Economic Participation, which states:
And
Respectively, I believe the current Metric system or Trust rating DOES NOT reflect these values as it only gives a fraction of the members the right to influence their opinions on budgets and rewards, which is supposed to be fundamental right of every coop member. |
@dckc I would like to address your subsequent questions here, Firstly, You asked about the discussion label, I add discussion labels to issues which do not involve any tasks or projects ( not all issues require tasks to be carried out and thus there have to be a way to separate issues based on this parameter). Secondly, you asked about the bounty-contract label, the aim of the issue is to propose a change to the bounty-contract with regards to the aspect of Trust ratings, I believe every member of RChain coop should be able to vote and influence rewards and budgets on issues they participated in, label guides should also be able to put a check on budgets too. Finally, you asked about the double quotes around the word fair, the double quotes were placed to show emphasis and nothing more. |
What happens if we reverse the system? Every member starts as journeyer until s/he blows it. |
Nice Observation @David405
|
@lapin7 I agree with your suggestion especially the part you said ALL MEMBERS I believe onE of the structures that should be changed in the bounty program is the structure of a few persons making all the decisions for everyone. The Cooperative Principles states
I think before any idea or suggestion is implemented in a cooperative, there should be a consensus by the majority of its members but for a while now in the RChain coop, ideas have been implemented without even proper information passed across to all members of the cooperative. In #653 the voters voted 0 as budget and rewards for @Pionelle and since the weight was more than 10x, she has lost her rewards in her other issues for the month, When was there a consensus that voting 0 with a weight of more than 10x on an issue cancels reward for the month?, who and who were present in such meeting? How many members agreed to it? I think it is high time we reform some of the structures we have in place if we truly refer to ourselves as a cooperative. I also think there should be another one or two persons working together with @dckc in implementing changes to the bounty App, and before any change is implemented, there should be a general agreement by the majority (since in practice it is difficult to get everyone to agree to one idea), there are other very good developers in the coop who should be able to review and implement changes to codes in php (xataface framework) and python (pandas). I strongly believe that there are other more effective ways to deal with fraud and excess rewards than allowing just a fraction of the members in a cooperative to influence their opinions |
Hey @David405 you seem to imply that right now we have a "structure of a few persons making all the decisions for everyone." I don't think that's currently the case: there are currently 46 people certified under the Trust Metric system to have a vote that counts, and more people can be added whenever those 46 people trust them. That's hardly elitist or centralised! As a matter of fact, I think the RChain Bounty System is an absolute front-runner when it comes to decentralised governance, and something to be proud of (even if it's still experimental). ... The idea of a majority of the members (which means 1000+ people) agreeing on every idea and concept is unworkable in practice. The concept of the Trust Metric has been in the works since February, and @dckc has always worked completely transparently, in the open. I think he's shown incredibly concern for the community with his work. As for slashing, this has similarly been in the works since early this year, and also completely transparently. Slashing was already in place in December last year, when @Mervyn853 effectively slashed @ResonanceXX. This is not by any means a new idea. ... Finally, I think @dckc would greatly appreciate any help on the bounty app he can get. If you know of any devs that are keen to help, send them our way so that they can help out! |
Personally I have been trying to relate RChain as a cooperative to the trust metric system too,don't understand how they both could work together without hampering the legitimacy and dictums of A COOPERATIVE.
in acceptance to lapins same proposition to prevent lateralization of power and function in the coop. Using issue #471 as a quick reference point to RChain's cooperative principles and values, I think the trust metric system and some other bias, domineering occurrences going on are gradually going against the laws of cooperative governing rchain as a coop. And as bad as that could be, it's obvious enough we might actually be contradicting ourselves by the use of the new trust metric system and maintaining of RChain being a cooperative #213.
Yes, although so far what is being done by some projects guides is in line and productive to an extent, but the issue of not giving detailed specifications as to what is fair in respect to the budgets is not helping the budget system work smoothly. just like I am trying to still figure out what is considered a fair percentage to be allocated for a review of a doc, what is fair for an issue and how is fair to reward contributors of an issue.since we are not seeming to accept the assumptions that those who worked on the issues knows what it's worth and who gets what. |
@lapin7 writes:
I don't see how to square that with the Apr 6 TOS decision by the board (#616). It seems to me that we tried the "everybody gets to vote on budgets and rewards" approach and the board decided it got abused. |
Yes, well, she voted (Tue May 8 10:59:46 2018) to reward thousands of dollars for work that is not done. I stand by my claim that this is not fair. As I said May 17, all she has to do is delete that vote:
|
@dckc says:
On Mar 31, #375, you suggested a $2000 for a work that was not yet done:
Does it mean that your action was a bad action and required slashing? |
I suggested a budget. I didn't claim a reward. |
Rather: I claimed a reward for the part that was done and available for all to see (talk given, slides shared, code written, etc.) And I made my suggestion plain for all to see an discuss, and all feedback was positive. |
@david writes on discord: |
without a consensus? Who objected? What did I miss? |
@dckc you ask that the budget be changed, on the basis of it not being fair even though you understood little to nothing as to the work done on it, and that I accepted and requested to know which budget was considered fair, but got no reply. Presently you said you even meant it being DELETED, which isn't straight forward, yet even when I asked to be cleared on your actual meaning, you sounded so sure of every word and decided giving a mute response. Although, I had to delete my votes to see what exactly the issue was, because the much biasness and high disregard for one's effort on a shallow basis was ad hominem. I don't consider it fair laying off people's honest efforts even to extent of making it a 0%, budget where is the the coops stand on equity? Wasn't even because the work wasn't done, because a fella lays an insincere comment on the basis of his unfair demands not being met?honestly it's just appalling. Please I request that issue #653 be redressed as the requirements have been met. |
I guess silence was assumed a consensus. |
Yes, well, I made a best effort to get a critical mass of support, and then I proceeded. There are different definitions of consensus, but the W3C definition is the one I have worked with the most:
|
What are the parameter for trust metric? |
Yes we tried the "everybody gets to vote on budgets and rewards" approach and the board decided it got abused. We would prefer a decentralized solution but there is no good model of that working. In most coops, such as enspiral it takes more than a year for new members to get budgetary voting power at the task/project level. If out of 48 trusted members you can get no support from any of them you should just give it up. You are not a cooperator with the active coop members if not one of them will take you on as an apprentice. On the other hand we are taking steps to improve the onboarding so that all new members can be given the chance at apprenticeship. The bounty system is on a fine line of being accepted or not by the board and Dan works tirelessly to try to keep it working honest. If it were not for him I do not believe the bounty system could have survived this long. If you have a proposal you think the board will accept for an alternative to the trust metric, do propose it. Complaining without giving a viable alternative with a team ready to implement is disruptive and counterproductive. Dan may not always be the most agreeable sort but give him a break. I believe he is sincere about helping you to make your issues compelling to the board as something that should be funded. I suggest to take his advice or engage other guides as to what is needed to be funded. You do not have to like his personality or manner. If any guide makes an accusation against you that you consider unfair all you need to do is show your work product or otherwise answer the concerns. Getting another guide involved can help resolve disputes. Do suggest improvement to the system. Find a team to implement the improvements. There is no system to my knowledge that rewards everyone fairly. People tend to disagree about what is fair. Unless there is some specific proposal I suggest this issue be closed. |
All members should have at least an apprentice vote until he or she blows it and certification should be based on vote and not on familiarity with some set of persons |
@jim why should it be closed? The unfairness in bounty trust metric is relatively high compare to the fairness and has to be address |
Allan will be entering a proposal to improve the system. Others may be the same. We know the current system is a compromise. Find your team and make it work or the bounty system may be cut off completely. To repeat. Everyone had a vote and the board squashed it. Now we have a committee of guides including as many people as we can. Anyone being cooperative will have no trouble finding a guide to certify them as an apprentice. Find your team and get working. Complaining does not solve anything. Making the best of what we have is necessary for the system to survive to evolve into something better. People tend to disagree on what is fair and that's okay. The system is decentralized enough that you will be stopped by any small set of guides. You may have to live with some delays in getting started but we are trying to minimize that and seek your help. |
I object to budgets by area that may squash or delay necessary and worthwhile projects. I suggest a liquid democracy where members can have their vote delegated to representatives on issues where they do not themselves vote. If I trust Kit and assign him as my representative then he would get his and my vote on issues I do not vote on. I expect the weighting of votes that results would be very close to the current trust metrics but who knows until we try it. It might centralize voting even further which is an outcome that concerns me. |
We have an issue on delegate / proxy voting (#484) I recently closed it with some regret...
The liquid democracy idea is something that has interested me for years. I'd love to see it happen. |
@dckc are there any parameter to justify the current trust metric? |
If you're asking why the current trust metric is in place, I have answered that to the best of my ability here and in #375 and #616 and in https://github.com/rchain/bounties/wiki/Trust-Ratings . I'm not sure I understand the question, though. |
I personally appreciate the collective and individual effort of the RAM in making attempts to checkmate bad actors and to ensure that the bounty system is working towards the vision of the RChain community. I wish to air my view on this issue. |
@dckc i mean what are the criteria used to justify those 46 person as trusted persons? |
Well, some coop members feels like the power to place budget and allocate rewards among them self has been deprived. They also think the system is no longer decentralized like it was on the first quarter of the year. They also think that the trust metric is very biased. @David405 comment
I observed that the problem is voting of reward. The rewards should be done among the collaborators or the participants of the issue. @David405 on his comment above, The votes reward awarded to @makys and @ICA3DaR5 wasn't on merit because they did not participate. Secondly, members who has SOW don't claim reward awarded to them by collaborating or participating on an issue, i suggest giving those reward to other supporting collaborator or participants. Thirdly, the trust metrics do not know the efforts of work done by other participant or collaborators of a particular issue. I do not see it fit for the trust metrics to allocate reward to the collaborators or participants on an issue, it should be done by the collaborators on how they see fit. The major reason of the trust metric is to cub the abuse of high budget placed on an issue. However, the slashing method has a way of doing that directly. When a budget is outrageously high compared to the work done, the slashing method can be used to adjust the budget appropriately. SUGGESTED PROPOSAL
|
@Keaycee your proposals are great and considerable I made similar suggestion earlier, The 'Trust metric' should have the powers to vote and adjust budgets only, while collaborators and participants should vote(rewards) on the approved budget on each issue. |
Well said!
…On Tuesday, June 5, 2018 3:14 PM, Keaycee ***@***.***> wrote:
Well, some coop members feels ...
|
@Keaycee writes:
I don't see how that's consistent with the board's Apr 6 decision (specifically steps 5, 6, and 7 of the Task Approval process regarding Project Bid Submissions); I trust you'll weigh in on #616 as to whether you think it is consistent or whether you're asking the board to reconsider their decision.
Yes, well, the whole point of slashing is to discourage unacceptable behavior. |
@jasoncruzzy suggested
I personally agreed with @jasoncruzzy . The trust metric system needs to be balanced where by those with trust metric should be the one to place budget vote and allow the creator of the issue and it's collaborators to place the reward vote. And if the the collaborators of the issue is less than three, some one with trust metric should follow the reward pattern of the 'creator' of the issue. @David405 comments
|
@Keaycee thanks for your comment and Proposal above. @dckc @kitblake @lapin7 I am wondering why our current system is designed for voters to separately vote on overall budget and then on individual contribution. As an outsider looking at a particular issue it makes sense to me to evaluate the overall impact of the work product, but I'd suppose that dividing up the reward would be primarily a job for the team to decide. Would it make sense for the issue statement to include specific allocation of reward to each contributor, and voting to be on the overall budget? Mechanistically this would probably require specific fields within the issue statement to generate the appropriate reward for each contributor (more work programming but less work voting). |
@tcezi writes:
Though you don't follow best practice in saying where you looked for the answer, I'll give you the benefit of the doubt that you read https://github.com/rchain/bounties/wiki/Trust-Ratings and #375 carefully and you still don't see an answer to that question. The answer is this part of the wiki page:
and #375 includes pointers to my slides and an Raph's academic style paper. If you're the sort that prefers to read code, the code we're running is https://github.com/dckc/rchain-dbr (currently cac6173 ). In particular, see net_flow.py which I borrowed wholesale and class TrustCert in social_coding_sync.py which calls it. |
@tony am sorry it was a mistake. Wanted to address your comment. |
@dckc On the with-holding all of my rewards based on irregularities on this #653. I find that a bit unfair, but if it is the rule. It must stand then, shouldnt it? I only found of this today and I was left confused since the other works were unrelated to this. |
I agree with @allancto that participant reward votes should count for observers working on issues. That will simplify the task for guides greatly as they would only need to vote on rewards for certain cases. The only issue is how compete the issue is and if the full budget rewarded by the team represents the actual work done. @dckc I agree with @Pionelle that withholding rewards on other issues because of disagreement on one issue is unfair and wrong. Work on each issue is a separate issue. |
@jimscarver @Pionelle but isn't that the whole idea of slashing: that you have actually something [significant] to lose for bad behavior? If people misbehave and abuse our trust, that should have more severe consequences than just not getting what you ask for. Otherwise there's nothing to lose from always asking for ridiculous rewards. The problem here is that @Pionelle is claiming tokens for bad work. Not that this governance system is broken! |
Slashing is not based on disagreement. It's based on unacceptable behavior, as judged by a quorum of the community. And having consequences beyond one issue is the whole point. You have to be fair everywhere or your whole "stake" (reward) for the pay period is at risk. (If there is more discussion on this point, I would prefer to have it in the slashing issue, #261). As noted in #653, I have removed two of my slashing votes because the respective unfair votes were removed. The process seems to be working quite well, in my experience. |
@pmoorman You could be correct. |
@allancto writes:
I don't know; it was like that when I got here. @jimscarver writes:
Please see the question I asked in my Jun 5 response to @Keaycee above about the Project Bid Submissions part of the TOS process. |
@pmoorman The issue has been resolved. The review never happened. |
As I said above on Jun 3, the most straightforward change to the existing system is to tweak the parameters for the expected number of masters, journeyers, and apprentices. Based on discussion in yesterday's RAM meeting (#758) and chatting with @jimscarver today, I did so: #375 (comment)
|
It should not be easy to discredit a person. Caring for the individual is a coop ethic. The current slashing rules seem worse than the the chinese social credit system https://www.facebook.com/NowThisFuture/videos/vb.1010847105623136/2164563946918107/?type=2&theater |
I suggest to slash per issue. Then the effects are less drastic |
This has been a useful discussion, but I don't see anyone taking the ball to follow up, so I'm closing this. I supported the $400 budget suggestion with a vote. If you seek a reward, add a reward vote for yourself and I'll consider matching it. Feel free to re-open this issue if you see more progress to be had. @Valentine-Mario it's not clear to me why you assigned yourself, but I don't see any progress in the last three weeks. Slashing@lapin7 I don't see any new information since last time that suggestion was made and rebutted by myself and @pmoorman :
@jimscarver I don't think getting a quorum of 10x weight is too easy. I don't see any cases where 10 has been too low. If you disagree, I suggest you follow up in #261. Number of apprentices etc.Note that I recently opened:
|
In 201805 pay period, the trust rating was put into place to be used for voting on budgets and rewards. Out of about 109 issues that have received votes for May pay period, about 80% of the rewards goes to
persons with trust ratings.
On issues where people with trust ratings and those without it collaborated, those with trust ratings usually have far higher percentages than the others (which is okay because voters will always seek to protect their rewards first before others).
On issues where none of the participants have trust ratings and they have to solicit for votes from people certified by the trust rating, there seems to be a lot of unfairness because the rewards votes are not fairly distributed among the participants as the voters did not partake in the task/project and so do not know who did the most work and who did a minimal job.
I understand that the Trust metric system is put in place to prevent fraud and encourage fairness but it's current state cannot accomplish this for some obvious reasons:
Humans tend to protect their interest first before any other and this is natural, so most times, during voting on collaborative projects, members with trust rating will naturally vote for themselves before any other. (This can be seen in democratic and cooperative societies)
On issues where none of the collaborators have the trust rating, it is difficult for an external voter to give a right judgement as to who gets what percentage of the reward simply because the external voter do not have an idea on the individual contributions.
Members without trust rating have to solicit for reward votes from members whose votes carry weight, sometimes these voters with trust rating take days (weeks in some cases while some never reply) before responding to their messages and then they end up voting based on their judgement and from an informed objective point of view.
Possible Amendments
I believe every member of the coop should have a right to protect his/her interest in rewards and voting. There may be people whose votes carry more weight but at least a member should be given the right to vote for himself/herself on a project he/she participated in.
Guides should be able to give suggestions as to how much budget an issue is worth.
Another possible amendment(I don't support this due to the stress involved but it can also be considered) Voters with trust rating should be tasked with voting for others without the trust rating.
This should be added to the agenda of 20180606 member meeting
Estimated Budget of Participation: $[400]
How will we measure completion? [Amendment to Trust metric system]
cc: @dckc @lapin7 @jimscarver @kitblake @ian-bloom @pmoorman @allancto and everyone
Legal
Task Submitter shall not submit Tasks that will involve RHOC being transacted in any manner that (i) jeopardizes RHOC’s status as a software access token or other relevant and applicable description of the RHOC as an “asset”—not a security— or (2) violates, in any manner, applicable U.S. Securities laws.
The text was updated successfully, but these errors were encountered: