From b647d53d55592e4fcea1b39a7e0dafd513870057 Mon Sep 17 00:00:00 2001 From: Shriya Palsamudram Date: Fri, 5 Jul 2024 09:43:33 -0700 Subject: [PATCH 1/4] Reword the review structure for fairness --- submission_rules.adoc | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/submission_rules.adoc b/submission_rules.adoc index a8e5cd0..f775ad3 100644 --- a/submission_rules.adoc +++ b/submission_rules.adoc @@ -28,19 +28,20 @@ The MLPerf review and publication process is overseen by a review committee. The review committee consists of the following people: -* The relevant working group chairs -* The relevant power working group chairs -* Optionally, the review chair of relevant working group -* The executive director -* The president of the board * Representatives of each submitter +* The review chair of relevant working group +* The relevant working group chairs † +* The relevant power working group chairs † +* The executive director † +* The president of the board † If representatives of a single organization, its parents, subsidiaries, or affiliates hold multiple positions on this list, at most one such representative is eligible to vote in the review committee. -The review committee is chaired by the relevant results working group chairs -unless a review chair is selected. +The review committee is chaired by the review chair, who should be selected among a group of candidates whose candidacy is approved by unanimous consent among the members who indicate intent to submit in the submission survey of the relevant round. The relevant working group chair or power working group chair may be a valid candidate as the review chair if approved by unanimous consent. If no candidates can be identified as acceptable by unanimous consent, the executive director must make best effort to appoint one from MLC staff who is best qualified in terms of impartiality and familiarity with the rules and subject of the relevant working group, or fulfill the role themselves. + +Any submitter may request that any individual serving a role demarcated by † to abstain from the review committee if the individual is deemed by the submitter to be associated with a competitor and the individual must abide by the request. ### Meetings @@ -78,7 +79,7 @@ from the following list who are able and willing to vote: If there are two outcomes, voting proceeds by simple majority. If there are more than two outcomes, voting proceeds by Condorcet poll with the minimax completion -rule. Votes are initiated by the chair and are cast openly. Votes may be cast +rule. Votes are initiated by the review chair and are cast openly. Votes may be cast verbally or using a shared spreadsheet or other voting software. The review committee operates on balance of interests rather than by avoiding From bf0a3649adc393d04e6dbd8879295098b7cb7ad1 Mon Sep 17 00:00:00 2001 From: Shriya Palsamudram Date: Thu, 11 Jul 2024 07:08:29 -0700 Subject: [PATCH 2/4] Minor updates --- submission_rules.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/submission_rules.adoc b/submission_rules.adoc index f775ad3..c666d3f 100644 --- a/submission_rules.adoc +++ b/submission_rules.adoc @@ -29,17 +29,17 @@ The MLPerf review and publication process is overseen by a review committee. The review committee consists of the following people: * Representatives of each submitter -* The review chair of relevant working group +* The review chairs of relevant working group +* The executive director +* The president of the board * The relevant working group chairs † * The relevant power working group chairs † -* The executive director † -* The president of the board † If representatives of a single organization, its parents, subsidiaries, or affiliates hold multiple positions on this list, at most one such representative is eligible to vote in the review committee. -The review committee is chaired by the review chair, who should be selected among a group of candidates whose candidacy is approved by unanimous consent among the members who indicate intent to submit in the submission survey of the relevant round. The relevant working group chair or power working group chair may be a valid candidate as the review chair if approved by unanimous consent. If no candidates can be identified as acceptable by unanimous consent, the executive director must make best effort to appoint one from MLC staff who is best qualified in terms of impartiality and familiarity with the rules and subject of the relevant working group, or fulfill the role themselves. +The review committee is driven by the review chairs, one of who is an MLC staff member. The other should be selected among a group of candidates whose candidacy is approved by unanimous consent among members who indicate intent to submit in the submission survey of the relevant round. The relevant working group chair or power working group chair may be a valid candidate as the review chair if approved by unanimous consent. If no candidates can be identified as acceptable by unanimous consent, the executive director must make best effort to appoint one from MLC staff who is best qualified in terms of impartiality and familiarity with the rules and subject of the relevant working group, or fulfill the role themselves. Any submitter may request that any individual serving a role demarcated by † to abstain from the review committee if the individual is deemed by the submitter to be associated with a competitor and the individual must abide by the request. From 835d47ad1a9d674f63f930de57bf7c18e186b2b8 Mon Sep 17 00:00:00 2001 From: Shriya Palsamudram Date: Fri, 12 Jul 2024 09:44:50 -0700 Subject: [PATCH 3/4] Update the rule with feedback from Training WG + head of MLPerf --- submission_rules.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/submission_rules.adoc b/submission_rules.adoc index c666d3f..bb7d6a9 100644 --- a/submission_rules.adoc +++ b/submission_rules.adoc @@ -30,7 +30,7 @@ The review committee consists of the following people: * Representatives of each submitter * The review chairs of relevant working group -* The executive director +* The head of MLPerf * The president of the board * The relevant working group chairs † * The relevant power working group chairs † @@ -39,9 +39,9 @@ If representatives of a single organization, its parents, subsidiaries, or affiliates hold multiple positions on this list, at most one such representative is eligible to vote in the review committee. -The review committee is driven by the review chairs, one of who is an MLC staff member. The other should be selected among a group of candidates whose candidacy is approved by unanimous consent among members who indicate intent to submit in the submission survey of the relevant round. The relevant working group chair or power working group chair may be a valid candidate as the review chair if approved by unanimous consent. If no candidates can be identified as acceptable by unanimous consent, the executive director must make best effort to appoint one from MLC staff who is best qualified in terms of impartiality and familiarity with the rules and subject of the relevant working group, or fulfill the role themselves. +The review committee is driven by review chairs who are selected among a group of candidates whose candidacy is approved by unanimous consent among members who indicate intent to submit in the submission survey of the relevant round. An impartial MLC staff member familiar with the rules and subject of the relavant working group can be a valid candidate. The relevant working group chair or power working group chair may be a valid candidate as well if approved by unanimous consent. If no candidates can be identified as acceptable by unanimous consent, the head of MLPerf must make best effort to appoint one, or fulfill the role themselves. -Any submitter may request that any individual serving a role demarcated by † to abstain from the review committee if the individual is deemed by the submitter to be associated with a competitor and the individual must abide by the request. +Any submitter may request that an individual serving a role demarcated by † to abstain from the review committee if the individual is deemed by the submitter to be associated with a competitor and the relavant working group along with the head of MLPerf must make best effort to address the request. ### Meetings From 799a4692aff30d869d883eab161459b4821e85ed Mon Sep 17 00:00:00 2001 From: Shriya Palsamudram Date: Thu, 18 Jul 2024 16:58:01 -0700 Subject: [PATCH 4/4] Updates based on Training WG discussion on 7/18/24 --- submission_rules.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/submission_rules.adoc b/submission_rules.adoc index bb7d6a9..66c1c85 100644 --- a/submission_rules.adoc +++ b/submission_rules.adoc @@ -39,7 +39,7 @@ If representatives of a single organization, its parents, subsidiaries, or affiliates hold multiple positions on this list, at most one such representative is eligible to vote in the review committee. -The review committee is driven by review chairs who are selected among a group of candidates whose candidacy is approved by unanimous consent among members who indicate intent to submit in the submission survey of the relevant round. An impartial MLC staff member familiar with the rules and subject of the relavant working group can be a valid candidate. The relevant working group chair or power working group chair may be a valid candidate as well if approved by unanimous consent. If no candidates can be identified as acceptable by unanimous consent, the head of MLPerf must make best effort to appoint one, or fulfill the role themselves. +The review committee is setup before submission and is driven by review chairs who are selected among a group of candidates whose candidacy is approved by unanimous consent among members who indicate intent to submit in the submission survey of the relevant round. An impartial MLC staff member familiar with the rules and subject of the relavant working group can be a valid candidate. The relevant working group chair or power working group chair may be a valid candidate as well if approved by unanimous consent. If no candidates can be identified as acceptable by unanimous consent, the head of MLPerf must make best effort to appoint one, or fulfill the role themselves. Any submitter may request that an individual serving a role demarcated by † to abstain from the review committee if the individual is deemed by the submitter to be associated with a competitor and the relavant working group along with the head of MLPerf must make best effort to address the request.