Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clarify rules regarding the review committee #180

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Clarify rules regarding the review committee #180

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b647d53
Reword the review structure for fairness
ShriyaPalsamudram Jul 5, 2024
bf0a364
Minor updates
ShriyaPalsamudram Jul 11, 2024
835d47a
Update the rule with feedback from Training WG + head of MLPerf
ShriyaPalsamudram Jul 12, 2024
799a469
Updates based on Training WG discussion on 7/18/24
ShriyaPalsamudram Jul 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions submission_rules.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,19 +28,20 @@ The MLPerf review and publication process is overseen by a review committee.

The review committee consists of the following people:

* The relevant working group chairs
* The relevant power working group chairs
* Optionally, the review chair of relevant working group
* The executive director
* The president of the board
* Representatives of each submitter
* The review chairs of relevant working group
* The head of MLPerf
* The president of the board
* The relevant working group chairs †
* The relevant power working group chairs †

If representatives of a single organization, its parents, subsidiaries, or
affiliates hold multiple positions on this list, at most one such representative
is eligible to vote in the review committee.

The review committee is chaired by the relevant results working group chairs
unless a review chair is selected.
The review committee is setup before submission and is driven by review chairs who are selected among a group of candidates whose candidacy is approved by unanimous consent among members who indicate intent to submit in the submission survey of the relevant round. An impartial MLC staff member familiar with the rules and subject of the relavant working group can be a valid candidate. The relevant working group chair or power working group chair may be a valid candidate as well if approved by unanimous consent. If no candidates can be identified as acceptable by unanimous consent, the head of MLPerf must make best effort to appoint one, or fulfill the role themselves.

Any submitter may request that an individual serving a role demarcated by † to abstain from the review committee if the individual is deemed by the submitter to be associated with a competitor and the relavant working group along with the head of MLPerf must make best effort to address the request.

### Meetings

Expand Down Expand Up @@ -78,7 +79,7 @@ from the following list who are able and willing to vote:

If there are two outcomes, voting proceeds by simple majority. If there are more
than two outcomes, voting proceeds by Condorcet poll with the minimax completion
rule. Votes are initiated by the chair and are cast openly. Votes may be cast
rule. Votes are initiated by the review chair and are cast openly. Votes may be cast
verbally or using a shared spreadsheet or other voting software.

The review committee operates on balance of interests rather than by avoiding
Expand Down
Loading