Skip to content

Commit

Permalink
Merge pull request #28 from mlcommons/peladodigital--R&D-SoC-update
Browse files Browse the repository at this point in the history
Update mobile_inference_rules.adoc
  • Loading branch information
Mostelk committed Aug 9, 2023
2 parents 680ec88 + 71c9ca8 commit 53737c4
Showing 1 changed file with 13 additions and 3 deletions.
16 changes: 13 additions & 3 deletions rules/mobile_inference_rules.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -80,12 +80,22 @@ but excludes rooted devices.

If you are measuring the performance of an experimental framework or system, you
must make the system and framework you use available upon demand for
replication by MLCommons. This class of systems will be called RDI (research,
development, internal). RDI systems include reference hardware and software
replication by MLCommons. This class of systems will be called R&D SoC. R&D SoC systems include reference hardware and software
modifications such as rooted phones.

R&D SoC will be considered as any physical device including SoC under test, where the device is not commercially available, and the SoC under test is currently not commercially available, but can be made commercially available at some point in the future.

R&D SoC submitter should submit that SoC under Available category with a commercially available device in the next official submission after the commercial device is available.

If submitter fails to do this submission, original R&D SoC submission will be marked as invalid.
The R&D SoC can be optionally integrated, depending on the vendor’s willingness, into the backend of the MLPerf app. Any other member can verify this integration giving credibility to this system.

In order to be published R&D SoC submissions will be audited with the backend code and the respective logs by examining any modifications to the backend code integrated in the MLPerf app. R&D SoC submitters can also optionally send their engineering R&D device to MLCommons to reproduce their results

R&D SoC submitter should submit that SoC under Available category with a commercially available device in the next official submission after the commercial device is available

For Available Systems, devices used for testing should be publicly available as
commercial devices and device rooting is not allowed. For RDI Systesm, hardware and
commercial devices and device rooting is not allowed. For R&D SoC Systems, hardware and
software changes such as rooting should be explicitly mentioned in the device name.

=== Benchmark implementations must be shared
Expand Down

0 comments on commit 53737c4

Please sign in to comment.