[RFC] Arm Ethos-U Integration #11
Conversation
This commit adds markdown for Arm Ethos-U Integration into TVM along with the diagrams used within the RFC. Change-Id: I56b8918628f93aadfc08068a4adefaf880664ca7
* fixing a typo in a link Change-Id: I9ab14c366f2c39ba36b8a43a83e4f15ea309712e
* adding the header for the RFC * modifying the paths to match the PR number Change-Id: Ice1c651c6619ae5d6a95fa70e739826b36a47af1
*fixing the path to assets with the PR number *adding a white bounding box to compiler_arch to render in a black github interface for better viewing experience Change-Id: If52e34583cbd81614f0ba176a6fce8f440d8041c
There was a problem hiding this comment.
hi @manupa-arm thanks for the RFC! left a round of comments, mostly clarifying things for now.
rfcs/0011_Arm_Ethos-U_Integration.md
Outdated
|
|
||
| ### C3. Ethos™-U TE/TIR Compiler Passes | ||
|
|
||
| At this stage, we should have a TE representation of all HW primitive operations that belong to the offloaded function. We will be scheduling the TE representation to TIR Primfunc that describes the intermediary storage and hardware operations that needed to be executed. In future, we are intending to add more TE/TIR passes make the Ethos™-U TE/TIR compiler perform memory and performance optimizations (See https://discuss.tvm.apache.org/t/rfc-cascade-scheduling/8119) . Therefore, its vital to have all the operations represented in TE/TIR. Its important to note that Ethos™-U hardware requires weights to be 'encoded' in a certain way to be readable by the hardware. Therefore, the weight encoding is performed here and represented in the TIR primfunc with post-encoding sizes as buffers. |
There was a problem hiding this comment.
could you clarify "We will be scheduling"--you're saying that this stage will schedule TE operators into TIR PrimFunc?
|
|
||
| Arm® Ethos™-U is a series of NPUs that will enable low-cost and highly efficient AI solutions for a wide range of embedded devices. This RFC introduces the port of Ethos-U into the uTVM compilation flow. The process of compilation relies on the multiple levels of abstraction in TVM and a variety of analysis and optimisation passes to produce c output. In the process of compilation, we rely on the many levels of TVM's IR (and the passes) to perform optimizations to create c-sources that can work with current microTVM deployments. | ||
|
|
||
| ## Scope: |
There was a problem hiding this comment.
could you explain the intent behind this RFC, so readers know whether it's comprehensive or whether to expect follow-on RFCs?
There was a problem hiding this comment.
I think its explained : "The scope for this RFC is to add support for offloading to the Arm® Ethos™-U55 NPU. The initial machine learning framework that we use for testing this is TensorFlow Lite. Future RFCs and pull requests will address additional NPUs such as the Ethos™-U65, optimization to the compilation pipeline and other frameworks as the port evolves."
* changed uNPU --> Ethos™-U Change-Id: I9b869396fef66772e70adc8cc90538434cc090ac
*addressed andrew's comments Change-Id: Id83f1d85d1105b524afd6fa33efa1051239edb03
|
Sorry for the delay! I addressed your comments and anwsered the questions. @areusch |
* further addressing comments Change-Id: I01c8f5721fc1aa650479b6d2fe4e0abde80b1de3
|
|
||
| ## Testing | ||
|
|
||
| Firstly, we will be providing unit tests for the components described above. | ||
|
|
||
| Secondly, we are planning to use Arm® Corestone™-300 Fixed Virtual Platform (FVP – https://developer.arm.com/ip-products/subsystem/corstone/corstone-300) in the CI to be able to simulate the codegen'd artifacts of TVM on a SoC that has Arm® Cortex™-M55 and Ethos™-U55. | ||
| Secondly, we are planning to use [Arm® Corstone™-300 reference system](https://developer.arm.com/ip-products/subsystem/corstone/corstone-300) in the CI to be able to simulate the codegen'd artifacts of TVM on a SoC that has Arm® Cortex™-M55 and Ethos™-U55. |
There was a problem hiding this comment.
Will there ever be plans to provide third party CI for testing on a hardware cloud?
There was a problem hiding this comment.
cc : @u99127, However I feel this question is out of scope for the RFC.
There was a problem hiding this comment.
Thanks @manupa-arm
We currently do not have any plans for adding hardware into a cloud. Ethos-U is licensed as a part of Arm’s IP portfolio. I expect SoCs and boards containing Ethos-U from the broader Arm ecosystem in due course but I’m not at liberty to speculate on time lines.
As part of the Ethos-U upstreaming work and the first PR that has been merged for docker images, Arm has contributed a reference system, what we call Fixed Virtual Platforms for the Corstone-300 subsystem in the TVM CI. We believe this is sufficient for continually testing the correctness of the Ethos-U port and indeed for testing correctness on Cortex-M.
The value of hardware based correctness testing is debatable beyond testing that comes from the simulator and is beyond the scope of this RFC or Pull Requests.
*defined acronyms *fixed spelling convention *reffered to tracking issue for the upstreaming plan Change-Id: I623edc7171c21bfe65509d89422698ff7959086c
|
Hi all, I think all the comments are addressed now. |
Fix code snippets
[uTVM] This commit adds markdown for Arm Ethos-U Integration into TVM along with the diagrams used within the RFC.
cc : @areusch @mbaret @tqchen