[EdgeTPU Backend] Organization of this project

[hohee-hee]

WHY

EdgeTPU is a processing unit which is optimized for tensor operation. It supports faster inference of deep learning models in on-device environment. However, it can’t inference every type of models. Only .tflite model that is already compiled with EdgeTPU Compiler can be executed on EdgeTPU. Therefore, EdgeTPU can run inference a TensorFlow Lite Model that is compiled with EdgeTPU Compiler provided by Google Coral.

Currently, however, the compiler is only performed in the CLI environment so it cannot provide a better developer experience. So we are going to add a new vscode extension to do above process on ONE-vscode which consists of GUI environment. Through this project, users are expected to use EdgeTPU more intuitively and conveniently.

WHAT

This project is to make vscode extension to use EdgeTPU compiler, which supports faster inference of deep learning model in on-device environment, in ONE conveniently.

Functions

1. Implementation of Backend Interface for EdgeTPU

   The main function of this project is to implement a Backend Interface for EdgeTPU Compiler in ONE-vscode. The goal is to provide an environment where users can install EdgeTPU compiler with clicking a button instead of entering command line. In addition, the extension has a capability to convert standard tflite model to EdgeTPU’s tflite model without entering CLI commands.

2. Visualization EdgeTPU’s tflite model in ONE Explorer

   One of the most important elements of extension of vscode is to visualize all models and files with tree structure. In this case of EdgeTPU, model files are compiled to tflite format but this appears same with general tflite models. Therefore, our new extension have to support visualization after considering how to distinguish naturally those two types of tflite models(general and EdgeTPU’s).

HOW

Mockup

1. on Toolchain View
   1. Support to install EdgeTPU Compiler with command palette menu
   2. Support to set Default Toolchain with EdgeTPU Compiler
   3. Support to uninstall EdgeTPU Compiler on Toolchain
2. on ONE Explorer
   1. show .tflite by Base Model
   2. after clicking the file, config window will be shown
   3. choose some compile options for EdgeTPU Compiler on the window
   4. with play button, compile the Base Model file with EdgeTPU Compiler
   5. show a compiled tflite file for EdgeTPU in the sub part of the cfg file
3. on Console
   1. print the same messages with Linux Console like Compiled Result, Error Message or Log

Some part of the HOW had to be discussed but there are some ambiguous parts to design the function. So we need some discussion about those parts like division of general tflite model and compiled tflite model with EdgeTPU or hiding some unnecessary steps. We are planning to create a new issue about it, so we would appreciate it if you could discuss those parts with us. Thank you.

RESULT

1. A GUI environment including explorer and config to execute EdgeTPU Compiler in ONE-vscode
2. A vscode extension for EdgeTPU Compiler modularized and isolated with ONE-vscode

Expectation

1. Because external compiler can be executed in ONE-vscode so it can support a seamless performance test.
2. People who are not familiar to CLI environment can use EdgeTPU Compiler in GUI environment via ONE-vscode.

[dayo09]

Thanks! 🎉
Addition of backend will help this project expanding to support many AI toolchains.

[dayo09]

Final Result

Well done ;-D