Status Update: 2022-08-31

[haixuanTao]

Status Update! 🚀

[haixuanTao]

Dora-Drives New Operators! 🤯

• yolop
  • 
  • 

• strong_sort
  • 
  • Strong sort took a bit of time due to not well maintained Open Source Implementation https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet , I had to extract the code and make it a package -> https://github.com/haixuanTao/yolov5_strongsort_package . This package will be an example on how to package a library in python and then import it in dora

[haixuanTao]

Investigating Operators

• Traffic sign recognition
  • Investigating: https://github.com/Kunkun-Jia/TSR-SA
• Trajectory Prediction (pedestrian and vehicles)
  • Investigating: https://github.com/ZikangZhou/HiVT
• Motion Planning

[haixuanTao]

• Bob: Let's see if we can put on real car.

[haixuanTao]

-> Lidar: For global depth estimation

[haixuanTao]

-> Control: Real IMU. ROS Topic. GNSS

[haixuanTao]

Docker or Natively? Dora-drives can definitely be run natively on Linux Ubuntu. Docker is just for sharing at the moment. Docker is essentially to contain Carla Simulator. If we remove the carla simulator, we will have less dependencies, and will make it easier to use natively.
Hardware Requirement? Weights can be tweaked to fit the GPU Memory and capabilities.

[haixuanTao]

Dora Core

Support for Windows and macOS

• Automatically add the shared library prefix and extension based on the platform
• Assume a .exe extension for all executables on Windows
• Figure out the correct flags for linking C and C++ nodes and operators on all platforms
• Run tests and examples for all three platfoms on the CI

New higher-level shared library API for operators

All types that pass the FFI-boundary between the runtime and operators compiled as shared libraries must be FFI-safe. One requirement for that is that the types have a specified layout in memory. As a result, we cannot use most of Rust's higher-level types and have to use raw pointers instead.

Previously, we defined this raw pointer API manually, which resulted in quite complex code. It was difficult to add new fields such as additional metadata. It was also difficult to maintain because we had to manually verify that the Rust API crate, the C header file, and the runtime stayed in sync.

To improve this situation, we rewrote the operator API in the past two weeks based on the safer-ffi crate. The crate features FFI-safe versions of common standard library types such as String or Vec, which allowed us to simplify the API significantly. By adding custom input and output structures, we are now prepared for adding additional message metadata in the future.

We also created a separate sub-crate to hold all the type information. This way, we can reuse the same definitions in the operator API and the runtime, thereby preventing dangerous type mismatch errors. We also autogenerate a C-header file based on these definitions to improve safety for operators written in C as well.

Plan: Use Iceoryx to improve IPC performance

Right now we use zenoh for all communication. While zenoh uses different techniques to optimize performance for both local and remote receivers, it still has a considerable overhead in many situations. Since we're only support local deployments in dora right now, we plan to evaluate Eclipse iceoryx™ as an alternative communication layer.

Iceoryx implements zero-copy IPC by passing messages via shared memory. The memory is handled by a central deamon, which makes it possible to send a message to multiple receivers without copying. Like zenoh, iceoryx provides a publish/subscribe-based interface, so the migration should be fairly simple.

If the iceoryx implementation turns out to be significantly faster, we will try to create a hybrid communication layer next. The idea is to select the communication backend based on the receiver's location. This would allow us to automatically use iceoryx for local nodes, while still supporting communication with remote nodes through zenoh.

Plan: Switch communication to synchronous API

Asynchronous Rust still has some rough edges. For example, the ecosystem is split between different async runtimes and the language is still missing support for async functions in traits. This leads to some additional complexity in our communication layer code and makes it more difficult to add support for other backends (e.g. iceoryx or mqtt). Because of this we decided to convert the communication layer trait to a synchronous trait for now. We don't expect that this will affect performance much.

Making the communication layer synchronous also affects the Rust node API, which features an asynchronous input stream right now. Our current plan is to migrate to a channel-based interface instead. The operator interface and the node interface for other nodes is already synchronous, so no changes are expected there.