Status Update: 2022-12-07

[haixuanTao]

🏃 🏃‍♀️

[haixuanTao]

Dora v0.1.1 Release

This release fixes some of the issue of deployment previously present in the 0.1.0.

This release contains fixes for:

• Python linking using pypi release but also a redesigned python thread model within the runtime to avoid deadlock of the GIL. This also fix an issue with patchelf.
• A deployment separation for ubuntu as the 20.04 version of dora and 22.04 version of dora are non-compatible.
• A better tagging of api for dora Rust API.

We have unfortunately removed macOS and Windows from prime support as we want to focus for Linux at the moment that is going to be used on small computer and cloud environment. Those OS can still use dora by compiling it from source.

Closed issues:

• Rust example build error when using 0.1.0 instruction  #150
• Errors when follow README for 0.1.0 release #147
• <package>.libs: cannot read file data: Is a directory when building package with release flag PyO3/maturin#1321

The release is available here:
https://github.com/dora-rs/dora/releases/tag/v0.1.1

Known Issues

• dora-coordinator has very high CPU usage
  • The dora-coordinator keeps on polling already closed connections, which can lead to high CPU load
  • Fixed in #155 (not released yet)

[haixuanTao]

Having generic Linux version might be better in the future.

[phil-opp]

Related: rust-lang/rust#57497

I think Rust always builds with the glibc version installed on the build system. Binaries compiled with older glibc versions should be compatible with newer glibc versions though, so shipping the 20.04 version should be enough.

[haixuanTao]

I see. Thanks for checking!

[haixuanTao]

So I think that we ca also include this fix in 0.1.2

[haixuanTao]

dora-drives

• Integration of dora v0.1.1 that remove the need to compile dora-rs from source.
• Integration of dora-rover logic
  • Using quaternions instead of euler angles.
  • Reduce the use of Carla API to reflect real environment scenarios where information is a lot restrained. E.g. positioning and velocity.
  • Use Frenet Optimal Planner.

[haixuanTao]

Having Carla Challenge leaderboard submission can be good.
But requires:

• Which Carla?
• Which Carla Leaderboard?
• Which Carla map or configuration?

[haixuanTao]

CentraleSupelec Master Thesis Project

Agreement in principle with school industry liaison to make dora-rs, a master thesis project for willing student.
This account for 200h of work per person as a group of 3 students.
This is for the scholar year of Sept 2023.

Other discussion of partnership with CentraleSupelec Tech Club are still on going.

[phil-opp]

Progress on dora-daemon

As discussed in the last status update, we plan to create a dora daemon executable that handles shared memory communication:

Our current progress is:

• We created a dora-daemon executable that listens for commands on a local TCP socket. Supported commands are:
  • Register: Nodes should send this message on startup to communicate their ID to the daemon
  • Subscribe: Instructs the daemon to send received inputs to the node (filtered by the node ID and the dataflow description)
  • PrepareOutputMessage: Allocate a shared memory region suitable for sending out a message. The caller can then map this region and fill in the desired data.
  • SendOutMessage: Sends out a prepared message to other nodes. Nodes on the same machine will receive this message through shared memory, so no copying occurs.
  • Stopped: Notifies the daemon that this node is exiting.
• We updated the Rust node API to use the above commands instead of communicating through iceoryx or zenoh

The next steps are:

• Parse the dataflow description in the daemon to determine which outputs should be mapped to which inputs
• Use our communication layer to send messages to non-local receivers in the daemon
• Instead of allocating a new shared memory region for each message, reuse a single, larger shared memory region
  • benchmark first, maybe new allocations are fast enough
  • requires management of allocations -> keep fragmentation down

[phil-opp]

We discussed whether we need locking (e.g. a Mutex) for the shared memory messages. Our conclusion was that locking should not be needed because the sender does not change the data anymore after sending it out. Thus, the receiving nodes can safely read the data concurrently, we just need to prevent them from modifying it.

To ensure that we don't cause any undefined behavior, we could protect the data with a RwLock. We don't expect any waiting, so we could just use the non-blocking try_* methods and panic if the lock acquisition fails unexpectedly.