Add computation of size contribution to info verb

[nicolaloi]

Addresses #1601 feature request (Show size contribution of each topic with ros2 bag info). This is a test draft.

Output of ros2 bag info test_bag -v --size-contribution:

Files:             test_bag.mcap
Bag size:          2.6 GiB
Storage id:        mcap
ROS Distro:        unknown
Duration:          39.541s
Start:             Jun 19 2024 23:40:38.345 (1718833238.345)
End:               Jun 19 2024 23:41:17.886 (1718833277.886)
Messages:          2310
Topic information: Topic: /parameter_events | Type: rcl_interfaces/msg/ParameterEvent | Count: 0 | Serialization Format: cdr
                   Topic: /velodyne_points | Type: sensor_msgs/msg/PointCloud2 | Count: 765 | Size Contribution: 557.0 MiB | Serialization Format: cdr
                   Topic: /events/read_split | Type: rosbag2_interfaces/msg/ReadSplitEvent | Count: 0 | Serialization Format: cdr
                   Topic: /image_raw_1 | Type: sensor_msgs/msg/Image | Count: 763 | Size Contribution: 1.0 GiB | Serialization Format: cdr
                   Topic: /image_raw_2 | Type: sensor_msgs/msg/Image | Count: 763 | Size Contribution: 1.0 GiB | Serialization Format: cdr
                   Topic: /events/write_split | Type: rosbag2_interfaces/msg/WriteSplitEvent | Count: 0 | Serialization Format: cdr
                   Topic: /rosout | Type: rcl_interfaces/msg/Log | Count: 19 | Size Contribution: 4.9 KiB | Serialization Format: cdr
Service:           0
Service information:

The total size contribution of each topic is computed by sequentially reading every message in the rosbag. As mentioned in #1601, this approach can be slow.

In a first test A (previous output), the computation of the size contribution took ~520ms for a bag with 2310 messages, 2.6GiB large, 39.5s long, with two sensor_msgs/msg/Image topics with 763 messages each, and one sensor_msgs/msg/PointCloud2 topic with 765 messages.

In a second test B, the size computation took ~78ms for a bag with 14577 messages, 5.2MiB large, 74.4s long, with one sensor_msgs/msg/Imu topic with 14559 messages.

So to me, it seems the computation time is not simply dependent on the number of messages, but on their types and individual sizes too. As a test, I was curious about skipping the message serialization step inside the reader. I tried a brute modification of the mcap_storage interface to directly access the messageView.message.dataSize variable while skipping the message serialization step. However, this only resulted in a 15-30% improvement in computation time.

As a test, removing the actual size computation code and just reading the rosbag in an empty while loop doesn't really change the timing (only real difference was in test B, 78ms->68ms).

In rosbag2_py/_info.cpp, the new function compute_topics_size_contribution can be combined with the existing read_service_info to avoid reading the rosbag twice.

@MichaelOrlov looking forward to your feedback and suggested changes.

[MichaelOrlov]

@nicolaloi Thanks for your contribution.
Without deserialization, the message's size calculation per topic will be pretty fast in the vast majority of cases. Perhaps no more than a few seconds. I think adding a separate CLI argument like --size-contribution doesn't make sense. It will be much simpler to make output for topics size contribution by default for the --verbose CLI option.

See my detailed comments and suggestions inline.

[nicolaloi]

Addressed review comments, added size contribution for services, and updated tests and design doc.

[MichaelOrlov]

@ros-pull-request-builder retest this please

[nicolaloi]

I will edit the file with the style divergence error. I have missed it since locally the tests no longer gave me any errors.

[MichaelOrlov]

@nicolaloi Thanks for addressing the review comments.
Overall, it looks good among a few nitpicks, mainly related to using const references.

[MichaelOrlov]

Thanks looks good to me if CI will pass green.

[MichaelOrlov]

@nicolaloi The Rpr job failed because need to make rebase.
Please rebase your branch on top of latest rolling branch.

[nicolaloi]

@MichaelOrlov I have rebased the branch and I have updated the new size contribution test I have added to match the latest changes in rolling, but now the Rpr test is failing due to an unrelated rcpputils::fs issue, as in the latest rolling merged PR #1740 (log L32773)

[MichaelOrlov]

@nicolaloi No worries. Those test failures are unrelated to the changes from this PR and test failing because the CI job from GitHub actions uses binaries for the other dependent packages from the latest release on rolling.
The test failure will go away when we will release a new version of the rcpputils package and do rolling synch.

Anyway, I will run CI jobs on the ROS 2 build farm, which compiles all packages from sources. If everything is correct, all tests should pass.

[MichaelOrlov]

Pulls: #1726
Gist: https://gist.githubusercontent.com/MichaelOrlov/5179a9d2b5bc8685c059758e5d9397de/raw/af0dc6b9cdc1954a535c28a5eeb3e5f43c031848/ros2.repos
BUILD args: --packages-above-and-dependencies ros2bag rosbag2_cpp rosbag2_py rosbag2_tests
TEST args: --packages-above ros2bag rosbag2_cpp rosbag2_py rosbag2_tests
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/14281

• Linux
• Linux-aarch64
• Linux-rhel
• Windows