Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix setup issues and improved documentation #39

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions doc/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,32 @@ The validation output reads for the other fields the same way as for the example
[0, 2717] StationaryObject.base.is_valid(None) does not comply in SensorView.global_ground_truth.stationary_object.base
[0, 2717] BaseStationary.base_polygon.is_set(None) does not comply in SensorView.global_ground_truth.stationary_object.base

Writing an OSI trace file
--------------------------

As already mentioned in the example, the OSI validator works with OSI trace files, which are a collection of, to byte code, serialized OSI messages. Upon start, the
OSI validator splits the trace file into the individual messages and validates them. This splitting is done slightly different for
the two allowed trace file formats .txt and .osi.

For a .txt trace file, the osi validator splits the messages by searching ``$$__$$`` separators within the file. To generate a valid
.txt trace file the file therefore has to start with a serialized OSI message, followed by a ``$$__$$`` separator (Note that the last message
also has to be followed by a separator!). Schematically, a .txt trace file would look like this:

.. code-block:: bash

[OSI Message]$$__$$[OSI Message]$$__$$[OSI Message]$$__$$...$$__$$

The .osi trace files are quite similar, but instead of a separator at the end of each message, each message is preceded by a 4-byte
integer (in byte code) that contains the length (in byte) of the following serialized OSI message. So a .osi trace file would look
like this:

.. code-block:: bash

[integer 1250 in byte][1250 byte long osi message][integer 3000 in byte][3000 byte long osi message]....

It has to be ensured that the validated trace files follow this convention, otherwise the OSI validator will not be able to
validate them correctly.

Custom Rules
--------------

Expand Down
2 changes: 1 addition & 1 deletion osivalidator/osi_trace.py
Original file line number Diff line number Diff line change
Expand Up @@ -320,7 +320,7 @@ def get_messages_in_index_range(self, begin, end):
for rel_index, rel_message_offset in enumerate(rel_message_offsets):
rel_begin = rel_message_offset + self._int_length
rel_end = (
rel_message_offsets[rel_index + 1] - self._int_length
rel_message_offsets[rel_index + 1]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anscheinend hat protobuf updates bekommen. Ich habs lokal getestet und es funktioniert, wenn man die subtraktion weglässt wie hier. Am besten man setzte auch für den osi-validator github actions auf.

if rel_index + 1 < len(rel_message_offsets)
else message_sequence_len
)
Expand Down
3 changes: 1 addition & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,11 @@
"sphinx_rtd_theme",
"recommonmark",
"open-simulation-interface",
"doxygen-interface",
"defusedxml",
"colorama",
"tabulate",
"progress",
"protobuf==3.9.1",
"protobuf",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Die Begründung (aus unserer internen Kommunikation):
Es sieht so aus, als hätte der osi-validator da ein größeres Problem mit Protobuf3: viele der Checks prüfen, ob ein Feld gesetzt ist, aber in proto3 kann man gar nicht mehr feststellen, ob eine Feld nicht gesetzt wurde, oder den Default-Wert hat.

Beispiel:

TrafficSign.MainSign.Classification.vertically_mirrored.is_set(None) does not comply in SensorView.global_ground_truth.traffic_sign.main_sign.classification

vertically_mirrored ist ein bool. Wenn der Wert bei der Generierung auf false gesetzt wird (der Default), dann wird er in proto3 nicht übertragen.

Damit ist ein größerer Teil der osi-validator checks im Zusammenhang mit proto3 hinfällig.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ich erinnere mich es gab vom osi-validator ein Problem bezüglich anderer protobuf Versionen. Deshalb habe ich auch explizit die funktionierende Version verwendet. Wie im Kommentar oben erwähnt würde es sich lohnen gh_actions für osi-validator aufzusetzen. Bevor man den PR merged um die test von dem PR zum laufen zu bringen.

],
dependency_links=[
"git+https://github.com/OpenSimulationInterface/"
Expand Down