Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor(perception_utils): add test in object_classification #2083

Merged
merged 12 commits into from
Nov 9, 2022
Merged

refactor(perception_utils): add test in object_classification #2083

Changes from 4 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
e678ec1
add test for isVehicle
YoshiRi Oct 15, 2022
a179ee3
add isCarLikeVehicle test
YoshiRi Oct 15, 2022
6d211d3
add isLargeVehicle test
YoshiRi Oct 15, 2022
86c4948
fix namespace range
YoshiRi Oct 17, 2022
695c7b5
ci(pre-commit): autofix
pre-commit-ci[bot] Oct 17, 2022
e1d667e
remove unnecessary forloop
YoshiRi Oct 17, 2022
8e74090
Merge branch
YoshiRi Oct 17, 2022
ba48343
fix cpplint message
YoshiRi Oct 17, 2022
7bdb3a3
added edge case to iscarlikevehicle
YoshiRi Oct 17, 2022
c1182c2
run precommit
YoshiRi Oct 17, 2022
62e8f55
add possibe edge cases with comment
YoshiRi Oct 17, 2022
f4689cb
Merge branch 'main' into refactor/add_test_perception_util
YoshiRi Oct 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 157 additions & 0 deletions common/perception_utils/test/src/test_object_classification.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,160 @@ TEST(object_classification, test_getHighestProbLabel)
EXPECT_EQ(label, ObjectClassification::CAR);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

pre-commit failed. Don't you know how to fix it on a github browser or locally with a command?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be due to unnecessary line break in the code.
I fixed it in local and push later.

Copy link
Contributor

@takayuki5168 takayuki5168 Oct 17, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant there is an auto-fix command in both github browser and local command.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I forgot pre-commit run -a command before git push.
(But I still had to fix some error manually)

Copy link
Contributor

@takayuki5168 takayuki5168 Oct 17, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it.

pre-commit install which enables running pre-commit every time we run git commit is also useful.

}
}

// Test isVehicle
TEST(object_classification, test_isVehicle)
{

using autoware_auto_perception_msgs::msg::ObjectClassification;
using perception_utils::isVehicle;

{// True Case with uint8_t
std::vector<std::uint8_t> vehicle_labels;
vehicle_labels.push_back(ObjectClassification::BICYCLE);
vehicle_labels.push_back(ObjectClassification::BUS);
vehicle_labels.push_back(ObjectClassification::CAR);
vehicle_labels.push_back(ObjectClassification::MOTORCYCLE);
vehicle_labels.push_back(ObjectClassification::TRAILER);
vehicle_labels.push_back(ObjectClassification::TRUCK);

for(auto label: vehicle_labels){
EXPECT_TRUE(isVehicle(label));
}
}

// False Case with uint8_t
{
std::vector<std::uint8_t> non_vehicle_labels;
non_vehicle_labels.push_back(ObjectClassification::UNKNOWN);
non_vehicle_labels.push_back(ObjectClassification::PEDESTRIAN);
for(auto label: non_vehicle_labels){
EXPECT_FALSE(isVehicle(label));
}
}

// True Case with object_classifications
{ // normal case
std::vector<autoware_auto_perception_msgs::msg::ObjectClassification> classification;
classification.push_back(createObjectClassification(ObjectClassification::CAR, 0.5));
classification.push_back(createObjectClassification(ObjectClassification::TRUCK, 0.8));
classification.push_back(createObjectClassification(ObjectClassification::BUS, 0.7));
EXPECT_TRUE(isVehicle(classification));
}

// False Case with object_classifications
{ // false case
std::vector<autoware_auto_perception_msgs::msg::ObjectClassification> classification;
classification.push_back(createObjectClassification(ObjectClassification::PEDESTRIAN, 0.8));
classification.push_back(createObjectClassification(ObjectClassification::BICYCLE, 0.7));
EXPECT_FALSE(isVehicle(classification));
}

} // TEST isVehicle


// TEST isCarLikeVehicle
TEST(object_classification, test_isCarLikeVehicle)
{

using autoware_auto_perception_msgs::msg::ObjectClassification;
using perception_utils::isCarLikeVehicle;

{// True Case with uint8_t
std::vector<std::uint8_t> carlike_labels;
carlike_labels.push_back(ObjectClassification::BUS);
carlike_labels.push_back(ObjectClassification::CAR);
carlike_labels.push_back(ObjectClassification::TRAILER);
carlike_labels.push_back(ObjectClassification::TRUCK);

for(auto label: carlike_labels){
EXPECT_TRUE(isCarLikeVehicle(label));
}
takayuki5168 marked this conversation as resolved.
Show resolved Hide resolved
}

// False Case with uint8_t
{
std::vector<std::uint8_t> non_carlike_labels;
non_carlike_labels.push_back(ObjectClassification::UNKNOWN);
non_carlike_labels.push_back(ObjectClassification::BICYCLE);
non_carlike_labels.push_back(ObjectClassification::PEDESTRIAN);
non_carlike_labels.push_back(ObjectClassification::MOTORCYCLE);
for(auto label: non_carlike_labels){
EXPECT_FALSE(isCarLikeVehicle(label));
}
}

// True Case with object_classifications
{ // normal case
std::vector<autoware_auto_perception_msgs::msg::ObjectClassification> classification;
classification.push_back(createObjectClassification(ObjectClassification::CAR, 0.5));
classification.push_back(createObjectClassification(ObjectClassification::TRUCK, 0.8));
classification.push_back(createObjectClassification(ObjectClassification::BICYCLE, 0.7));
EXPECT_TRUE(isCarLikeVehicle(classification));
}

// False Case with object_classifications
{ // false case
std::vector<autoware_auto_perception_msgs::msg::ObjectClassification> classification;
classification.push_back(createObjectClassification(ObjectClassification::MOTORCYCLE, 0.8));
classification.push_back(createObjectClassification(ObjectClassification::BICYCLE, 0.8));
Comment on lines +139 to +140
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding a test to see what will happen if the object classification includes car-like vehicle and not car-like vehicle with the same probability.
This will be the edge case for isCarLikeVehicle function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. I'll do it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This edge case depends following function getHighestProbLabel, which extract highest-scored label from classification.

  • This function do not considering multiple highest probability
  • It returns an earlier label with maximum probability. (It depends labels order!)
  • For example, if an objects has CAR and MOTORCYCLE label with the same probability,
    • isCarLikeVehicle return true when CAR label is added first
    • else it returns false
inline std::uint8_t getHighestProbLabel(
  const std::vector<ObjectClassification> & object_classifications)
{
  std::uint8_t label = ObjectClassification::UNKNOWN;
  float highest_prob = 0.0;
  // TODO(Satoshi Tanaka): It might be simple if you use STL or range-v3.
  for (const auto & object_classification : object_classifications) {
    if (highest_prob < object_classification.probability) {
      highest_prob = object_classification.probability;
      label = object_classification.label;
    }
  }
  return label;
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Even if the current implementation is not good, in order to show how it works with the edge case in the current implementation, adding this edge-case test seems useful.
What do you think?

Copy link
Contributor Author

@YoshiRi YoshiRi Oct 17, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Frankly, I think it is a rare situation that the label probabilities become the same since it is float value.
But adding edge case seems to be not harmful and it is easy to implement, I added some edge cases to this PR.

Thanks for your help.

EXPECT_FALSE(isCarLikeVehicle(classification));
}

} // TEST isCarLikeVehicle



// TEST isLargeVehicle
TEST(object_classification, test_isLargeVehicle)
{

using autoware_auto_perception_msgs::msg::ObjectClassification;
using perception_utils::isLargeVehicle;

{// True Case with uint8_t
std::vector<std::uint8_t> large_vehicle_labels;
large_vehicle_labels.push_back(ObjectClassification::BUS);
large_vehicle_labels.push_back(ObjectClassification::TRAILER);
large_vehicle_labels.push_back(ObjectClassification::TRUCK);

for(auto label: large_vehicle_labels){
EXPECT_TRUE(isLargeVehicle(label));
}
}

// False Case with uint8_t
{
std::vector<std::uint8_t> non_large_vehicle_labels;
non_large_vehicle_labels.push_back(ObjectClassification::UNKNOWN);
non_large_vehicle_labels.push_back(ObjectClassification::BICYCLE);
non_large_vehicle_labels.push_back(ObjectClassification::PEDESTRIAN);
non_large_vehicle_labels.push_back(ObjectClassification::MOTORCYCLE);
non_large_vehicle_labels.push_back(ObjectClassification::CAR);

for(auto label: non_large_vehicle_labels){
EXPECT_FALSE(isLargeVehicle(label));
}
}

// True Case with object_classifications
{ // normal case
std::vector<autoware_auto_perception_msgs::msg::ObjectClassification> classification;
classification.push_back(createObjectClassification(ObjectClassification::CAR, 0.5));
classification.push_back(createObjectClassification(ObjectClassification::TRUCK, 0.8));
classification.push_back(createObjectClassification(ObjectClassification::TRAILER,0.7));
EXPECT_TRUE(isLargeVehicle(classification));
}

// False Case with object_classifications
{ // false case
std::vector<autoware_auto_perception_msgs::msg::ObjectClassification> classification;
classification.push_back(createObjectClassification(ObjectClassification::MOTORCYCLE, 0.8));
classification.push_back(createObjectClassification(ObjectClassification::BICYCLE, 0.8));
classification.push_back(createObjectClassification(ObjectClassification::CAR, 0.8));
EXPECT_FALSE(isLargeVehicle(classification));
}

} // TEST isLargeVehicle