feat(elevation_map_loader): reduce memory usage of elevation_map_loader #2571

Shin-kyoto · 2022-12-22T15:18:03Z

Description

When you use large pointcloud map, elevation_map_loader use so much amount of memory and keep it after making elevation_map. So in this PR, remove unused object from member of ElevationMapLoaderNode.
Before PR, 10GB memory is allocated after generating elevation_map. (Elevation_map is generated in the memory peak with 13+αGB.)
After PR, 1+αGB memory is allocated after generating elevation_map. (Elevation_map is generated in the memory peak with 12GB.)

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.
The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

The PR follows the pull request guidelines.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

Signed-off-by: Shin-kyoto <[email protected]>

codecov · 2022-12-22T15:34:14Z

Codecov Report

Base: 11.39% // Head: 10.54% // Decreases project coverage by -0.85% ⚠️

Coverage data is based on head (38d78b0) compared to base (33ecdb5).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2571      +/-   ##
==========================================
- Coverage   11.39%   10.54%   -0.86%     
==========================================
  Files        1279     1267      -12     
  Lines       89170    87680    -1490     
  Branches    23587    20938    -2649     
==========================================
- Hits        10161     9242     -919     
- Misses      68239    68573     +334     
+ Partials    10770     9865     -905

Flag	Coverage Δ		*Carryforward flag
differential	`0.00% <0.00%> (?)`
total	`10.55% <0.00%> (-0.85%)`	⬇️	Carriedforward from 4336653

*This pull request uses carry forward flags. Click here to find out more.

Impacted Files	Coverage Δ
..._localizer/include/ekf_localizer/ekf_localizer.hpp	`0.00% <ø> (ø)`
localization/ekf_localizer/src/ekf_localizer.cpp	`0.00% <ø> (ø)`
...elevation_map_loader/elevation_map_loader_node.hpp	`0.00% <ø> (ø)`
...ation_map_loader/src/elevation_map_loader_node.cpp	`0.00% <0.00%> (ø)`
...r/scene_module/avoidance/avoidance_module_data.hpp	`0.00% <ø> (ø)`
...ne_module/lane_following/lane_following_module.hpp	`0.00% <ø> (ø)`
...nner/scene_module/pull_out/pull_out_parameters.hpp	`0.00% <ø> (ø)`
...er/scene_module/pull_over/pull_over_parameters.hpp	`0.00% <ø> (ø)`
...nner/scene_module/side_shift/side_shift_module.hpp	`0.00% <ø> (ø)`
...or_path_planner/src/behavior_path_planner_node.cpp	`0.18% <0.00%> (+0.01%)`	⬆️
... and 229 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

kosuke55 · 2022-12-23T10:49:14Z

perception/elevation_map_loader/include/elevation_map_loader/elevation_map_loader_node.hpp

+  void createElevationMapFromPointcloud(
+    const pcl::shared_ptr<grid_map::GridMapPclLoader> & grid_map_pcl_loader_);


Suggested change

void createElevationMapFromPointcloud(

const pcl::shared_ptr<grid_map::GridMapPclLoader> & grid_map_pcl_loader_);

void createElevationMapFromPointcloud(

const pcl::shared_ptr<grid_map::GridMapPclLoader> & grid_map_pcl_loader);

kosuke55 · 2022-12-23T10:50:13Z

perception/elevation_map_loader/src/elevation_map_loader_node.cpp

+  pcl::shared_ptr<grid_map::GridMapPclLoader> grid_map_pcl_loader_ =
+    pcl::make_shared<grid_map::GridMapPclLoader>(grid_map_logger);
+  grid_map_pcl_loader_->loadParameters(this->param_file_path);


Suggested change

pcl::shared_ptr<grid_map::GridMapPclLoader> grid_map_pcl_loader_ =

pcl::make_shared<grid_map::GridMapPclLoader>(grid_map_logger);

grid_map_pcl_loader_->loadParameters(this->param_file_path);

pcl::shared_ptr<grid_map::GridMapPclLoader> grid_map_pcl_loader =

pcl::make_shared<grid_map::GridMapPclLoader>(grid_map_logger);

grid_map_pcl_loader->loadParameters(param_file_path_);

kosuke55 · 2022-12-23T10:51:22Z

perception/elevation_map_loader/src/elevation_map_loader_node.cpp

@@ -187,15 +189,16 @@ void ElevationMapLoaderNode::createElevationMap()
  } else {
    grid_map_pcl_loader_->setInputCloud(data_manager_.map_pcl_ptr_);
  }
-  createElevationMapFromPointcloud();
+  createElevationMapFromPointcloud(grid_map_pcl_loader_);


Suggested change

createElevationMapFromPointcloud(grid_map_pcl_loader_);

createElevationMapFromPointcloud(grid_map_pcl_loader);

kosuke55 · 2022-12-23T10:51:38Z

perception/elevation_map_loader/src/elevation_map_loader_node.cpp

@@ -187,15 +189,16 @@ void ElevationMapLoaderNode::createElevationMap()
  } else {
    grid_map_pcl_loader_->setInputCloud(data_manager_.map_pcl_ptr_);
  }
-  createElevationMapFromPointcloud();
+  createElevationMapFromPointcloud(grid_map_pcl_loader_);
  elevation_map_ = grid_map_pcl_loader_->getGridMap();


Suggested change

elevation_map_ = grid_map_pcl_loader_->getGridMap();

elevation_map_ = grid_map_pcl_loader->getGridMap();

kosuke55 · 2022-12-23T10:51:48Z

perception/elevation_map_loader/src/elevation_map_loader_node.cpp

+void ElevationMapLoaderNode::createElevationMapFromPointcloud(
+  const pcl::shared_ptr<grid_map::GridMapPclLoader> & grid_map_pcl_loader_)


Suggested change

void ElevationMapLoaderNode::createElevationMapFromPointcloud(

const pcl::shared_ptr<grid_map::GridMapPclLoader> & grid_map_pcl_loader_)

void ElevationMapLoaderNode::createElevationMapFromPointcloud(

const pcl::shared_ptr<grid_map::GridMapPclLoader> & grid_map_pcl_loader)

kosuke55 · 2022-12-23T10:52:03Z

perception/elevation_map_loader/include/elevation_map_loader/elevation_map_loader_node.hpp

@@ -97,7 +98,7 @@ class ElevationMapLoaderNode : public rclcpp::Node
  bool use_inpaint_;
  float inpaint_radius_;
  bool use_elevation_map_cloud_publisher_;
-  pcl::shared_ptr<grid_map::GridMapPclLoader> grid_map_pcl_loader_;
+  std::string param_file_path;


Suggested change

std::string param_file_path;

std::string param_file_path_;

or not making a member, using this->declare_parameter("param_file_path", "path_default"); in createElevationMap() may be good

kosuke55 · 2022-12-23T10:58:16Z

perception/elevation_map_loader/src/elevation_map_loader_node.cpp

@@ -50,7 +50,7 @@ ElevationMapLoaderNode::ElevationMapLoaderNode(const rclcpp::NodeOptions & optio
 : Node("elevation_map_loader", options)
 {
  layer_name_ = this->declare_parameter("map_layer_name", std::string("elevation"));
-  std::string param_file_path = this->declare_parameter("param_file_path", "path_default");
+  this->param_file_path = this->declare_parameter("param_file_path", "path_default");


Suggested change

this->param_file_path = this->declare_parameter("param_file_path", "path_default");

param_file_path_ = this->declare_parameter("param_file_path", "path_default");

it is better to align with other variables.

Signed-off-by: Shin-kyoto <[email protected]>

Shin-kyoto · 2022-12-26T12:30:08Z

@kosuke55
Thank you for your review! I fixed variables' name.

kosuke55

thanks very much, I did not test but the code looks good.

Shin-kyoto · 2023-01-12T11:24:39Z

thanks very much, I did not test but the code looks good.

I checked code by generating elevation map with sample_map.
Checked by zooming elevation_map and found no broken points.
No large holes in elevation_map from top view.
The elevation_map does not deviate significantly from the point cloud.
And this PR does not modify main algorithm, so it is enough to check for major breakdowns. I visualized elevation_map and checked it, and there was no problem. So I merge this PR.

…er (autowarefoundation#2571) * feat: reduce memory usage of elevation_map_loader Signed-off-by: Shin-kyoto <[email protected]> * chore: remove unnecessary comment Signed-off-by: Shin-kyoto <[email protected]> * fix: modify variables' name Signed-off-by: Shin-kyoto <[email protected]> Signed-off-by: Shin-kyoto <[email protected]>

…arefoundation#2571, autowarefoundation#2885, #943) (#337) * feat(elevation_map_loader): reduce memory usage of elevation_map_loader (autowarefoundation#2571) * feat: reduce memory usage of elevation_map_loader Signed-off-by: Shin-kyoto <[email protected]> * chore: remove unnecessary comment Signed-off-by: Shin-kyoto <[email protected]> * fix: modify variables' name Signed-off-by: Shin-kyoto <[email protected]> Signed-off-by: Shin-kyoto <[email protected]> * feat(elevation_map_loader): use polygon iterator to speed up (autowarefoundation#2885) * use grid_map::PolygonIterator instead of grid_map::GridMapIterator Signed-off-by: Shunsuke Miura <[email protected]> * formatting Signed-off-by: Shunsuke Miura <[email protected]> * use use_lane_filter option Signed-off-by: Shunsuke Miura <[email protected]> * delete unused use-lane-filter option Signed-off-by: Shunsuke Miura <[email protected]> * change use_lane_filter to True, clarify the scope Signed-off-by: Shunsuke Miura <[email protected]> * change to use grid_map_utils::PolygonIterator Signed-off-by: Shunsuke Miura <[email protected]> * Add lane margin parameter Signed-off-by: Shunsuke Miura <[email protected]> * use boost geometry buffer to expand lanes Signed-off-by: Shunsuke Miura <[email protected]> * Change use_lane_filter param default to false Signed-off-by: Shunsuke Miura <[email protected]> * update README Signed-off-by: Shunsuke Miura <[email protected]> --------- Signed-off-by: Shunsuke Miura <[email protected]> * perf(behavior_velocity_planner): add faster PolygonIterator (#943) * Add grid_map_utils pkg with faster implementation of PolygonIterator Signed-off-by: Maxime CLEMENT <[email protected]> * suppress error --------- Signed-off-by: Shin-kyoto <[email protected]> Signed-off-by: Shunsuke Miura <[email protected]> Signed-off-by: Maxime CLEMENT <[email protected]> Co-authored-by: Shintaro Tomie <[email protected]> Co-authored-by: Shunsuke Miura <[email protected]> Co-authored-by: Maxime CLEMENT <[email protected]>

…er (autowarefoundation#2571) * feat: reduce memory usage of elevation_map_loader Signed-off-by: Shin-kyoto <[email protected]> * chore: remove unnecessary comment Signed-off-by: Shin-kyoto <[email protected]> * fix: modify variables' name Signed-off-by: Shin-kyoto <[email protected]> Signed-off-by: Shin-kyoto <[email protected]>

Shin-kyoto added 2 commits December 23, 2022 00:02

feat: reduce memory usage of elevation_map_loader

94fdb62

Signed-off-by: Shin-kyoto <[email protected]>

chore: remove unnecessary comment

927b6f4

Signed-off-by: Shin-kyoto <[email protected]>

Shin-kyoto requested review from kosuke55, taichiH and a team as code owners December 22, 2022 15:18

github-actions bot added the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label Dec 22, 2022

Shin-kyoto changed the title ~~feat: reduce memory usage of elevation_map_loader~~ feat(elevation_map_loader): reduce memory usage of elevation_map_loader Dec 22, 2022

Merge branch 'main' into feature/reduce_memory_usage

263086e

kosuke55 reviewed Dec 23, 2022

View reviewed changes

fix: modify variables' name

4336653

Signed-off-by: Shin-kyoto <[email protected]>

kosuke55 approved these changes Dec 27, 2022

View reviewed changes

Merge branch 'main' into feature/reduce_memory_usage

38d78b0

Shin-kyoto merged commit 214912a into autowarefoundation:main Jan 12, 2023

tier4-autoware-public-bot bot mentioned this pull request Jan 13, 2023

chore: sync upstream tier4/autoware.universe#221

Merged

h-ohta mentioned this pull request Mar 24, 2023

feat(elevation_map_loader): large size pointcloud map (backport #2571, #2885, #943) tier4/autoware.universe#337

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(elevation_map_loader): reduce memory usage of elevation_map_loader #2571

feat(elevation_map_loader): reduce memory usage of elevation_map_loader #2571

Shin-kyoto commented Dec 22, 2022 •

edited

Loading

codecov bot commented Dec 22, 2022 •

edited

Loading

kosuke55 Dec 23, 2022

kosuke55 Dec 23, 2022

kosuke55 Dec 23, 2022

kosuke55 Dec 23, 2022

kosuke55 Dec 23, 2022

kosuke55 Dec 23, 2022

kosuke55 Dec 23, 2022

kosuke55 Dec 23, 2022

kosuke55 Dec 23, 2022

Shin-kyoto commented Dec 26, 2022

kosuke55 left a comment

Shin-kyoto commented Jan 12, 2023

		void createElevationMapFromPointcloud(
		const pcl::shared_ptr<grid_map::GridMapPclLoader> & grid_map_pcl_loader_);

	createElevationMapFromPointcloud(grid_map_pcl_loader_);
	createElevationMapFromPointcloud(grid_map_pcl_loader);

	elevation_map_ = grid_map_pcl_loader_->getGridMap();
	elevation_map_ = grid_map_pcl_loader->getGridMap();

		void ElevationMapLoaderNode::createElevationMapFromPointcloud(
		const pcl::shared_ptr<grid_map::GridMapPclLoader> & grid_map_pcl_loader_)

	this->param_file_path = this->declare_parameter("param_file_path", "path_default");
	param_file_path_ = this->declare_parameter("param_file_path", "path_default");

feat(elevation_map_loader): reduce memory usage of elevation_map_loader #2571

feat(elevation_map_loader): reduce memory usage of elevation_map_loader #2571

Conversation

Shin-kyoto commented Dec 22, 2022 • edited Loading

Description

Related links

Pre-review checklist for the PR author

In-review checklist for the PR reviewers

Post-review checklist for the PR author

codecov bot commented Dec 22, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Shin-kyoto commented Dec 26, 2022

kosuke55 left a comment

Choose a reason for hiding this comment

Shin-kyoto commented Jan 12, 2023

Shin-kyoto commented Dec 22, 2022 •

edited

Loading

codecov bot commented Dec 22, 2022 •

edited

Loading