Test the new AWSIM Labs parking lot map

[armaganarsln]

We have recently created a new parking lot map here:

• Create a new map: Parking Lot AWSIM-Labs#121

The link is here: https://github.com/autowarefoundation/AWSIM-Labs/releases/download/v1.2.0/awsim_labs_ParkinArea.zip

We would like the following new Parking Planning PRs to be tested before merging to confirm they work in the AWSIM Labs simulation environment.

• feat(freespace_planning_algorithms): implement support for multiple goal candidates in A star planner #8092
• feat(freespace_planning_algorithms): implement option for backward search from goal to start #8091
• feat(freespace_planning_algorithms): use distance to nearest obstacle to improve path planning #8089

Expected outcomes:

• Existing parking algorithms are tested and a simple report is created with accompanying videos.
• New parking algorithms are tested and and a simple report is created with accompanying videos.

cc. @xmfcx @maxime-clem @Aysenayilmaz @mitsudome-r @takayuki5168

[maxime-clem]

@armaganarsln thank you for providing the map.
We will prepare a report comparing the parking behavior before/after all the PRs. We will do our best to prepare this report by the next Planning & Control working group.
If you do not mind, we will evaluate all PRs together as they were split to make the reviews easier, but the behavior improvements is mostly noticeable with all PRs.

I also understood that during the working group it was agreed that you would provide test scenarios for scenario_simulator_v2. Is this still planned ?

[mkquda]

@armaganarsln

In response to your request, the following PR has been tested using the provided parking lot map: (this PR includes all the changes from other PR's as well)

• feat(freespace_planning_algorithms): implement support for multiple goal candidates in A star planner #8092

Few things to note:

• Due to the duplicate lanes drawn on top of each other (which I assume are to allow two way movement), It was difficult to run the simulations smoothley using the provided lanelet map, in many cases the mission planner fails to find a route.
• In current Autoware configurations, costmap_generator has buffer size of 1.0 m around object polygons, and freespace_planner has a safety margin of 0.5 m added to ago vehicle shape for collision detection, so free space path planning requires at least 1.5 m between ego and detected objects. In the layout of the provided map the spacing between the parking slots is not sufficient to satisfy these settings when trying to park next to a vehicle.
• Due to above 2 points I modified the provided lanelet map to remove the duplicate lanelets, and increase the spacing between the parking slots. The modified lanelet map is attached below.

lanelet2_map.zip

Test 1: (L parking next to parked vehicle)

• Goal pose is placed next to a parked vehicle
• Other vehicles are parked on the opposite side
• Ego can park successfully

parking_tests_1.mp4

Test 2: (L parking between 2 parked vehicles)

• Goal pose is placed between 2 parked vehicles
• Other vehicles are parked on opposite side
• Ego can park successfully

parking_tests_2.mp4

Test 3: (Angled parking between 2 parked vehicles)

• Goal pose is placed between 2 parked vehicles on angled parking
• Other vehicles are parked on the opposite side
• Goal pose is far from ego, exceeding occupancy grid range
• Autoware successfully switches to lane driving to move closer to goal
• Ego can park successfully

parking_tests_3.mp4

Test 4: (Parallel parking)

• Goal pose is placed in parallel parking
• Vehicle parked infront of target slot
• Vehicle parked behind target slot
• Other vehicles parked on the side
• Ego cannot park successfully

parking_tests_4.mp4

Test 5: (L parking with obstacle)

• Goal pose is place next to parked vehicle
• Vehicle is parked infront of Ego
• Ego can park successfully

parking_tests_5.mp4

[armaganarsln]

@armaganarsln thank you for providing the map. We will prepare a report comparing the parking behavior before/after all the PRs. We will do our best to prepare this report by the next Planning & Control working group. If you do not mind, we will evaluate all PRs together as they were split to make the reviews easier, but the behavior improvements is mostly noticeable with all PRs.

I also understood that during the working group it was agreed that you would provide test scenarios for scenario_simulator_v2. Is this still planned ?

@Aysenayilmaz started working on the scenarios for AWSIM Labs Scenario-based testing but @mkquda already made the above tests and there are changes made to the lanelet. So first thing we should test the current planner with the above lanelet and further discuss the details of why those changes to lanelet are needed and if these changes are necessary. I have made the parking area with a relaxed setting and using the biggest parking lot standard available so I am not convinced that there are changes required to these sizes. I will like to discuss the details during the planning meeting to learn from you (planning team) why these changes must be made and why there is no other way to park.

[armaganarsln]

I have checked the tests above and apparently, they are made in the planning simulator not in AWSIM therefore could you also publish your results in AWSIM Labs for us to see the performance. For example the area that you parked has a vehicle in that place .

[xmfcx]

You can find the AWSIM Labs binary in Releases section:

• https://github.com/autowarefoundation/AWSIM-Labs/releases/tag/v1.2.1
• Simulation executable: https://github.com/autowarefoundation/AWSIM-Labs/releases/download/v1.2.1/awsim_labs_parking_v1.2.1.zip
• Lanelet map (which you downloaded): https://github.com/autowarefoundation/AWSIM-Labs/releases/download/v1.2.1/awsim_labs_ParkinArea.zip

[xmfcx]

• Due to the duplicate lanes drawn on top of each other (which I assume are to allow two way movement), It was difficult to run the simulations smoothley using the provided lanelet map, in many cases the mission planner fails to find a route.

I understand you've removed one of the overlapping lanes to make it work, we can fix it in the future but still this is unrealistic. This parking lot has those spaces bidirectional like most real parking lots. We can leave this part to future.

• In current Autoware configurations, costmap_generator has buffer size of 1.0 m around object polygons, and freespace_planner has a safety margin of 0.5 m added to ago vehicle shape for collision detection, so free space path planning requires at least 1.5 m between ego and detected objects. In the layout of the provided map the spacing between the parking slots is not sufficient to satisfy these settings when trying to park next to a vehicle.

This is not realistic as well. The parking lot was created by following regulations and standards.

• Create a new map: Parking Lot AWSIM-Labs#121 (comment)

Due to above 2 points I modified the provided lanelet map to remove the duplicate lanelets, and increase the spacing between the parking slots. The modified lanelet map is attached below.

Instead of modifying the parking spaces, please consider changing the default parameters of the Autoware.

If the current safety margins in the Autoware are very unrealistic, try reducing them and let's focus on improving the safety without losing the ability to park in this lot.

Parking when there is this much space in all those parking spaces should be possible.
They are all easy parking spots with lots of room for error.

[Aysenayilmaz]

Park Testing Report

I've conducted a series of tests on parking algorithms using AwsimLabs and Autoware.
I'm on the main branch in Autoware ( a742e82 ) and this pr (#8089) has been merged into the current main branch.
Below are the observations and results:

Test 1: Parallel Parking Attempt

• I attempted a parallel parking maneuver.
• The generated path was not suitable for parallel parking.
• Resulting in a failed parking attempt.

https://www.youtube.com/watch?v=f6jN62Wny-I

2024-08-14.15-09-44.mp4

Test 2: Goal Pose Behind the Vehicle

• I set a goal pose in a parking spot behind the vehicle.
• While the vehicle stayed within the parking area, it failed to complete the parking maneuver successfully.
• This might require further tuning, but I am not sure about the specifics.

https://www.youtube.com/watch?v=tmlBhRD0oFc

2024-08-14.15-55-26.mp4

Test 3: Parallel & Angled Parking

• When setting a goal pose for parallel parking, the system successfully generated a path.
• For angled parking spots, the system failed to generate a path.
• The error "Component_state_diagnostic " was encountered.
• The vehicle could perform autonomous driving for parallel parking but could not set a goal pose for L-shaped or angled parking.

https://www.youtube.com/watch?v=JdTxU79LLG4

2024-08-14.16-21-28.mp4

Test 4: L Parking Issue

• In another location, I attempted to set a goal pose for L-shaped parking.
• The system did not exit the stop mode when the goal pose was set, and no path was generated.

https://www.youtube.com/watch?v=0bYq64MQT3w

2024-08-14.16-26-28.mp4

cc : @xmfcx @armaganarsln @mkquda

[mkquda]

@armaganarsln @xmfcx @Aysenayilmaz

I conducted the following tests in AWSIM with modified costmap_generator & freespace_planner parameters to reduce safety margins.

Test 1: (L parking next to vehicle)

awsim_tests_1_.mp4

Test 2: (L parking between 2 vehicles)

awsim_tests_2_.mp4

Test 3: (Angled parking next to vehicle)

awsim_tests_3_.mp4

Test 4: (Parallel parking)

awsim_tests_4_.mp4

[xmfcx]

@mkquda thanks for the tests. Could you also post the parking behavior for these tests before your changes?

So we can see how it has improved with your additions.

[Aysenayilmaz]

I created the tests conducted by @mkquda using the scenario simulator, and ran them with AwsimLabs and ss2 together. I'm sharing the results as a youtube playlist. I'm also adding the yaml files of the scenarios to the drive.

Youtube playlist : https://www.youtube.com/playlist?list=PL-g7xKnyjxjggO6xgFJ_n3Labd8OFk93J

Scenarios files : https://drive.google.com/drive/folders/1CbLekZoej3XICKTSgkTt0zQyZ_g8_lhA?usp=drive_link

[Aysenayilmaz]

Park Testing Report

I've conducted a series of tests on parking algorithms using AwsimLabs and Autoware. I'm on the main branch in Autoware ( a742e82 ) and this pr (#8089) has been merged into the current main branch. Below are the observations and results:

Test 1: Parallel Parking Attempt

* I attempted a parallel parking maneuver.

* The generated path was not suitable for parallel parking.

* Resulting in a failed parking attempt.

https://www.youtube.com/watch?v=f6jN62Wny-I
2024-08-14.15-09-44.mp4

Test 2: Goal Pose Behind the Vehicle

* I set a goal pose in a parking spot behind the vehicle.

* While the vehicle stayed within the parking area, it failed to complete the parking maneuver successfully.

* This might require further tuning, but I am not sure about the specifics.

https://www.youtube.com/watch?v=tmlBhRD0oFc
2024-08-14.15-55-26.mp4

Test 3: Parallel & Angled Parking

* When setting a goal pose for parallel parking, the system successfully generated a path.

* For angled parking spots, the system failed to generate a path.

* The error _"Component_state_diagnostic "_ was encountered.

* The vehicle could perform autonomous driving for parallel parking but could not set a goal pose for L-shaped or angled parking.

https://www.youtube.com/watch?v=JdTxU79LLG4
2024-08-14.16-21-28.mp4

Test 4: L Parking Issue

* In another location, I attempted to set a goal pose for L-shaped parking.

* The system did not exit the stop mode when the goal pose was set, and no path was generated.

https://www.youtube.com/watch?v=0bYq64MQT3w
2024-08-14.16-26-28.mp4

cc : @xmfcx @armaganarsln @mkquda

I had to change 'obstacle_threshold: 101' and 'expand_polygon_size: 0.0' to encounter the issue I mentioned, but I know this means it won't detect obstacles. I did it just to draw the path and test.