- Carla will be tested at less than 10 miles per hours speed.
- Rosbag image samples are real world samples for classifier training/testing
- Carla should not go any further than the last waypoint given (no looping).
- Safety driver will be present when testing with Carla in the field.
- For safety reasons, throttle commands are limited to 0.025% for actuation for Carla and 80% for the simulator. This may limit the vehicle's ability to reach its intended velocity. These are implemented as parameters in the launch files, so they may be adjusted by the testers in the field without code modifications.
- The Green Light in the simulator is still too quick (around 2 seconds). In order to ensure that the vehicle does not cross the intersection on a red light, the system will test if at its current velocity, it can reach the traffic light in 2 seconds. If not, it will prefer to slow down for a second chance later when it reaches within 5 meters of the traffic light and stops.
- Tested with Simulator with 40kmph, 20mph and 10mph speed settings. NOTE: Because of the throttle and braking command limits set to 80% for the simulator, some speed settings could not be reached during testing.
- Note: When manually moving the vehicle to a position in the track, make sure to start the system up with the
Manualcheckbox still checked. Once the system is completely initialized with its classifier loaded and working, you may uncheck theManualcheckbox to engage the system.
- dumpCurrentPos.py This tool dumps out the current position of the vehicle in the simulator into a CSV file.
- dumpCurrentTwist.py This tool dumps out the current twist command from the waypoint follower into a CSV file.
- dumpWaypoints.py This tool dumps out the static waypoints from the waypoint loader into a CSV file.
- grabFrontCameraImage.py This tool grabs a single image from the front camera of the vehicle from the simulator.
- viewFrontCamera.py This tool creates a pygame display of the front camera of the vehicle from the simulator.
- viewFrontCameraWithClassifier.py This tool creates a pygame display of the front camera of the vehicle from the simulator and also predicts the traffic light classification.
- dumpCurrentPosSteer.py This tool dumps out the current position and steering angle of the vehicle in the simulator into a CSV file.
- dumpFinalWaypoints.py This tool dumps out the final waypoints from the waypoint updater into a CSV file.
- fakeGreenLight.py This tool force sends -1 waypoint message to force the vehicle to move during testing when the traffic light classifier is misbehaving.
- autoTLDataCollector.py This tool will automatically collect pictures of the front vehicle camera and labels them for classification while it is driving in the simulator.
- diagScreen.py This tool is a combination of most of the tools above including charts of the vehicle's speed, throttle, brake and steering.
- diagScreenWithClassifier.py This tool is a combination of most of the tools above including charts of the vehicle's speed, throttle, brake, steering and the traffic light classifier to predict the traffic lights.
- view_rosbag_video.py This tool allows you to view the front vehicle camera images recorded from Carla at the test track in the form of a rosbag in a pygame window, and has an integrated GAN classifier to show its traffic light predictions.
- rosbag_video_2_jpg.py This tool allows you to dump the rosbag camera images into a set of JPEG files. We use this for training the classifier for the SITE test.
- rosbag_video_2_mp4.py This tool allows you to convert the rosbag camera images into a MP4 video file using the moviepy python module.
- rosbag_video_2_gif.py This tool allows you to convert the rosbag camera images into a moving GIF file using the moviepy python module.
Here are some notes that were gathered during development and testing of this project.
The following were observed during development on the project relating to performance.
We first discover a latency issue when we tried to move the car forward by just applying the throttle. Once we started developing the tools above (especially the diagScreen), we discovered that there were certain messages that were being broadcast more often then necessary. We found various ways to reduce the traffic in the message bus to increase the response time of the system.
Where possible, we tried to decrease the amount of time spinning in a ROS node, and except where it is critical, we reduce the periodic processing to 2 Hz (2 times a second). This sped up the processing and allow the simulator to receive its topic messages at low latency. This is necessary to insure that messages where not getting dropped due to timing issues.
The following were observed during development on the project relating to waypoints.
A flowchart of the class WaypointUpdater is shown as follows.
We found that the waypoint updater were reading from the /base_waypoints more often than it should - particularly since the data it contain were static. We made the system read the topic once and then unregister from the topic so it would no longer recieve any further messages until it is restarted.
A flowchart of the class DBWNode is shown as follows.
The drive by wire (dbw) node was tough to correctly impliment. The diagScreen tool help us tremendously by showing us how the actuators application effected the vehicle's velocity and direction of motion. Using this information, we were able to better update both the waypoint updater and the dbw to handle the vehicle better.
Generating the smooth velocity to stop at the traffic light was tough! We had issues with being able to stop within 5 meters of the traffic light. A lot of the time we would miss and go into the intersection. What we eventually did was to made different stop zones 2, 4, 6 and 10 meters from the traffic light. This helped us to force the vehicle to slow down enough to apply the brake and stop. This worked most of the time until we integrated the traffic light detector. After that we have to start planning to stop the vehicle even further back from 40 meters away.
A flowchart of the class TLDetector is shown as follows.
The calculating of the current traffic light and the set of 200 waypoints that are part of the final waypoints sent to the drive by wire node were taking too long. We eventually settled on mapping the static location of the traffic lights to the static location of their nearest waypoint. This mapping was done once at the beginning during initialization and help us reduce the amount of processing the system needed.
Test drive to all intersections in simulator with traffic lights and test if system will stop the car in the simulator. This test completed successfully as shown below:
This is a separate report that can be viewed here: classifier/readme.md
We updated the traffic light detector so as to make it not retrieve camera images until it is within 40 meters of a traffic light. Otherwise, it would unregister from the /raw_camera topic, and only resubscribe when it gets within range.
As requested by Udacity, the system will stop Carla at the last waypoint, as shown below:
The ROS Graph was updated as follows
as we were informed by Udacity
that the obstacle node is currently not implemented in the simulator.
We tested this configuration in the Simulator at 40kmph and 10mph speed settings. You may watch our simulator test videos below:
Our first site testing was not completely satisficatory. Our throttle setting was set too high and made Carla very hard to control as can be seen in our first attempt to make Carla move:
In addition, we use the given site waypoints churchlot_with_cars.csv which made a counter-clockwise loop around the test track instead of clockwise as recorded in the sample rosbag:
We did a complete analysis of the two Udacity rosbags using the following Jupyter Notebook: tools/data_visualization.ipynb
SDC System Integration Carla Test Course Waypoints and Map from rosbag sample:
