{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TODOs\n",
    "\n",
    "#todo: remove this section when all todos are finished \n",
    "\n",
    "- update code lines if these change \n",
    "- insert real images un the image references, now i just use the example images provided in the example-folder \n",
    "- output images\n",
    "- camera calibration \n",
    "- identified lane-line pixels and fit their positions with a polynomial\n",
    "- radius of curvature of the lane and the position of the vehicle with respect to center.\n",
    "- discussion \n",
    "- search through all the 'todo's in this file and solve questions/todos\n",
    "- save this as a markdown file when it's done :)\n",
    "\n",
    "---\n",
    "## Advanced Lane Finding Project\n",
    "\n",
    "The goals / steps of this project are the following:\n",
    "\n",
    "* Compute the camera calibration matrix and distortion coefficients given a set of chessboard images.\n",
    "* Apply a distortion correction to raw images.\n",
    "* Use color transforms, gradients, etc., to create a thresholded binary image.\n",
    "* Apply a perspective transform to rectify binary image (\"birds-eye view\").\n",
    "* Detect lane pixels and fit to find the lane boundary.\n",
    "* Determine the curvature of the lane and vehicle position with respect to center.\n",
    "* Warp the detected lane boundaries back onto the original image.\n",
    "* Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.\n",
    "--- \n",
    "[//]: # (Image References)\n",
    "[image1]: ./examples/undistort_output.png \"Undistorted\"\n",
    "[image2]: ./test_images/test1.jpg \"Road Transformed\"\n",
    "[image3]: ./examples/binary_combo_example.jpg \"Binary Example\"\n",
    "[image4]: ./examples/warped_straight_lines.jpg \"Warp Example\"\n",
    "[image5]: ./examples/color_fit_lines.jpg \"Fit Visual\"\n",
    "[image6]: ./examples/example_output.jpg \"Output\"\n",
    "[video1]: ./project_video.mp4 \"Video\"\n",
    "\n",
    " \n",
    "\n",
    "## Code Structure \n",
    "The objective of this section is to give the reader a quick overview of the code, it will not go in depth of how the code is written, \n",
    "\n",
    "#todo: is the code structure changed since i wrote this? If yes, this section should be updated \n",
    "\n",
    "The notebook consists of nine code blocks:\n",
    "1. **Camera Calibration**: Handles the camera calibration. \n",
    "2. **Parameters**: Here all the parameters used in the code is decleared. It was helpful to have all the parameters in the same code block when tunning the parameters.\n",
    "3. **Helper-functions**: To keep the pipeline as short and clear as possible, several helper functions was created. Here you can find functions that converts an image to gray scale, undistorts an image, calculates and applies different thresholds, defines region of interest, performs perspective transformation, projects detected lanes onto an image, measure radius of curvature, measure road width and performs sanity check.\n",
    "4. **Line Class**: Defines a line class that is used to store useful information about previous detections between frames in videos \n",
    "5. **Finding lanes**: Code that detect the lanes from an processed image. There are two lane detection functions, `sliding_windows()` and `search_around_poly()`. The former method detects lane lines in an image by looking at peaks in a histogram and searches within relevant \"window\" areas. The latter method detects lane lines more efficiently by exploiting information from earlier detections. The last method in the block - `find_lane_pixels()` - decides weather you should use sliding_window or search_around_poly to detect the lanes. \n",
    "6. **Pipeline**: This is where the pipeline is written. It contains one method `process_image()`, that takes an image as input and returns the image with the detected lane drawn onto it. If the method is run with `debug=True`, it will show all the steps in the pipeline as output. \n",
    "7. **Test Pipeline on Images**: Runs the pipeline on a list of images \n",
    "8. **Test Pipeline on Video**: Runs the pipeline on a video, and saves the video to the project \n",
    "9. **Play video**: Plays the video in the notebook\n",
    "\n",
    "---\n",
    "\n",
    "## Output Images \n",
    "#todo: in the *output_images*-folder there was a txt-file asking us to put all output_images here and to have a section in our readme describing what each image is showing \n",
    "\n",
    "---\n",
    "\n",
    "## Rubrics Points\n",
    "Here I will consider the rubric points individually and describe how I addressed each point in my implementation.  \n",
    "\n",
    "---\n",
    "### Camera Calibration\n",
    "**How:** todo\n",
    "\n",
    "**Where:** The code is defined in the first code block, *Camera Calibration* (lines 7-39 todo: need to update?).\n",
    "\n",
    "**Example:** todo\n",
    "\n",
    "### Pipeline (single images)\n",
    "The pipeline can be found in the sixth code block, *Main Pipeline*. The pipeline consists of multiple steps and will be explained step by step below. \n",
    "\n",
    "Note: if you run the `process_image(img, debug=True)` with the debug parameter set to True, all the steps will be shown.\n",
    "\n",
    "#### 1. Distortion-corrected image\n",
    "\n",
    "**How:** First, the image is undistorted. I did this by utilizing cv2's `undistort()`. `undistort()` takes three parameters: a distorted image, a camera calibration matrix and distortion coefficients. The two latter was calculated in the camera calibration. \n",
    "\n",
    "**Where:** The code is defined in the `undistort()` function in the *Helper Function* block (lines <todo: insert code lines>).\n",
    "\n",
    "**Example**: Undistorting test image *test1.jpg*:\n",
    "![alt text][image2]  \n",
    "\n",
    "\n",
    "#### 2. Binary image\n",
    "\n",
    "**How:** To create a binary image I used color transformations, gradient transformations and definion a region of interest in the image. The threshold values for the different transformations was set by trial and error on the test images. \n",
    "- Gradient transformations and threshold values: \n",
    "    - Sobel in x direction, [30, 100] \n",
    "    - Sobel in y direction, [20, 100] \n",
    "    - Magnitude of gradient, [50, 150] \n",
    "    - Diractional gradient, [0.7, 1.3] \n",
    "- Color transformation and threshold values: \n",
    "    - Saturation [100, 255]\n",
    "- Region of interest: I defined an area in the images where it made sense to look for lanes. The rest of the image was set to black. \n",
    "\n",
    "todo: did we use the L_channel? If yes, this should be filled in \n",
    "\n",
    "**Where:** \n",
    "- Color and gradient transformation is calculated and applied in `apply_thresholds` and some helper functions. The code can be found in the *Helper Function* code block (lines 11-97 todo)\n",
    "- The region of interest is calculated in `region_of_interest()` in the *Helper Function* code block (lines 99-122 todo)\n",
    "- The vertices for the region of interest can be found in the *Pipeline* code block (line 34 todo)\n",
    "\n",
    "**Example:** Binary image of test image *test1.jpg*  \n",
    "![alt text][image3]\n",
    "\n",
    "\n",
    "#### 3. Perspective transform\n",
    "**How:** First, the code chooses four source points in the image, and four destination points the source points will correspond to after the transformation. Then it uses *cv2'*s `getPerspectiveTransform()` to get the transformation matrix based on the source and destination points. Finally, it uses *cv2*'s `warpPerspective()` to transform the image given the transformation matrix and image shape. The source and destination points were hardcoded in the following manner:\n",
    "\n",
    "```python\n",
    "    src = np.float32([[570,465],[720,465],[273,665],[1033,665]])\n",
    "    xmax =img.shape[1]\n",
    "    ymax=img.shape[0]\n",
    "    dst = np.float32([[xmax*0.3, ymax*0.3], [xmax*0.7, ymax*0.3],[xmax*0.3, ymax],[xmax*0.7, ymax]])\n",
    "```\n",
    "\n",
    "Which resulted in the following points: \n",
    "\n",
    "| Source        | Destination   | \n",
    "|:-------------:|:-------------:| \n",
    "| (570, 465)    | (384, 216)    | \n",
    "| (720, 465)    | (896, 216)    |\n",
    "| (273, 665)    | (384, 720)    |\n",
    "| (1033,665)    | (896, 720)    |\n",
    "\n",
    "\n",
    "**Where:** The code is defined in `transform_perspective()` in the *Helper Function* code block (lines 142-156)\n",
    "\n",
    "**Example:** Perspective transformation on test image *test_1.jpg\n",
    "![alt text][image4]\n",
    "\n",
    "#### 4. Identified lane-line pixels and fit their positions with a polynomial\n",
    "**How:** todo\n",
    "\n",
    "**Where:** The code is defined in `sliding_windows()`, `search_around_poly()` and `find_lane_pixels()` in the *Finding Lanes* code block (lines 1-171 todo). It also uses the helper function `fit()` \n",
    "\n",
    "**Example:** \n",
    "- Identified lane-line pixels in test image *test1.jpg* #todo: insert image \n",
    "- Estimated lane from identified pixels in test image *test1.jpg* #todo: insert image \n",
    "\n",
    "![alt text][image5]\n",
    "\n",
    "#### 5. Radius of curvature of the lane and the position of the vehicle with respect to center.\n",
    "**How:** #todo: missing the position of the vehicle with respect to the senter part \n",
    "The radius of curvature was calculated based on this formula: \n",
    "<img src=\"https://latex.codecogs.com/svg.latex?\\Large&space;R_{curve}=\\frac{[1+f'(y)^2]^{3/2}}{|f''(y)^2|}\"/>\n",
    "Where f(y) is the fitted lane line given by: \n",
    "<img src=\"https://latex.codecogs.com/svg.latex?\\Large&space;f(y)=Ay^2+By+C\"/>\n",
    "\n",
    "So the code calculated the radius of curvature by calculating: \n",
    "<img src=\"https://latex.codecogs.com/svg.latex?\\Large&space;R_{curve}=\\frac{[1+(2Ay+B)^2]^{3/2}}{|2A|}\"/>\n",
    "\n",
    "**Where:** The code is defined in `measure_curvature()` in the *Helper Function* code block (lines todo) \n",
    "\n",
    "**Example:** todo \n",
    "\n",
    "\n",
    "#### 6. Result plotted back down onto the road such that the lane area is identified clearly.\n",
    "**How:** To visualize the detected lane onto the original image I used the code suggested in the course. This takes a perspective transformed image and transforms it back to the normal image using *cv2*'s `warpPerspective()` and the **inverse** transformation matrix. Then it recasts the points of the detected lane to fit the new image, and fills this area with a green color.\n",
    "\n",
    "**Where:** The code is defined in `project_lanes_onto_original_image()` in the *Helper Function* code block (lines 158-181 todo)\n",
    "\n",
    "**Example:** Drawn detected lanes on test image *test1.jpg*\n",
    "![alt text][image6]\n",
    "\n",
    "---\n",
    "\n",
    "### Pipeline (video)\n",
    "\n",
    "The final output for the project video can be found [here](./test_videos_output/project_video.mp4), or by navigating to *./test_videos_output/project_video.mp4*. \n",
    "#todo: chech that the final video is full length and looks good \n",
    "\n",
    "---\n",
    "\n",
    "### Discussion\n",
    "\n",
    "#### 1. Briefly discuss any problems / issues you faced in your implementation of this project.  Where will your pipeline likely fail?  What could you do to make it more robust?\n",
    "\n",
    "#todo: remove line 179 when Discussion is finished\n",
    "\n",
    "The pipeline performs quite well on the project_video.mp4, but has a much harder time identifying the lanes on the challange_video.mp4. Two possible approaches to make it more robust could be: \n",
    "1. Add a color threshold on lightness, to hopefully remove shadow edges \n",
    "2. Add a color threshhold for red and green, to hopefully only keep white and yellow colors\n",
    "\n",
    "#todo: discuss more problems. possible issues to discuss: parameter tuning or sanity check\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}