The problem
Lane departure warning and lane keeping start with knowing where the lane is. Deep networks dominate production, but classical pipelines are still used as fallbacks, as teaching baselines and on low-cost hardware, and they show clearly why faded paint, shadows, curves and missing markings on Indian roads make the task hard.
Objectives
- Define the operating conditions (road type, lighting, speed) the detector is meant to handle.
- Implement colour masking, Canny edge detection, a region of interest and Hough line extraction.
- Fit left and right lane boundaries and smooth them across frames.
- Measure detection rate and lateral position error on labelled frames.
- Document the failure cases and their causes.
What you need
Tools and toolboxes
- Python — Pipeline scripting, evaluation and plotting.
- OpenCV — Colour conversion, Gaussian blur, Canny, HoughLinesP, perspective drawing and video I/O.
- NumPy — Line fitting with polyfit and array maths for masks and metrics.
- Matplotlib — Plots of detection rate, error histograms and intermediate images for the report.
Software
- Python 3 with OpenCV, NumPy and Matplotlib
- A simple labelling tool or a script that records clicked lane points
- BDD100K lane annotations (optional) for extra test frames
Extra parts and data
- Dashcam or phone footage of marked roads (recorded as a passenger) — Local footage shows faded markings and mixed traffic that public datasets under-represent; record it as a passenger or use existing footage, never while driving.
Key concepts
- Canny edge detector
- Finds edges from intensity gradients using two hysteresis thresholds, after Gaussian smoothing to suppress noise. Lane paint produces strong edges against asphalt.
- Hough transform
- Maps edge pixels into a parameter space (distance and angle) where collinear points vote for the same line. The probabilistic version returns line segments directly and is faster.
- Region of interest
- A polygon mask covering the road ahead so edges from the sky, buildings and the bonnet are ignored.
- Temporal smoothing
- Averaging lane parameters over recent frames (for example an exponential moving average) reduces jitter and bridges frames where markings vanish briefly.
Step-by-step guide
- 1
Define the scope and test set
Write down the operating conditions: daytime, highways and urban roads with visible markings, speeds up to about 80 km/h, camera mounted centrally behind the windscreen. List what is out of scope, such as night, heavy rain and unmarked roads. Gather 15 to 30 minutes of footage, split it into clips by condition (clear, shadowed, faded, curved) and set aside about one third of the clips as a test set that is never used for tuning thresholds.
Checkpoint: A scope note and a clip list with each clip tagged by condition and by tuning or test split.
- 2
Label ground truth
Sample one frame per second from the test clips and, for each, mark two or three points along the left and right lane boundaries at fixed image rows near the bottom of the frame. Store the x positions at those rows in a CSV with the frame index. Mark frames where a boundary is genuinely absent so they can be scored separately. Two people labelling the same 50 frames gives an estimate of label noise in pixels.
Checkpoint: A CSV of labelled boundary positions for at least 300 test frames, with measured labeller disagreement.
- 3
Build the colour and edge stage
Convert each frame to HLS and threshold the lightness and saturation channels to keep white and yellow paint, combining the masks with a bitwise OR. Convert to greyscale, apply a 5×5 Gaussian blur and run cv2.Canny with a low-to-high threshold ratio near 1:3. Combine the edge map with the colour mask so only painted edges remain. Save intermediate images for a few representative frames so each stage can be shown in the report.
Checkpoint: Edge images in which lane paint stands out clearly on clear-condition frames, saved for the report.
- 4
Apply the region of interest and Hough transform
Define a trapezoidal region of interest from the bottom corners to points just below the horizon and mask the edge image with cv2.fillPoly. Run cv2.HoughLinesP and tune rho, theta, the vote threshold, minimum line length and maximum gap on the tuning clips only. Reject segments whose slope is near horizontal, then classify the rest as left or right by the sign of their slope and their position relative to the image centre.
Checkpoint: Overlay videos showing Hough segments grouped into left and right on tuning clips.
- 5
Fit and extrapolate lane lines
For each side, fit a straight line x = a·y + b to the segment end points using np.polyfit weighted by segment length, then extrapolate from the bottom of the image to the top of the region of interest. For curved roads, try a second-order fit on edge pixels inside a band around the previous lane position. Draw the lane area as a filled polygon on the frame so the result is easy to judge visually.
Checkpoint: Both boundaries drawn as stable lines on straight sections and the polygon follows gentle curves.
- 6
Add temporal tracking and sanity checks
Keep an exponential moving average of each lane's parameters with a smoothing factor around 0.2. Reject a new fit if the lane width at the bottom row changes by more than a set fraction or if the slope jumps beyond a limit, and hold the previous estimate for up to a few frames before declaring the lane lost. Record a per-frame confidence flag (detected, held or lost) for use in evaluation and for a simple departure warning.
Checkpoint: Jitter visibly reduced and the lost flag raised on frames where markings disappear.
- 7
Evaluate on the test set
Run the frozen pipeline on the test clips and compare predicted x positions with the labels at each labelled row. Count a boundary as correctly detected when the error is below a tolerance, for example 20 pixels, and report detection rate, mean absolute error and false detections on frames where the boundary is absent. Break the results down by condition tag so the effect of shadows, faded paint and curves is visible.
Checkpoint: A results table of detection rate and pixel error per condition on unseen clips.
- 8
Measure speed and analyse failures
Time each stage with time.perf_counter over the full test set and report frames per second on a laptop CPU. Collect the 20 worst frames by error and classify their causes: shadows across the lane, worn paint, vehicles occluding markings, tar joints creating false edges or sharp curves. Try one targeted fix, such as adaptive thresholds from CLAHE-equalised lightness, and re-run the test set to show whether it helps.
Checkpoint: FPS figures, a failure taxonomy with counts and a before-and-after result for one fix.
- 9
Write the report
Document the scope, labelling method, each pipeline stage with intermediate images, the parameter values chosen and why, and the per-condition results. State the limitations openly: straight-line fits on sharp curves, no night or rain data, and a single camera position. Include the code, configuration and the labelled CSV so another student can reproduce the numbers. Close with a comparison of what a learned segmentation model would be expected to improve.
Checkpoint: A report and repository from which the test-set results can be regenerated with one command.
Testing & validation
- Detection rate and mean pixel error against labelled test frames, per condition.
- False detection rate on frames with no visible boundary.
- Frames per second on a laptop CPU, per stage.
- Labeller disagreement as a floor on achievable error.
Expected results
A working lane detector with overlay videos, quantified accuracy per road condition, speed figures and an honest catalogue of where classical methods fail on Indian roads.
Take it further
- Add a bird's-eye-view warp and sliding-window polynomial fitting for curves.
- Compute lateral offset from lane centre and raise a lane departure warning.
- Compare against a small learned lane segmentation network on the same test set.
- Port the pipeline to a Raspberry Pi and measure frame rate.
Viva and review questions
Why apply a Gaussian blur before Canny?
Canny works on gradients, which amplify pixel noise. Smoothing first suppresses noise-induced edges so the detector responds mainly to real boundaries such as paint.
What does each Hough parameter control?
Rho and theta set the resolution of the parameter grid, the threshold sets how many votes make a line, and minimum length and maximum gap control how segments are formed from edge pixels.
Why separate left and right lines by slope?
In a forward camera image the left boundary leans one way and the right the other, so the slope sign is a cheap and reliable way to group segments before fitting.
Why keep a test set that is never used for tuning?
Thresholds tuned on the same frames they are scored on give optimistic results. Unseen clips show how the pipeline generalises.
Learn it at DIYguru
These DIYguru courses and programmes teach what this project needs.