Lidar Point-Cloud Ground Removal and Obstacle Clustering

The problem

Before a lidar point cloud can be used for detection or free-space estimation, ground points must be separated from obstacles, and obstacle points grouped into objects. Roads are rarely perfectly flat, so a single plane can cut off low objects or leave slopes as false obstacles. A classical pipeline makes these trade-offs visible and remains useful as a fast baseline.

Objectives

• Load, crop and downsample lidar scans.
• Remove ground with global RANSAC and a patch-wise plane fit.
• Cluster obstacles with DBSCAN and fit oriented bounding boxes.
• Evaluate ground segmentation and obstacle detection against labels.
• Measure runtime per scan.

What you need

Key concepts

RANSAC plane fitting

Repeatedly fits a plane to three random points and keeps the plane with the most inliers within a distance threshold; robust to many outliers such as obstacles.

Voxel downsampling

Replaces all points in each small cube with their centroid, reducing point count and evening out density while keeping shape.

DBSCAN

Density-based clustering: points with enough neighbours within a radius form clusters, and isolated points are marked as noise. It needs no preset number of clusters.

Patch-wise ground fitting

Dividing the area into sectors or grid cells and fitting a local plane in each, which follows slopes and camber better than one global plane.

Step-by-step guide

1. 1

   Define scope and evaluation

   Define the region of interest, for example 0 to 40 m ahead and ±15 m to the side, and the objects of interest (cars, pedestrians, cyclists). Decide the metrics: ground precision and recall if point labels are available, and obstacle detection recall and precision against KITTI boxes using bird's-eye-view IoU or centre distance. Choose a set of scans for tuning and a separate set for testing.

   Checkpoint: A scope note with ROI, metrics and tuning and test scan lists.

2. 2

   Load and visualise scans

   Read KITTI velodyne .bin files as N×4 float arrays (x, y, z, reflectance), convert to an Open3D point cloud and visualise it. Load the matching labels and calibration and transform the 3-D boxes from camera to lidar coordinates so they can be drawn on the cloud. Confirm alignment by checking that labelled cars sit around clusters of points. Write a small helper that draws any scan with its boxes from a frame index, since you will use it repeatedly for debugging.

   Checkpoint: Visualisations with label boxes correctly placed around objects in the point cloud.

3. 3

   Crop and downsample

   Crop the cloud to the region of interest with a NumPy mask or an axis-aligned bounding box and remove points on the ego vehicle. Apply voxel_down_sample with a voxel size around 0.1 to 0.2 m and record the point count before and after. Compare the visual quality and later clustering at two voxel sizes to choose a balance between speed and detail for small objects such as pedestrians.

   Checkpoint: Point counts and runtime before and after downsampling for the chosen voxel size.

4. 4

   Remove ground with global RANSAC

   Run segment_plane with a distance threshold around 0.15 to 0.25 m, three points per sample and around 1,000 iterations. Check that the fitted plane normal is close to vertical and reject the fit otherwise. Colour inliers as ground and the rest as non-ground. Inspect scans with slopes, kerbs and camber, noting where the global plane cuts into low objects or leaves sloped road as obstacles.

   Checkpoint: Ground-coloured scans with notes on where a single plane fails.

5. 5

   Implement patch-wise ground fitting

   Divide the area into polar sectors and range rings or a grid of cells several metres wide. In each cell take the lowest points as seeds, fit a plane with RANSAC or least squares and classify points within a threshold as ground, rejecting planes that are too steep. Smooth transitions by checking neighbouring cell heights. Compare visually and quantitatively with the global plane on the same scans.

   Checkpoint: Side-by-side comparisons showing better ground following on sloped scans.

6. 6

   Cluster obstacles

   Run DBSCAN with cluster_dbscan on non-ground points, tuning eps around 0.5 to 0.8 m and min_points around 5 to 10 on the tuning scans. Discard clusters that are too small or implausibly large. Because point density falls with range, test a range-dependent eps or run clustering in distance bands and compare the effect on distant pedestrians and cars. Count merged and split clusters against the labels on the tuning scans, so the parameter choice rests on numbers and not only on visual inspection.

   Checkpoint: Coloured clusters that separate nearby cars and pedestrians on tuning scans.

7. 7

   Fit bounding boxes

   For each cluster compute an axis-aligned and an oriented bounding box (get_oriented_bounding_box or a minimum-area rectangle in the ground plane with fixed vertical extent). Classify clusters roughly by size into car-like, pedestrian-like and other. Draw the boxes with the KITTI labels so matches and misses are obvious, and save a few bird's-eye images for the report. Note that oriented boxes from partial views often have the wrong heading, because the lidar sees only one or two sides of a vehicle; record how often this happens on the tuning scans.

   Checkpoint: Boxes drawn over labels in bird's-eye view, with most cars matched.

8. 8

   Evaluate on the test scans

   Match each labelled object in the ROI to a cluster box by bird's-eye IoU above a threshold or centre distance below about 1 m and compute recall and precision per class and by distance band. If using SemanticKITTI, compute ground precision and recall for both ground methods. Time each pipeline stage and report the total per scan. Freeze all parameters before running the test set.

   Checkpoint: Recall and precision by class and distance, ground metrics and per-stage timing.

9. 9

   Report

   Present the pipeline, parameter choices, ground-removal comparison, clustering results, test metrics and timing, with bird's-eye figures. Analyse failures: merged clusters for adjacent pedestrians, split clusters for trucks, missed distant objects and kerbs detected as obstacles. Explain where a learned detector such as PointPillars would help and where this classical pipeline remains useful, for example as a fast free-space check. Include the final parameter values in a table so the pipeline can be rerun on other scans with the same settings.

   Checkpoint: A report with reproducible scripts and a failure analysis.

Testing & validation

• Ground precision and recall for global and patch-wise methods.
• Obstacle recall and precision by class and distance band.
• Runtime per stage and per scan.
• Visual inspection of sloped and crowded scenes.

Expected results

A classical lidar pipeline that removes ground, clusters obstacles and fits boxes, with measured ground accuracy, detection rates by class and range, and timing.

Take it further

• Add a simple tracker across consecutive scans.
• Compute a free-space boundary from ground points.
• Implement the same pipeline as a ROS 2 node.
• Compare against PointPillars on the same test scans.

Viva and review questions

Learn it at DIYguru

These DIYguru courses and programmes teach what this project needs.