Drivable-Area Segmentation for Unstructured Indian Roads

The problem

Many Indian roads lack clear markings, mix asphalt with dirt and have pedestrians, animals and parked vehicles on the carriageway. Lane-based perception fails there, so a vehicle needs pixel-level understanding of where it can drive. Datasets such as Cityscapes describe structured European streets, while IDD captures the unstructured case.

Objectives

• Define a drivable-area class scheme on IDD's label hierarchy.
• Train at least two segmentation architectures under identical conditions.
• Evaluate mIoU, per-class IoU, boundary F-score and inference speed.
• Test transfer between Cityscapes and IDD in both directions.
• Analyse failure modes on unstructured scenes.

What you need

Key concepts

Semantic segmentation

Assigning a class label to every pixel. For drivable area, the key classes are road, drivable fallback such as dirt shoulders, non-drivable regions and obstacles.

mIoU

Mean intersection over union across classes: for each class the overlap of predicted and true pixels divided by their union, then averaged.

Encoder–decoder architecture

An encoder extracts features at reduced resolution and a decoder recovers spatial detail. DeepLabV3 uses atrous convolution; lighter models trade accuracy for speed.

Boundary F-score

Precision and recall measured on predicted class boundaries within a small pixel tolerance, important because drivable-area edges decide path planning.

Step-by-step guide

1. 1

   Define the problem and research questions

   Frame the questions: which architecture gives the best drivable-area IoU within a real-time budget, and how much does training on structured data hurt on unstructured roads? Define the ODD as daytime Indian urban and rural roads. Set a latency budget, for example 50 ms per frame on a given GPU. Write the experimental plan with fixed seeds and identical training budgets so comparisons are fair.

   Checkpoint: A written plan with research questions, ODD, latency budget and experiment matrix.

2. 2

   Build the label scheme

   Use IDD's hierarchy to map fine labels into a compact scheme: road, drivable fallback, sidewalk and non-drivable ground, vehicles, vulnerable road users, other obstacles and background. Write a remapping lookup table and apply it to all label images. Do the same for Cityscapes for the transfer experiment, noting classes that have no counterpart. Compute pixel frequencies per class to plan for imbalance.

   Checkpoint: Remapped label sets for IDD and Cityscapes with a class frequency table.

3. 3

   Set up data loading and splits

   Use IDD's official training and validation splits and hold out part of validation, chosen by sequence or location, as a test set if no public test labels are available. Resize to a fixed resolution such as 1280×720 or crops of 768×768 and apply augmentations: random scaling, cropping, horizontal flip, colour jitter. Verify by overlaying labels on images for a random sample that remapping and augmentation keep images and labels aligned.

   Checkpoint: Visual checks of aligned image–label pairs and documented splits.

4. 4

   Train the baseline models

   Train two architectures with the same schedule: a DeepLabV3 model with a ResNet-50 backbone from torchvision and a lighter real-time model such as an LR-ASPP MobileNetV3. Use cross-entropy with class weights or online hard example mining, a poly learning-rate schedule and the same number of iterations for both. Log loss, validation mIoU and GPU memory. Repeat key runs with two or three seeds to estimate variance.

   Checkpoint: Trained checkpoints and validation curves for both models, with seed variance.

5. 5

   Evaluate accuracy

   On the test split compute per-class IoU, mIoU and the IoU of the combined drivable classes. Compute a boundary F-score on the drivable-area edge with a tolerance of a few pixels. Produce confusion matrices to see whether road is confused with drivable fallback or sidewalk. Report the mean and spread across seeds rather than a single best run. Save colour-coded prediction overlays for a fixed set of test images so qualitative comparisons between models use the same scenes.

   Checkpoint: A results table of per-class IoU, drivable IoU and boundary F-score with spread.

6. 6

   Measure speed and model size

   Export each model to ONNX and time inference at the chosen resolution on the GPU, with warm-up and averaging over several hundred frames. Report latency, frames per second, parameter count and memory. Plot accuracy against latency so the trade-off is visible and mark the latency budget. If a model exceeds the budget, test a lower input resolution and report the accuracy cost.

   Checkpoint: An accuracy–latency plot with the budget line and measured points for each configuration.

7. 7

   Run the transfer experiment

   Train the faster model on Cityscapes only and evaluate on the IDD test split, then train on IDD and evaluate on Cityscapes validation, using the shared class scheme. Compare with in-domain results to quantify the domain gap in each direction. Then fine-tune the Cityscapes model with progressively more IDD data to show how much local data closes the gap. Use the same number of training iterations in each direction so the comparison is not distorted by unequal training effort.

   Checkpoint: A transfer matrix and a curve of IDD accuracy against amount of IDD fine-tuning data.

8. 8

   Analyse failure modes

   Tag a sample of test images by scene type (no markings, dirt shoulders, waterlogging, dense traffic, construction, animals on road) and compute drivable IoU per tag. Inspect the worst predictions and describe errors such as shoulders labelled as road or crowds eroding the drivable region. Link each failure type to a possible remedy, for example more data, higher resolution or a boundary-aware loss.

   Checkpoint: Per-scene IoU table and a gallery of characteristic failures with causes.

9. 9

   Try one targeted improvement

   Choose the most promising remedy from the failure analysis, such as adding a boundary loss term, multi-scale inference or class-balanced sampling for drivable fallback. Implement it on the faster model, retrain with the same schedule and seeds and compare against the baseline using the same metrics. Report whether the improvement is larger than the seed variance. Change only one thing at a time; if two changes are combined, it becomes impossible to say which one produced the effect you observe.

   Checkpoint: A controlled comparison showing the effect of one change against seed variance.

10. 10

    Write the thesis chapter

    Present the label scheme, experimental setup, accuracy and speed results, transfer study, failure analysis and the improvement experiment. Discuss what drivable-area quality would be needed for a path planner and how segmentation could combine with detection. State limitations such as daytime data, annotation ambiguity between drivable classes and single-frame models. Release code, remapping tables and configuration files. Add an appendix listing every training run with its configuration, seed and result, so the reported means can be traced back.

    Checkpoint: A complete chapter with reproducible configs and released code.

Testing & validation

• Per-class IoU, mIoU and drivable-area IoU on held-out IDD data, across seeds.
• Boundary F-score on drivable edges.
• Latency and frame rate after ONNX export.
• Cross-dataset transfer results between Cityscapes and IDD.

Expected results

A controlled comparison of segmentation architectures for unstructured roads with accuracy, boundary quality, speed and transfer results, plus a failure analysis by scene type.

Take it further

• Add temporal consistency across video frames.
• Deploy the faster model on a Jetson board with TensorRT.
• Combine segmentation with a free-space boundary output for planning.
• Study semi-supervised learning with unlabelled local video.

Viva and review questions

Learn it at DIYguru

These DIYguru courses and programmes teach what this project needs.