Traffic Sign Recognition with a CNN on Indian and GTSRB Signs

The problem

Traffic sign recognition supports speed-limit information and driver alerts. Public datasets are mostly European, while Indian signs differ in design, condition and placement, so a model that scores well on GTSRB may fail locally. Measuring that gap and reducing it is a practical lesson in dataset shift.

Objectives

• Train a CNN classifier on GTSRB with a clean train, validation and test split.
• Collect and label a small Indian sign test set for the classes that overlap.
• Measure accuracy, per-class recall and confusion on both sets.
• Reduce the domain gap with augmentation and fine-tuning on a few local images.
• Export the model and measure inference time.

What you need

Key concepts

Convolutional neural network

Stacks of learned filters that detect edges, shapes and then sign-specific patterns, followed by layers that map these features to class scores.

Class imbalance

Some sign classes have many more images than others, which biases the model. Weighted loss or balanced sampling helps.

Domain shift

A drop in accuracy when test images come from a different distribution (country, camera, weather) from the training images.

Confusion matrix

A table of true class against predicted class that shows which signs are mistaken for which, for example between similar speed limits.

Step-by-step guide

1. 1

   Define the task and class mapping

   Decide the task: classify a cropped sign image into one of the GTSRB classes. List which GTSRB classes have a clear Indian equivalent (speed limits, stop, no entry, give way, pedestrian crossing and so on) and define a mapping table. Record which Indian signs have no GTSRB equivalent; they are out of scope for the classifier but should be noted. Set target metrics: overall accuracy and the lowest acceptable per-class recall on safety-relevant signs.

   Checkpoint: A class mapping table and target metrics agreed with the guide.

2. 2

   Prepare GTSRB

   Download GTSRB, keep its official test set untouched and split the training images into training and validation sets by track rather than by image, because consecutive images of the same physical sign are near-duplicates. Resize crops to 32×32 or 48×48, normalise by channel mean and standard deviation and plot the class histogram to see the imbalance. Write a data loader that returns tensors and labels.

   Checkpoint: A class histogram and a loader that iterates over track-separated train and validation sets.

3. 3

   Collect the Indian test set

   Photograph signs from footpaths or as a passenger, never while driving, in different lighting and from different angles. Crop each sign with a small margin, label it with the mapped class and note conditions such as faded, dirty, partly occluded or night. Aim for at least 15 to 20 crops per overlapping class. Keep this set for testing only, except for a small separate pool reserved for the fine-tuning experiment.

   Checkpoint: A labelled Indian crop set with condition tags and a separate fine-tuning pool.

4. 4

   Train a baseline CNN

   Build a small network: three convolution blocks (convolution, batch normalisation, ReLU, max pooling) with 32, 64 and 128 filters, dropout and a fully connected classifier. Train with cross-entropy loss and the Adam optimiser at a learning rate near 1e-3, using class weights for imbalance, and stop early on validation loss. Log training and validation curves so overfitting is visible. Record the training time per epoch and the final parameter count, as both are needed when choosing a model for embedded use.

   Checkpoint: Validation accuracy curves that have levelled off, with the best checkpoint saved.

5. 5

   Evaluate on GTSRB and Indian sets

   Run the frozen model on the GTSRB test set and on the Indian test set. Report overall accuracy, macro-averaged recall and a confusion matrix for each. Pay attention to safety-relevant mistakes, such as one speed limit predicted as another or stop predicted as something else. Look at misclassified Indian crops and group them by cause: design differences, damage, lighting or viewing angle.

   Checkpoint: Two confusion matrices and a quantified accuracy gap between GTSRB and Indian signs.

6. 6

   Reduce the gap with augmentation

   Retrain with augmentations that mimic local conditions: random brightness and contrast, blur, small rotations and perspective changes, colour jitter and random occlusion patches. Do not use horizontal flips, because they change the meaning of some signs. Compare the new model with the baseline on both test sets using the same metrics, so the effect of augmentation alone is isolated. Keep the random seed and epoch count identical to the baseline run, so any change in accuracy can be attributed to augmentation.

   Checkpoint: A table showing baseline against augmented model on both test sets.

7. 7

   Fine-tune with a few local images

   Fine-tune the augmented model on the small Indian pool, freezing the convolution layers first and training only the classifier with a lower learning rate, then optionally unfreezing the last block. Repeat with 5, 10 and all available images per class to show how accuracy grows with local data. Evaluate only on the Indian test crops, which have never been used for training.

   Checkpoint: A plot of Indian test accuracy against number of fine-tuning images per class.

8. 8

   Export and build a demo

   Export the final model to ONNX and time inference per crop on a laptop CPU with ONNX Runtime. Build a simple demo that runs on recorded video: detect sign candidates with colour and shape cues (red circles, triangles) or a pretrained detector, crop them and classify each crop, displaying the label and confidence. Present it as a demonstration of the classifier, not as a complete detector.

   Checkpoint: An ONNX model with measured latency and a demo video with labelled signs.

9. 9

   Write the report

   Present the dataset preparation, network design, training curves, both confusion matrices, the augmentation and fine-tuning experiments and the inference timing. Discuss what the remaining errors mean for a speed-limit assist function and why a classifier alone cannot be trusted without a detector and temporal confirmation. Include the class mapping table and describe the Indian set so others can extend it.

   Checkpoint: A report with reproducible numbers and released training and evaluation scripts.

Testing & validation

• Accuracy and macro recall on the untouched GTSRB test set.
• Accuracy and confusion on the Indian test set, by condition.
• Effect of augmentation and of fine-tuning set size on Indian accuracy.
• Inference time per crop on a CPU after ONNX export.

Expected results

A trained sign classifier with measured accuracy on European and Indian signs, a quantified domain gap, evidence of how much augmentation and a few local images close it, and a video demo.

Take it further

• Train a YOLO detector for signs so detection and classification run end to end.
• Add temporal voting across frames to stabilise the displayed speed limit.
• Compare a small CNN with a pretrained MobileNet backbone.
• Deploy on a Raspberry Pi or Jetson board and measure frame rate.

Viva and review questions

Learn it at DIYguru

These DIYguru courses and programmes teach what this project needs.