The problem
Machine-learning perception has no complete specification and can fail silently on inputs unlike its training data, such as night scenes, heavy rain or different road environments. ISO 21448 (SOTIF) asks for measures that reduce such insufficiencies. A runtime monitor that flags unreliable outputs lets the system degrade safely. Many monitors work on benchmarks but are not evaluated against real detection errors or downstream safety impact. This project addresses that gap.
Objectives
- Train or fine-tune an object detector and define what counts as a safety-relevant error.
- Build in-distribution and distribution-shifted test sets from public datasets.
- Implement uncertainty, feature-distance and learned monitors.
- Evaluate monitors at frame and object level against actual detection errors.
- Measure the effect of monitor-triggered degradation on a simulated AEB function.
What you need
Tools and toolboxes
- Python with PyTorch — Implements monitors, feature extraction and evaluation.
- Ultralytics YOLO — Provides the object detector to monitor, fine-tuned on driving data.
- OpenCV — Generates synthetic corruptions such as fog, rain, blur and low light.
- CARLA — Simulates closed-loop driving with the monitored detector feeding a simple AEB function.
- ONNX Runtime or TensorRT — Measures the runtime cost of monitors alongside the detector.
- pandas, scikit-learn and matplotlib — Computes AUROC, precision-recall and plots results.
Software
- Python with PyTorch and Ultralytics YOLO
- OpenCV
- CARLA simulator
- ONNX Runtime or TensorRT
- scikit-learn, pandas and matplotlib
Extra parts and data
- A GPU workstation or cloud GPU — Fine-tuning the detector, running ensembles and CARLA simulation need GPU compute.
Key concepts
- Out-of-distribution input
- An input that differs significantly from the training data, where the model's behaviour has not been validated and errors are more likely.
- Predictive uncertainty
- An estimate of how unsure the model is, from confidence scores, Monte Carlo dropout or an ensemble of models disagreeing.
- Feature-distance monitor
- Compares internal network features of a new input with those of training data, for example with Mahalanobis distance, and flags inputs that lie far away.
- Safety-relevant error
- A detection error that could affect the downstream function, such as a missed pedestrian in the ego lane, as opposed to errors that do not matter, like a misclassified distant parked car.
Step-by-step guide
- 1
Define errors and the monitoring goal
Define the monitored component: a YOLO detector for vehicles and pedestrians on front-camera images. Define safety-relevant errors: a missed object or a localisation error above an IoU threshold within an ego-corridor region and a distance range. Define the monitor's task as predicting, per frame and per object, whether such an error is present. Write research questions and metrics before building anything.
Checkpoint: A written error definition with region, distance, IoU thresholds and research questions.
- 2
Build datasets
Use one dataset for training, for example clear-weather daytime frames from BDD100K, and build test sets: in-distribution held-out frames, shifted frames from BDD100K night and rain subsets, a different-domain set from IDD (India Driving Dataset) and synthetic corruptions with OpenCV at several severities. Map all labels to common classes. Keep a separate calibration set for monitor thresholds. Record licence terms for each dataset and keep a script that rebuilds the splits.
Checkpoint: Documented train, calibration and test sets with class mapping and frame counts.
- 3
Train the detector
Fine-tune an Ultralytics YOLO model on the training set with standard settings and fixed seeds, keeping feature maps accessible for monitoring. Evaluate mAP on every test set to quantify how much performance drops under shift. Compute safety-relevant error rates per test set using your error definition, which may differ strongly from mAP. Train a small ensemble of detectors with different seeds for uncertainty monitors.
Checkpoint: mAP and safety-relevant error rates for every test set, showing the performance drop under shift.
- 4
Implement uncertainty monitors
Implement monitors from detector outputs: low maximum confidence, number of low-confidence boxes, entropy of class scores, Monte Carlo dropout variance if dropout layers can be enabled, and ensemble disagreement measured by box matching between ensemble members. Produce a frame-level score and an object-level score for each monitor. Record computational cost for each. Keep monitor code separate from the detector so each monitor can be switched on and off.
Checkpoint: Monitor scores computed for all test frames, with runtime cost per monitor.
- 5
Implement feature-distance monitors
Extract pooled features from chosen backbone layers for training frames, fit class-conditional or global Gaussian models and compute Mahalanobis distance for new frames. Also implement a k-nearest-neighbour distance in feature space with an efficient index. Choose layers using the calibration set only. Compare frame-level scores across test sets to see whether shifted data lies further away. Save the fitted statistics so later runs reuse them exactly.
Checkpoint: Feature-distance scores with distributions separated between in-distribution and shifted sets on calibration data.
- 6
Implement a learned monitor
Train a small classifier that predicts safety-relevant error presence from monitor features and detector outputs, using a held-out part of the training domain plus synthetic corruptions as training data. Do not use the real shifted test sets for training, so their evaluation stays fair. Compare it with the hand-crafted monitors, and check that it is not simply learning dataset identity.
Checkpoint: A trained learned monitor with performance on calibration data and a check against dataset-identity shortcuts.
- 7
Evaluate against real errors
For every monitor and test set, compute AUROC and precision-recall for predicting safety-relevant errors at frame and object level. Also report the classic out-of-distribution detection AUROC for comparison, and show where the two disagree: a monitor can detect shifted data well but still miss the actual errors. Set thresholds on the calibration set for a fixed false-alarm rate and report the error catch rate.
Checkpoint: Tables of AUROC and catch rate at fixed false-alarm rate for every monitor and test set.
- 8
Closed-loop evaluation in CARLA
In CARLA, build scenarios with pedestrians crossing and stopped vehicles in clear, rain and night conditions. Feed the monitored detector's output to a simple AEB function. When the monitor flags low reliability, switch to a degraded mode such as reduced speed and larger braking distance thresholds. Compare collision rate, minimum TTC and unnecessary braking events with and without monitoring. Simulation only.
Checkpoint: Collision rate and TTC results with and without monitoring across conditions.
- 9
Measure runtime cost
Export the detector and monitors to ONNX Runtime or TensorRT and measure latency and memory on a GPU and, if available, on an embedded board such as a Jetson. Ensemble monitors multiply inference cost; feature-distance monitors add little. Plot error catch rate against runtime cost to show the practical choices for an embedded ADAS system. Measure with the same input resolution used in the evaluation to keep figures consistent.
Checkpoint: A cost-versus-benefit plot of catch rate against monitor latency.
- 10
Write the thesis
Present the error definition, datasets, detector, monitors, open-loop and closed-loop evaluations and runtime cost. Answer the research questions, especially whether out-of-distribution scores predict actual safety-relevant errors. Release code, splits and thresholds. Relate the results to SOTIF arguments for perception, without claiming a monitor makes a system safe on its own. Add a short discussion of which monitor you would choose for an embedded system and why.
Checkpoint: A thesis draft with released code and an argued answer to each research question.
Testing & validation
- Detector mAP and safety-relevant error rates on every test set.
- Monitor AUROC and catch rate at fixed false-alarm rate against real errors.
- Comparison of OOD detection performance with error prediction performance.
- Closed-loop collision rate and TTC with and without monitoring.
- Runtime cost of each monitor.
Expected results
An evaluation of uncertainty, feature-distance and learned runtime monitors for a driving object detector, judged against actual safety-relevant errors and in closed-loop CARLA scenarios, with runtime costs.
Research angle for your PhD
Research questions: do out-of-distribution scores predict the detection errors that matter for a downstream ADAS function, and which monitor gives the best catch rate per unit of runtime cost? Metrics: AUROC and catch rate at fixed false-alarm rate for safety-relevant errors, closed-loop collision rate and minimum TTC, and latency. Baselines: confidence thresholding and no monitor. Evaluating monitors against task-relevant errors and in closed loop, rather than against dataset labels alone, is the contribution, suitable for conference or journal publication.
Take it further
- Add temporal monitors that check detection consistency across frames.
- Monitor a lidar or radar fusion output in addition to camera detection.
- Use conformal prediction to give calibrated error guarantees under exchangeability.
- Feed monitor flags into a degraded-mode manager designed with Stateflow.
Viva and review questions
Why is OOD detection performance not enough?
A monitor can tell that an image looks unusual while the detector still works on it, or miss subtle failures on normal-looking images; what matters is catching the errors that affect the function.
Why not use the shifted test sets to train the learned monitor?
That would make the evaluation optimistic; in deployment the monitor meets shifts it has not seen, so the test must reflect that.
What does the monitor do when it flags a frame?
It triggers a degraded mode, such as reduced speed and earlier braking, rather than trusting the detector's output unchanged.
Why define safety-relevant errors by region and distance?
Not every detection error matters; a missed object far outside the ego path does not affect braking, so counting all errors would mislead the evaluation.
Learn it at DIYguru
These DIYguru courses and programmes teach what this project needs.