Fault-Tolerant Sensor Fusion under Sensor Degradation

The problem

ADAS sensors degrade in real use: cameras are blinded by low sun or mud, radar is misaligned after a minor knock, lidar loses range in heavy rain and any sensor can drop out. ISO 21448 (SOTIF) asks designers to identify such triggering conditions and handle them, and ISO 26262 requires detection of faults. A fusion system that blindly trusts all sensors produces confident but wrong tracks. Detecting degradation and adapting the fusion is an open research problem.

Objectives

• Define a fault and degradation catalogue with triggering conditions for each sensor.
• Inject these faults into simulated sensor data in repeatable scenarios.
• Design detectors based on innovation statistics and cross-sensor consistency.
• Adapt the fusion by reweighting or excluding degraded sensors.
• Measure detection delay, false alarms and fused tracking quality against a non-adaptive baseline.

What you need

Key concepts

Triggering condition

In SOTIF, a specific condition such as glare or heavy rain that can make an otherwise correct function behave unsafely. Listing them drives the scenario design.

Innovation-based fault detection

If a sensor's measurements keep disagreeing with the tracker's prediction beyond what its noise model allows, the normalised innovation squared rises, flagging a possible fault.

Cross-sensor consistency

Comparing what different sensors report about the same object. A sensor that disagrees with the others persistently is likely degraded.

Graceful degradation

Continuing with reduced capability when part of the system fails, for example lowering ACC speed or handing back control, rather than failing silently.

Step-by-step guide

1. 1

   Build the fault and degradation catalogue

   List degradation modes for each sensor: camera blinding and blur, radar misalignment of a few degrees, radar ghost targets near barriers, lidar range reduction in rain and total dropout of any sensor. For each, describe the triggering condition, its effect on measurements and its hazard for an ACC or AEB function. Use SOTIF and ISO 26262 terms correctly and link each mode to a safety goal.

   Checkpoint: A catalogue table of at least eight modes with effects, hazards and linked safety goals.

2. 2

   Set up baseline scenarios and sensors

   Create highway and urban scenarios in drivingScenario with a lead vehicle, a cut-in, a stationary vehicle and a pedestrian. Mount a camera (visionDetectionGenerator), a radar (drivingRadarDataGenerator) and a lidar model, with realistic noise and fields of view. Run the healthy baseline and save detections and ground truth. Tune a central fusion tracker on healthy data first, so later degradation is measured against a good reference.

   Checkpoint: Healthy-condition GOSPA and RMSE for the fused tracker across all scenarios.

3. 3

   Implement fault injection

   Write MATLAB functions that modify detections to imitate each mode: for misalignment, rotate radar detections by an angle growing over time; for blinding, drop camera detections and add spurious ones in a sector; for rain, reduce lidar detection probability with range; for dropout, remove a sensor entirely. Parameterise each by onset time and severity, and keep it reproducible with fixed seeds.

   Checkpoint: Plots of degraded against healthy detections for every mode at two severities.

4. 4

   Measure the baseline under faults

   Run the non-adaptive fused tracker on every scenario and fault mode at each severity. Record GOSPA, position error of the lead vehicle, false tracks and missed objects, and compute downstream effects such as a false AEB trigger or a missed TTC threshold. This shows how badly a fusion system that trusts all sensors behaves and sets the baseline the adaptive design must beat.

   Checkpoint: A table of baseline degradation per mode and severity, including downstream ADAS effects.

5. 5

   Design per-sensor degradation detectors

   For each sensor, compute a moving window of normalised innovation squared from its associated detections and compare it with chi-square bounds. Add cross-sensor checks: for objects seen by two or more sensors, compute the average position disagreement and its expected spread. Add simple availability checks for missing data. Combine these into a health score per sensor with hysteresis, to avoid flicker.

   Checkpoint: Health-score plots that fall when each fault starts and stay steady in healthy runs.

6. 6

   Tune detectors on separate data

   Use a tuning set of scenarios and seeds to choose window lengths and thresholds that balance detection delay and false alarms. Plot receiver operating characteristic curves for each detector and mode. Keep the test scenarios and seeds separate. Record the chosen operating points, since they define the safety behaviour and must be justified in the report. Prefer short detection delay for modes with severe hazards and fewer false alarms for minor ones.

   Checkpoint: ROC curves per detector and mode with chosen operating points marked.

7. 7

   Adapt the fusion

   Implement two adaptation strategies. In the first, inflate the measurement noise of a degraded sensor in proportion to its health score. In the second, exclude the sensor when its health falls below a threshold and fall back to the remaining sensors. For radar misalignment, also try an online angle bias estimate from cross-sensor comparison. Keep the tracker otherwise identical to the baseline.

   Checkpoint: Both strategies run on every fault scenario without breaking healthy-condition performance.

8. 8

   Add a degradation-mode manager

   In Stateflow, model function states such as full ACC, ACC with reduced maximum speed, warning only and driver takeover request, with transitions based on sensor health and fused track confidence. Define which sensor combinations allow which mode. This shows how degraded perception should limit the function's behaviour, in line with ISO 26262 safe-state thinking. Give each transition a minimum dwell time so the function does not toggle between modes. Test each transition with a dedicated Simulink Test case.

   Checkpoint: Stateflow simulations showing the correct mode for each fault and its recovery when the fault clears.

9. 9

   Run the full evaluation

   Run baseline, noise-inflation and exclusion strategies over every scenario, fault mode, severity and at least 30 seeds. Record detection delay, false-alarm rate, GOSPA, lead-vehicle error, false AEB triggers and missed warnings. Present results by fault mode, because averages hide the cases that matter for safety. Include confidence intervals. Run every strategy on identical fault timings and seeds, so differences come only from the adaptation. Keep tuning scenarios out of this evaluation. Store raw logs so any later metric can be computed without rerunning the simulations.

   Checkpoint: A complete results table per mode and strategy with confidence intervals.

10. 10

    Analyse residual risk

    Identify the fault modes where detection was too slow or the adapted fusion still caused unsafe outcomes. Classify them as known unsafe scenarios in SOTIF terms and propose measures, such as an additional sensor, a plausibility check from vehicle dynamics, or restricting the operational design domain. This links the experiments back to a structured safety argument. Rank the open items by severity and how often their triggering condition is expected in the intended operating domain.

    Checkpoint: A residual-risk list with proposed measures for each unresolved mode.

11. 11

    Write up the research

    Present the catalogue, fault injection method, baseline results, detectors, adaptation strategies, mode manager, evaluation and residual risk. Clearly separate the healthy-data tuning from the test results. Discuss limitations of simulated degradation compared with real sensor failure data. Release the fault-injection library and scenarios so others can reuse them. Explain how the fault catalogue and results would feed a real SOTIF and ISO 26262 work product, such as a hazard analysis or a verification report. Document the fault-injection library's interface so other students can add new modes. Include a list of open questions for future work.

    Checkpoint: Thesis chapters with reproducible results and a released fault-injection toolkit.

Testing & validation

• Healthy-condition fused tracking performance as a reference.
• Baseline degradation per fault mode and severity, including downstream ADAS effects.
• Detection delay and false-alarm rate for every detector from ROC analysis.
• Fused tracking quality for noise-inflation and exclusion strategies.
• Correct degradation-mode transitions in Stateflow.

Expected results

A fault-aware fusion system with a reusable fault-injection library, measured detection performance and quantified recovery of tracking quality under sensor degradation, tied to a structured SOTIF residual-risk analysis.

Take it further

• Learn degradation detection from sensor data with a small neural network and compare with statistical detectors.
• Use recorded rain and night sequences from nuScenes to validate the camera and radar detectors.
• Estimate camera–radar extrinsic drift online and correct it.
• Generate degradation scenarios automatically to search for the worst cases.

Viva and review questions

Learn it at DIYguru

These DIYguru courses and programmes teach what this project needs.