The problem
Fatigue and distraction contribute to many crashes, and driver monitoring systems are increasingly fitted to vehicles. A camera-based monitor must detect real drowsiness quickly without nagging an alert driver, work across faces, spectacles and lighting, and respect privacy because it records people.
Objectives
- Extract eye, mouth and head-pose features from a webcam stream in real time.
- Define PERCLOS, blink duration, yawn and gaze-off-road measures with thresholds.
- Implement a staged alert logic with hysteresis.
- Record and label sessions to measure detection delay and false alarm rate.
- Document privacy handling and limitations.
What you need
Tools and toolboxes
- Python — Feature extraction, alert logic, logging and evaluation.
- MediaPipe — Face Mesh landmarks for eyes, mouth and face orientation.
- OpenCV — Camera capture, solvePnP head-pose estimation, drawing and video recording.
- NumPy — Eye aspect ratio, rolling windows and statistics.
Software
- Python 3 with MediaPipe, OpenCV and NumPy
- pandas and Matplotlib for analysis
- A simple event-labelling script (key presses during recording)
Extra parts and data
- USB webcam (an infrared-capable camera optional) — A cabin-facing camera is the sensor; an IR camera with IR illumination extends tests to low light.
Key concepts
- Eye aspect ratio (EAR)
- The ratio of vertical to horizontal distances between eye landmarks. It drops sharply when the eye closes, so a threshold detects closure.
- PERCLOS
- The percentage of time in a window (for example 60 seconds) that the eyes are mostly closed. It is a widely used drowsiness measure because long, slow closures matter more than single blinks.
- Head pose
- Yaw, pitch and roll of the head estimated by fitting 2-D landmarks to a 3-D face model with solvePnP. Sustained yaw or pitch away from the road indicates distraction.
- Hysteresis
- Using a higher threshold to raise an alert than to clear it, so the alert does not flicker on and off near the boundary.
Step-by-step guide
- 1
Define requirements and consent
Write the functional requirements: detect eyes closed for more than about two seconds, rising PERCLOS, frequent yawning and gaze away from the road for more than a set time, and alert in two stages (advisory, then urgent). Set targets for detection delay and false alarms per hour of alert driving. Prepare a consent form for participants and decide how recordings are stored and deleted. All tests are done in a parked vehicle or at a desk, never while driving.
Checkpoint: A requirements sheet with numeric targets and signed consent from each participant.
- 2
Capture landmarks in real time
Open the webcam with OpenCV at 640×480 and pass RGB frames to MediaPipe Face Mesh with refine_landmarks enabled so iris points are available. Draw the eye and mouth landmarks on the frame and display the frame rate. Handle frames where no face is found by counting them, because a missing face over several seconds is itself a distraction cue. Log timestamps and landmark coordinates to a file for offline analysis.
Checkpoint: A live view with landmarks drawn at a stable frame rate and a landmark log on disk.
- 3
Compute eye and mouth features
From six landmarks per eye compute EAR as the sum of two vertical distances divided by twice the horizontal distance, then average both eyes. Compute a mouth aspect ratio in the same way for yawns. Calibrate per person: record 30 seconds of normal open-eye driving posture and set the closure threshold as a fraction (for example 70 per cent) of that person's median EAR, since eye shape differs between people and with spectacles.
Checkpoint: EAR plots showing clear dips for blinks and a per-person calibrated threshold.
- 4
Estimate head pose and gaze-off-road
Select landmarks such as the nose tip, chin, eye corners and mouth corners, map them to a generic 3-D face model and call cv2.solvePnP with an approximate camera matrix from the image size. Convert the rotation vector to yaw, pitch and roll. Define the on-road gaze zone from the calibration recording and flag frames where yaw or pitch leaves the zone. Smooth the angles with a short moving average to reduce jitter.
Checkpoint: Head-pose angles that track deliberate head turns and a working gaze-off-road flag.
- 5
Build drowsiness and distraction measures
Compute rolling measures over time windows: PERCLOS over 60 seconds, the longest closure in the last few seconds, blink rate and mean blink duration, yawns per minute and continuous gaze-off-road duration. Use timestamps rather than frame counts so measures stay correct if the frame rate varies. Log every measure each second so thresholds can be tuned offline without re-recording. Plot the measures for one recording with acted events marked, so you can see which measure responds to which behaviour.
Checkpoint: A per-second log of all measures for a test recording.
- 6
Design the staged alert logic
Write a small state machine with states normal, advisory and urgent. Move to advisory when PERCLOS or yawn rate exceeds a first threshold, and to urgent when a single closure exceeds about two seconds or gaze-off-road exceeds a limit. Clear alerts only when measures fall below lower thresholds for a set time, giving hysteresis. Drive an on-screen banner and an audio beep, and log each state change with a timestamp.
Checkpoint: Alerts that rise and clear cleanly in staged tests without flickering.
- 7
Record and label test sessions
Record sessions with several participants, with and without spectacles and in different lighting, in which they act normally and then simulate long closures, slow blinking, yawns, phone glances and mirror checks on cue. Press keys during recording to mark the start and end of each event. Keep separate recordings for tuning thresholds and for final testing. Simulated drowsiness is a limitation to state, since real fatigue looks different.
Checkpoint: Labelled sessions from several people, split into tuning and test sets.
- 8
Evaluate detection and false alarms
On the test sessions, compute for each labelled event whether the correct alert was raised and how long after the event start. Count false alerts during labelled normal periods and normalise per hour. Check that ordinary mirror checks and blinks do not trigger alerts. Report results per participant and per condition so a poor result for spectacles or low light is visible rather than averaged away.
Checkpoint: A table of detection rate, median delay and false alarms per hour by participant and condition.
- 9
Report, privacy and limits
Document the features, calibration, thresholds, alert logic and results. Describe how video was handled: stored locally, deleted after analysis and never shared without consent. State the limitations: acted rather than real drowsiness, few participants, visible-light camera only and a generic 3-D face model. Suggest how an infrared camera, more participants and a validated sleepiness scale would make the evaluation stronger.
Checkpoint: A report with reproducible analysis scripts and a clear privacy section.
Testing & validation
- Detection rate and delay for labelled closure, yawn and distraction events.
- False alerts per hour during normal behaviour.
- Results split by participant, spectacles and lighting.
- Frame rate of the full pipeline on a laptop.
Expected results
A real-time driver monitor with calibrated eye, yawn and head-pose measures, staged alerts with hysteresis and measured detection delay and false alarm rate across participants.
Take it further
- Use an infrared camera and illuminator for night-time tests.
- Train a small classifier on the feature log and compare with fixed thresholds.
- Add phone-use detection with an object detector.
- Publish alerts on a CAN or serial link to a dashboard display.
Viva and review questions
Why calibrate the EAR threshold per person?
Open-eye EAR differs between people and with spectacles, so a fixed threshold misses closures for some people and fires constantly for others.
Why use PERCLOS instead of counting blinks?
Normal blinks are short and frequent; drowsiness shows up as more time with the eyes closed and slower closures, which PERCLOS captures over a window.
How do you estimate head pose from a single camera?
By matching 2-D landmark positions to points on a generic 3-D face model with solvePnP, which returns the rotation and translation of the head relative to the camera.
What is the main weakness of your evaluation?
The drowsiness was acted on cue, so real fatigue may look different; the results show the pipeline works but not that it detects genuine sleepiness reliably.
Learn it at DIYguru
These DIYguru courses and programmes teach what this project needs.