The problem
Planners that treat other road users' predicted paths as fixed are either too timid, freezing in dense traffic, or too aggressive when others do not behave as predicted. Real traffic involves negotiation: others slow down or speed up depending on what the ego does. Interaction-aware planning models this, but it raises questions of tractability, safety and evaluation.
Objectives
- Train or adapt a multi-modal trajectory predictor conditioned on the ego's plan.
- Design a planner that evaluates ego candidates against conditional predictions.
- Build closed-loop evaluation with reactive agents from real data.
- Compare interaction-aware, prediction-only and conservative planners.
- Quantify safety, progress and the freezing-robot problem.
What you need
Tools and toolboxes
- Python — Data pipelines, planner implementation and evaluation.
- PyTorch — Training the conditional trajectory predictor.
- nuScenes and Waymo Open Dataset — Real-world scenes with tracked agents and maps for training and evaluation.
- CARLA (optional) — Additional closed-loop tests with scripted reactive agents.
Software
- Python 3 with PyTorch, NumPy and pandas
- nuScenes devkit and Waymo Open Dataset tools
- CARLA (optional)
Extra parts and data
- GPU workstation or cloud GPU — Training prediction networks on large datasets needs a GPU.
Key concepts
- Multi-modal prediction
- Predicting several possible futures with probabilities, for example yielding or not yielding, rather than one average path.
- Conditional prediction
- Predicting other agents' futures given a candidate ego plan, which models how they might react.
- Freezing robot problem
- When uncertainty about others grows, a cautious planner finds no safe action and stops, even when proceeding would be acceptable.
- Closed-loop evaluation
- Testing where agents react to the ego in simulation, unlike open-loop replay where logged agents ignore the ego's actions.
Step-by-step guide
- 1
Frame research questions
Review interaction-aware planning: game-theoretic, conditional-prediction and joint-prediction approaches. Frame questions: does conditioning predictions on the ego plan reduce freezing in merges and unprotected turns without raising collision risk, and how should uncertainty in reactions be handled? Define scenario types and metrics: collisions, minimum TTC, time to complete, progress and comfort. Decide early how freezing will be measured, for example as waiting time beyond what the logged human driver needed.
Checkpoint: A research proposal with questions, scenario types and metrics.
- 2
Prepare data
Extract interactive scenes from nuScenes and the Waymo Open Dataset motion data: merges, lane changes in dense traffic and unprotected turns. Convert agents' tracks and map features into a common format centred on the ego. Split by scene and location. Record statistics such as the number of interacting agents and the distribution of yielding behaviours. Check the dataset licence terms before planning any release of processed data.
Checkpoint: Processed train, validation and test splits with interaction statistics.
- 3
Train a baseline predictor
Train a multi-modal predictor that takes agent histories and map context, such as a vectorised encoder with a multi-hypothesis decoder. Evaluate with minADE, minFDE and miss rate on the test split. Compare against a constant-velocity baseline to confirm the learned predictor adds value. Report results per agent type and per scenario type, not just overall, since interactive scenes are a minority of the data and averages can hide poor performance there.
Checkpoint: Prediction metrics clearly better than constant velocity.
- 4
Add ego conditioning
Extend the model to take a candidate ego future as an input, training on logged ego futures. Evaluate whether predictions change sensibly when the ego plan changes, for example an agent's predicted yielding probability rising when the ego enters the junction earlier. Check that conditioning does not exploit unrealistic correlations, such as agents always yielding. Test on held-out scenes with ego plans that differ from the log.
Checkpoint: Qualitative and quantitative evidence that predictions respond plausibly to ego plans.
- 5
Design the planners
Implement three planners sharing a candidate generator, such as Frenet sampling: a prediction-only planner using unconditional predictions, an interaction-aware planner scoring each candidate against its conditional predictions, and a conservative planner using worst-case reachable sets. Use the same cost terms for comfort and progress. Add a safety check with a minimum distance under all predicted modes above a probability threshold.
Checkpoint: All three planners run on replayed scenes and produce candidate rankings.
- 6
Build reactive closed-loop simulation
Replay scenes with non-ego agents controlled by reactive policies, such as an intelligent driver model with gap acceptance seeded from logged behaviour, or the learned predictor itself used as a simulator with sampling. Validate that agents behave plausibly when the ego acts differently from the log. Avoid evaluating a planner only against the same model it uses for prediction, or results will be optimistic.
Checkpoint: A reactive simulator with documented agent models and plausibility checks.
- 7
Evaluate planners
Run each planner on hundreds of interactive scenes with several random seeds. Measure collision rate, minimum TTC, scene completion time, number of stalls (freezing) and jerk. Present results per scenario type with confidence intervals. Separately evaluate against agent models different from the planner's own predictor to test robustness to model mismatch. Keep the planner settings frozen during evaluation and report results with and without the safety check.
Checkpoint: A results table with confidence intervals and a model-mismatch study.
- 8
Analyse safety implications
Interaction-aware planners assume others will respond, which can be unsafe if they do not. Analyse failures where agents did not yield as predicted and quantify how a safety check bounds the risk. Relate this to SOTIF triggering conditions and propose rules, such as never relying on another agent's yielding beyond a certain deceleration. Show examples of each failure type with timelines of predicted and actual agent behaviour.
Checkpoint: A failure analysis with proposed safety constraints.
- 9
Write the thesis
Write up the literature, data, predictor, planners, simulator, results and safety analysis. Release code and processed scene lists where dataset licences allow. State limitations: simulated reactions, dataset coverage and the gap to deployment. Position the contribution relative to existing benchmarks. Document the reactive agent models in detail, including their parameters and plausibility checks, because readers will judge the closed-loop results by how realistic those agents are.
Checkpoint: Thesis chapters and a paper draft with reproducible results.
Testing & validation
- Prediction minADE, minFDE and miss rate against baselines.
- Collision rate, minimum TTC and stall count per planner.
- Completion time and comfort in interactive scenes.
- Robustness to agent model mismatch.
Expected results
A conditional prediction model and interaction-aware planner evaluated in reactive closed-loop simulation built from real scenes, with evidence on freezing, safety and robustness to model mismatch.
Research angle for your PhD
Research questions: does conditional prediction reduce freezing in dense interactions without raising collision risk, and how robust is that benefit when agents behave differently from the model? Metrics: collision rate, minimum TTC, stalls, completion time, jerk and prediction metrics. Evaluation in reactive closed loop with deliberate agent-model mismatch addresses a known weakness in the literature and is suitable for top robotics and intelligent vehicle venues.
Take it further
- Add game-theoretic reasoning for two-agent negotiations.
- Include Indian mixed-traffic recordings with two- and three-wheelers.
- Bound risk with formal reachability analysis.
Viva and review questions
Why is open-loop replay not enough to evaluate interaction-aware planning?
Logged agents follow their recorded paths whatever the ego does, so the planner's effect on others cannot be measured; reactive simulation is needed.
What is the risk of conditional prediction?
The planner may assume others will yield and act aggressively; if they do not, collisions can result, so safety checks must bound such assumptions.
How do you measure the freezing robot problem?
By counting stalls or excess waiting time in scenes where a human driver proceeded safely.
Learn it at DIYguru
These DIYguru courses and programmes teach what this project needs.