Containerised Vehicle Software: Isolation, Startup Time and Real-Time Limits

The problem

Software-defined vehicle platforms increasingly consider containers for isolation and independent updates, but ADAS and control tasks have real-time deadlines. Containers share the kernel, so scheduling, memory and I/O interference can affect timing. Clear measurements of startup cost, latency and isolation under load are needed before deciding what can run in a container. This project provides them on a reproducible benchmark.

Objectives

• Build a benchmark pipeline with perception, fusion and control stages with deadlines.
• Run it natively, in Docker containers and in containers with real-time configuration.
• Measure startup time, end-to-end latency, jitter and deadline misses.
• Test isolation under CPU, memory, cache and I/O interference.
• Compare a PREEMPT_RT kernel with a standard kernel.

What you need

Key concepts

Container

A process group isolated with Linux namespaces and limited with cgroups, sharing the host kernel. It starts faster than a virtual machine but offers weaker isolation.

PREEMPT_RT

A set of Linux kernel changes that make most kernel code pre-emptible, reducing worst-case scheduling latency for real-time tasks.

CPU isolation

Reserving CPU cores for chosen tasks with isolcpus, cpusets and interrupt affinity, so other work cannot run on them.

Mixed criticality

Running functions with different safety and timing needs on the same hardware, which requires isolation so less critical functions cannot disturb critical ones.

Step-by-step guide

1. 1

   Define the benchmark pipeline

   Define three stages: Perception runs a small ONNX detection model on recorded frames at 20 Hz; Fusion combines detections with synthetic radar objects at 20 Hz; Control computes a braking request at 100 Hz from the fused objects. Set deadlines per stage and an end-to-end latency target. Add an infotainment-like noisy neighbour workload. Write the research questions and hypotheses before measuring.

   Checkpoint: A pipeline specification with rates, deadlines, interference workloads and written hypotheses.

2. 2

   Implement and instrument the stages

   Write each stage in C++ as a separate process communicating over shared memory or local UDP. Timestamp every message at creation, receive and output with CLOCK_MONOTONIC, and log to memory, writing to disk only at the end to avoid I/O disturbing timing. Give Control a SCHED_FIFO priority with locked memory using mlockall. Verify the native pipeline meets deadlines with no load.

   Checkpoint: The native pipeline runs for one hour with no deadline misses at idle and complete timing logs.

3. 3

   Characterise the platform

   Run cyclictest on both standard and PREEMPT_RT kernels, idle and under stress-ng load, for at least one hour each, to measure scheduling latency. Record CPU model, frequency governor, kernel versions and BIOS power settings, and fix the CPU frequency. These baseline numbers show what the platform can do before containers are involved. Repeat runs on different days to check the baselines are stable before relying on them.

   Checkpoint: A table of cyclictest maximum and percentile latencies for both kernels, idle and loaded.

4. 4

   Containerise the stages

   Write Dockerfiles for each stage on a minimal base image, with the same binaries as the native run. Run with host networking or shared memory mounts so communication paths match the native case. Create three variants: default containers, containers with cpuset and memory limits, and real-time containers with SCHED_FIFO allowed, a cpu-rt-runtime setting and memory locking. Pin image versions and record image sizes, since size affects cold start time.

   Checkpoint: All three container variants run the pipeline and produce timing logs in the same format.

5. 5

   Measure startup time

   Measure time from start command to first valid output for each stage and the whole pipeline: native process launch, cold container start with images not cached, warm container start, and container restart after a crash. Repeat 50 times each. This matters for boot time and for recovery after a stage fails, both important in vehicles. Clear caches between cold-start runs so each one truly starts from scratch.

   Checkpoint: Startup time distributions for each case with median and worst values.

6. 6

   Measure latency without interference

   Run each configuration, native and three container variants on both kernels, for one hour at nominal load. Compute end-to-end latency, per-stage execution time, jitter and deadline misses. Plot cumulative distributions on a log scale so tails are visible. Test whether differences between native and container runs are statistically significant with an appropriate non-parametric test. Use the same recorded input frames for every run so the workload is identical.

   Checkpoint: Latency distributions and miss counts for every configuration with significance results.

7. 7

   Test isolation under interference

   Add interference in a separate container: CPU hogs, memory bandwidth and cache thrashing with stress-ng, disk I/O and network floods. Run each type with and without CPU isolation of the Control stage on reserved cores. Record how each interference type changes Control latency and misses. Cache and memory bandwidth interference are not fully controlled by cgroups, so check this specifically.

   Checkpoint: A matrix of interference type against configuration with latency tails and deadline misses.

8. 8

   Test fault containment and updates

   Crash Fusion, leak memory in Perception until its limit, and fork-bomb the noisy neighbour, each in container and native setups. Record whether other stages keep meeting deadlines and how fast a supervisor restarts the failed one. Then update the Perception image to a new version and measure downtime and steps needed, compared with replacing a native binary. Record how the supervisor detected each fault and how long detection took.

   Checkpoint: Fault containment results and update downtime for container and native setups.

9. 9

   Repeat on an embedded board

   Port the benchmark to an Arm board such as a Raspberry Pi 5 or Jetson, using the same configurations where the kernel allows. Compare relative overheads with the x86 results: absolute values will differ, but if container overhead and interference effects keep the same pattern, conclusions are more general. Note any configuration that could not be reproduced. Use the same container images where the architecture allows, rebuilding only when necessary.

   Checkpoint: A cross-platform comparison table of overhead and interference effects.

10. 10

    Write the thesis

    Present the benchmark, platform characterisation, startup, latency, isolation, fault and update results. Answer the research questions with evidence: which configurations keep the control deadline under which interference, and what containers gain in fault containment and update effort. Release the benchmark code, Dockerfiles and scripts. State the limits: one pipeline, general-purpose hardware and no safety certification claims. Include the interference matrix as a single summary figure that readers can scan quickly.

    Checkpoint: A thesis with reproducible benchmark, answered research questions and stated limits.

Testing & validation

• cyclictest baselines for both kernels idle and loaded.
• Startup time distributions for native and container cases.
• End-to-end latency tails and deadline misses for each configuration.
• Interference matrix with and without CPU isolation.
• Fault containment and update downtime results.

Expected results

A reproducible benchmark quantifying startup cost, timing overhead, isolation and fault containment of containerised ADAS-style software on standard and real-time Linux, on x86 and an embedded board.

Take it further

• Compare with a hypervisor-based partition for the Control stage.
• Use memory bandwidth regulation or cache partitioning where the hardware supports it.
• Orchestrate the containers with a lightweight orchestrator and measure its overhead.
• Add a health monitor that degrades Perception rate when Control deadlines are at risk.

Viva and review questions

Learn it at DIYguru

These DIYguru courses and programmes teach what this project needs.