Sensor fusion

Sensor fusion, also called sensor data fusion or information fusion, is the practice of combining data from several sensors and other sources so that the result is more complete or more reliable than any single input. The output is usually a situation picture, meaning a machine-built estimate of what is happening in an area, assembled from measurements that are often imperfect, late or in disagreement with one another. In a NATO science and technology lecture note, Wolfgang Koch describes fusion systems as cognitive tools that extend human perception, much as conventional tools extend physical strength.

How it works

Inputs can come from radars, optical and infrared cameras, passive receivers, networks of distributed sensors on fixed or moving platforms, databases holding background knowledge, and reports from people. These inputs differ widely in range, update rate, precision and quality. Fusion software has to align them in time and space, decide which measurements refer to the same object, and maintain a track for each object as new data arrives. Probabilistic estimation is the usual mathematical basis, because each measurement carries uncertainty that must be carried forward rather than ignored.

The same ideas apply inside a single platform. A NATO science and technology paper on navigation without satellite positioning, for example, combines inertial sensors and cameras to estimate where an aircraft is, because no single source is dependable on its own.

Origins

The earliest large fusion projects were built for air defence, where operators faced time-critical decisions and large volumes of track data, and they influenced civil air traffic control. In 1984 the Joint Directors of Laboratories, an advisory board to the US Department of Defense, introduced the term sensor data and information fusion and began to structure the field. Its functional model, known as the JDL model, remains the common reference, and later authors have generalised it. It describes a chain from sensors, databases and human reports through to the users and the decisions they can take, with feedback loops at several levels.

Who uses it and why it matters

Fusion sits behind most modern surveillance, air and missile defence, maritime picture building and intelligence, surveillance and reconnaissance. It is the step that turns many sensor feeds into something a command and control system can display and a commander can act on. Exchanging fused tracks between platforms and nations depends on shared formats, which is where tactical data links come in.

Fusion also shapes procurement. If the picture is built in software, the value of a sensor depends partly on how openly its data can be integrated, and on who controls the fusion layer.

What to watch

• Interoperability: whether nations can combine each other’s sensor data without bespoke integration for every pairing.
• Automation: the more of the picture is built by algorithms, the more verification and the handling of errors matter.
• Human oversight: the research literature treats fusion as support for human judgement, not a replacement, and how that balance is written into requirements will matter.
• Degraded conditions: navigation and sensing that continue to work when satellite positioning or communications are disrupted.

Sources

• NATO STO, Target Tracking and Data Fusion for Ground Situational Awareness (STO-EN-SET-191)
• NATO STO, Resilient Navigation with Multi-Sensor Data Fusion in GNSS-Denied Environment (STO-MP-SET-312)