UAV Anomaly Detection & Localization

Pipeline explorer

Full pipeline view

Three-panel UAV anomaly detection figure showing original image, anomaly map, and anomaly mask.

Full pipeline view

The example summarizes the full logic of the pipeline: the network starts from the original aerial image, produces an anomaly score map, then a thresholded mask that can be post-processed into localized detections.

Stage explanation

Original image: the raw aerial crop containing small targets over a mostly static background.
Anomaly map: a heatmap of reconstruction error or anomaly score. Brighter regions indicate content the model fails to reconstruct well.
Mask: the thresholded anomaly output, which is much easier to convert into connected components and bounding boxes.

Why this approach is interesting

Efficient: it can generate candidate regions without applying a heavy detector to the whole image.
Interpretable: the anomaly map makes it easier to understand where the signal comes from.
Modular: anomaly localization and final classification can be separated into two stages.

AE / VAE model view

The model learns to reconstruct what looks normal. Objects or texture changes that are not reconstructed well become residual signal, then a cleaner mask for localization.

Input

Aerial crop

Encoder Compress normal patterns

Latent AE code or VAE sample

Decoder Reconstruct the scene

Residual Error map → mask

AE x → z → x′ Use reconstruction error as the anomaly score.

VAE μ, σ → z Sample the latent code to make the representation smoother.

Post-process heatmap → boxes Thresholding and components turn signal into localization.

Pipeline structure

Reconstruct

Train an AE / VAE on normal or background-dominated imagery so the model learns to reproduce the expected scene.

Score anomalies

Compute a pixel-wise or patch-wise anomaly score from the difference between input and reconstruction.

Threshold & clean

Convert the anomaly map into a mask and apply denoising / connected-components logic to isolate likely targets.

Localize & crop

Extract bounding boxes and pass crops to a downstream classifier if you want a full detection + classification pipeline.

What I learned

The appealing part of this project is that it uses structure in the problem. When the background is relatively stable, anomaly models can work as a lightweight proposal mechanism.

At the same time, the hard part is noise: VAEs may detect useful anomalies but produce messy heatmaps, so post-processing matters a lot. This is where thresholding, denoising, and connected components become important.