What you are building
Extracting building footprints from high-resolution aerial imagery is a classic geospatial deep learning problem, and one of the best end-to-end exercises for anyone moving from computer vision into geospatial work. The task is concrete, the data is freely available, and the failure modes are visual enough to debug by eye.
This walkthrough designs a complete workflow on NAIP, the National Agriculture Imagery Program, which provides roughly 60 cm/pixel aerial imagery across the continental United States. The goal is to take raw raster and vector inputs, train a U-Net semantic segmentation model, run sliding-window inference on an unseen scene, and turn the predicted masks into clean, regularized building polygons. Along the way, we also compare the U-Net output to a pretrained Mask R-CNN instance model and explore zero-shot segmentation with Grounding DINO and SAM.
Environment, data, and chip generation
The pipeline starts with environment setup: install the geoai library, verify that PyTorch can see a GPU (Colab's T4 is fine), and define a single configuration dictionary for paths, training parameters, and inference parameters. That single source of truth saves a lot of debugging later.
The training raster and vector pair come from public NAIP tiles on Source Cooperative, with corresponding building footprint labels in GeoJSON. After downloading, the first step is inspection: check coordinate reference systems, raster dimensions and band count, vector feature counts, and overlay the labels on the imagery to make sure the alignment is correct. A common early failure is assuming the raster and vector share a CRS when they do not.
With alignment confirmed, the source imagery is divided into overlapping georeferenced chips (typically 512x512 with 256 overlap), and matching binary raster masks are generated for each chip. These paired tiles are what the model actually trains on.
Training U-Net with a ResNet-34 encoder
The model itself is a standard U-Net with a ResNet-34 encoder pretrained on ImageNet. geoai.train_segmentation_model handles validation splitting, early stopping, checkpointing, and performance monitoring. The two things to watch during training are the validation IoU curve and the loss curve. A model that converges on training loss while validation IoU plateaus early is overfitting; a model whose validation IoU never crosses 0.5 is probably learning the wrong features.
After training, plot the learning curves and identify the epoch with the highest validation IoU. That is the checkpoint you use for inference, not necessarily the last checkpoint saved.
Inference, vectorization, and regularization
For inference, sliding-window inference with the same window size and overlap as training produces a seamless prediction raster for the full test scene. The output is a binary mask plus an optional probability map. From the mask, the next steps are cleanup: remove small noisy regions below an area threshold, convert the cleaned mask to vector polygons, and then regularize the geometries into orthogonal, building-shaped footprints.
The regularization step is the part that turns "a segmentation result" into "a usable GIS layer." Raw polygonized outputs are jagged, have stair-stepped edges from the raster grid, and include tiny sliver artifacts. An orthogonalize pass with a small epsilon (around 2 pixels) produces clean rectangles that match what a human would have drawn.
Geometric properties like area, perimeter, and compactness can then be added as columns on the resulting GeoDataFrame, and an interactive map view makes the result explorable.
Zero-shot and instance comparison
The interesting side-experiment is comparing the trained U-Net to a pretrained Mask R-CNN. Mask R-CNN does instance segmentation, which means each building gets its own mask and confidence score, and it can be applied zero-shot to a new scene without any fine-tuning. The trade-off is precision: a domain-trained U-Net will usually win on IoU for the specific building class, but Mask R-CNN gives you per-building scores for free, which is useful when downstream applications need to filter by confidence.
Grounding DINO plus SAM gives a third option: prompt-based zero-shot segmentation. You describe the target in text ("building"), Grounding DINO proposes bounding boxes, SAM fills in the masks. It is the most flexible approach, and the slowest, but it works on classes the model has never seen.
Going beyond the demo
The full pipeline extends to real-world areas using NAIP imagery from Microsoft Planetary Computer and building labels from Overture Maps. The same configuration object, the same training call, the same inference utility, all work on the new data with only the input paths changed. That portability is the test of whether the pipeline is actually a pipeline, or just a notebook that happens to work once.
The end product is a clean GeoJSON layer of building footprints with area, perimeter, and compactness attributes, ready to drop into a downstream GIS workflow. For anyone working on urban planning, disaster response, or change detection, this is the most useful 200 lines of geospatial deep learning code you can write.