Geocoding and Address Data Validation

Geocoding turns a string into a coordinate, and the coordinate always looks authoritative. Nothing in a point layer distinguishes a rooftop match from a postcode centroid: both are two numbers, both plot on a map, and both feed the same density surface. The defects that matter in geocoded data are therefore not geometric — the points are valid, they are in the right country, they pass every check in Geometry Validity Checks for Vector Data — they are semantic. This topic sets out the checks that recover that lost information: normalisation before matching, match-level and confidence accounting, fallback detection, accuracy measurement against reference points, and the fitness gate that decides whether a layer may be used for the analysis someone has planned for it.

Prerequisites

• Python 3.10+ with geopandas 0.14+ and shapely 2.0+, plus one of usaddress, pyap or libpostal bindings for address parsing depending on the locale you work in.
• A geocoder whose response you can inspect fully — match level, confidence, matched address string and the identifier of the matched reference record. A geocoder that returns only coordinates cannot be validated, and that is a procurement decision as much as a technical one.
• An authoritative address reference for accuracy measurement: a national address file, a surveyed address-point layer, or a parcel layer with verified centroids. Without one, accuracy can only be described relatively.
• A declared intended use. “Good enough” for a service-area analysis at postcode resolution is not good enough for emergency dispatch, and the same layer can pass one gate and fail the other. Write the intended use into the specification alongside the rules, as described in Defining Spatial Data Quality Policies.
• The canonical CRS decision from Coordinate Reference System Precision Standards. Geocoders return longitude and latitude; displacement measurements need a projected system with metres.

Conceptual Foundation

A geocoding result carries three separable pieces of information, and most quality problems come from storing only the first.

The coordinate is what everyone keeps. The match level describes what kind of thing was matched — a rooftop or parcel point, an interpolated position along a street segment, a street centroid, a postcode centroid, or a locality centroid. The confidence expresses how well the input string matched the reference record, independent of what kind of record it was. A postcode-level match with 0.98 confidence and a rooftop match with 0.71 confidence are both plausible, and they mean completely different things: the first is precisely the wrong scale, the second is approximately the right one.

Match level maps directly onto positional uncertainty, and that mapping is empirical. Rooftop matches in a well-maintained national address file are typically within a few metres. Street interpolation depends on how evenly numbered a street is and can be tens of metres out, worse on long rural roads. Postcode centroids vary from tens of metres in a dense urban postcode to kilometres in a rural one. Measuring these distributions for your reference data — rather than accepting the vendor’s marketing figure — is the single most useful thing a geocoding QC programme does.

Centroid fallback is the failure mode that hides in plain sight. When a geocoder cannot resolve the address it returns the centre of the containing feature it could resolve, and the result is a legitimate coordinate that is shared by every other unresolvable address in the same container. The statistical signature is unmistakable once you look for it: a handful of coordinates each repeated dozens or hundreds of times. The visual signature is a cluster of perfectly coincident points, which analysts routinely mistake for a genuine density peak.

Finally, geocoding is not idempotent across time. Reference data is updated, geocoders change interpolation logic, and the same input string produces a different coordinate six months later. Any geocoded layer that feeds a longitudinal analysis needs the geocoder version, the reference-data vintage and the run date stored with each record — the same reproducibility argument made in Observability and Lineage for Validation.

Step-by-Step Implementation

Step 1 — Normalise the input before you blame the geocoder

# geocode_qc/normalise.py
import re

import usaddress

SUFFIX = {"st": "Street", "rd": "Road", "ave": "Avenue", "av": "Avenue",
          "dr": "Drive", "ln": "Lane", "ct": "Court", "blvd": "Boulevard"}


def normalise(raw: str) -> dict:
    """Parse an address string into components and rebuild it in a canonical form."""
    cleaned = re.sub(r"\s+", " ", raw.strip().rstrip(","))
    try:
        parts, kind = usaddress.tag(cleaned)
    except usaddress.RepeatedLabelError:
        return {"input": raw, "parsed": False, "reason": "ambiguous repeated component"}

    street_type = parts.get("StreetNamePostType", "")
    parts["StreetNamePostType"] = SUFFIX.get(street_type.lower().strip("."), street_type)

    canonical = " ".join(
        parts.get(k, "") for k in (
            "AddressNumber", "StreetNamePreDirectional", "StreetName",
            "StreetNamePostType", "OccupancyType", "OccupancyIdentifier",
            "PlaceName", "StateName", "ZipCode",
        ) if parts.get(k)
    )
    return {"input": raw, "parsed": True, "kind": kind,
            "components": dict(parts), "canonical": canonical}

Verification: run the normaliser over a sample of a thousand input addresses and count the unparsed ones. A parse failure rate above a few percent means the input pipeline, not the geocoder, is the bottleneck — and geocoding unparsed strings inflates the fallback rate for reasons that have nothing to do with the reference data.

Step 2 — Persist the whole geocoding result, not the coordinate

# geocode_qc/record.py
from dataclasses import dataclass, asdict
from datetime import date


@dataclass(frozen=True)
class GeocodeResult:
    record_id: str
    input_address: str
    matched_address: str | None
    match_level: str          # rooftop | parcel | interpolated | street | postcode | locality | none
    confidence: float | None  # 0..1 as reported by the provider
    lon: float | None
    lat: float | None
    provider: str
    provider_version: str
    reference_vintage: str
    run_date: str

    def to_row(self) -> dict:
        return asdict(self)


def build(record_id: str, input_address: str, response: dict, provider: str,
          version: str, vintage: str) -> GeocodeResult:
    return GeocodeResult(
        record_id=record_id,
        input_address=input_address,
        matched_address=response.get("formatted"),
        match_level=response.get("accuracy", "none"),
        confidence=response.get("confidence"),
        lon=response.get("lon"),
        lat=response.get("lat"),
        provider=provider,
        provider_version=version,
        reference_vintage=vintage,
        run_date=date.today().isoformat(),
    )

Verification: confirm that every column is populated for a sample of results. If match_level is empty for a provider, that provider cannot support a quality programme — the level is not derivable after the fact.

Step 3 — Detect fallback by finding repeated coordinates

# geocode_qc/fallback.py
import geopandas as gpd
import pandas as pd


def coincident_clusters(gdf: gpd.GeoDataFrame, min_count: int = 5) -> pd.DataFrame:
    """Coordinates shared by many records — the signature of centroid fallback."""
    key = gdf.geometry.apply(lambda g: (round(g.x, 6), round(g.y, 6)))
    counts = key.value_counts()
    repeated = counts[counts >= min_count]

    rows = []
    for coord, n in repeated.items():
        subset = gdf[key == coord]
        rows.append({
            "lon": coord[0],
            "lat": coord[1],
            "record_count": int(n),
            "distinct_match_levels": sorted(subset["match_level"].unique()),
            "example_input": subset["input_address"].iloc[0],
        })
    return pd.DataFrame(rows).sort_values("record_count", ascending=False)

Verification: the clusters found here should correlate with non-rooftop match levels. If a coordinate is shared by fifty records that all claim rooftop, the provider is mislabelling fallbacks — a far more serious finding than the fallback itself, because it defeats every downstream filter.

Step 4 — Measure displacement against reference points

# geocode_qc/accuracy.py
import geopandas as gpd
import numpy as np


def displacement_stats(geocoded: gpd.GeoDataFrame, reference: gpd.GeoDataFrame,
                       key: str = "record_id", metric_epsg: int = 27700) -> dict:
    """Displacement in metres between geocoded points and authoritative address points."""
    a = geocoded.to_crs(metric_epsg).set_index(key)
    b = reference.to_crs(metric_epsg).set_index(key)
    common = a.index.intersection(b.index)
    if len(common) == 0:
        return {"compared": 0}

    d = a.loc[common].geometry.distance(b.loc[common].geometry)
    out = {
        "compared": int(len(common)),
        "mean_m": float(d.mean()),
        "median_m": float(d.median()),
        "p90_m": float(np.percentile(d, 90)),
        "p95_m": float(np.percentile(d, 95)),
        "max_m": float(d.max()),
        "rmse_m": float(np.sqrt((d ** 2).mean())),
    }
    by_level = a.loc[common].assign(dist=d).groupby("match_level")["dist"]
    out["by_match_level"] = {
        level: {"n": int(g.size), "median_m": float(g.median()), "p95_m": float(np.percentile(g, 95))}
        for level, g in by_level
    }
    return out

Verification: the by_match_level breakdown is the deliverable. It converts a categorical label into metres, which is what every downstream fitness decision actually needs. Recompute it whenever the provider or the reference vintage changes.

Step 5 — Gate the layer on fitness for its intended use

# geocode_qc/gate.py
FITNESS = {
    # analysis -> (allowed match levels, minimum share of records that must qualify)
    "emergency_dispatch":   ({"rooftop", "parcel"}, 0.98),
    "service_area":         ({"rooftop", "parcel", "interpolated"}, 0.95),
    "density_mapping":      ({"rooftop", "parcel", "interpolated", "street"}, 0.90),
    "regional_reporting":   ({"rooftop", "parcel", "interpolated", "street", "postcode"}, 0.85),
}


def assess(counts: dict[str, int], intended_use: str) -> dict:
    allowed, floor = FITNESS[intended_use]
    total = sum(counts.values()) or 1
    qualifying = sum(n for level, n in counts.items() if level in allowed)
    share = qualifying / total
    return {
        "intended_use": intended_use,
        "qualifying_share": round(share, 4),
        "required_share": floor,
        "passes": share >= floor,
        "excluded_levels": sorted(set(counts) - allowed),
    }

Verification: run the same layer through two intended uses and confirm it can pass one and fail the other. That asymmetry is the point: fitness is a property of the pairing between data and purpose, not of the data alone.

Common Failure Modes & Fixes

Performance & Scale Considerations

Geocoding is a network-bound operation and the validation around it is not, so the two should be separated in the pipeline. Geocode once, persist the full result set with all its metadata, and run the quality checks over the stored results. Re-geocoding because a check needs a field that was not stored is the most expensive mistake available here — a million-address batch costs hours and, with a commercial provider, real money.

Cache aggressively on the normalised address string. Duplicate addresses are common in operational data — the same building appearing under several account records — and a cache keyed on the canonical form from Step 1 typically eliminates 10–30% of calls.

The clustering check in Step 3 is a value_counts over rounded coordinates and scales linearly; on ten million rows it is a minute of work. The accuracy measurement in Step 4 is a join on a sample, so it stays small by design — resist the temptation to compare every record against the reference layer, which turns a sampling exercise into a full spatial join for no additional insight.

Integration with the Validation Pipeline

Address validation sits at the ingestion stage of the pipeline described in Validation Pipeline Architecture, before the geometry rules run. Its output is not a repaired geometry but an annotated one: every point carries its match level, its confidence and the vintage of the reference data it came from.

Those annotations then drive the rule stage. A topology rule that expects address points inside parcel polygons should ignore postcode-centroid matches, because the failure it would report is a geocoding artefact rather than a data defect. Encoding that exclusion as a selector on the rule — the pattern described in Building Rule Engines with GeoPandas — keeps the two concerns separate and keeps the defect counts honest.

Frequently Asked Questions

Related

• Measuring Geocoding Accuracy Against Reference Points — the sampling design and displacement statistics in detail
• Detecting Centroid Fallback in Geocoded Results — finding and quantifying collapsed matches
• Validating Address Normalization Before Geocoding — parsing and standardising input so match rates mean something
• Attribute Schema Mapping for Spatial Datasets — the attribute contract that address components must satisfy

Back to Core Spatial QC Fundamentals & Standards