Testing Spatial Validation Code

A validation pipeline is a machine for making claims about other people’s data. When it is wrong, the failure is quiet and expensive: a rule that silently never fires means a defect class has been invisible for months, and nobody looks for a bug in the thing that reports no problems. Testing the validator therefore matters more than testing most application code, and it has its own difficulties — geometry does not compare cleanly, real defects are awkward to obtain, and the interesting cases are precisely the ones nobody thought to write down. This topic covers fixture design, paired positive and negative tests, tolerance-aware assertions, property-based coverage and the tiering that keeps a suite fast enough to run on every commit. It supports the rule work in Building Rule Engines with GeoPandas and the pipelines described across this section.

Prerequisites

• pytest 8.x, shapely 2.0+, geopandas 0.14+ and numpy 1.24+.
• hypothesis 6.x for the property-based tier.
• A rule set with a stable result contract — feature identifier, rule identifier, severity, message. Testing is much harder when every rule invents its own output shape, which is one more argument for the contract described in the rule-engine topic.
• An ephemeral PostGIS container for the database tier, provisioned as in Docker-Based PostGIS Validation Containers.
• A convention for where fixtures live. Mixing generated fixtures and committed files without a rule about which is which produces a directory nobody trusts.

Conceptual Foundation

Spatial validation code has four testable layers, and each fails differently.

The predicate layer is pure: given a geometry, does the rule fire? These are the easiest tests to write and the ones most often written badly, because a test that only proves the rule fires on a bad geometry says nothing about false positives. Every predicate needs at least two cases — one that must fire and one that must not — and the second is the one that catches an over-broad rule.

The classification layer maps a fired predicate onto a severity and a message. It is pure logic and deserves ordinary unit tests, but it is frequently untested because it looks trivial. It is not: severity decides routing, and a mis-tiered rule sends blockers to a batch review queue where they sit for a week.

The aggregation layer turns findings into a report — grouping, deduplication, ordering, roll-ups. This is where golden tests earn their place, because the behaviour is structural rather than per-feature and unit tests over individual findings cannot see it.

The integration layer is where the geometry meets a real engine: PostGIS predicates, index behaviour, driver quirks. These tests are slow, need a container, and are the only ones that catch a GEOS version difference. Keep them in a separate tier and run them in continuous integration rather than on every save.

The other structural decision is how defects are obtained. Real defective data is the most convincing fixture and the least usable one: it is large, often confidential, and rarely contains the specific defect a new rule targets. Synthetic defects — constructed deliberately, minimal, and named after the thing they demonstrate — make better tests, and generating them is a skill worth investing in, as covered in Generating Synthetic Invalid Geometries for Tests.

Step-by-Step Implementation

Step 1 — Build minimal fixtures in code

# tests/fixtures/geometry.py
import pytest
from shapely.geometry import Polygon, LineString, Point


@pytest.fixture
def valid_square() -> Polygon:
    """The control case: every rule must leave this alone."""
    return Polygon([(0, 0), (10, 0), (10, 10), (0, 10)])


@pytest.fixture
def bowtie() -> Polygon:
    """Self-intersecting ring — the canonical invalid polygon."""
    return Polygon([(0, 0), (10, 10), (10, 0), (0, 10)])


@pytest.fixture
def sliver() -> Polygon:
    """Thin and small: 40 m long, 8 cm wide."""
    return Polygon([(0, 0), (40, 0), (40, 0.08), (0, 0.08)])


@pytest.fixture
def touching_pair() -> tuple[Polygon, Polygon]:
    """Shares an edge exactly — must never be reported as an overlap."""
    return (Polygon([(0, 0), (10, 0), (10, 10), (0, 10)]),
            Polygon([(10, 0), (20, 0), (20, 10), (10, 10)]))


@pytest.fixture
def overlapping_pair() -> tuple[Polygon, Polygon]:
    """Genuine overlap of 20 square units."""
    return (Polygon([(0, 0), (10, 0), (10, 10), (0, 10)]),
            Polygon([(8, 0), (18, 0), (18, 2), (8, 2)]))

Verification: each fixture should be readable as a shape from its coordinates alone. If you cannot picture it, simplify it — a fixture that needs a comment explaining what it looks like will be misunderstood by the next person to touch the rule.

Step 2 — Pair every positive with a negative

# tests/test_overlap_rule.py
from rules.topology import must_not_overlap


def test_overlap_is_reported(overlapping_pair):
    a, b = overlapping_pair
    findings = must_not_overlap([a, b])
    assert len(findings) == 1
    assert findings[0]["rule"] == "TOPO_OVERLAP_001"
    assert findings[0]["severity"] == "blocker"


def test_shared_edge_is_not_an_overlap(touching_pair):
    """Touching is legal; a rule that flags it makes every coverage unusable."""
    a, b = touching_pair
    assert must_not_overlap([a, b]) == []


def test_identical_geometries_report_once(overlapping_pair):
    a, _ = overlapping_pair
    findings = must_not_overlap([a, a])
    assert len(findings) == 1, "a symmetric relation must not be reported twice"

Verification: the third test is the one that fails on most first implementations. A self-join over a symmetric predicate reports each pair twice unless the comparison is ordered, and a doubled violation count undermines every metric downstream.

Step 3 — Assert geometry with tolerance, never as text

# tests/test_repair.py
from shapely import normalize
from shapely.geometry import Polygon

from rules.repair import repair_geometry

TOL = 1e-9


def test_repair_preserves_area(bowtie):
    repaired = repair_geometry(bowtie)
    assert repaired.is_valid
    assert abs(repaired.area - 100.0) < 1e-6


def test_repair_is_idempotent(bowtie):
    once = repair_geometry(bowtie)
    twice = repair_geometry(once)
    assert normalize(once).equals_exact(normalize(twice), TOL)


def test_repair_leaves_valid_geometry_untouched(valid_square):
    result = repair_geometry(valid_square)
    assert normalize(result).equals_exact(normalize(valid_square), TOL)

Verification: try replacing equals_exact with a WKT string comparison and watch the tests become environment-dependent. Coordinate formatting differs between GEOS builds, and a suite that compares WKT will pass locally and fail in continuous integration for reasons that have nothing to do with the code.

Step 4 — State invariants and let a generator attack them

# tests/test_properties.py
from hypothesis import given, settings, strategies as st
from shapely.geometry import Polygon

from rules.repair import repair_geometry


@st.composite
def random_ring(draw, n: int = 6):
    coords = draw(st.lists(
        st.tuples(st.floats(-1e4, 1e4, allow_nan=False, allow_infinity=False),
                  st.floats(-1e4, 1e4, allow_nan=False, allow_infinity=False)),
        min_size=n, max_size=n))
    return Polygon(coords)


@given(random_ring())
@settings(max_examples=300, deadline=None)
def test_repair_always_produces_valid_output(poly):
    result = repair_geometry(poly)
    assert result.is_valid or result.is_empty


@given(random_ring())
@settings(max_examples=300, deadline=None)
def test_repair_never_grows_area(poly):
    """Repair may lose area by splitting; it must never invent it."""
    before = poly.buffer(0).area          # a comparable valid interpretation
    after = repair_geometry(poly).area
    assert after <= before + 1e-6

Verification: run with a high example count once. Property tests earn their keep on the shapes nobody thinks to write — degenerate rings, coincident vertices, near-collinear points — and hypothesis will find them within a few hundred examples.

Step 5 — Tier the suite by cost

# tests/conftest.py
import pytest


def pytest_configure(config):
    config.addinivalue_line("markers", "db: needs a live PostGIS container")
    config.addinivalue_line("markers", "slow: full-dataset or property-heavy test")


def pytest_collection_modifyitems(config, items):
    if config.getoption("-m") or config.getoption("--runslow", default=False):
        return
    skip_slow = pytest.mark.skip(reason="slow tier: run with --runslow or -m slow")
    for item in items:
        if "slow" in item.keywords or "db" in item.keywords:
            item.add_marker(skip_slow)

Verification: the default pytest run should finish in a couple of seconds. If it does not, developers stop running it, and a suite that is not run on every change provides documentation rather than protection.

Common Failure Modes & Fixes

Performance & Scale Considerations

The fast tier should be pure geometry and pure logic, and it should run in under five seconds for a rule set of a hundred rules. That is achievable because each test operates on a handful of vertices; a test that loads a real dataset has silently joined the slow tier.

Property-based tests need a budget. Three hundred examples per property is a reasonable default in the fast tier; ten thousand belongs in a nightly job. Hypothesis’s example database makes this cheaper than it sounds — once a counterexample is found it is replayed on every subsequent run, so the expensive search happens once.

Database tests should share one container across the whole session rather than starting one per test. Use a transaction rollback per test for isolation: it is an order of magnitude faster than recreating the schema, and it keeps the tests independent.

Finally, resist the temptation to test the rule engine by running the whole pipeline. End-to-end tests are valuable as a small number of golden cases, but they are slow, they fail for many reasons at once, and a suite dominated by them takes longer to diagnose than the bug it caught.

Integration with the Validation Pipeline

The test suite is what makes the rule set safe to change, and it belongs in the same repository as the rules, versioned together. A rule change and its test change should arrive in one commit, which is also what makes the rule-set version in the run metadata meaningful — the version identifies both the logic and the evidence that the logic works.

In continuous integration, the fast tier runs on every push and the database tier on every pull request, as described in Running Spatial Validation in GitHub Actions. The golden report test is the natural place to detect an unintended change in output shape before it reaches the consumers who parse it.

Frequently Asked Questions

Related

• Generating Synthetic Invalid Geometries for Tests — constructing each defect class deliberately
• Property-Based Testing of Spatial Rules with Hypothesis — invariants, strategies and shrinking
• Snapshot Testing Validation Reports in Pytest — golden reports without the churn
• Building Rule Engines with GeoPandas — the rule contract these tests assert against
• Continuous Integration for Spatial Validation — where each test tier runs

Back to Validation Pipeline Architecture