Integrating Great Expectations with GeoPandas

You already validate spatial data in code, but a steward, auditor, or compliance officer needs to see that validation happened — which checks ran, what passed, what failed, and against which dataset version — in a report they can open in a browser. That is the gap Great Expectations fills for a GeoDataFrame: a versioned expectation suite, a persistent run history, and rendered Data Docs. This page shows how to bridge Great Expectations to GeoPandas with custom geometry expectations, render the evidence, and run a checkpoint inside a pipeline. It is the batch-and-evidence companion to the Declarative Validation Frameworks for Geospatial Data overview and targets in-memory datasets validated on a single node.

Prerequisites

• Python 3.10+ and Great Expectations 0.18+ (pip install "great-expectations>=0.18,<0.19"). The 0.18 fluent application programming interface used here differs from both the older 0.15 batch-request style and the newer 1.x line; pin the minor version so the context, data-source, and checkpoint calls behave as written.
• GeoPandas 0.14+ and Shapely 2.0+. Every geometry expectation is precomputed with the vectorised shapely.is_valid. Confirm shapely.__version__ returns 2.x.
• A canonical coordinate reference system (CRS) fixed in advance. The CRS-match expectation compares each layer’s projection to one target EPSG code; decide it before authoring the suite. See Coordinate Reference System Precision Standards.
• A writable project directory. Great Expectations persists the context, suites, and rendered Data Docs to disk. Use a file-backed context (gx.get_context(mode="file")) so runs accumulate a browsable history rather than vanishing with the process.
• Familiarity with the shared error contract. Checkpoint output is mapped into the pipeline’s ValidationResult shape so failures route like any other spatial error, as covered in Categorizing and Prioritizing Spatial Errors.

Step-by-Step Procedure

Step 1 — Precompute geometry expectations with Shapely

Great Expectations asserts on scalar columns, not geometry objects. Derive the boolean columns the suite will check. This is where all spatial truth is computed.

import geopandas as gpd
import shapely
from shapely.geometry import box

def add_geometry_flags(gdf: gpd.GeoDataFrame,
                       target_epsg: int,
                       bounds: tuple[float, float, float, float]) -> gpd.GeoDataFrame:
    out = gdf.copy()
    out["geometry_valid"] = shapely.is_valid(gdf.geometry.values)
    out["geometry_nonempty"] = ~shapely.is_empty(gdf.geometry.values)
    out["crs_match"] = (gdf.crs is not None) and (gdf.crs.to_epsg() == target_epsg)
    bbox = box(*bounds)
    out["within_bounds"] = gdf.geometry.intersects(bbox).values
    return out

Verification: on a fixture with one bad geometry, confirm exactly one row reports geometry_valid == False.

from shapely.geometry import Polygon
valid  = Polygon([(0, 0), (0, 1), (1, 1), (1, 0), (0, 0)])
bowtie = Polygon([(0, 0), (1, 1), (1, 0), (0, 1), (0, 0)])
gdf = gpd.GeoDataFrame(
    {"parcel_id": ["P-001", "P-002"], "area_m2": [1000.0, 2500.0],
     "geometry": [valid, bowtie]}, crs="EPSG:4326")
flagged = add_geometry_flags(gdf, target_epsg=4326, bounds=(-180, -90, 180, 90))
assert (~flagged["geometry_valid"]).sum() == 1

Step 2 — Initialise a context and register the batch

Create a file-backed context, add a pandas data source, and define a whole-dataframe batch over the flagged frame.

import great_expectations as gx

context = gx.get_context(mode="file")   # persists suites and Data Docs to disk

data_source = context.data_sources.add_or_update_pandas("parcels_source")
asset = data_source.add_dataframe_asset(name="parcels_asset")
batch_definition = asset.add_batch_definition_whole_dataframe("parcels_batch")

Verification: fetch the batch and confirm the derived columns are present before building any expectation.

batch = batch_definition.get_batch(batch_parameters={"dataframe": flagged})
cols = batch.columns()
assert {"geometry_valid", "crs_match", "within_bounds"}.issubset(set(cols))

Step 3 — Build the expectation suite

Assert the geometry flags are all True, then add attribute expectations. Each expectation is a declarative, versionable statement about the batch.

suite = context.suites.add_or_update(gx.ExpectationSuite(name="parcel_geometry_contract"))

# Geometry contract — backed by the Shapely-derived boolean columns
for col in ["geometry_valid", "geometry_nonempty", "crs_match", "within_bounds"]:
    suite.add_expectation(
        gx.expectations.ExpectColumnValuesToBeInSet(column=col, value_set=[True])
    )

# Attribute contract
suite.add_expectation(
    gx.expectations.ExpectColumnValuesToBeBetween(column="area_m2", min_value=0, strict_min=True)
)
suite.add_expectation(
    gx.expectations.ExpectColumnValuesToBeUnique(column="parcel_id")
)

Verification: validate the batch directly and confirm the invalid-geometry row makes the suite fail.

result = batch.validate(suite)
print(result.success)   # False — the bowtie fails geometry_valid

Step 4 — Run a checkpoint and render Data Docs

A checkpoint bundles the batch, the suite, and post-run actions into one callable — the unit an orchestrator invokes. Its action list rebuilds Data Docs so evidence regenerates on every run.

checkpoint = context.checkpoints.add_or_update(
    gx.Checkpoint(
        name="parcels_checkpoint",
        validation_definitions=[
            context.validation_definitions.add_or_update(
                gx.ValidationDefinition(
                    name="parcels_validation",
                    data=batch_definition,
                    suite=suite,
                )
            )
        ],
        actions=[gx.checkpoint.actions.UpdateDataDocsAction(name="update_docs")],
    )
)

results = checkpoint.run(batch_parameters={"dataframe": flagged})
print("Checkpoint success:", results.success)
context.open_data_docs()   # opens the browsable HTML report

Verification: open Data Docs and confirm the geometry_valid expectation is listed as failed with one unexpected value; the HTML is the artifact a steward or auditor reads.

Step 5 — Adapt checkpoint output to the pipeline contract

Translate the run into the shared ValidationResult shape so a failed expectation routes like any other spatial error.

from dataclasses import dataclass

@dataclass
class ValidationResult:
    feature_id: str
    rule_name: str
    passed: bool
    failure_reason: str | None = None
    severity: str = "warning"

def checkpoint_to_results(results, dataset: str) -> list[ValidationResult]:
    out = []
    for run_result in results.run_results.values():
        for exp in run_result["validation_result"]["results"]:
            cfg = exp["expectation_config"]
            out.append(ValidationResult(
                feature_id=dataset,
                rule_name=f"ge::{cfg['type']}::{cfg['kwargs'].get('column', '')}",
                passed=exp["success"],
                failure_reason=None if exp["success"] else str(exp["result"]),
                severity="blocker" if not exp["success"] else "informational",
            ))
    return out

Verification: confirm each emitted result carries a rule_name, passed flag, and severity, matching the schema the downstream router consumes.

Interpreting Results

The checkpoint returns a result object whose success flag is True only if every expectation passed. Drill into run_results for per-expectation detail: each entry records the expectation type, the target column, the observed count of unexpected values, and a sample of failing rows. The geometry_valid expectation failing with one unexpected value maps straight back to the single invalid geometry Shapely flagged in Step 1 — the boolean column is the join between spatial truth and the report.

Data Docs render this same information as HTML: a run summary, a table of expectations with pass/fail badges, and expandable failure details. This is the deliverable for non-engineers. Where Categorizing and Prioritizing Spatial Errors consumes the machine-readable ValidationResult stream to score and route failures, Data Docs serve the human audit trail — the two are produced from one checkpoint run.

Gotchas & Edge Cases

Expectations cannot read Shapely objects. Passing a raw geometry column to any built-in expectation validates the string representation, not the geometry. Always precompute boolean flags with Shapely first; the geometry column itself is never the direct subject of an expectation.

An empty batch reports success. Every column-values expectation is vacuously true over zero rows, so a checkpoint on an accidentally filtered-away GeoDataFrame passes green. Add ExpectTableRowCountToBeBetween with a realistic floor so an empty batch fails loudly rather than silently.

crs_match is a per-batch scalar, not a per-row property. Because CRS applies to the whole layer, add_geometry_flags writes the same boolean to every row. That is intentional — but do not interpret a crs_match failure as “some features have the wrong CRS”; it means the entire layer is in the wrong projection.

Version drift breaks the fluent API. The context, data-source, and checkpoint calls shown here target Great Expectations 0.18. The 0.15 batch-request style and the 1.x rewrite both use different method names. Pin the minor version in your lockfile and in the continuous integration / continuous delivery (CI/CD) environment, or the suite that passes locally will error in the pipeline.

Data Docs go stale without an action. Building a suite or running a validation does not refresh the HTML on its own. Include an UpdateDataDocsAction in the checkpoint’s action list so every run regenerates the evidence; otherwise auditors read yesterday’s result.

When to Escalate

Great Expectations is a column-level contract and evidence layer. Move beyond it when:

• You need cross-feature topology checks. Overlap detection, gap analysis, and network connectivity require spatial joins that no single-column expectation can express. Keep that logic in the engine described in Building Rule Engines with GeoPandas, and run the checkpoint as the front gate.
• The dataset outgrows a single node. The pandas backend holds the whole frame in memory. Above roughly two million features, switch to the Spark backend or validate Dask-GeoPandas partitions, mirroring the scale-out in Declarative Validation Frameworks for Geospatial Data.
• Record-level API validation is the real need. If features arrive one at a time and you want per-field errors at the boundary rather than a batch report, use the model approach in Declarative Schema Validation with Pydantic instead.
• Invalid features must be repaired, not just reported. Great Expectations flags failures; it does not fix geometry. Route failing rows through the make_valid repair patterns in Implementing Shapely Geometry Checks in Python.

Frequently Asked Questions

Related:

• Declarative Validation Frameworks for Geospatial Data — how Great Expectations, pandera, and Pydantic complement each other
• Declarative Schema Validation with Pydantic — the record-level counterpart for boundary validation
• Categorizing and Prioritizing Spatial Errors — scoring and routing the failures a checkpoint surfaces

Back to Declarative Validation Frameworks for Geospatial Data