Writing Declarative Rule Configs in YAML

Rules encoded as Python functions are easy to write and hard to govern. A data steward who wants to raise a sliver threshold from 0.5 to 0.8 square metres must open a pull request against the validator; a compliance officer who wants to know which rules apply to the parcel layer must read the source. Moving the decisions — scope, thresholds, severity, enablement — into a versioned YAML file while keeping the logic in reviewed Python separates those concerns cleanly. This guide designs that config: a rule schema, a predicate registry instead of expression evaluation, declarative selectors, CI validation, and an escape hatch for the rules that genuinely need code. It builds on the rule contract in Building Rule Engines with GeoPandas.

Prerequisites

  • Python 3.10+, PyYAML 6.x, jsonschema 4.x, geopandas 0.14+.
  • An error code registry, as described in Designing an Error Code Taxonomy for Spatial Defects — the config references codes rather than inventing them.
  • A vectorised predicate library: functions taking a GeoDataFrame and returning a boolean mask.
  • A CI pipeline that can run the config validation on every commit.

Step-by-Step Procedure

Step 1 — Design the rule schema

# rules/parcels.yaml
version: "2026.08.1"
defaults:
  severity_source: registry        # severity comes from the error code registry
  enabled: true

rules:
  - id: GEOM_VALID_001
    description: "Parcel geometry must satisfy OGC validity"
    selector:
      layers: [parcels]
    check: geometry_is_valid
    params: {}

  - id: TOPO_SLIVER_001
    description: "Flag thin, small polygons that are probably overlay artefacts"
    selector:
      layers: [parcels]
      exclude_where: "right_of_way == true"
    check: sliver_polygon
    params:
      max_area_m2: 25.0
      max_compactness: 0.15

  - id: ATTR_RANGE_001
    description: "Assessed value must be plausible"
    selector:
      layers: [parcels]
      where: "record_date >= '2020-01-01'"
    check: numeric_between
    params:
      column: assessed_value
      minimum: 0
      maximum: 500000000
      inclusive: both

  - id: CRS_MISMATCH_001
    description: "Layer must be in the canonical CRS"
    selector:
      layers: [parcels, road_network, water_mains]
    check: crs_equals
    params:
      epsg: 27700
    scope: layer                    # evaluated once per layer, not per feature

  - id: TOPO_OVERLAP_001
    description: "Parcels must not overlap"
    selector:
      layers: [parcels]
    check: no_self_overlap
    params:
      min_overlap_m2: 0.5
      ignore_touching: true
    scope: pairwise

Five fields do the work: id ties to the code registry, selector says where the rule applies, check names a registered predicate, params parameterises it, and scope tells the engine how to evaluate it — per feature, per layer or pairwise. Severity is deliberately absent, because it lives in the registry.

What moves into the config, and what stays in codeTwo-part stack. Configuration holds the decisions: which layers a rule applies to, thresholds, severity source and whether a rule is enabled. Code holds the implementation: the vectorised predicates, the selector evaluator and the engine itself.Config: scopewhich layers, feature classes and time windows — a steward decisioneditable by stewardsConfig: parametersthresholds, tolerances, column names — a steward decisioneditable by stewardsConfig: enablementon, off, or expiring — a steward decisioneditable by stewardsCode: predicatesthe vectorised check functions — reviewed pull requestsengineering changeCode: engineselector evaluation, scheduling, result contract — reviewedengineering changeThe dividing line is whether the change is a decision or an implementation. Everything above the line can ship without a deploy; nothing below it can.
Configuration is for decisions, code is for logic — and an expression evaluator in the config erases the distinction.

Step 2 — Register predicates rather than evaluating expressions

# engine/registry.py
from typing import Callable

import geopandas as gpd
import pandas as pd
import shapely

PREDICATES: dict[str, Callable] = {}


def predicate(name: str, scope: str = "feature"):
    def wrap(fn):
        fn._scope = scope
        PREDICATES[name] = fn
        return fn
    return wrap


@predicate("geometry_is_valid")
def geometry_is_valid(gdf: gpd.GeoDataFrame) -> pd.Series:
    """True where the feature FAILS the rule — the engine reports the True rows."""
    return ~gdf.geometry.map(shapely.is_valid)


@predicate("numeric_between")
def numeric_between(gdf: gpd.GeoDataFrame, column: str, minimum: float,
                    maximum: float, inclusive: str = "both") -> pd.Series:
    series = pd.to_numeric(gdf[column], errors="coerce")
    inside = series.between(minimum, maximum, inclusive=inclusive)
    return ~inside.fillna(False)          # non-numeric counts as a failure


@predicate("sliver_polygon")
def sliver_polygon(gdf: gpd.GeoDataFrame, max_area_m2: float,
                   max_compactness: float) -> pd.Series:
    area = gdf.geometry.area
    perimeter = gdf.geometry.length
    compactness = (4 * 3.141592653589793 * area) / perimeter.pow(2).replace(0, pd.NA)
    return (area < max_area_m2) & (compactness < max_compactness)


@predicate("crs_equals", scope="layer")
def crs_equals(gdf: gpd.GeoDataFrame, epsg: int) -> bool:
    return gdf.crs is None or gdf.crs.to_epsg() != epsg


@predicate("no_self_overlap", scope="pairwise")
def no_self_overlap(gdf: gpd.GeoDataFrame, min_overlap_m2: float,
                    ignore_touching: bool = True) -> pd.DataFrame:
    joined = gpd.sjoin(gdf[["geometry"]], gdf[["geometry"]],
                       predicate="intersects", how="inner")
    joined = joined[joined.index < joined["index_right"]]
    pairs = []
    for left, right in joined["index_right"].items():
        a, b = gdf.geometry.loc[left], gdf.geometry.loc[right]
        if ignore_touching and a.touches(b):
            continue
        overlap = a.intersection(b).area
        if overlap >= min_overlap_m2:
            pairs.append({"feature_ids": [str(left), str(right)],
                          "overlap_m2": round(float(overlap), 3)})
    return pd.DataFrame(pairs)

Verification: every predicate returns “True means this feature violates the rule”, consistently. Mixed conventions — some returning pass, some returning fail — is the single most common bug in a home-grown rule engine, and it inverts findings silently.

Step 3 — Selectors, evaluated safely

# engine/selector.py
import geopandas as gpd


def apply_selector(gdf: gpd.GeoDataFrame, layer_id: str, selector: dict):
    """Restrict the frame to the rows a rule applies to. Returns None if it does not apply."""
    if layer_id not in selector.get("layers", [layer_id]):
        return None

    subset = gdf
    if "where" in selector:
        subset = subset.query(selector["where"], engine="python")
    if "exclude_where" in selector:
        excluded = subset.query(selector["exclude_where"], engine="python").index
        subset = subset.drop(index=excluded)
    return subset if len(subset) else None

Verification: DataFrame.query is a restricted expression language over columns, not arbitrary Python — it cannot import, call out or mutate state. That makes it safe enough for a config file, unlike eval. Test a selector referencing a missing column and confirm it raises clearly rather than silently selecting everything.

Step 4 — Validate the config in CI

# engine/validate_config.py
import json

import jsonschema
import yaml

from engine.registry import PREDICATES
from taxonomy.step4_enforce import load_registry

CONFIG_SCHEMA = {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "required": ["version", "rules"],
    "properties": {
        "version": {"type": "string", "pattern": r"^\d{4}\.\d{2}\.\d+$"},
        "rules": {
            "type": "array", "minItems": 1,
            "items": {
                "type": "object",
                "required": ["id", "description", "selector", "check"],
                "additionalProperties": False,
                "properties": {
                    "id": {"type": "string", "pattern": "^[A-Z]+_[A-Z]+_[0-9]{3}$"},
                    "description": {"type": "string", "minLength": 10},
                    "selector": {
                        "type": "object",
                        "required": ["layers"],
                        "properties": {
                            "layers": {"type": "array", "items": {"type": "string"},
                                       "minItems": 1},
                            "where": {"type": "string"},
                            "exclude_where": {"type": "string"},
                        },
                    },
                    "check": {"type": "string"},
                    "params": {"type": "object"},
                    "scope": {"enum": ["feature", "layer", "pairwise"]},
                    "enabled": {"type": "boolean"},
                },
            },
        },
        "defaults": {"type": "object"},
    },
}


def validate(path: str) -> list[str]:
    doc = yaml.safe_load(open(path, encoding="utf-8"))
    problems = []

    try:
        jsonschema.validate(doc, CONFIG_SCHEMA)
    except jsonschema.ValidationError as exc:
        problems.append(f"schema: {exc.message} at {list(exc.absolute_path)}")
        return problems                       # structure first; semantics are meaningless otherwise

    registry = load_registry()
    seen = set()
    for rule in doc["rules"]:
        if rule["check"] not in PREDICATES:
            problems.append(f"{rule['id']}: unknown check {rule['check']!r}")
        if rule["id"] in seen:
            problems.append(f"{rule['id']}: duplicate rule id")
        seen.add(rule["id"])
        try:
            registry.require(rule["id"])
        except (KeyError, ValueError) as exc:
            problems.append(f"{rule['id']}: {exc}")
        fn = PREDICATES.get(rule["check"])
        if fn is not None:
            declared = rule.get("scope", "feature")
            if getattr(fn, "_scope", "feature") != declared:
                problems.append(
                    f"{rule['id']}: scope {declared!r} does not match predicate "
                    f"scope {getattr(fn, '_scope')!r}")
    return problems
# .github/workflows/rules.yml (fragment)
- name: Validate rule configuration
  run: |
    python -c "
    from engine.validate_config import validate
    import sys, glob
    bad = [(f, p) for f in glob.glob('rules/*.yaml') for p in validate(f)]
    for f, p in bad: print(f'{f}: {p}')
    sys.exit(1 if bad else 0)"

Verification: break the config four ways — unknown check, duplicate id, unregistered code, mismatched scope — and confirm each is reported distinctly. A config validator that reports only “invalid” is barely better than the runtime error it was meant to prevent.

Step 5 — Keep an escape hatch, and mark it

  - id: TOPO_CADASTRAL_SEAM_001
    description: "Boundary must match the neighbouring authority's registered edge"
    selector:
      layers: [parcels]
      where: "adjoins_authority_boundary == true"
    check: custom.cadastral_seam_match     # namespaced: this is Python, not config
    params:
      reference_layer: "ref.authority_boundaries"
      tolerance_m: 0.05
# engine/custom_rules.py
from engine.registry import predicate


@predicate("custom.cadastral_seam_match")
def cadastral_seam_match(gdf, reference_layer: str, tolerance_m: float):
    """Genuinely custom: needs a reference layer join and authority-specific logic.
    Registered like any predicate, but namespaced so config reviewers can see that
    changing its parameters is not the same as changing a standard threshold."""
    ...

Verification: the custom. prefix is a documentation device with teeth — a config review can filter on it and ask whether each one still needs to be custom. Escape hatches that are not visible accumulate until the config is a thin wrapper over bespoke code.

Interpreting Results

Config review signal What it means Action
Many rules with the same check, different params The parameterisation is working Nothing — this is the goal
Growing number of custom. predicates Logic is leaking back into code Review whether a general predicate would cover several
Rules disabled but not deleted Temporary suppression that became permanent Set an expiry convention; review quarterly
Selectors with long where clauses Business logic migrating into the selector Consider a derived column instead
Config version unchanged across rule edits Version discipline broken Enforce a version bump in CI
Identical rules duplicated per layer Missing multi-layer selector Collapse into one rule with a layer list

The last row is worth catching early. A config with TOPO_OVERLAP_001 repeated four times for four layers means four places to change a threshold, and they will diverge.

How a config becomes a running ruleSequence diagram between the loader, the JSON Schema validator, the predicate registry and the engine. The config is parsed once at startup, validated structurally, resolved against the registry, and only then are callables handed to the workers.loaderschemaregistryenginevalidate structurestructural errors, if anyresolve check names to callablesbound predicate + declared scoperesolved rules, parsed onceParsing once at startup is what keeps declarative configuration free at runtime — re-parsing per partition turns a negligible cost into a dominant one.
The registry resolution step is where an unknown check name fails, and it fails at startup rather than mid-run.

Gotchas & Edge Cases

YAML type coercion surprises. no parses as boolean false, 27700 as an integer, 01 as a string in YAML 1.2 and as an octal in some 1.1 parsers. Quote anything ambiguous and validate types in the schema rather than trusting the parser.

Four config errors, and which layer catches eachGrid of four config mistakes with the layer that detects each and the message it produces: a mistyped key, an unknown check name, a duplicate rule identifier and a scope mismatch.Caught byMessageMistyped keyJSON SchemaadditionalProperties not allowedUnknown check nameregistry resolutionunknown check "sliver_polgyon"Duplicate rule idconfig validatorduplicate rule id GEOM_VALID_001Scope mismatchregistry resolutiondeclared "feature", predicate is "pairwise"Only the first is a structural error. The other three need the registry, which is why schema validation alone is not enough.
Two validation layers, because a config can be perfectly well-formed and still reference something that does not exist.

Parse the config once. Building callables per partition or per feature turns a negligible cost into a dominant one. Parse at startup, hold the resolved rule objects, and pass those to workers.

Config changes are rule-set changes. Bumping a threshold changes the findings, so the config version must travel into the run metadata exactly like a code version — otherwise a step in the trend has no explanation, as discussed in Observability and Lineage for Validation.

DataFrame.query with engine="python" is slower but predictable. The numexpr engine does not support all operations and silently falls back; pinning the engine avoids behaviour that differs between environments.

A disabled rule is not the same as a deleted one. Disabling leaves the code in the registry and the history intact, which is right for a temporary suppression and wrong as a permanent state. Set an expiry date in the config and check it in CI.

Do not let the config express ordering. Rules should be independent; a config where rule B depends on rule A having run is a pipeline, not a rule set, and belongs in the orchestration layer.

When to Escalate

  • A steward asking for a rule that no predicate supports is a feature request for the engine, not a config change. Handle it through the normal development path rather than by adding an expression evaluator.
  • Pressure to allow inline expressions should be resisted with the security argument first and the tooling argument second. A config that executes code has the blast radius of code and none of its safeguards.
  • Config and registry disagreeing — a rule referencing a deprecated code, for instance — should fail the build. If that is blocking a release, fix the config, not the check.
  • Rules whose scope nobody can state usually indicate an unclear data contract. That belongs with the steward, following Defining Spatial Data Quality Policies.

Frequently Asked Questions

Why not allow Python expressions in the YAML?

Because a config that can execute arbitrary code is code with worse tooling — no type checking, no tests, no debugger, and a serious security problem if the file is ever editable by someone who should not run code on your infrastructure. Register named predicates instead: the config selects from a vetted set, and new behaviour arrives through a reviewed pull request.

What belongs in the config and what stays in code?

Anything a data steward should be able to change without an engineer: which rules apply to which layers, thresholds, severities, and whether a rule is enabled. Anything requiring new logic — a predicate that does not exist yet — stays in code. The dividing line is whether the change is a decision or an implementation.

How are config errors caught before production?

With a JSON Schema for structure and a registry check for semantics, both running in continuous integration. Structure catches a missing severity or a mistyped key; the registry check catches a reference to a predicate that does not exist, which is by far the more common mistake and the one a schema alone cannot detect.

Does declarative configuration slow the engine down?

No, if the config is parsed once into callables at startup. The per-feature work is identical because it is the same vectorised predicate underneath. What does slow things down is re-parsing the config per partition or building a new closure per feature — both easy to avoid, and both worth checking for in a profile.

Related

Back to Building Rule Engines with GeoPandas