Shapefile vs GeoPackage Schema Enforcement

You have a Shapefile from a supplier or a legacy system, and you need its schema to actually mean something — field names that survive, numerics that distinguish a real zero from a missing value, and constraints that reject bad data on write rather than flagging it three stages later. The Shapefile cannot give you any of that; a GeoPackage can. This page walks through exactly where Shapefile schema enforcement fails, what GeoPackage enforces in its place, and how to migrate one to the other with ogr2ogr and SQL so the resulting schema is enforced, not merely documented. It sits under validating spatial file formats and assumes you have already read the declared schema of your source.

Prerequisites

• GDAL 3.8+ with ogr2ogr and ogrinfo on the path. Confirm with ogrinfo --version. The GPKG driver and its constraint handling are stable from 3.8; earlier versions may not preserve every field type on conversion.
• Python 3.10+ with Fiona 1.9+ for reading schemas, and the standard-library sqlite3 module for constraint DDL. No third-party SQLite package is required — a GeoPackage is an ordinary SQLite database.
• A documented source contract. You need to know the intended full-length field names and the intended nullability, because the Shapefile itself cannot tell you either. If no contract exists, derive one first via attribute schema mapping for spatial datasets.
• Gotcha to flag now: ogr2ogr creating a GeoPackage table does not copy CHECK constraints from anywhere — there is nowhere in a Shapefile to copy them from. You add constraints explicitly after the load, or by pre-creating the target table. Plan for a two-phase migration.

Step-by-Step Procedure

Step 1 — Inspect the Shapefile schema and flag losses

import fiona

def inspect_shapefile(path: str) -> dict:
    with fiona.open(path) as src:
        schema = dict(src.schema)          # {'geometry': ..., 'properties': {...}}
        crs = src.crs
    props = schema["properties"]           # {'parcel_ide': 'str:80', 'value': 'float', ...}

    truncation_suspects = [name for name in props if len(name) == 10]
    collisions = [name for name in props if name[-2:-1] == "_" and name[-1].isdigit()]

    return {
        "properties": props,
        "geometry": schema["geometry"],
        "crs": crs,
        "truncation_suspects": truncation_suspects,
        "collisions": collisions,
    }

report = inspect_shapefile("data/parcels.shp")
print("Suspected truncations:", report["truncation_suspects"])
print("Collision suffixes:   ", report["collisions"])

Verification: every field name reported in truncation_suspects is exactly ten characters — the Shapefile ceiling. If collisions is non-empty, the original export had two names sharing a prefix, and you must recover both intended names from your contract, not from the file.

Step 2 — Build a field mapping to full names and target types

The Shapefile lost your names and blurred your types. Declare the intended target explicitly.

# Left: name as it exists in the Shapefile. Right: (intended name, GeoPackage type)
FIELD_MAP = {
    "parcel_ide": ("parcel_identifier", "TEXT"),
    "owner_na":   ("owner_name",        "TEXT"),
    "owner_n_1":  ("owner_name_alt",     "TEXT"),   # collision-recovered
    "value":      ("assessed_value",     "REAL"),   # NULL-capable in GPKG
    "flag":       ("is_exempt",          "BOOLEAN"),# real bool, not "T"/"F"
    "zoning":     ("zoning_code",        "TEXT"),
}

# Build the SQL SELECT that ogr2ogr will apply during conversion.
select_cols = ", ".join(
    f'"{src}" AS "{dst}"' for src, (dst, _type) in FIELD_MAP.items()
)
SELECT_SQL = f"SELECT {select_cols}, GEOMETRY FROM parcels"
print(SELECT_SQL)

Verification: the generated SELECT_SQL contains one AS alias per source column and preserves GEOMETRY. Every truncated or collided name on the left maps to a distinct full name on the right.

Step 3 — Convert with ogr2ogr

Run the conversion, normalising the CRS and promoting to multi-geometry so mixed single/multi polygons do not fail the write.

import subprocess

def convert(src_shp: str, dst_gpkg: str, layer: str, select_sql: str) -> None:
    cmd = [
        "ogr2ogr",
        "-f", "GPKG",
        "-t_srs", "EPSG:4326",
        "-nln", layer,
        "-nlt", "PROMOTE_TO_MULTI",
        "-dialect", "SQLite",
        "-sql", select_sql,
        dst_gpkg,
        src_shp,
    ]
    subprocess.run(cmd, check=True)

convert("data/parcels.shp", "data/parcels.gpkg", "parcels", SELECT_SQL)

Verification: run ogrinfo -so data/parcels.gpkg parcels and confirm the full-length names appear, the geometry type is Multi Polygon, and the CRS resolves to EPSG:4326. At this point the schema is correct but still unenforced — constraints come next.

Step 4 — Add enforced constraints with SQL

Rebuild the table with NOT NULL, CHECK, and UNIQUE, then add a trigger for a rule a CHECK cannot express. In SQLite the clean way to add table constraints is to create a constrained table and copy rows into it.

import sqlite3

def enforce_constraints(gpkg: str) -> None:
    con = sqlite3.connect(gpkg)
    cur = con.cursor()
    cur.executescript("""
        -- 1. Create a constrained twin of the loaded table.
        CREATE TABLE parcels_enforced (
            fid              INTEGER PRIMARY KEY,
            geom             MULTIPOLYGON,
            parcel_identifier TEXT    NOT NULL UNIQUE,
            owner_name        TEXT    NOT NULL,
            owner_name_alt    TEXT,
            assessed_value    REAL    CHECK (assessed_value IS NULL OR assessed_value >= 0),
            is_exempt         BOOLEAN CHECK (is_exempt IN (0, 1)),
            zoning_code       TEXT    NOT NULL CHECK (length(zoning_code) BETWEEN 1 AND 12)
        );

        -- 2. Copy the loaded rows into the constrained table.
        INSERT INTO parcels_enforced
            (fid, geom, parcel_identifier, owner_name, owner_name_alt,
             assessed_value, is_exempt, zoning_code)
        SELECT fid, geom, parcel_identifier, owner_name, owner_name_alt,
               assessed_value, is_exempt, zoning_code
        FROM parcels;

        -- 3. A trigger for a cross-column rule CHECK cannot enforce:
        --    an exempt parcel must not carry a positive assessed value.
        CREATE TRIGGER trg_exempt_value
        BEFORE INSERT ON parcels_enforced
        FOR EACH ROW WHEN NEW.is_exempt = 1 AND NEW.assessed_value > 0
        BEGIN
            SELECT RAISE(ABORT, 'exempt parcel cannot have a positive assessed_value');
        END;
    """)
    con.commit()
    con.close()

enforce_constraints("data/parcels.gpkg")

Verification: the script runs without error and parcels_enforced contains the same row count as parcels. (For a production GeoPackage you would also register parcels_enforced in the gpkg_contents and gpkg_geometry_columns metadata tables so OGR recognises it as a spatial layer; register it, then drop the unconstrained parcels table.)

Step 5 — Prove the constraints reject bad data

Enforcement is only real if a violating write fails. Test each constraint deliberately.

import sqlite3

def prove_enforcement(gpkg: str) -> None:
    con = sqlite3.connect(gpkg)
    cur = con.cursor()

    cases = [
        ("negative value",
         "INSERT INTO parcels_enforced (parcel_identifier, owner_name, assessed_value, is_exempt, zoning_code) "
         "VALUES ('P-NEG', 'Test', -5.0, 0, 'R1')"),
        ("null required name",
         "INSERT INTO parcels_enforced (parcel_identifier, owner_name, is_exempt, zoning_code) "
         "VALUES ('P-NULL', NULL, 0, 'R1')"),
        ("duplicate identifier",
         "INSERT INTO parcels_enforced (parcel_identifier, owner_name, is_exempt, zoning_code) "
         "VALUES ('P-NEG', 'Dup', 0, 'R1')"),
        ("exempt with value (trigger)",
         "INSERT INTO parcels_enforced (parcel_identifier, owner_name, assessed_value, is_exempt, zoning_code) "
         "VALUES ('P-EX', 'Test', 100.0, 1, 'R1')"),
    ]

    for label, sql in cases:
        try:
            cur.execute(sql)
            print(f"FAIL: {label} was accepted — constraint not enforced")
        except sqlite3.IntegrityError as exc:
            print(f"OK:   {label} rejected → {exc}")
        con.rollback()

    con.close()

prove_enforcement("data/parcels.gpkg")

Verification: all four cases print OK and are rejected. If any prints FAIL, the corresponding constraint or trigger did not apply — re-check the DDL in Step 4.

Interpreting Results

The migration converts three categories of silent Shapefile loss into enforced GeoPackage guarantees:

• Naming. Ten-character truncation and collision suffixes disappear because the AS aliases restored full names, and UNIQUE now guarantees identifier integrity on every future write.
• Nullability. assessed_value can hold a true SQL NULL distinct from 0, and the CHECK (... IS NULL OR ... >= 0) predicate permits missing values while still rejecting negatives — a distinction the Shapefile could not represent at all.
• Typing. is_exempt is a real boolean constrained to (0, 1) rather than a character "T", so downstream code can rely on the type instead of parsing strings.

A clean run of Step 5 means the schema is now enforced by the storage engine. That is the difference that matters: in a Shapefile the schema is advisory documentation; in a constrained GeoPackage it is a gate that every writer — GDAL, QGIS, a Python ETL job — must pass. The same logical contract you enforce here as CHECK constraints can be expressed for interchange formats too, as shown in mapping attribute constraints to GeoJSON schemas.

Gotchas & Edge Cases

ogr2ogr field renaming needs the SQLite dialect. Applying AS aliases through -sql requires -dialect SQLite when the source is a Shapefile, because the default OGR SQL dialect has weaker expression support. Omitting it can pass the truncated names straight through.

Type coercion is one-directional and lossy. Once a numeric NULL was written as 0 in the .dbf, no conversion can distinguish it from a real zero. If the source used a sentinel such as -9999 for missing, translate it during Step 2 with a CASE WHEN value = -9999 THEN NULL ELSE value END expression; otherwise the information is unrecoverable.

A GeoPackage boolean is stored as an integer. SQLite has no dedicated boolean type; BOOLEAN is an affinity that stores 0/1. The CHECK (is_exempt IN (0, 1)) constraint is what actually prevents a stray 2 or "T" from entering — declaring the column BOOLEAN alone does not.

Constraints do not apply retroactively. Adding a CHECK to a table does not re-validate existing rows in SQLite; it only governs future inserts and updates. That is why Step 4 copies rows into a freshly constrained table — the INSERT ... SELECT forces every existing row through the constraints, surfacing any pre-existing violation immediately.

Registering the spatial layer matters for OGR round-trips. If you create parcels_enforced with raw SQL and forget to register it in gpkg_contents and gpkg_geometry_columns, OGR and GeoPandas will not see it as a spatial layer even though the data is present. Register it before dropping the original table.

When to Escalate

The ogr2ogr plus SQL path is right for single-file, moderate-volume migrations where the constraint set is expressible in SQLite. Move to a heavier approach when:

• The dataset is columnar and analytical, or very large. For wide attribute tables scanned by column across millions of rows, a typed columnar store validates faster and pushes range checks into metadata. See the sibling guide validating GeoParquet schemas with PyArrow.
• Constraints span multiple tables or need transactional integrity across datasets. A single GeoPackage file is convenient but a full PostGIS database gives you referential integrity across many layers, richer constraint expressions, and concurrent write control — evaluate that when the QC schema outgrows one file.
• You need distributed enforcement over partitioned data. When validation runs across workers on partitioned inputs, per-file constraints are not enough; coordinate schema enforcement at the batch layer described in batch processing large spatial datasets.

Related:

• Validating Spatial File Formats — the parent overview of how each format constrains schema enforcement
• Validating GeoParquet Schemas with PyArrow — the columnar alternative when GeoPackage is not the right sink
• Attribute Schema Mapping for Spatial Datasets — deriving the contract the enforced constraints implement

Back to Validating Spatial File Formats