Validating GPS Tracks for Speed and Teleport Outliers

A GPS trace is a sequence of claims about where something was. Most are approximately right, a few are badly wrong, and the wrong ones are not distinguishable by inspection: a fix two kilometres off the road looks exactly like a fix on it. What separates them is the sequence — the distance to the previous position, the time it took, and whether the implied movement is physically possible for the thing being tracked. This guide implements those checks: ordering and deduplication, implied-speed computation, accuracy-aware thresholds, teleport classification and stationary drift suppression. It is the sequence-rule layer of Streaming Spatial Data Validation, and the same code runs unchanged over a stored track.

Prerequisites

• Python 3.10+ with pandas 2.x; pyproj 3.6+ if you want geodesic distances rather than the haversine approximation used here.
• Fixes carrying at least an entity identifier, a timestamp with a time zone, longitude, latitude, and ideally a reported horizontal accuracy in metres. Accuracy is the difference between a usable filter and a guess.
• A vehicle or entity class per track, with a maximum plausible speed. Without it, thresholds are arbitrary.
• A clear statement of what the track is used for. Distance-billing, insurance telematics and service-coverage analysis have different tolerances for a dropped fix, and the severity assignment should reflect that.

Step-by-Step Procedure

Step 1 — Order, deduplicate, and detect clock problems

# gps/step1_order.py
import pandas as pd


def prepare(fixes: pd.DataFrame) -> tuple[pd.DataFrame, list[dict]]:
    """Sort by entity and time, drop exact duplicates, and report clock anomalies."""
    findings: list[dict] = []
    df = fixes.copy()
    df["ts"] = pd.to_datetime(df["ts"], utc=True)

    dup_mask = df.duplicated(subset=["entity_id", "ts", "lon", "lat"], keep="first")
    if dup_mask.any():
        findings.append({"rule": "GPS_DUP_001", "severity": "informational",
                         "count": int(dup_mask.sum()),
                         "message": "exact duplicate fixes removed before analysis"})
    df = df[~dup_mask]

    # Two different positions claiming the same instant is a device or ingest fault.
    clash = df.duplicated(subset=["entity_id", "ts"], keep=False)
    if clash.any():
        findings.append({"rule": "GPS_CLOCK_001", "severity": "warning",
                         "count": int(clash.sum()),
                         "message": "multiple distinct positions share one timestamp"})

    df = df.sort_values(["entity_id", "ts"]).reset_index(drop=True)
    return df, findings

Verification: count rows before and after. A device that re-sends its buffer after a reconnect produces large duplicate blocks; seeing them here rather than as phantom stationary periods later is the point of doing this first.

Step 2 — Compute implied speed between consecutive fixes

# gps/step2_speed.py
import numpy as np
import pandas as pd

R_EARTH_M = 6371008.8


def haversine_series(lon1, lat1, lon2, lat2) -> np.ndarray:
    lon1, lat1, lon2, lat2 = map(np.radians, (lon1, lat1, lon2, lat2))
    dlon, dlat = lon2 - lon1, lat2 - lat1
    h = np.sin(dlat / 2) ** 2 + np.cos(lat1) * np.cos(lat2) * np.sin(dlon / 2) ** 2
    return 2 * R_EARTH_M * np.arcsin(np.sqrt(h))


def add_kinematics(df: pd.DataFrame) -> pd.DataFrame:
    """Distance, time delta and implied speed against the previous fix per entity."""
    out = df.copy()
    g = out.groupby("entity_id")
    out["prev_lon"] = g["lon"].shift()
    out["prev_lat"] = g["lat"].shift()
    out["dt_s"] = g["ts"].diff().dt.total_seconds()

    out["dist_m"] = haversine_series(out["prev_lon"], out["prev_lat"], out["lon"], out["lat"])
    out["speed_ms"] = np.where(out["dt_s"] > 0, out["dist_m"] / out["dt_s"], np.nan)
    return out

Verification: plot the speed distribution for one fleet. It should be dominated by a plausible operating range with a thin tail; a spike at implausibly high speeds is the population this guide is about, and a spike near zero-but-not-zero is stationary drift.

Step 3 — Make the threshold accuracy-aware

Raw displacement over-reports movement whenever the fix is noisy. Subtracting the reported accuracy before judging removes most false positives at rest.

# gps/step3_accuracy.py
import numpy as np
import pandas as pd


def effective_movement(df: pd.DataFrame, default_accuracy_m: float = 10.0) -> pd.DataFrame:
    """Displacement net of the positional uncertainty of both fixes."""
    out = df.copy()
    acc = out.get("accuracy_m", pd.Series(default_accuracy_m, index=out.index)).fillna(default_accuracy_m)
    prev_acc = acc.groupby(out["entity_id"]).shift().fillna(default_accuracy_m)

    # Movement we can be confident about: distance minus the combined uncertainty.
    out["confident_dist_m"] = np.maximum(0.0, out["dist_m"] - (acc + prev_acc))
    out["confident_speed_ms"] = np.where(out["dt_s"] > 0,
                                         out["confident_dist_m"] / out["dt_s"], np.nan)
    out["is_stationary"] = out["confident_dist_m"] <= 0
    return out

Verification: take a track from a vehicle you know was parked for an hour. Every fix in that hour should come out with is_stationary true. If a parked hour still shows movement, the reported accuracy is too optimistic — common on consumer devices — and the default should be raised for that device class.

Step 4 — Classify the outlier type

One threshold produces one undifferentiated pile of findings. Four rules produce something a fleet manager can act on.

# gps/step4_classify.py
import pandas as pd

CLASS_MAX_SPEED_MS = {"pedestrian": 3.0, "cyclist": 12.0, "van": 40.0,
                      "hgv": 30.0, "rail": 90.0}
TELEPORT_MIN_DIST_M = 500.0
TELEPORT_MAX_DT_S = 30.0
GAP_MIN_DT_S = 900.0


def classify(df: pd.DataFrame, entity_class: str) -> pd.DataFrame:
    limit = CLASS_MAX_SPEED_MS[entity_class]
    out = df.copy()
    out["finding"] = None
    out["severity"] = None

    gap = out["dt_s"] >= GAP_MIN_DT_S
    out.loc[gap, ["finding", "severity"]] = ["GPS_GAP_001", "informational"]

    teleport = ((out["confident_dist_m"] >= TELEPORT_MIN_DIST_M)
                & (out["dt_s"] <= TELEPORT_MAX_DT_S))
    out.loc[teleport, ["finding", "severity"]] = ["GPS_TELEPORT_001", "blocker"]

    overspeed = (~teleport) & (~gap) & (out["confident_speed_ms"] > limit)
    out.loc[overspeed, ["finding", "severity"]] = ["GPS_SPEED_001", "warning"]

    # Sustained over-speed across three or more consecutive fixes is a different problem
    # from a single bad fix: it usually means a clock or unit error, not GPS noise.
    run = overspeed.groupby((~overspeed).cumsum()).transform("sum")
    out.loc[overspeed & (run >= 3), ["finding", "severity"]] = ["GPS_SPEED_002", "blocker"]

    drift = out["is_stationary"] & (out["dist_m"] > 0)
    out.loc[out["finding"].isna() & drift, ["finding", "severity"]] = [
        "GPS_DRIFT_001", "informational"]
    return out

Verification: the class distribution of findings is diagnostic in itself. A track dominated by GPS_TELEPORT_001 singletons has a noisy receiver; one dominated by GPS_SPEED_002 runs has a unit or clock problem — for instance, timestamps in milliseconds treated as seconds, which multiplies every speed by a thousand.

Step 5 — Emit per-fix findings and a track summary

# gps/step5_emit.py
import pandas as pd


def emit(df: pd.DataFrame) -> tuple[pd.DataFrame, dict]:
    """One row per suspect fix, plus a per-track summary for the run report."""
    findings = df[df["finding"].notna()][
        ["entity_id", "ts", "lon", "lat", "dist_m", "dt_s",
         "confident_speed_ms", "finding", "severity"]
    ].copy()

    summary = {
        "entity_count": int(df["entity_id"].nunique()),
        "fix_count": int(len(df)),
        "suspect_fixes": int(len(findings)),
        "suspect_share": round(len(findings) / len(df), 4) if len(df) else 0.0,
        "by_rule": findings["finding"].value_counts().to_dict(),
        "blocker_entities": sorted(
            findings.loc[findings["severity"] == "blocker", "entity_id"].unique().tolist()),
    }
    return findings, summary

Verification: the summary should be small enough to log on every run and rich enough to trend. Watching suspect_share per fleet over weeks catches a degrading receiver long before anyone reports a problem.

Interpreting Results

The distinction to preserve is between a bad fix and a bad track. Isolated teleports affect one position and leave the rest of the trace usable, so the right response is a per-fix exclusion in the derived product. Sustained over-speed, systematic clock issues or a mis-declared entity class invalidate the whole track, and patching individual fixes there produces a plausible trace that is wrong end to end.

Distance totals deserve particular care. Summing raw segment distances over a noisy stationary period can add kilometres to a vehicle that never moved, which matters directly when the total drives billing or emissions reporting. Sum confident_dist_m rather than dist_m for any figure that leaves the team.

Gotchas & Edge Cases

Haversine underestimates on long segments. For fixes seconds apart it is exact enough. For a gap of an hour spanning a hundred kilometres, use a geodesic distance from pyproj.Geod — the difference reaches several tenths of a percent, which matters if the number is billed.

Reported accuracy is a device’s opinion. Some hardware reports a constant value regardless of conditions, which makes the accuracy-aware filter a no-op. Compare reported accuracy against observed scatter during known-stationary periods, and override the default per device model where they disagree.

Time zones and clock skew masquerade as speed errors. A device reporting local time while the pipeline assumes UTC produces an hour-long negative time delta at every boundary. Parse timestamps as timezone-aware and reject naive ones at ingestion.

Interpolating across a gap invents data. Filling a fifteen-minute reporting gap with a straight line is convenient and false — the vehicle was on roads, not on a chord. If a continuous path is needed, map-match it and record that the path is inferred.

Altitude is usually worse than horizontal position. GPS vertical error is typically two to three times the horizontal, so speed computed in three dimensions is noisier than the two-dimensional version. Use horizontal distance unless the vertical component is genuinely required.

When to Escalate

• A whole fleet’s suspect_share rises at once — that is a platform or firmware change, not a data problem. Escalate to the tracker vendor with the date and the affected device models.
• Sustained over-speed on a specific route may be a road-network or map-matching issue rather than a GPS one; check whether the fixes are plausible and the reference network is wrong.
• Blocker-severity findings on tracks used for billing or compliance must go through the exception process rather than being filtered locally, under the accountability model in Data Stewardship Roles and Responsibilities.
• Findings requiring neighbour context — a vehicle reported inside a building, or two vehicles in the same place — are cross-entity questions. They belong in a windowed stage, described in Handling Late and Out-of-Order Spatial Events.

Frequently Asked Questions

Related

• Streaming Spatial Data Validation — where sequence rules sit in a streaming design
• Running Geometry Checks in a Kafka Consumer — executing these rules per event with bounded state
• Categorizing and Prioritizing Spatial Errors — the severity model these findings are assigned under

Back to Streaming Spatial Data Validation