Publishing Validation Dashboards with Grafana

Q: Should the dashboard read the findings table directly?

No. Findings tables grow into the tens of millions of rows and a dashboard that scans them on every refresh becomes slow and, in a metered warehouse, expensive. Point Grafana at the daily aggregate: it is small, it is fast, and it is the same table the scorecard and the executive report read, which keeps every view consistent.

The operational view of spatial quality answers two questions in order: did the run work, and is the data getting worse. Most dashboards answer the second one first, which is how a team ends up investigating a “defect spike” that turns out to be a partial run over a tenth of the usual features. This guide builds the dashboard the other way round — run health at the top, rates with their denominators below, alerts against a rolling baseline, annotations for rule and scope changes — and provisions the whole thing as code so it is versioned with the pipeline. It renders the metric set defined in Spatial Data Quality Metrics and Reporting.

Prerequisites

Grafana 10 or 11, with a data source for wherever your metrics live — PostgreSQL, BigQuery, Snowflake or Prometheus. The SQL below is written for a PostgreSQL-compatible source.
A daily aggregate table with one row per run per layer: run date, layer, features validated, defect counts by severity, weighted score and rule-set version.
A run status table recording whether each scheduled run started, succeeded and how long it took. Without this, “no data” and “no defects” look identical.
Provisioning access — a directory Grafana reads for dashboards and alert rules, so the dashboard is deployed rather than clicked together.

Step-by-Step Procedure

Step 1 — Point Grafana at the aggregate, not the findings

-- The view Grafana reads. Small, indexed, one row per run per layer.
CREATE OR REPLACE VIEW qa.metrics_daily_v AS
SELECT
  run_date::timestamptz          AS "time",
  layer_id,
  rule_set_version,
  features_validated,
  blocker_count,
  warning_count,
  info_count,
  1000.0 * blocker_count / GREATEST(features_validated, 1) AS blocker_per_1k,
  1000.0 * warning_count / GREATEST(features_validated, 1) AS warning_per_1k,
  clean_share,
  run_seconds,
  run_status
FROM qa.metrics_daily;

CREATE INDEX IF NOT EXISTS metrics_daily_time_layer
  ON qa.metrics_daily (run_date DESC, layer_id);

Verification: time a query for the last 90 days across all layers. It should return in well under a second; if it does not, the dashboard will feel broken regardless of how the panels are arranged.

Run health first. A defect spike means something entirely different after a partial run.

Step 2 — Run health above data health

{
  "title": "Spatial validation — operations",
  "panels": [
    {
      "type": "stat", "title": "Runs completed (last 24h)", "gridPos": {"h": 4, "w": 4, "x": 0, "y": 0},
      "targets": [{"rawSql": "SELECT count(*) FROM qa.metrics_daily WHERE run_date >= now() - interval '1 day' AND run_status = 'success'", "format": "table"}],
      "fieldConfig": {"defaults": {"thresholds": {"steps": [
        {"color": "red", "value": null}, {"color": "green", "value": 12}]}}}
    },
    {
      "type": "stat", "title": "Layers with no run today", "gridPos": {"h": 4, "w": 4, "x": 4, "y": 0},
      "targets": [{"rawSql": "SELECT count(*) FROM qa.expected_layers e WHERE NOT EXISTS (SELECT 1 FROM qa.metrics_daily m WHERE m.layer_id = e.layer_id AND m.run_date = current_date)", "format": "table"}],
      "fieldConfig": {"defaults": {"thresholds": {"steps": [
        {"color": "green", "value": null}, {"color": "red", "value": 1}]}}}
    },
    {
      "type": "timeseries", "title": "Features validated per layer", "gridPos": {"h": 8, "w": 16, "x": 8, "y": 0},
      "targets": [{"rawSql": "SELECT \"time\", layer_id, features_validated FROM qa.metrics_daily_v WHERE $__timeFilter(\"time\") ORDER BY 1", "format": "time_series"}],
      "description": "The denominator behind every rate below. A dip here explains most rate spikes."
    }
  ]
}

Verification: the “layers with no run today” panel is the one that earns its place. A missing run is invisible on a defect chart — the line simply stops — and this panel turns that silence into a red number.

Step 3 — Rates, always with their denominator

-- Blocker rate with the denominator on a second axis, per layer.
SELECT
  "time",
  layer_id || ' — blocker/1k' AS metric,
  blocker_per_1k              AS value
FROM qa.metrics_daily_v
WHERE $__timeFilter("time") AND layer_id IN ($layer)
UNION ALL
SELECT
  "time",
  layer_id || ' — features (right axis)' AS metric,
  features_validated          AS value
FROM qa.metrics_daily_v
WHERE $__timeFilter("time") AND layer_id IN ($layer)
ORDER BY 1;

-- Which rules are driving today's blockers — the drill-down table panel.
SELECT rule_id,
       count(*)                                            AS findings,
       count(DISTINCT feature_id)                          AS features_affected,
       round(1000.0 * count(*) /
             GREATEST(max(m.features_validated), 1), 2)    AS per_1k
FROM qa.findings f
JOIN qa.metrics_daily m
  ON m.run_date = f.run_date AND m.layer_id = f.layer_id
WHERE f.run_date = current_date
  AND f.severity = 'blocker'
  AND f.layer_id IN ($layer)
GROUP BY rule_id
ORDER BY findings DESC
LIMIT 20;

Verification: click through from the rate chart to the drill-down table for a day with a spike. If the table does not immediately name the responsible rule, the panels are not linked usefully — add a data link from the time series to the table filtered by the clicked day.

Step 4 — Alert on rates against a rolling baseline

# provisioning/alerting/spatial-qc.yaml
apiVersion: 1
groups:
  - orgId: 1
    name: spatial-quality
    folder: Data Quality
    interval: 5m
    rules:
      - uid: blocker-rate-above-baseline
        title: Blocker rate above its 30-day 95th percentile
        condition: breach
        for: 30m                      # one noisy run must not page anyone
        annotations:
          summary: >-
            {{ $labels.layer_id }} blocker rate {{ $values.rate }} per 1k exceeds
            its 30-day p95 of {{ $values.baseline }}.
          runbook_url: https://runbooks.internal/spatial-qc/blocker-rate
        labels:
          severity: page
          team: geospatial-platform
        data:
          - refId: rate
            datasourceUid: pg-metrics
            model:
              rawSql: |
                SELECT layer_id, blocker_per_1k AS value
                FROM qa.metrics_daily_v
                WHERE run_date = current_date
          - refId: baseline
            datasourceUid: pg-metrics
            model:
              rawSql: |
                SELECT layer_id,
                       percentile_cont(0.95) WITHIN GROUP (ORDER BY blocker_per_1k) AS value
                FROM qa.metrics_daily_v
                WHERE run_date BETWEEN current_date - 30 AND current_date - 1
                GROUP BY layer_id
          - refId: breach
            type: math
            model: {expression: "$rate > $baseline * 1.5"}

      - uid: run-missing
        title: Expected validation run did not complete
        condition: missing
        for: 0m
        labels: {severity: page, team: geospatial-platform}
        data:
          - refId: missing
            datasourceUid: pg-metrics
            model:
              rawSql: |
                SELECT count(*) AS value
                FROM qa.expected_layers e
                WHERE NOT EXISTS (
                  SELECT 1 FROM qa.metrics_daily m
                  WHERE m.layer_id = e.layer_id AND m.run_date = current_date
                    AND m.run_status = 'success')

Verification: two alerts is the right order of magnitude for a starting point — one for data health, one for pipeline health. Add a third only when a specific incident proves it was needed; alerts added speculatively are the ones that get muted.

Step 5 — Annotate changes and provision as code

-- Annotations come from the same database, so a rule deployment shows up on every chart.
CREATE TABLE IF NOT EXISTS qa.annotations (
  at            timestamptz NOT NULL,
  kind          text NOT NULL,          -- 'rule_change' | 'scope_change' | 'incident'
  layer_id      text,
  text          text NOT NULL,
  rule_set_version text
);

-- Grafana annotation query
SELECT at AS "time", text, kind AS tags
FROM qa.annotations
WHERE $__timeFilter(at) AND (layer_id IS NULL OR layer_id IN ($layer));

# Dashboards live in the repository and deploy with the pipeline.
grafana/
  provisioning/
    dashboards/spatial-qc.yaml        # loader config
    alerting/spatial-qc.yaml          # the rules above
  dashboards/
    spatial-qc-operations.json
    spatial-qc-scorecard.json

Verification: delete the dashboard in the Grafana UI and redeploy. If it comes back identical, the provisioning is genuinely the source of truth. If it does not, someone has been editing in the UI and those edits will be lost eventually anyway — better to discover that during a test.

Interpreting Results

What you see	Read it as	Next step
Rate spike, denominator dip	A partial run, not a data event	Check the pipeline before the data
Rate step, annotation at the same point	A rule or scope change	Nothing to investigate; the annotation is the answer
Gradual rate climb over weeks	Genuine upstream drift	Steward escalation; this is the real signal
Flat rate, rising absolute counts	The dataset is growing	Working as intended — this is why rates are used
Alert fires and clears within an hour	Threshold too tight, or `for` too short	Widen the baseline multiplier before muting
Clean share falling while blocker rate is flat	Warnings spreading across more features	Look at warning breadth, not just severity mix

The rate-with-denominator pairing does most of the interpretive work here. Once every rate panel carries its denominator, the “is this real?” question is answered on the same screen, and the number of investigations that end in “oh, the run was short” drops to near zero.

Two alerts is the right order of magnitude to start with; add a third only when an incident proves it was needed.

Gotchas & Edge Cases

Missing data renders as a gap, and gaps are invisible at a glance. Configure the time series to show points as well as lines, or add the explicit “layers with no run” panel — a line that simply stops does not draw the eye.

Annotate every rule and scope change — an unexplained step costs a month of investigation.

Template variables silently narrow alerts. A dashboard variable filtering to one layer is fine for viewing; an alert rule that inherits a variable evaluates against whatever the last saved state was. Alert queries must be fully specified with no variables.

Timezone mismatches shift every daily boundary. Grafana renders in the browser’s timezone by default while the aggregate is keyed on a date computed in UTC. Set the dashboard timezone explicitly to match how the pipeline defines a day.

Percentile baselines need enough history. A 30-day p95 computed over eight runs is not a baseline. Guard the alert with a minimum-sample condition or it fires wildly during the first month.

Panels that query raw findings will eventually time out. They work fine at ten thousand rows and fail at ten million, usually on the day somebody is presenting.

Dashboard sprawl dilutes attention. One operational dashboard and one scorecard dashboard is usually the right number. A folder of twenty dashboards means nobody knows which one is authoritative, and the answer diverges between them.

When to Escalate

An alert that has fired more than twice without action should be either fixed or deleted. A permanently red panel trains people to ignore the whole dashboard.
Metrics disagreeing with the scorecard means two views are computing from different sources; that must be resolved immediately, because it destroys trust in both.
Sustained rate climbs belong with the layer’s steward, under the escalation path in Data Stewardship Roles and Responsibilities — the dashboard’s job ends at making the trend visible.
Requests for a new panel per stakeholder are usually a request for a different report. Point them at the executive view described in Reporting Data Quality to Non-Technical Stakeholders rather than growing the operational dashboard.

Frequently Asked Questions

Should the dashboard read the findings table directly?

No. Findings tables grow into the tens of millions of rows, and a dashboard that scans them on every refresh becomes slow and, in a metered warehouse, expensive. Point Grafana at the daily aggregate: it is small, fast, and the same table the scorecard and the executive report read, which keeps every view consistent.

What belongs on the first screen?

Whether the run happened, how long it took and how many features it saw — before any defect number. A rising defect rate means one thing after a normal run and something entirely different after a run that processed a tenth of the usual data. Ordering the panels this way prevents the most common misreading.

How do I avoid alert fatigue?

Alert on few things, on rates rather than counts, against a rolling baseline rather than a fixed number, and with a for duration so a single noisy run does not page anyone. Everything else goes on the dashboard and into a weekly digest. An alert that fires weekly and is dismissed weekly is worse than no alert.

Why annotate rule changes on the charts?

Because a tightened rule produces a step in the defect rate that looks exactly like a data regression. An annotation at the deployment point turns a month of confused investigation into a glance. The same applies to scope changes, which move the denominator.

Related

Spatial Data Quality Metrics and Reporting — the metric definitions behind every panel
Building a Spatial Data Quality Scorecard — the second dashboard, aimed at stewards
Observability and Lineage for Validation — the run signals that feed the operations panels

Back to Spatial Data Quality Metrics and Reporting