Rollback Triggers & Versioning for Indoor Mapping Pipelines
Indoor map artifacts are deployable objects, and like any deployable object they occasionally need to be reverted in seconds rather than rebuilt over hours. This is one of the four operational concerns inside Production-Ready Indoor Map Deployment: a corrupted routing graph, a misaligned floor level, or a stale POI coordinate set can break wayfinding the moment it reaches a device. This page covers how to version spatial assets so a revert is deterministic, how to define the telemetry thresholds that trigger a revert automatically, and how to execute that revert atomically without interrupting an active navigation session.
The Problem: a Bad Floor Plan Is Already Live
The failure mode is specific and visible. A new release of a hospital’s third floor ships at 08:30. By 08:34 the routing engine starts returning null paths for any query that crosses the east elevator lobby, because the new export snapped two corridor nodes to the same coordinate and silently disconnected a graph component. Wayfinding kiosks show “no route found,” the facilities team’s space-utilization dashboard flatlines, and — worst case — an emergency egress route now points at a wall. None of this was caught downstream of CI Gating for Map Updates because the regression only manifests under live query load, not in static validation.
A revert has to be possible without a human reading logs, diffing geometry, and re-running a build pipeline. That requires two things working together: every artifact must be addressable by an identity that cannot drift (so you always know what to roll back to), and the system must measure its own health against explicit thresholds (so it knows when to roll back). Versioning without triggers is a manual fire drill; triggers without content-addressed versioning revert to an artifact you cannot prove is intact.
Prerequisites & Dependencies
Before implementing rollback automation, the following must already be in place:
- A canonical artifact envelope. Every map export is a GeoJSON
FeatureCollectionwith a top-levelmetadatablock, conforming to JSON Schema Design for Indoor Maps. Versioning fields live in that block, not in filenames. - A consistent coordinate frame. Reverting between versions only round-trips cleanly if both artifacts share the same building-local frame from Indoor Coordinate Reference Systems. A rollback that crosses a CRS change will re-introduce coordinate drift.
- Python 3.10+ with
requestsfor the orchestration calls. Hashing and manifest handling use only the standard library (hashlib,json,pathlib,dataclasses). - Git LFS for tracking large binaries (
.geojson,.mbtiles,.imdf) so the primary repository holds lightweight pointers, not multi-megabyte tilesets. - A telemetry source — a metrics API, a log stream, or Kubernetes probe output — that exposes routing success rate, path-computation latency, and POI lookup failures per release.
How Content-Addressed Versioning Works
The core idea is that an artifact’s identity is derived from its bytes, not assigned to it. A topology_hash (SHA-256 over the canonical serialization) is the immutable identity; the human-readable map_version is a label that maps onto a hash in a manifest. Reverting is then a pointer swap from one hash to another — no rebuild, because the target artifact already passed every gate when it was first promoted.
Layer a spatially-aware version scheme on top so the label itself signals how disruptive a change is to client SDKs:
- MAJOR — topology restructuring (adding or removing a floor level, re-routing corridors, changing the connectivity of the routing graph). Requires SDK reinitialization and full cache invalidation.
- MINOR — attribute and POI updates, metadata corrections, or style adjustments. Backward-compatible with existing routing engines; supports incremental cache updates.
- PATCH — coordinate precision fixes, minor geometry snapping, or cache manifest updates. Safe to hot-patch without interrupting active navigation.
Version identifiers are embedded directly in the spatial payload as top-level metadata, so an edge node can validate identity before serving and never silently mix an old map_version label with new bytes:
{
"type": "FeatureCollection",
"metadata": {
"map_version": "2.4.1",
"schema_revision": "v1.3.0",
"generated_at": "2026-05-12T08:30:00Z",
"topology_hash": "sha256:a1b2c3d4e5f6",
"coordinate_system": "EPSG:4326"
},
"features": [
{
"type": "Feature",
"id": "room-101",
"geometry": { "type": "Polygon", "coordinates": [[[0, 0], [10, 0], [10, 8], [0, 8], [0, 0]]] },
"properties": { "category": "office", "floor_level": 3 }
}
]
}
Embedding map_version and topology_hash enables deterministic validation at edge nodes and prevents silent drift between origin storage and client caches.
Step-by-Step Implementation
Step 1: Track binaries and generate an integrity manifest
GeoJSON, MBTiles, and CAD-derived vector exports frequently exceed standard repository size limits, so route them through Git LFS. Configure .gitattributes at the repository root:
*.geojson filter=lfs diff=lfs merge=lfs -text
*.mbtiles filter=lfs diff=lfs merge=lfs -text
*.imdf filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
Pair LFS tracking with SHA-256 verification. Every promotion to staging or production includes a manifest.json mapping each version string to its binary hash, which prevents partial uploads and guarantees byte-for-byte reproducibility. The utility below generates and validates manifests using only the standard library:
import hashlib
import json
import logging
from datetime import datetime, timezone
from pathlib import Path
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger("map_manifest")
def compute_sha256(file_path: Path) -> str:
"""Compute the SHA-256 topology hash for a spatial binary."""
sha256 = hashlib.sha256()
try:
with open(file_path, "rb") as fh:
for chunk in iter(lambda: fh.read(8192), b""):
sha256.update(chunk)
except FileNotFoundError:
logger.error("Cannot hash missing file: %s", file_path)
raise
except PermissionError:
logger.error("Permission denied reading: %s", file_path)
raise
return sha256.hexdigest()
def generate_manifest(assets_dir: Path, version: str, output_path: Path) -> dict:
"""Generate a versioned manifest mapping filenames to topology hashes."""
manifest: dict = {
"version": version,
"assets": {},
"generated_at": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
}
for pattern in ("*.geojson", "*.mbtiles", "*.imdf"):
for file in sorted(assets_dir.glob(pattern)):
manifest["assets"][file.name] = compute_sha256(file)
output_path.write_text(json.dumps(manifest, indent=2))
logger.info("Manifest generated for %s (%d assets)", version, len(manifest["assets"]))
return manifest
def verify_manifest(manifest_path: Path, assets_dir: Path) -> bool:
"""Validate local binaries against a published manifest."""
try:
manifest = json.loads(manifest_path.read_text())
except (FileNotFoundError, json.JSONDecodeError) as exc:
logger.error("Unreadable manifest %s: %s", manifest_path, exc)
return False
for filename, expected_hash in manifest["assets"].items():
file_path = assets_dir / filename
if not file_path.exists():
logger.error("Missing asset: %s", filename)
return False
if compute_sha256(file_path) != expected_hash:
logger.error("Hash mismatch for %s", filename)
return False
logger.info("All %d assets verified", len(manifest["assets"]))
return True
Reference the official Git Large File Storage documentation for pointer migration and bandwidth optimization.
Step 2: Define thresholds and evaluate the rollback trigger
Rollback triggers must be deterministic, measurable, and decoupled from manual intervention. Monitor three telemetry streams: routing engine health (graph traversal success rate, path-computation latency, cycle detection), spatial integrity (coordinate drift beyond tolerance, orphaned nodes/edges, POI lookup failures), and cache and edge delivery (CDN hit ratio, version-mismatch errors, stale manifest fetches).
Gate promotions and trigger automatic reverts on explicit thresholds:
| Metric | Warning Threshold | Rollback Trigger |
|---|---|---|
| Routing success rate | < 98.5% | < 95.0% sustained for 30s |
| Path computation P95 | > 450ms | > 800ms sustained for 60s |
| POI lookup failures | > 0.5% | > 2.0% sustained for 30s |
| Graph topology errors | > 0 | > 0 (immediate halt) |
The evaluator below ingests telemetry snapshots and returns a structured rollback decision over a sliding window:
import logging
import time
from dataclasses import dataclass
logger = logging.getLogger("rollback_trigger")
ROUTING_FLOOR = 0.95
P95_CEILING_MS = 800.0
POI_FAILURE_CEILING = 0.02
@dataclass
class TelemetrySnapshot:
timestamp: float
routing_success_rate: float
path_p95_ms: float
poi_failure_rate: float
topology_errors: int
def evaluate_rollback_trigger(snapshots: list[TelemetrySnapshot], window_seconds: int = 60) -> bool:
"""Return True when recent telemetry crosses a rollback threshold."""
cutoff = time.time() - window_seconds
recent = [s for s in snapshots if s.timestamp >= cutoff]
if not recent:
logger.warning("No telemetry in the last %ds; holding current version", window_seconds)
return False
avg_success = sum(s.routing_success_rate for s in recent) / len(recent)
avg_p95 = sum(s.path_p95_ms for s in recent) / len(recent)
max_poi_fail = max(s.poi_failure_rate for s in recent)
topology_error = any(s.topology_errors > 0 for s in recent)
breach = (
avg_success < ROUTING_FLOOR
or avg_p95 > P95_CEILING_MS
or max_poi_fail > POI_FAILURE_CEILING
or topology_error
)
if breach:
logger.error(
"Rollback armed: success=%.3f p95=%.0fms poi_fail=%.3f topo_err=%s",
avg_success, avg_p95, max_poi_fail, topology_error,
)
return breach
Integrate this evaluator into your CI pipeline or Kubernetes readiness probes to block deployments that fail pre-flight spatial validation.
Step 3: Degrade gracefully before reverting
When a map service degrades, hard failures should be avoided. A spatial circuit breaker tracks consecutive routing failures or manifest-fetch timeouts and, once the failure threshold is breached, routes requests to a fallback payload rather than crashing the wayfinding application:
- Primary — the latest validated
vX.Y.Ztopology and POI dataset. - Fallback — the previous stable
v(X-1).Y.Zsnapshot, or a simplified 2D floor plan with static POI markers.
Client SDKs must handle version pinning and fallback routing transparently. Align the breaker behaviour with established SDK Integration Patterns so mobile and kiosk clients keep navigating during a backend revert, and with the server-side approaches in Fallback Routing Architectures so the degraded graph is still navigable rather than empty.
Step 4: Execute the atomic swap and invalidate caches
Executing a rollback requires atomic swaps, strict cache invalidation, and SDK version synchronization. The orchestration sequence is:
- Identify the target. Query the artifact registry for the last known-good
map_versionwith a verifiedtopology_hash. - Atomic swap. Update the CDN origin pointer or object-store symlink to the fallback version.
- Cache purge. Issue
PURGE/INVALIDATEfor versioned endpoints (/maps/v{version}/). - SDK notification. Broadcast a WebSocket or push message to active clients with the new
map_versionandcache_ttl. - Verification. Re-run graph connectivity checks against the rolled-back dataset.
import logging
import requests
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
logger = logging.getLogger("map_rollback")
class MapRollbackOrchestrator:
def __init__(self, cdn_api_url: str, api_key: str, artifact_registry: str) -> None:
self.cdn_api_url = cdn_api_url
self.headers = {"Authorization": f"Bearer {api_key}"}
self.registry_url = artifact_registry
def get_last_stable_version(self) -> str | None:
"""Fetch the last verified version from the artifact registry."""
try:
resp = requests.get(f"{self.registry_url}/stable", headers=self.headers, timeout=5)
resp.raise_for_status()
return resp.json().get("version")
except requests.Timeout:
logger.error("Registry timed out resolving last stable version")
return None
except requests.RequestException as exc:
logger.error("Failed to fetch stable version: %s", exc)
return None
def execute_rollback(self, target_version: str) -> bool:
"""Atomically swap the CDN pointer and invalidate versioned caches."""
try:
swap = requests.post(
f"{self.cdn_api_url}/maps/swap",
json={"version": target_version, "action": "rollback"},
headers=self.headers, timeout=10,
)
swap.raise_for_status()
logger.info("Swapped CDN origin to %s", target_version)
purge = requests.post(
f"{self.cdn_api_url}/cache/purge",
json={"paths": [f"/maps/{target_version}/*"]},
headers=self.headers, timeout=10,
)
purge.raise_for_status()
logger.info("Purged versioned cache for %s", target_version)
return True
except requests.Timeout:
logger.error("CDN API timed out during rollback to %s", target_version)
return False
except requests.RequestException as exc:
logger.error("Rollback execution failed: %s", exc)
return False
Edge Cases & Gotchas
| Symptom | Root cause | Resolution |
|---|---|---|
Routing returns null paths |
Graph cycles or disconnected components | Run networkx.is_strongly_connected() on the topology; repair orphaned edges before re-promoting |
| POI coordinates drift after revert | Coordinate-system mismatch (WGS84 vs. building-local CRS) | Verify coordinate_system metadata; re-apply the stored affine transform from the prior frame |
| LFS pointer corruption | Incomplete git lfs push or network timeout |
Run git lfs fetch --all; re-verify each SHA-256 against the manifest |
| SDK fails to load map | map_version label points at the wrong hash, or schema_revision missing |
Re-verify the manifest mapping; enforce strict schema validation before serving |
| High cache-miss ratio after swap | CDN not purged for the versioned path | Trigger a manual PURGE for /maps/v*/; confirm Cache-Control and ETag headers |
| Revert “succeeds” but serves corrupt bytes | Edge node skipped hash re-verification | Make topology_hash re-validation mandatory at the edge before the object is served |
The most dangerous case is the last one: a pointer swap that lands on a partially-uploaded object. Content-addressing only protects you if the edge actually re-computes and compares the hash before serving — otherwise the rollback restores a label, not a known-good artifact.
Validation Output
A correct manifest verification is unambiguous, and the revert path should assert on it rather than trust an HTTP 200. The pattern below confirms that the version the registry claims is stable is the version actually being served:
# Correct: stable label resolves to an intact, re-hashed artifact
assert verify_manifest(Path("manifest.json"), Path("./dist")) is True
served = requests.get("https://cdn.example.com/maps/v2.3.0/floor3.geojson", timeout=5)
assert served.json()["metadata"]["topology_hash"] == "sha256:a1b2c3d4e5f6"
# Incorrect: label advanced but bytes never propagated — this MUST fail loudly
# AssertionError: served hash 'sha256:deadbeef...' != manifest 'sha256:a1b2c3d4e5f6'
A pre-deployment gate makes the same guarantees before promotion, so a revert target is always provably intact:
#!/usr/bin/env bash
set -euo pipefail
echo "[1/4] Validating schema compliance..."
python -m jsonschema --instance map_export.json schema/v1.3.0.json
echo "[2/4] Verifying LFS pointer integrity..."
git lfs fsck
echo "[3/4] Checking topology connectivity..."
python scripts/validate_topology.py --input map_export.geojson
echo "[4/4] Generating deployment manifest..."
python scripts/generate_manifest.py --dir ./dist --version "$VERSION" --output manifest.json
echo "[ok] Pipeline validation complete. Ready for staging promotion."
Performance & Scale Notes
- Hashing cost is linear in bytes. SHA-256 over an 8 KB chunk stream runs at roughly 500 MB/s on a modern core, so a 200 MB campus tileset hashes in well under a second. Hash once at promotion and store the digest in the manifest; never re-hash on the serving hot path beyond the edge’s single integrity check.
- The trigger window is the latency floor. A 30–60s sustained-breach window deliberately trades detection speed for false-positive resistance. A transient spike during a CDN cache fill should not revert a healthy release, so size the window to at least two scrape intervals of your metrics source.
- Revert is O(1) in artifact size. Because a rollback is a pointer swap plus a scoped cache purge — not a rebuild — its wall-clock cost is dominated by CDN purge propagation (typically single-digit seconds), independent of how large the floor plan is.
- Scope purges per building, not globally. Invalidating
/maps/v*/site-wide forces every kiosk and phone to re-fetch. Key invalidation on the changed building or floor level, in line with Cache Invalidation Strategies, so a single-floor revert does not stampede the origin. - Keep the last N stable manifests hot. Retaining the previous three to five promoted hashes in the registry makes multi-step reverts (when the immediately-prior version is also bad) instant rather than a cold rebuild.
Frequently Asked Questions
What is the difference between a rollback trigger and a CI gate?
A CI gate runs before promotion against a static artifact — it catches schema violations, unclosed rings, and disconnected graphs that are visible without traffic. A rollback trigger runs after promotion against live telemetry, catching regressions that only appear under real query load, such as latency cliffs or POI lookup failures concentrated on one floor. They are complementary: gating from CI Gating for Map Updates reduces how often triggers fire, but it can never replace them, because no static check reproduces production load.
Why content-address artifacts by topology_hash instead of trusting the version label?
Because labels drift and bytes do not. A map_version like 2.4.1 is assigned by humans and can point at a re-built, partially-uploaded, or accidentally-mutated object. The topology_hash is derived from the bytes themselves, so a revert target can be re-verified at the edge before it is served. If the recomputed hash does not match the manifest, the edge refuses to serve — which means a rollback can never silently restore a corrupted artifact.
How do I choose the sustained-breach window for a trigger?
Set it to at least two scrape intervals of your metrics source and tune from there. Too short and a transient spike during a cache fill or a deploy ripple reverts a healthy release; too long and users feel the regression before the system reacts. Topology errors are the exception — a disconnected routing graph is never transient, so that trigger fires immediately with no window.
What should clients show while a rollback is in flight?
A pinned fallback, not an error. The spatial circuit breaker should serve the previous stable snapshot or a simplified 2D floor plan with static POI markers so navigation continuity is preserved. The SDK pins the last-good map_version until it receives the broadcast with the reverted version and cache_ttl, following the patterns in SDK Integration Patterns.
Related Pages
- CI Gating for Map Updates — the pre-promotion gate that reduces how often a revert is needed.
- JSON Schema Design for Indoor Maps — the metadata envelope that carries
map_versionandtopology_hash. - Cache Invalidation Strategies — scoping the purge that follows an atomic swap.
- SDK Integration Patterns — how clients pin versions and consume fallback payloads.
- Fallback Routing Architectures — keeping the degraded graph navigable during a revert.
This page is part of the Production-Ready Indoor Map Deployment reference.