| « 7. Matching polygon features to Overture | Home |
This page builds on the two matching demos in lessons 6 and 7. It covers the conceptual foundation for cardinality-based matching decisions, how to iterate on a matching pipeline, and how the demo methodology relates to the production pipelines that power Overture’s official releases.
The word “cardinality” appears throughout data engineering and GIS work, usually in the context of database relationships. Esri’s ArcGIS documentation defines it plainly: cardinality describes how records in two different tables relate to one another – one-to-one, one-to-many, many-to-one, or many-to-many – and treats it as foundational for any relate or relationship class.[1] Their usage, though, is about modeling relationships you already understand. You know a fire station has many personnel, so you declare that cardinality in the geodatabase schema.
What makes cardinality useful in a matching context is different: you don’t know the relationship in advance. You discover it from the geometry. And what you discover determines what you hand the end user.
This idea has a formal counterpart in the record linkage literature. The
standard framing, going back to Fellegi and Sunter (1969) and codified
in Peter Christen’s Data Matching (2012), treats linking as a
classification problem: pairs of records are assigned to a match set or
a non-match set based on comparison scores.[2] The Python Record Linkage
Toolkit, a widely used open-source implementation of these methods,
makes the distinction explicit in its API: OneToOneLinking and
OneToManyLinking are separate post-classification steps, because the
shape of the match – not just the score – determines how to handle
the output.[3]
Research in large-scale record linkage has established why this matters operationally. Moretti, Valentino, and Tuoto (2019) describe the “selection of unique links” as a distinct phase of a record linkage pipeline in official statistics – a constrained optimization problem that specifically enforces 1:1 cardinality when that’s what the application requires.[4] The key insight is that a matching algorithm can produce high-confidence scores while still generating a match structure that violates the cardinality the application actually needs. Zhang, Rubinstein, and Gemmell (2014) formalize this further, showing that enforcing 1:1 matching through bipartite graph optimization significantly improves precision and recall compared to simply thresholding similarity scores.[5]
The framework in this demo extends that logic into a decision tool. We don’t enforce any cardinality upfront – we let the geometry produce whatever structure it produces, then observe the result. Observed cardinality at the feature-code level drives the adoption recommendation:
This is what the GERS adoption matrix is doing: it’s cardinality, summarized at the feature-code level, translated into a recommendation about what kind of crosswalk artifact the downstream user actually needs. The matching algorithm is the means; cardinality observation is the decision mechanism.
This framing isn’t standard in the geospatial literature, which tends to treat matching as an end in itself and evaluate it on recall and precision. The contribution here is using observed match cardinality as the primary axis for an ID adoption decision – treating the shape of the match as the variable that determines what you build, not just whether the match succeeded.
The cross-tab is informative but dense. The two-rate diagnostic compresses it into two numbers per feature code:
The two move independently, and the combination is more informative than either alone:
| Pattern | Example (this cell) | What it means |
|---|---|---|
| High match, high clean | AL015 Building (85% / 99%) | Most polygons match; almost all are clean. Direct GERS ID works. |
| Low match, high clean | BH080 (32% / 100%) | When it matches, it matches well. Coverage gap, not schema mismatch. |
| High match, low clean | BA030 Island (66% / 3%) | Coverage is fine, but matches fragment. Needs a link table. |
| Zero match | BA040 Tidal Water, BH140 River | No polygon counterpart at all. Defer to a different geometry or theme. |
Codes that fall in between on both axes (EC030 Trees at 45% / 41%, DA010 Soil Surface Region at 27% / 48%) need case-by-case judgment. Clean rate is undefined when match rate is zero; the matrix in the next section handles this by checking match rate first.
The two rates per feature code, combined with a minimum-sample
threshold, produce a categorical assignment. The notebook computes this
with explicit thresholds (MATCH_RATE_HIGH = 80, CLEAN_RATE_HIGH = 80,
MIN_SAMPLE = 5) that classify each fcode into one of five buckets:
(mgcp_uid,
gers_id) crosswalk handles the many-to-one or one-to-many cases.The notebook shows the full matrix. One finding from that run is worth surfacing in prose.
The skeleton anticipated this bucket would be substantial: feature codes
that match often but fragment when they do. The matrix run disagrees.
The codes that fragment in this cell (BA030 Island, EC030 Trees) have
moderate match rates, not high ones, so they land in review rather
than link table. The codes with high match rates don’t fragment.
This is a finding about the data, not a flaw in the methodology. At the 1:100K capture scale of the Bahamas cell, the cardinality problem appears to be largely binary: feature codes either match cleanly or don’t match at all, with the “matches frequently but messily” middle ground appearing more rarely than expected. Whether this generalizes to other MGCP cells, to TDS v7, or to denser capture scales is exactly the kind of question the next iteration should investigate.
The 80/80 thresholds are conservative. Lowering MATCH_RATE_HIGH to 50
would pull moderate-match codes into link table, making the matrix
look more like the skeleton expected, at the cost of recommending
direct or link-table adoption for codes that match less than two-thirds
of the time. The notebook makes the thresholds visible at the top of
the classifier cell.
The two demos in this workshop – LSIB boundary matching and MGCP building matching – are deliberate simplifications. They’re designed to be readable and runnable on a laptop, and to expose the decision points clearly. Understanding what they leave out, and why, is part of understanding what production conflation actually involves.
The Overture buildings production pipeline uses the same core matching signal as this demo: Intersection over Union (IoU), with a threshold of 0.5. That’s not a coincidence – it’s the right signal for buildings because geometry is the only reliable cross-source signal available. Building names are sparse or absent across most sources, and the largest source volumes come from ML-derived datasets (Microsoft, Google) that have no stable or meaningful name attributes at all.
The production pipeline adds significant operational scaffolding the demo doesn’t have:
The matching algorithm itself – IoU, threshold, best-match selection, cardinality filtering – is structurally the same. The difference is everything around it.
The LSIB boundary demo and the Overture divisions production pipeline share a goal (linking boundary features across datasets) but use substantially different approaches. The divisions pipeline adds three things the demo has no equivalent of.
H3-based blocking. The production pipeline partitions candidate
pairs spatially using H3 cells before any scoring runs. The H3
resolution varies by administrative subtype: countries are blocked at
a coarse resolution covering hundreds of thousands of square kilometers,
while neighborhoods and microhoods are blocked at a resolution covering
less than a square kilometer. This means a candidate pair is only scored
if both features fall in overlapping H3 cells at the resolution
appropriate for their subtype. In the LSIB demo, the equivalent of
blocking is the pair_key join – you only compare features that share
a country-pair code. H3 blocking is the spatial generalization of that
idea, applicable to any administrative feature regardless of whether it
carries an explicit country-code attribute.
Multi-signal composite scoring. The LSIB demo scores pairs on two geometric signals: buffer overlap and length ratio. The divisions pipeline scores on a composite of name similarity and geographic similarity, combined with configurable weights. The composite is evaluated under multiple weighting scenarios and the minimum score is taken – a conservative strategy that penalizes pairs where one signal is strong but the other is weak. A near-perfect geometric match with a very different name scores poorly, and vice versa. Geographic overlap is still a signal, but it’s one input into the composite rather than the whole answer.
Multilingual name embeddings. Name similarity in production is computed using a cross-lingual sentence transformer model (XLM-RoBERTa) applied to all name variants for a feature, with average pooling across variants. This means “München” and “Munich” produce similar embeddings without requiring an explicit translation table. The LSIB demo has no name signal at all – LSIB boundaries are identified by country-pair code, not by name.
The reason divisions needs names and buildings doesn’t comes back to the data. Every division has a name; it’s the primary human-recognizable signal for whether two features represent the same entity. Most buildings don’t have names, and for the ones that do, the name is often absent from ML-derived sources. The matching signal has to match what’s reliably present in the data.
Both the divisions and buildings production pipelines implement a form of history-first matching: before any geometric or attribute scoring runs, the pipeline checks whether a candidate feature was already assigned a GERS ID in a prior release. If it was, it gets that ID back directly, without re-running the full scoring logic.
This is the production equivalent of the iterative matching approach described earlier in this lesson. In the demo, iteration is a manual loop: run the matcher, inspect the unresolved cases, apply a different strategy, append results to the crosswalk. In production, the first iteration already happened in a previous release, and the history pass captures its output automatically. New or changed features fall through to the full scoring pipeline; stable features are matched by identity.
The crosswalk file this demo produces is, structurally, a hand-built version of that history register – a record of which external IDs have already been resolved to GERS IDs, which can be checked before running any geometry comparison on subsequent passes.
This demo is what it is: a methodology demonstration against one MGCP cell. What it shows is encouraging but not generalizable.
base/land_cover and similar passes.The matrix doesn’t tell you what to do; it tells you which decisions to make and on what evidence. For the data shown here, the decisions are: direct GERS adoption for buildings, defer water-line features to a different theme or geometry type, and review the middle band case by case.
buildings/building and the base theme types.[1] Esri, “Relates and Relationship Classes Explained,” ArcGIS Training Blog, February 2022. https://community.esri.com/t5/esri-training-blog/relates-and-relationship-classes-explained/ba-p/900757
[2] Christen, Peter. Data Matching: Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection. Springer, 2012. Referenced via the Python Record Linkage Toolkit documentation.
[3] Python Record Linkage Toolkit, Classification reference (v0.15).
OneToOneLinking and OneToManyLinking classes.
https://recordlinkage.readthedocs.io/en/latest/ref-classifiers.html
[4] Moretti, Diego, Luca Valentino, and Tiziana Tuoto. “Optimization Routines for Enforcing One-to-One Matches in Record Linkage Problems.” The R Journal 11/01, June 2019. https://journal.r-project.org/archive/2019/RJ-2019-008/RJ-2019-008.pdf
[5] Zhang, Duo, Benjamin I. P. Rubinstein, and Jim Gemmell. “Principled Graph Matching Algorithms for Integrating Multiple Data Sources.” arXiv preprint, 2014. https://arxiv.org/abs/1402.0282
| « 7. Matching polygon features to Overture | Home |