Precision and Recall

Formulas

The confusion matrix

Precision uses the top row; recall uses the left column. True negatives, which dominate any matching problem, appear in neither.

Why accuracy is not used

Record linkage compares far more non-matching pairs than matching ones. Hand and Christen (2018) point out that this class imbalance makes accuracy and misclassification rate meaningless for judging linked records, as the worked example shows. The same reasoning applies to other rare-event detection, such as flagging relevant filings or extracting events from news: a detector that flags nothing looks almost perfect on accuracy.

Thresholds and the trade-off

Most matchers produce a confidence score and link pairs above a threshold. Moving the threshold trades precision against recall. The Fellegi-Sunter model of probabilistic matching, covered under entity resolution, uses two thresholds: pairs above the upper one are linked, pairs below the lower one are not, and pairs between go to clerical review. The thresholds are set from the error rates the user will tolerate. Precision and recall are the measures of those error rates.

Measuring in practice

• Ground truth. Both measures need labelled pairs. Registry identifiers, such as the legal entity identifier, can supply a truth set where they exist; elsewhere, reviewers label a sample.
• Sample design. A sample drawn only from linked pairs estimates precision but says nothing about recall. Recall needs a sample of unlinked pairs or an independent list of known duplicates.
• Pairs or clusters. Matching that groups records into clusters can be scored on pairs or on whole clusters; results differ and the choice should be stated.
• Context. A published figure states the dataset, the threshold, the sample, the period and the date. Precision and recall from different datasets are not comparable.

Which error costs more in private-markets data

A false merge combines two different organisations: one allocator's commitments are attributed to another, outreach reaches the wrong institution, and figures are summed across entities that should be separate. It is hard to see and hard to undo. A missed duplicate splits one organisation across records, a failure of uniqueness, one of the data quality dimensions, which causes double counting and fragmented history but is usually visible and easy to fix. For merges, practitioners therefore favour precision; for review queues, which a person will check, recall matters more.

How Altss applies this (Altss methodology)

In Altss's entity-resolution methodology, no similarity score merges records. Two organisation records are merged only on a shared registry identifier of the same scheme, scope and jurisdiction, or a filing or registry record that names both as one entity; two person records are resolved automatically only on deterministic identity evidence or on several independent strong signals that agree, and otherwise stay separate. Precision and recall of a probabilistic matcher therefore measure how well it finds and ranks candidate pairs for registry lookup and review, not how well it merges. A match score has derivation status DERIVED and is used only to rank candidates: it is never an evidence origin or a validation status, and never evidence of identity. Any published precision or recall figure would state its population, sample size, period and date; none is implied here. See the Entity Resolution Methodology.

Worked examples

Examples are illustrative; figures are not market data.

Not the same as

• Confidence Score: A confidence or match score is assigned to each item; precision and recall evaluate the decisions made from those scores across a labelled sample.
• Data Quality Dimensions: The accuracy dimension of data quality asks whether stored values match reality; precision and recall evaluate a matcher's decisions, leaving out the true negatives that make classification accuracy uninformative.

Common mistakes

• Reporting accuracy for a matching or detection task where non-matches dominate.
• Reporting F1 alone, which hides whether precision or recall is weak.
• Estimating recall from a sample of linked pairs only.
• Comparing precision and recall across different datasets or thresholds as if they were the same test.
• Ignoring errors in the ground truth itself.
• Confusing precision of a classifier with precision in the sense of numerical granularity (a date known only to the year).

Edge cases

• If nothing is flagged, precision is undefined (0/0), not zero or one.
• Pairwise scoring can count one wrong cluster of five records as several errors; cluster-level scoring counts it once.
• Entities that legally merge or split over time change the ground truth; labels need an as-of date.
• A matcher can have high precision on a sample dominated by easy pairs and lower precision on hard cases such as related entities with similar names.

Questions

Sources

1. A note on using the F-measure for evaluating record linkage algorithms. David Hand; Peter Christen, Statistics and Computing (Springer), Vol. 28, p. 539 onward (2018); published online 2017. Status: Published (checked 2026-10-01). Abstract — supports: Record linkage as classification of pairs; class imbalance makes accuracy meaningless; precision, recall and F-measure; F-measure as a method-dependent weighted sum; relative weight should be set by the problem and user
2. A Theory for Record Linkage. Ivan P. Fellegi; Alan B. Sunter, Journal of the American Statistical Association, Vol. 64(328), pp. 1183-1210. Status: Published (paywalled) (checked 2026-10-01). Vol. 64(328), pp. 1183-1210 — supports: Link, possible link (clerical review) and non-link decisions with error rates fixed in advance

• Entity Resolution
• Confidence Score
• Data Quality Dimensions
• Triangulation (Source Triangulation)