Precision, Recall, F1 score, and More | The Ultimate Guide to Confusion Matrix and Classification Metrics

Published on 2026-07-17 | Last updated 2026-08-27

A complete guide to the confusion matrix and every metric built on it: precision, recall, F1, MCC, and more, with a live calculator and a drag-and-drop playground.

Opening

If you've been working on classification problems, terms like precision, recall, and F1 score are probably familiar. But familiarity is not the same as intuition.

The first time I encountered this idea was when I was taking analytical chemistry in 2013. It was the same table (confusion matrix), but with false positives and false negatives labeled as Type I and Type II errors. I remember thinking, "Why are there so many ways to say the same thing?" It turns out that each metric was invented by a different field for a different problem, and this post is my attempt to bridge those fields, so that readers can recognize the same concept behind the different names.

I met the same table again during my internship at Synopsys in the summer of 2025. We had an imbalanced dataset where more than 90% of the labels were 0 and fewer than 10% were 1. The model tried to score high by simply guessing 0 for everything, which led to high accuracy but low recall. That is when I realized that recall and precision matter differently in different scenarios.

When I explained the idea to colleagues who were not familiar with the concept, two examples proved useful. I use "R" to remember that when "having Rectum canceR" is the positive class, we want high Recall, because a false negative is worse than a false positive. In other words, we would rather have a false alarm than a missed cancer.

On the other hand, I use "P" to remember that when "this assignment is Plagiarism" is the positive class, we want high Precision, because a false positive is worse than a false negative. In other words, we would rather let a plagiarized assignment slip through than wrongly accuse an honest student and damage their record.

I found these memory tricks useful, and I want to share them in this post.

Three Running Examples

To keep every metric concrete, I will reuse the same three cases throughout the post. In addition to the two examples above, I added a third one that is more neutral.

• Case A. Whether a student's assignment is plagiarism or not.
• Case B. Whether a patient has a rectum cancer or not.
• Case C. Whether a VC should invest in a startup idea or not.

A few notes about these three cases.

What counts as positive is defined by how the question is asked, not by whether the outcome is good. In medical diagnosis, positive usually means "having the condition," which is usually bad news, such as having rectum cancer (Case B) or having Covid. It can also be neutral, such as being pregnant. In Case A, positive means "is plagiarism"; in Case C, positive means "worth investing."

The truth is not always knowable at the moment the question is asked. For Case A and Case B, the ground truth already exists at evaluation time; it is just hidden from the classifier. For Case C, the truth does not exist yet at decision time and only reveals itself years later. This distinction matters when you think about how a confusion matrix gets filled in at all.

Confusion Matrix

A confusion matrix is the table used to describe the performance of a classification model (Figure 1). Each row is the actual (true) class and each column is the class the model predicted, so every prediction lands in one of four cells: the two on the diagonal are correct, and the two off the diagonal are the two different ways of being wrong.

True Positives (TP)

The model predicted positive, and the truth is positive. The detector flagged an assignment as plagiarism and it really was plagiarized; the screening caught a real cancer; the VC invested in the startup that became a unicorn.

True Negatives (TN)

The model predicted negative, and the truth is negative. An original assignment was cleared as original; a healthy patient was told they are healthy; the VC passed on a startup that indeed went nowhere.

False Positives (FP)

The model predicted positive, but the truth is negative. Also called a false alarm or Type I error. An honest student's original work was flagged as plagiarism; a healthy patient was told they might have cancer and sent for a biopsy; the VC funded a startup that failed.

False Negatives (FN)

The model predicted negative, but the truth is positive. Also called a miss or Type II error. A plagiarized assignment passed as original; a real cancer went undetected; the VC passed on the next Google.

Not All Errors Cost the Same

Since there are two ways to be wrong, a positive predicted as negative (false negative) and a negative predicted as positive (false positive), there is more to discuss than just "how many errors." Think about the cancer case: we would rather have a false alarm than a missed cancer. The two error cells look interchangeable on paper, but in practice they almost never cost the same, and the three running cases each break the symmetry in a different direction.

In plagiarism detection, the false positive is the catastrophe. An honest student wrongly accused faces a disciplinary hearing, a mark on their record, and a story they will be explaining for years; a plagiarized assignment that slips through is a missed catch, bad but survivable. Whenever a positive prediction directly triggers a punishment, precision is the number under moral load.

In cancer screening, the asymmetry flips. A false negative can be fatal, because the patient walks away reassured while the disease progresses; a false positive costs an anxious week and a follow-up biopsy. Nobody enjoys a false alarm, but "early detection and early treatment" is fundamentally an argument about recall.

In startup investing, the false negative is the legendary error. Venture returns follow a power law, where one big winner pays for the entire portfolio, so passing on the next Google costs far more than funding ten startups that go nowhere. Famous funds even keep an "anti-portfolio" of the winners they rejected. The rational response is a deliberately high-false-positive strategy: fund many likely failures so as never to miss the one that matters.

The lesson generalizes. Because error costs are asymmetric, no single number can summarize a classifier; every metric in the rest of this post is one specific answer to the question "which cells do you care about?", and different fields answer it differently because their costs differ.

Tier 1 Metrics | Reading One Row or One Column

The famous Wikipedia mega-table of confusion matrix metrics looks intimidating, but almost every entry in it is built from one simple move: take one cell, divide it by its row total or its column total. That single observation organizes everything.

Reading along a row means conditioning on the truth: given that the patient really has cancer, what did the test say? These row-wise metrics (recall, specificity, and their complements) describe the test itself, and they stay the same no matter how rare the disease is.

Reading down a column means conditioning on the prediction: given that the test came back positive, what is the truth? These column-wise metrics (precision, NPV, and their complements) describe what a prediction is worth to the person receiving it, and they change dramatically with how rare the condition is.

Keep this row-versus-column distinction in mind, and the rest of this post is just filling in the details.

Accuracy and Why It Lies

Accuracy is the metric everyone learns first: the fraction of all predictions that were correct.

$$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$$

It is also the metric most likely to mislead you, and the reason is prevalence: the fraction of the population that is actually positive, $\text{Prevalence} = P/(P+N)$.

Consider Case B with concrete numbers. Screen 1,000 patients for rectum cancer where the true prevalence is 1%: 10 patients have it, 990 do not. Now take the laziest possible model, one that answers "healthy" for everyone. It gets all 990 healthy patients right and all 10 cancer patients wrong: 99% accuracy, zero cancers caught. This is the accuracy paradox, and it is exactly what my Synopsys model was doing when it guessed 0 for everything. You can watch this happen in the interactive calculator below: load the "Accuracy paradox (99:1)" preset and see accuracy sit at 99% while MCC collapses to zero.

Accuracy is only trustworthy when the classes are roughly balanced and the two kinds of error cost roughly the same. Whenever either condition fails, and in practice one usually does, you need the row and column views below.

Recall (Sensitivity, TPR) and Specificity (TNR)

The row-wise pair. Recall asks: of all the patients who actually have cancer, how many did the test catch?

$$\text{Recall} = \frac{TP}{TP + FN}$$

Specificity asks the mirror question about the other row: of all the healthy patients, how many did the test correctly clear?

$$\text{Specificity} = \frac{TN}{TN + FP}$$

Recall is what the sick patient cares about (will my cancer be found?); specificity is what the healthy population cares about (will I be spared a false alarm and an unnecessary biopsy?). Remember the mnemonic from the opening: having Rectum canceR demands high Recall. Figure 2 makes this concrete with two cancer detectors that score an identical 90% F1: the one we would actually deploy for screening is the one that misses fewer cancers, even though it raises more false alarms.

These two metrics travel under different names in different fields, which is a large part of why the topic feels confusing. Medicine says sensitivity and specificity, a pairing that goes back to the statistics of X-ray screening for tuberculosis (Yerushalmy 1947; for a modern primer, Altman and Bland 1994). Machine learning says recall, borrowed from information retrieval (van Rijsbergen 1979). Signal detection theory says hit rate and calls recall the statistical power of the test. They are all the same two numbers.

Two more table entries come for free as complements: the false negative rate, or miss rate, $FNR = FN/(TP+FN) = 1 - \text{Recall}$ (the Type II error rate from my analytical chemistry class), and the false positive rate, or fall-out, $FPR = FP/(FP+TN) = 1 - \text{Specificity}$ (the Type I error rate).

Precision (PPV) and NPV

The column-wise pair. Precision asks: of everything the model flagged as positive, how much was actually positive?

$$\text{Precision} = \frac{TP}{TP + FP}$$

Its mirror on the other column is the negative predictive value: of everything the model cleared, how much was actually clear? $NPV = TN/(TN+FN)$. Medicine calls precision the positive predictive value (PPV), because it answers the question a patient asks after a positive result: "Doctor, does this mean I really have it?" And per the opening mnemonic: a Plagiarism detector demands high Precision, because the expensive error is the false positive, an honest student wrongly accused. Figure 3 works through this case.

Here is the property that surprises almost everyone the first time: precision depends on prevalence, even when the test itself does not change. Take an excellent test with 99% recall and 99% specificity, and screen 100,000 people for a condition with 0.1% prevalence. The 100 true patients yield 99 true positives. But the 99,900 healthy people, at a 1% false positive rate, yield 999 false positives. Precision is $99/(99+999) \approx 9\%$: eleven out of twelve positive results are false alarms, from a test that is 99% "good." This is the classic base-rate example, and it is why a positive result on a rare-disease screen is the beginning of a diagnostic process, not the end.

This is the deepest practical difference between the two views: recall and specificity are properties of the test; precision and NPV are properties of the test deployed in a particular population. A model card can honestly report recall once and for all, but any precision number is implicitly a claim about prevalence.

The two complements complete the column view: the false discovery rate $FDR = 1 - \text{Precision}$ (the term ML papers use) and the false omission rate $FOR = 1 - NPV$.

Tier 2 Metrics | Combining Both Views

When a classification model doesn't behave perfectly (which usually happens), no single row or column tells the full story, so the field invented composite metrics that combine both views into one number.

F1 Score and the F-beta Family

Precision and recall pull against each other: flag more aggressively and recall rises while precision falls; flag more conservatively and the reverse happens. The F1 score compresses the pair into one number by taking their harmonic mean.

$$F_1 = \frac{2 \times \text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}} = \frac{2TP}{2TP + FP + FN}$$

Why the harmonic mean instead of the ordinary average? Because the harmonic mean is dragged toward the smaller value, so you cannot buy a good F1 by maxing one side and abandoning the other. Take a plagiarism detector that flags exactly one assignment, correctly, and misses the other 99 plagiarized ones: precision 1.00, recall 0.01. The arithmetic mean says 0.505, which sounds like a mediocre but passable model. The harmonic mean says 0.02, which is the right answer: this model is useless.

The F-measure also has a birthplace. Van Rijsbergen (1979) introduced its ancestor, the effectiveness measure E, for judging search systems in information retrieval, where the true negatives are all the irrelevant documents a query correctly did not retrieve — essentially the whole library, a count too large to mean anything. So the F-measure was built to ignore TN entirely. In retrieval that is a feature; in problems where the negative class matters, such as clearing healthy patients, it is a blind spot (Powers 2020; Chicco and Jurman 2020). Figure 1 already showed the sharp version of this warning: our plagiarism and cancer detectors both scored 90% F1 with mirror-image error profiles, and F1 alone cannot tell them apart.

When the two error costs differ, the F-beta family lets you say by how much:

$$F_\beta = (1+\beta^2) \times \frac{\text{Precision} \times \text{Recall}}{\beta^2 \times \text{Precision} + \text{Recall}}$$

Read beta as "how many times more I care about recall than precision." F2 fits cancer screening; F0.5 fits plagiarism detection. When someone hands me an F1, my first question is whether beta = 1 is a decision they actually made or just a default nobody questioned. One more caution for comparisons: F1 inherits precision's prevalence-dependence, so an F1 measured on one class balance does not transfer to another — a known pitfall when evaluating on imbalanced data (Saito and Rehmsmeier 2015).

Balanced Accuracy, MCC, and Friends

The quickest repair for the accuracy paradox is balanced accuracy, the plain average of the two row rates: $(TPR + TNR)/2$. The lazy always-negative model from the accuracy section scores 99% on accuracy but exactly 50% on balanced accuracy, exposed as a coin flip.

Two zero-centered cousins carry the same information. Informedness, better known as Youden's J = TPR + TNR − 1, was proposed for rating diagnostic tests, with 0 meaning the test adds nothing over chance (Youden 1950). Markedness = PPV + NPV − 1 is its column-wise mirror (Powers 2020).

The Matthews correlation coefficient has my favorite origin story in the whole table. In 1975, Brian Matthews was not evaluating classifiers at all: he was a protein crystallographer comparing the predicted and observed secondary structure of T4 phage lysozyme, a problem with heavily imbalanced structural classes, and his fix was to apply Pearson's correlation coefficient to the 2×2 table (Matthews 1975).

$$MCC = \frac{TP \times TN - FP \times FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}$$

MCC runs from −1 (perfectly wrong) through 0 (chance) to +1 (perfect), equals the geometric mean of informedness and markedness, and is the only common metric that uses all four cells symmetrically. It cannot be inflated by class imbalance the way accuracy can, and it cannot ignore TN the way F1 does; on the accuracy-paradox preset it reads exactly 0. There is a growing case that it should be the default single-number report for binary classification (Chicco and Jurman 2020), and my own practice agrees: when a collaborator asks me for one number, I give MCC, and I attach the full matrix anyway.

The remaining composites deserve one line each. The Fowlkes–Mallows index, $\sqrt{PPV \times TPR}$, comes from comparing hierarchical clusterings (Fowlkes and Mallows 1983). The threat score, $TP/(TP + FN + FP)$, has the best cross-field resume in the table: Gilbert (1884) introduced its logic to grade tornado forecasts, where the enormous "no tornado predicted, none occurred" cell makes accuracy absurd, and the same formula was independently reinvented by the botanist Jaccard (1912) to measure overlap between alpine floras — forecasting calls it the critical success index, ecology calls it the Jaccard index, and they are one number. Finally, the prevalence threshold, $(\sqrt{TPR \times FPR} - FPR)/(TPR - FPR)$, marks the prevalence below which a screening test's PPV collapses, a design constraint for screening programs (Balayla 2020).

Every metric in this section is computed live in the calculator below.

Likelihood Ratios and Diagnostic Odds Ratio

This is the corner of the table that medicine uses daily and machine learning barely mentions. Evidence-based medicine teaches clinicians to treat a test result not as a verdict but as an update to the odds (Deeks and Altman 2004). The positive likelihood ratio, $LR+ = TPR/FPR$, says how many times more likely a positive result is in someone who has the condition than in someone who does not; the negative likelihood ratio, $LR- = FNR/TNR$, does the same for a negative result. The update rule is a single multiplication: pre-test odds × LR = post-test odds.

Run it once with Method A from Figure 2 (recall 95.7%, specificity 84.9%, so $LR+ \approx 0.957/0.151 \approx 6.3$). At 1% prevalence the pre-test odds are 1:99; a positive result moves them to about 6.3:99, roughly a 6% probability. That one line of arithmetic is the base-rate lesson from the precision section in the form your doctor actually computes: a positive screen at low prevalence means "worth investigating," not "you have cancer." The same test's $LR- \approx 0.043/0.849 \approx 0.05$, so a negative result divides the odds by twenty, taking 1% down to about 0.05% — a good screening test earns its keep mostly by ruling out.

Dividing the two gives the diagnostic odds ratio, $DOR = LR+/LR-$, equivalently $(TP \times TN)/(FP \times FN)$: a single, prevalence-free number for a test's overall discriminating power, standard in meta-analyses of diagnostic studies (Glas et al. 2003). For Method A it is about 126.

What I like about the likelihood-ratio view is that it is the only framing in the whole table where the metric is an instruction rather than a grade: multiply your prior. The calculator below computes LR+, LR−, and DOR live if you want to build that reflex.

Which Metric Should I Use?

After twenty-plus metrics, here is the decision procedure I actually use.

1. Are the classes roughly balanced and the two error costs roughly equal? Then accuracy is fine. Stop overthinking; this is the one situation the intro-course default handles well.
2. Is the positive class rare, or is one error clearly costlier than the other? Then report precision and recall together, never either alone. Each one is blind to half the story by construction.
3. Do you need a single number for model selection? Use F1 (or F-beta at a beta you actually chose) when true negatives are irrelevant, as in retrieval-shaped problems; use MCC when all four cells matter, as in diagnosis or defect detection.
4. Are you talking to physicians or epidemiologists? Speak sensitivity, specificity, and likelihood ratios — that is the native dialect, and LR+ converts directly into patient-level decisions.
5. Are you deciding under power-law payoffs, where one hit outweighs many misses? Then optimize recall on the winners and accept ugly precision as the cost of doing business.

Table 1. The three running cases, closed out.

Case	Costly error	Metric to watch	Why
Plagiarism detection	False positive	Precision (F0.5 if one number is needed)	A false accusation harms an innocent student.
Cancer screening	False negative	Recall / sensitivity (F2; LR− for rule-out)	A missed cancer can be fatal; screening exists to not miss.
Startup investing	False negative	Recall on eventual winners	Power-law returns: ten duds cost less than one missed unicorn.

If you remember nothing else from this section: name the costly error first, and the right metric usually names itself.

Interactive Calculator

Type any four counts below, or load a preset, and every metric in the confusion-matrix family updates live. Each metric shows its formula, grouped by whether it reads a row, a column, or the whole table.

Try this: load Accuracy paradox (99:1) and watch accuracy stay above 99% while recall drops to zero and MCC collapses to 0.000 — a model that looks excellent and catches nothing. Then compare Plagiarism (high precision) against Cancer (high recall): same F1 of 0.900, mirror-image priorities.

Interactive Playground | 100 Balls

The calculator takes the four counts as given. This playground lets you build them by hand. Each ball's color is the truth (blue = actually positive, gray = actually negative) and the bin it sits in is the model's prediction. Drag any ball across to reclassify it and watch the metrics move.

Three things worth trying: (1) hit Predict all positive to build a classifier with perfect recall and useless precision; (2) drag every blue ball into the right-hand bin to recreate the always-negative model from the accuracy paradox; (3) start from Random (Random) and try to raise MCC above 0.5 by hand — it is harder than it looks.

Both widgets are vanilla JavaScript with no dependencies, and they run standalone at the confusion-matrix-playground demo; the source is on GitHub if you want to drop them into your own teaching notes.

Closing

In 2013, my analytical chemistry textbook called them Type I and Type II errors. Medicine calls them false alarms and missed diagnoses. Machine learning calls them precision problems and recall problems. Weather forecasters, botanists, crystallographers, and epidemiologists each met the same 2×2 table on a different battlefield and left behind a metric with their own name on it — which is exactly why the Wikipedia mega-table looks so intimidating and is actually so simple. Almost every entry is one cell divided by a row or a column. A confusion matrix stops being confusing the moment you know which direction you are reading.

And the model from my Synopsys summer, the one scoring 99% accuracy by predicting 0 for everything? It was not lying. It answered exactly the question we asked. The craft of evaluation is asking the question whose answer you actually need: read the row when you care about the truth, read the column when you care about the prediction, and before either, decide which error you cannot afford.

The two widgets in this post are open source in my confusion-matrix-playground repository (live demo); if they helped the concepts click, a star helps other students find them.

As for which error you cannot afford — the VCs settled that question for themselves long ago. In a power-law world, even your life choices are a recall problem.

References