How the k-nearest neighbors algorithm works
A plain idea from machine learning: to judge something new, look at the most similar examples you already know and let them vote.
Written by Amili, an AI writer, from the sources listed below · 19 October 2026 · 5 min read
The k-nearest neighbors algorithm (k-NN) labels a new item by finding the k most similar items whose labels are already known and taking a vote among them. It needs no training phase and works both for categories and for numbers, which makes it one of the easiest learning methods to understand.
In short
- A new point gets the most common label among its k closest known examples.
- For numeric predictions, k-NN averages the neighbours' values instead of voting.
- Choosing k is a trade-off: small k chases noise, large k blurs real boundaries.
- Because it relies on distance, features measured on different scales must be normalised.
- It stores every example and compares against all of them, which gets slow on large data.
What is the idea behind k-nearest neighbors?
Most learning methods try to boil data down into a rule or a formula. k-NN does something more modest: it keeps every labelled example it has seen and, when a new case arrives, simply asks which stored examples sit closest to it. Whatever label dominates among those neighbours becomes the answer.
Evelyn Fix and Joseph Hodges devised the method in 1951; Thomas Cover later extended it. Statisticians call it non-parametric, because it makes no assumption about the shape of the underlying pattern. It is a supervised method, meaning it learns from examples that already carry correct labels.
How does it work step by step?
First, every example is turned into a list of numbers, a point in what is called feature space. The 'training' consists only of storing those points with their labels. There is no fitting and no model to build.
When a query point arrives, the algorithm measures its distance to every stored point. For continuous measurements the usual choice is ordinary straight-line (Euclidean) distance; for text and other discrete data a different measure, such as counting mismatched positions, can be used. It then sorts the stored points by distance, keeps the closest k, and returns the most frequent label among them. With k equal to 1, the new point just copies the label of its single nearest neighbour.
For regression, where the answer is a number rather than a category, the k neighbours' values are averaged. A common refinement weights each neighbour by the inverse of its distance, so that close neighbours count for more than distant ones.
What does a worked example look like?
Picture a small orchard dataset: ten fruits, each described by weight and colour on a numeric scale, and each labelled apple or pear. A new fruit arrives. Plot it among the ten, find the three closest, and suppose two are apples and one is a pear. With k set to 3, the new fruit is called an apple.
Now note a trap. If weight is recorded in grams and colour on a scale from 0 to 1, weight differences will swamp everything else and colour will barely matter to the distance. Rescaling each feature to a comparable range before measuring distance can improve accuracy a great deal. Note also that 3 is odd: in a two-class problem an odd k avoids a tied vote.
Where is k-NN used?
It is used for classification, its most frequent role, and for regression, where it is sometimes called nearest neighbour smoothing. Researchers have applied it to gene expression data using correlation measures as the notion of distance. Recommender systems use close relatives of it, and one variant even flips the idea, looking at the most dissimilar users to suggest less obvious items, usually at some cost in predictive accuracy.
There is a theoretical reason for its staying power. As the amount of data grows without limit, the error rate of the single-nearest-neighbour rule is guaranteed to stay within a factor of two of the Bayes error rate, the lowest error any method could achieve for that data.
Where does it fail?
Plain majority voting struggles when one class is much more common than another. The common class tends to crowd every neighbourhood and win votes it should not. Weighting votes by distance is one remedy.
Accuracy also suffers badly when the data contain noisy or irrelevant features, or features whose scale does not reflect their importance; much research effort has gone into choosing and scaling features. And the naive version is computationally heavy, since every prediction compares the query with every stored example. Approximate nearest neighbour search methods exist to make large datasets workable.
What does it teach about thinking?
k-NN is reasoning by resemblance made explicit. People do this constantly: a new situation reminds us of a few past ones, and we expect a similar outcome. The algorithm shows when that works and when it misleads. It works when the features we compare are the ones that truly matter. It misleads when superficial similarity stands in for relevant similarity, or when the most familiar category drowns out the rest.
The choice of k carries a broader lesson too. Trusting a single closest precedent makes us hostage to one odd case; averaging over too many makes us ignore real differences. Good judgement sits between the two.
Questions people ask
How do you choose k in k-NN?
The best k depends on the data. Larger values reduce the effect of noise but make the boundaries between classes less sharp; smaller values follow local detail but can be fooled by odd points. In practice k is picked by testing several values, often with resampling methods such as the bootstrap. For two-class problems an odd number avoids tied votes.
Does k-nearest neighbors need training?
Not in the usual sense. The training phase only stores the labelled examples. All the computation is postponed until a prediction is requested, when distances to the stored points are calculated. This makes k-NN quick to set up but slower to use, because every prediction has to search the stored data for the closest points.
Why does k-NN need feature scaling?
Because it decides everything by distance. If one feature is measured in large units and another in small ones, the large one dominates the distance and the others are effectively ignored. Normalising features to comparable ranges lets each one contribute fairly, and can greatly improve accuracy when features represent different physical units.
The thinking behind it
It introduces k-nearest neighbours with friendly, illustrated examples alongside other beginner algorithms.
Read or listen to Grokking Algorithms
Hear the whole book free: start an Audible trial and your first audiobook — this one, if you like — is on the house.
As an Amazon Associate, ReadGlobe earns from qualifying purchases and Audible trials — at no extra cost to you.
Sources
- K-nearest neighbors algorithm — Wikipedia
- Statistical classification — Wikipedia
How this was made: Amili, an AI writer, wrote this article in its own words from the sources above. Every link was checked before publishing. Spotted an error? Tell us and we will correct it.