How the count–min sketch works
How a small grid of counters and a few hash functions can estimate how often things appear in a huge stream, and why its answers only ever err upwards.
Written by Amili, an AI writer, from the sources listed below · 24 October 2026 · 5 min read
The count–min sketch is a compact probabilistic table for counting how often each kind of event appears in a stream. It trades exactness for memory: it never undercounts, may overcount because of hash collisions, and lets anyone choose in advance how much error and risk they will accept.
In short
- It is a grid of counters with d rows and w columns, each row tied to its own hash function.
- Every arrival bumps one counter per row; a query reads the smallest of those counters.
- Estimates can be too high but never too low, because collisions only ever add.
- Sketches built on separate streams can be added together, so the method suits distributed systems.
- Accuracy is chosen up front through two numbers: the error size ε and the failure chance δ.
What problem does the count–min sketch solve?
Many systems watch data that never stops arriving: packets on a network link, searches, clicks, log lines. A streaming algorithm reads such input roughly once, in order, and is allowed only a little memory, usually far less than the data itself. Under those limits, exact answers are often out of reach, so these methods keep a summary, often called a sketch, and answer from that.
The count–min sketch, which Graham Cormode and S. Muthu Muthukrishnan devised in 2003 and published in a 2005 paper, answers one common question: how many times has a given item shown up so far? An ordinary hash table could answer exactly, but it needs space that grows with the number of distinct items. The sketch uses sub-linear space instead and accepts some overcounting as the price.
How does it work, step by step?
The structure is a two-dimensional array of counters with d rows and w columns, all starting at zero. Each row has its own hash function, and those functions must be pairwise independent. The two sizes are fixed when the sketch is created. Setting w to e divided by ε (rounded up) and d to the natural log of 1/δ (rounded up) means a query's error stays within an additive factor of ε, measured against the total number of items seen, with probability 1 − δ.
When an event of some type arrives, the sketch runs it through every row's hash function. Each function points to one column in its row, and that counter goes up by one. So every arrival touches exactly d counters.
To ask how often a type has appeared, the same hashes locate its d counters, and the sketch reports the smallest. Each of those counters holds the true count plus whatever other items happened to land in the same cell. Taking the minimum picks the row where that extra noise was least.
What does a small worked example look like?
Imagine a toy sketch with 3 rows and 5 columns tracking fruit sold at a market stall. An apple arrives; row one sends it to column 2, row two to column 4, row three to column 1, and each of those cells becomes 1. Later a pear arrives and, by chance, row one also sends it to column 2, while its other rows pick different cells.
Now ask how many apples were sold. The three apple cells read 2, 1 and 1. Row one's cell is inflated because the pear shared it. The minimum, 1, is the right answer here. Had the pear collided with the apple in all three rows, the sketch would have said 2: an overestimate, never an underestimate.
Where is it used?
Streaming summaries of this kind are common in networking, for jobs such as spotting elephant flows that dominate a link, counting distinct flows and estimating the spread of flow sizes. They also appear in databases, for example to estimate how large a join will be before running it.
A further strength is that the sketch is linear. Build one sketch per stream, add them cell by cell, and the result matches a single sketch built on the combined stream. That makes it natural to count on many machines and merge the tallies afterwards. Beyond single counts, two sketches can also estimate the inner product of the histograms they summarise.
Where does it fail or mislead?
The basic minimum estimator is biased: it can report too much but never too little. Rare items suffer most, since a few collisions with popular items can swamp a small true count. The method also shines when the data is highly skewed; when it is not skewed enough, sketches based on averages, such as the count sketch, tend to be more accurate.
Researchers have proposed fixes. The hCount* estimator samples random cells to estimate the typical bias and subtracts it. A maximum likelihood estimator can match or beat the plain minimum and copes with flatter distributions. Conservative updating raises only the counters that need it, which loses linearity but keeps the sketch mergeable.
What does it teach about thinking?
The sketch is a lesson in choosing your error deliberately. Rather than chasing a perfect count it cannot afford, it fixes a bound on how wrong it may be and how often, then spends just enough memory to honour that promise. Knowing which direction an estimate can be wrong is almost as useful as knowing its size: a figure that can only be too high is a safe ceiling, not a mystery.
Questions people ask
Can a count–min sketch ever underestimate a count?
No. Each counter an item touches contains that item's true count plus any extra from other items that collided with it in the same cell. Since collisions only add, every counter is at least the true value, and so is the smallest of them. The sketch can overcount, but in its standard form it never reports less than the real frequency.
How is a count–min sketch different from a Bloom filter?
It can be seen as an implementation of a counting Bloom filter, but the two are sized for different jobs. A counting Bloom filter is usually given about as many cells as there are elements in the set. A count–min sketch typically has a sub-linear number of cells, chosen from the approximation quality wanted rather than from the number of items.
How do you choose the width and depth?
Pick the error tolerance ε and the acceptable failure chance δ first. The width is then e divided by ε, rounded up, and the depth is the natural log of 1 divided by δ, rounded up. A wider grid shrinks the error, while more rows make a large error less likely. Both choices also set the memory and time costs.
The thinking behind it
Sources
- Count–min sketch — Wikipedia
- Streaming algorithm — Wikipedia
- An improved data stream summary: the count-min sketch and its applications — Internet Archive
How this was made: Amili, an AI writer, wrote this article in its own words from the sources above. Every link was checked before publishing. Spotted an error? Tell us and we will correct it.