RankquantRQ

Theory & Derivations

The 90% CI-floor: the uncertainty figure Rankquant publishes

1. Why a mean on its own is not the whole story

The most common instinct — "sort by average rating" — says nothing about how much evidence stands behind the average. A product with one 5-star review has a mean of 5.0 and a sample of one. A product with 500 reviews averaging 4.6 is obviously the safer pick. Sample size has to be accounted for somewhere, and reported plainly wherever it has not been.

There are two well-established fixes: Bayesian shrinkage (pull thin-sample items toward a prior) and confidence-interval bounds (state a defensibly pessimistic estimate alongside the point estimate). Rankquant uses both — in different places, kept deliberately separate:

  • The global and in-cohort percentiles report the measurement as made. Both are the empirical-CDF rank of Ẑ itself: no floor, no shrinkage. A number corrected twice before you see it is no longer a measurement.
  • The AI-adjusted percentile is where the sample is charged for. It re-percentiles Ẑ · n/(n + 53), discounting each item in proportion to how little evidence stands behind it. Shrinkage arithmetic with one published constant — no model, no training, no inference.
  • The 90% CI-floor is published per item as the diagnostic. It quantifies the same uncertainty the adjustment acts on, in the units of the score, and it is derived below. It sets no percentile.

The floor earns its place as the published diagnostic for three reasons. No prior to specify: it needs only a confidence level. Scale-free: it applies identically to z-scores, percentages or counts. Transparent: a reader can recompute it by hand — take the mean, subtract 1.645 divided by the square root of the effective sample size. That's it.

2. The CI-floor formula, and what each published figure actually ranks

One mean, Ẑ, computed once per item. Two of the three published figures rank it directly and the third shrinks it; the 90% floor is computed from the same mean and published beside them.
Global percentileThe empirical-CDF rank of Ẑ itself — the unweighted mean of the qualifying reviewers' z-scores (n_u ≥ 2, σ_u > 0), over N of them. No floor and no shrinkage is applied before the ranking. Every reviewer counts once and equally, so there is no weight to correct for and N_eff = N.
In-cohort percentileIdentical Ẑ, identical N. The cohort figure re-ranks the same numbers inside a smaller peer set — same category, price band or city depending on the vertical. No second interval, because there is no second estimate.
AI-adjusted percentileThe same reviewer-normalized mean, discounted for the confidence its sample supports: Ẑ · n / (n + 53), re-percentiled on the same scale. This is the figure that acts on sample size, in every vertical. Shrinkage arithmetic, not a model.
90% CI-floor (published diagnostic)floor = Ẑ − 1.645 · (1/√N), computed per item and shipped with it — ciFloorZ in the catalog exports, zCiLower and zMargin on products. The SE scale factor is 1 because z-scores have unit variance by construction. It is there to be read, not sorted on: no published percentile is a function of it.
One mean, Ẑ, computed once per item. Two of the three published figures rank it directly and the third shrinks it; the 90% floor is computed from the same mean and published beside them.

3. Worked example — three products, three reviewer counts

Product A:  4 reviewers,   mean z = +2.10
   SE       = 1/√4  = 0.500
   90% floor = 2.10 − 1.645·0.500 = +1.28   ← published diagnostic
   adjusted  = 2.10 · 4/(4+53)    = +0.15   ← what the AI-adjusted figure ranks

Product B:  20 reviewers,  mean z = +1.80
   SE       = 1/√20 = 0.224
   90% floor = 1.80 − 1.645·0.224 = +1.43
   adjusted  = 1.80 · 20/(20+53)  = +0.49

Product C:  80 reviewers,  mean z = +1.60
   SE       = 1/√80 = 0.112
   90% floor = 1.60 − 1.645·0.112 = +1.42
   adjusted  = 1.60 · 80/(80+53)  = +0.96

Global and in-cohort percentiles rank the mean z:  A > B > C   (2.10 > 1.80 > 1.60)
AI-adjusted percentile ranks the shrunk mean:      C > B > A   (0.96 > 0.49 > 0.15)
The 90% floor is printed for each and ranks nothing.

The re-ordering on the middle line is the point of the third figure, and it is the only one of the three where sample size changes the order. Product A might be exceptional, but with 4 reviewers we can't distinguish "exceptional quality" from "lucky small sample," so the adjustment lets it keep only 4/57 of its measured distance from the corpus average. Products B and C have earned the confidence that their means aren't accidents, and keep far more of theirs. Meanwhile the headline percentile still reports A's +2.10 as measured, with its reviewer count and its 90% floor beside it — measurement and uncertainty shown separately rather than pre-blended into one number. Read as a diagnostic, the 0.01 gap between B and C at the floor is a statistical tie (we flag it on-site); the 0.15 gap between B and A is a real separation.

4. Why 90% and not 95% or 99%

The confidence level sets how conservative the published floor is. At higher confidence the floor sits further below the mean — a broader statement of what we don't know about a thin sample. At lower confidence it hugs the mean and stops telling a reader anything they couldn't read off Ẑ directly.

One-tailed critical values:
  80% →  z = 0.842
  85% →  z = 1.036
  90% →  z = 1.645     ← Rankquant's published floor
  95% →  z = 1.960
  99% →  z = 2.326

Distance the floor sits below the mean, per 1/√N of SE,
relative to 90%:
  80% → 0.51×   (much softer)
  90% → 1.00×   (our choice)
  95% → 1.19×   (19% wider)
  99% → 1.41×   (41% wider — uninformative at the sample sizes we hold)

We publish at 90% because it is the conventional level for a one-tailed directional claim and because it stays informative at the sample sizes we actually hold. At 95% the floor collapses so far for items under ~15 reviewers that it barely distinguishes one thin sample from another; at 85% it sits close enough to Ẑ to add nothing the headline figure isn't already saying. The constant is published and version-stable at 1.645 across every vertical; any change requires a public version bump. Note what that change would not do: it would not reorder a single ranking, because no published percentile reads the floor.

5. Relationship to other small-sample techniques

The floor and the n/(n+53) shrinkage sit in the same well-known family of small-sample-safe techniques. Rankquant publishes the first and ranks with the second.
Wilson score lower bound (binary)Reddit's "best" sort and Yelp's internal ranking use the Wilson lower bound on up/down-vote binomials, as the sort key. Rankquant's CI-floor is the continuous-scale analog on z-scores — same construction, published as a diagnostic rather than used to sort.
IMDb Top 250 (Bayesian shrinkage)IMDb's Top 250 formula W = (v/(v+m))·R + (m/(v+m))·C shrinks thin-sample movies toward the global mean C. This is the family the AI-adjusted percentile belongs to: Ẑ · n/(n + 53) is the same weight with m = 53 and the prior set at the corpus average, which is zero on the z scale.
Hodges-Lehmann estimatorA robust rank-based estimator of a location parameter. Rankquant's empirical-CDF percentile step is rank-based (Hodges-Lehmann flavored) even though Ẑ and its floor are both mean-based.
Meta-analysis random-effects modelsDerSimonian-Laird and related random-effects estimators combine within-study and between-study variance. Rankquant's reviewer-level aggregation is a fixed-effects approximation; a random-effects variant is a future methodology upgrade.
The floor and the n/(n+53) shrinkage sit in the same well-known family of small-sample-safe techniques. Rankquant publishes the first and ranks with the second.

6. What users see on the product page

  • Global percentile: where the item lands against everything we rank, as one number. "89 / 100 global."
  • In-cohort percentile: the same quantity re-ranked among its peer set, shown beside the global figure with the tagline interpreting any spread.
  • AI-adjusted percentile: the same reviewer-normalized mean after the shrinkage Ẑ · n/(n + 53), re-percentiled on the same scale, with the reviewer count it rests on.
  • The 90% floor, as a figure to read: published per item beside the three percentiles — "mean +1.60, floor +1.42, SE 0.112, N 80" — so the uncertainty is legible without being baked into a rank.
  • Statistical ties: products within 0.05 of each other on that published floor are bracketed as "statistically similar" rather than read as ordinally separated.

7. A note on one-tailed vs two-tailed

The CI-floor uses the one-tailed 90% critical value z = 1.645. Why one- tailed? Because the floor answers a one-tailed question: "what's a defensibly-low estimate of this product's quality?" The upper bound isn't interesting — a lucky ceiling is not evidence of anything, and no figure on this site reads one. Two-tailed would correspond to z = 1.960, giving a 95% symmetric CI; quoting that as the floor would overstate how little we know about items whose uncertainty we only care about in one direction.

Frequently asked questions

Why 90% and not 95%?+
Consumer review reporting is not a drug trial. At 95% the floor collapses so far for products with fewer than 15 reviewers that it stops distinguishing one thin sample from another, and an uninformative diagnostic is the only real cost, since no ranking reads it. 90% is the conventional one-tailed level, it stays legible at the sample sizes we hold, and the constant is published and version-stable.
Does anything on the site rank on the CI-floor?+
No — not since 29 July 2026 for products, and not on any vertical now. The global and in-cohort percentiles are the empirical-CDF rank of the mean reviewer z-score itself. The floor is still computed and still shipped per item, so the retired CI-ranked figures stay reproducible and so readers can see the uncertainty, but no published percentile is a function of it. The sample-size correction moved to the AI-adjusted percentile, which shrinks the same mean by n/(n + 53).
Where is the CI-floor published?+
On the item, in the data. It ships per row in the catalog exports as ciFloorZ, and on products as zCiLower with its margin zMargin, alongside the mean z and the calibrated-reviewer count. /llms-full.txt states the same thing in prose for answer engines: the floor is computed, it is shipped, and it sets no published percentile.
What's the CI-floor of a cohort percentile?+
Trick question — the in-cohort figure is a ranking view, not a separate estimator. It is a deterministic re-ranking of the same global mean z-scores among cohort members. There's no second sample, so there's no second interval. If you want uncertainty on the cohort score, read the item's published floor.
How does CI-floor width change with N?+
SE scales as 1/√N, so the margin 1.645 × SE halves when N quadruples. Going from N=4 to N=16 cuts it in half. N=100 gives a margin of 0.165 — small but not zero. N=1000 gives 0.052 — negligible. The floor approaches the mean asymptotically as N grows, which is exactly the shape of the statement the diagnostic is making: with enough reviewers there is nothing left to discount.
Does this penalize new products too harshly?+
The floor penalizes nothing — it reports. A new product with genuinely good reviews but few of them gets the global percentile its mean earns, printed with the reviewer count it rests on and the floor that says how far the truth could sit below it. The figure that does move against it is the AI-adjusted percentile, deliberately: at n = 4 the item keeps 4/57 of its measured distance from the corpus average. That is by design and we own it, and it unwinds on its own as coverage deepens.

← Degrees of freedom · Next: Inter-rater reliability →