Theory & Derivations

Theory & derivations

Short answer

Rankquant's methodology is four operations in sequence: (1) normalize every reviewer onto their personal z-scale, (2) average those z-scores with every qualifying reviewer counting equally — no reviewer and no source is weighted above another, (3) convert that mean to a 0–100 percentile via the empirical CDF, unfloored and unshrunk, so the headline figure reports the measurement as made, (4) publish two further readings of the same mean: its 90% CI floor, Ẑ − 1.645·SE(Ẑ), as the per-item uncertainty diagnostic — computed and shipped, never ranked on — and the AI-adjusted percentile, which re-percentiles Ẑ · n/(n + 53). That produces exactly three published rankings per item: the global percentile, the in-cohort percentile— the same quantity re-ranked among an item's peer set, a view rather than a second computation — and the AI-adjusted percentile, the same reviewer-normalized mean discounted for how much confidence the sample supports. Each sub-page below unpacks one of the three load-bearing statistical concepts in detail. That estimator scores all six verticals — Films & TV, Hotels, Wines, Cruises, Books and, since 29 July 2026, Products. Products differs in reach, not in estimator: the Amazon reviewer graph is sparse enough that only the minority of listings with overlapping reviewers gets a percentile, and the rest are published with no rating at all — not a percentile, and not the retailer's star average standing in for one. Scope is spelled out below.

Start here: the founding metrics

New to the methodology? Read Founding metrics — the five statistics primitives behind every Rankquant percentile first. It is a first-principles tour of the five textbook ingredients (z-score, standard error, 90% CI-floor, empirical CDF, Kish design effect) the rest of the pipeline composes.

The three load-bearing concepts

Each concept has a dedicated sub-page with derivation and worked examples.
Founding metrics (start here)	First-principles tour of the five primitives the pipeline rests on: z-score, standard error, 90% CI-floor, empirical CDF, Kish design effect. Each primitive paired with its canonical statistical reference.
Degrees of freedom (df)	Why reviewer σ_u uses n_u − 1 (Bessel), why we require n_u ≥ 2, why effective sample size N_eff equals the raw reviewer count when every reviewer counts equally, and why a reviewer with df = 1 is admitted on equal terms while thinness is reported in the published floor and charged at the AI-adjusted step.
Confidence intervals (the published 90% floor)	What the floor measures, and why it is published per item rather than ranked on. Derivation of SE(Ẑ) = 1/√N under equal weights. Why 90% and not 95%. Analytical comparison to Wilson score and to IMDb-style Bayesian shrinkage — the family the AI-adjusted percentile's n/(n + 53) belongs to.
Inter-rater reliability (reviewer-level)	ICC(1,1) variance decomposition across reviewers, how per-reviewer normalization removes reviewer main-effects, Cohen's κ for binary reviewer decisions, and why low agreement is published as a diagnostic rather than converted into a reviewer weight.

Each concept has a dedicated sub-page with derivation and worked examples.

Which verticals this estimator scores

All six verticals run the pipeline derived on these pages, and the schema id shipped with every dataset records which build produced a given number. What differs between them is how much of each catalog the estimator can reach — not how anyone's reviewers are counted, which is equally everywhere.

Score schema by vertical. Every vertical is now a percentile of the same statistic, computed with every qualifying reviewer counting equally; what differs is how much of each catalog it reaches.
Films & TV · Hotels · Wines · Cruises · Books	Reviewer-z schema. Per-reviewer z-score → equal-weight mean across qualifying reviewers → empirical-CDF percentile of that mean. Hotels joined that shape on 6 August 2026 — it was the last vertical whose headline ranked a confidence floor rather than the mean itself — so a published hotel with no calibrated reviewer now carries a null percentile instead of a floor-derived one. The 90% CI floor is computed for every item whose sample defines one and published with it as a diagnostic; the correction for sample size lives in the AI-adjusted percentile, which re-percentiles the same mean times n/(n + 53). Everything on this page and its sub-pages describes this estimator.
Products (Amazon) — schema id reviewer-z-r3-mean-v2	The same estimator: per-reviewer z-score (reviewers with 2+ ratings) → equal-weight mean → empirical-CDF percentile of that mean, re-ranked within the category bucket for the in-cohort figure. It replaced reviewer-z-r3-ci90-v1 on 2026-07-29 — which ranked the 90% floor of the same mean — and that had replaced bayesian-site-v1, a percentile of a prior-shrunk Amazon star average, earlier the same day. Three different scales, not three vintages of one. Both predecessors are retired, and since 6 August 2026 the retailer's star average and rating count are not published anywhere on this site — not on a page, not in the data files, not in structured data, and not as a sort key. Coverage, not method, is the limit here: 1,215,077 of 1,352,809 published products have enough reviewer overlap to be scored, and the rest carry no rating at all rather than a borrowed one.

Score schema by vertical. Every vertical is now a percentile of the same statistic, computed with every qualifying reviewer counting equally; what differs is how much of each catalog it reaches.

The limit is capture depth, not the estimator. Each ASIN scrape reads review page 1 only and the crawl is sharded ASIN-disjoint, so the same Amazon reviewer is often observed on just one product: 1,594,196 of 4,721,328 reviewers rated two or more and can therefore supply a baseline, which is what leaves 1,215,077 of 1,352,809 listings with a computable percentile. The rest are published as openly unscored pages — no percentile, and nothing borrowed from the retailer standing in for one — and gain a score as the review capture goes deeper. One exclusion rule survives, and it is about identifiers rather than ratings: a 10-character ASIN that does not begin with B is an ISBN, i.e. published media, and belongs to the Books catalog. The two integrity cuts that used to withhold roughly a quarter of the inventory — an implausible 5.0 star average, a ratings count repeated verbatim across distinct ASINs — were retired on 6 August 2026. Both detect real scraper corruption and both are kept as scrape-health diagnostics, but neither field is published, exported, sorted on or filtered on any more, so a wrong value in one cannot reach a reader, and withholding a product on that basis would let the retailer decide our catalog through a side door. Full derivation and the exclusion rules: /methodology#scope.

Each step of the pipeline, with its statistical justification

Rankquant's methodology (/methodology) is a compositional estimator: three steps to the headline percentile, plus two further readings of the same mean and one re-ranking of it. The table below maps each to the statistical property that keeps it rigorous, and points at the deep-dive page that justifies it.

Step	Estimator	Statistical property
1	z_u,i = (r_u,i − μ_u) / σ_u	Per-reviewer studentization. σ_u uses Bessel correction (df = n_u − 1). Removes reviewer main-effects; dimensionless output.
2	Ẑ(i) = mean_u z_u,i	Unweighted reviewer-fixed-effects aggregation. Every qualifying reviewer counts once and equally — no source multiplier, so N_eff = N. Unbiased estimator of the product's population mean standardized rating.
3	p = empirical-CDF rank of Ẑ	Nonparametric ranking of the mean itself — no floor and no shrinkage applied first, so the headline figure reports what was measured. Robust to non-normality of the distribution of means. Global p uses everything we rank; cohort p uses the peer set.
Diag	floor = Ẑ − 1.645 · SE(Ẑ)	One-tailed 90% CI lower bound. SE(Ẑ) = 1/√N_eff under unit-variance z-scores. Computed for every item whose sample defines one and published with it as the uncertainty diagnostic — no percentile is a function of it, on any vertical, since 6 August 2026. Same family as Wilson score / Reddit's "best" sort.
Adj	adj(i) = Ẑ(i) · n / (n + 53)	The AI-adjusted percentile, and the one figure that acts on sample size: the same equal-weight mean, shrunk toward the corpus average in proportion to how thin the sample is, then re-percentiled on the same scale. Shrinkage arithmetic — no model, no training. K = 53 is the median sample size across everything we rank.
Cohort	re-rank the same Ẑ within Cohort(i)	No new estimator. The in-cohort percentile is a view, not a computation. Guarantees internal consistency: cohort rank is a deterministic function of the same mean z-scores.

Each sub-tab above unpacks one of the three load-bearing concepts in full. Read in order (Degrees of freedom → Confidence intervals → Inter-rater reliability), or jump to the one you need. All pages cite primary statistical sources; nothing here is proprietary theory — what's proprietary is publishing the details, committing to the constants, and running the pipeline on real data at scale.

Prerequisites: basic familiarity with mean, standard deviation, and the normal distribution. For term-by-term definitions see /glossary/statistics-terms.