Methodology changelog

Per-reviewer z-score + CI-floor ranking (complete methodology overhaul — the CI-floor ranking and the R2/R3 lenses were later withdrawn; see the 2026-08-06 corrections at the end of this entry)

• Replaced v0.1's source-level six-step pipeline with reviewer-level four-step pipeline: per-reviewer z-score → aggregate three ways → 90% CI-floor → empirical-CDF percentile.
• Per-reviewer normalization: z_{u,i} = (r_{u,i} − μ_u) / σ_u, pooled cross-category per reviewer. Admission rule n_u ≥ 2 and σ_u > 0 for R1 and R2; σ_u = 0 reviewers admitted to R3 via imputed σ̃_s.
• Three aggregation lenses specified: R1 (pure relative, unweighted reviewer mean — headline), R2 (source-weighted with published w_s), R3 (broadened to include consistent-rater reviewers). SUPERSEDED: only the unweighted R1 aggregate was ever computed; R2 and R3 were withdrawn in the 2026-08-06 correction below.
• Ranking specified to move from the raw aggregate mean to the 90% one-tailed CI-floor: floor = Ẑ − 1.645 · SE(Ẑ), where SE = 1/√N_eff. For R2, N_eff uses Kish's design-effect formula. SUPERSEDED: that move never shipped — see the 2026-08-06 correction below. Ranking is on the raw mean z-score.
• Output scale changed from 1–5 normalized score to 0–100 empirical-CDF percentile. A 90 means the product's CI-floor is higher than 90% of all products in the database.
• Cohort definition simplified: same category × ±20% of list price. Cohort percentile is a pure re-ranking of the same CI-floor among cohort members — not a separate computation. Removes the v0.1 peer-set-hierarchy overhead.
• Bayesian prior strength k retired from the pipeline. Its role (penalizing thin samples) is now performed by the CI-floor's SE term, which is parameter-free.
• Internal reference implementation bumped to v0.2.0: Reviewer record, aggregate_r1/r2/r3, percentile_from_ci_floors, cohort_percentile, is_in_cohort. It was not distributed as a package or a public repository — the specification at /methodology/ was the public artifact. SUPERSEDED: aggregate_r1/r2/r3 and percentile_from_ci_floors describe the v0.2.0 specification as it was written in April 2026, not code that ever ran; both were withdrawn in the 2026-08-06 corrections below and neither survives in the current pipeline.
• v0.2 is not backward-compatible with the v0.1 six-step formulation: the two produce different numbers from the same inputs. Scores published under v0.1 are read against the v0.1 specification, which stays on this page.
• Correction (2026-08-06): the three-lens R1/R2/R3 framing described in this entry was never implemented. No Rankquant score has ever applied a source weight or any other per-source multiplier — the pipeline computes one unweighted aggregate in which every qualifying reviewer counts equally. The three figures actually published per item are the Global percentile (empirical-CDF rank of that mean z-score against everything we rank), the In-cohort percentile (the same z-score re-ranked inside a smaller peer cohort — a re-ranking, not a second computation) and the AI-adjusted percentile (the same mean corrected for how much confidence its sample supports: adj = mean_z · n / (n + 53)). R2, R3, the source weights w_s and the imputed σ̃_s are withdrawn, and with equal weights throughout there is no design effect left for Kish's formula to correct.
• Correction (2026-08-06): the move from a raw aggregate mean to the 90% one-tailed CI-floor described in this entry was likewise never shipped, so every reference to a CI-floor above is withdrawn. The Global percentile is the empirical-CDF rank of the raw mean z-score itself; the In-cohort percentile re-ranks that same raw mean inside the peer cohort. The 90% confidence-interval lower bound is still computed and published per item as a diagnostic field, but no percentile is ranked on it. Protection against thin samples is carried by the AI-adjusted percentile's n / (n + 53) shrinkage instead — arithmetic, not a model, and not machine learning.

Initial public methodology + source weights (legacy)

• Six-step normalization method published at /methodology/ (rescale → source-weighted mean → Bayesian adjust → z-score → percentile → 1-5).
• Source-weighted mean uses w_s · √n_s (meta-analysis inverse-variance approximation) to keep large-sample crowd sources influential but not dominant.
• Bayesian prior strength set to k = 30 across all categories. Chosen via simulation across 10,000 synthetic review distributions.
• Source weights published for wine, movies/TV, books, Amazon consumer products, hotels, and restaurants. Those tables were withdrawn in the 2026-08-06 correction below and no longer appear at /methodology/ or /glossary/wine-terms/.
• Adaptive peer-set hierarchy for wines (style × grape × region × vintage × price tier) with minimum-30-peers backoff rule.
• Correction (2026-07-28): this entry previously listed an affiliate routing formula (price_attractiveness × commission_rate) as committed and published. No such formula was ever implemented. Outbound links are built per-category from the record's own identifiers; see /methodology/#affiliate.
• Reference implementation specified in prose rather than released as a package: every constant it depended on (source weights, n_u >= 2 and sigma_u > 0 admission rules, the 90% CI-floor, the empirical-CDF percentile mapping) was written out at /methodology/ at the time. SUPERSEDED: the source weights and the CI-floor were both withdrawn in the 2026-08-06 corrections and no longer appear there; the admission rules and the empirical-CDF mapping still do.
• Correction (2026-08-06): no published Rankquant score has ever applied one of the source weights described in this entry. The shipped pipeline has only ever computed an unweighted per-reviewer z-score aggregate, so the source-weighted mean above was specification prose rather than running code. Every qualifying reviewer counts equally and no source is weighted above any other; the weight tables have been removed site-wide.