How Rankquant handles natural wines in review aggregation

By Ryan Siegal · Founder and Principal

Published 2026-04-23

Why natural wines are a methodology test

Natural wines — wines produced with minimal intervention, native yeasts, minimal or no sulfite addition — have become a substantial category in the last 15 years. They're particularly strong in Loire Valley whites, Jura, Beaujolais, and increasingly New York, Oregon, and California.

They pose three distinct challenges for aggregated scoring:

Thin professional coverage. Wine Spectator, Wine Advocate, and Suckling cover natural wines less comprehensively than conventionally-made wines. Decanter and Jancis Robinson cover them more. CellarTracker and Vivino cover them substantially.
Stylistic polarization. Some professional critics penalize natural wines for stylistic choices (brettanomyces, volatile acidity, unfined cloudiness) that the natural-wine community considers features rather than flaws. The same wine can get 88 from Wine Spectator and 16/20 from Jancis Robinson.
Category growth outpacing coverage. New natural-wine producers enter the market faster than professional critics can cover them. The review-coverage gap widens year over year.

How Rankquant handles it — principle by principle

1. Natural wines aren't their own category

We don't create a separate "natural wines" peer set. A natural Burgundy Chardonnay is compared against other Burgundy Chardonnays in its price tier and vintage — conventional and natural alike. This prevents the stylistic-polarization problem from distorting the score: we don't create an artificially-friendly or artificially-hostile peer set.

2. Thin reviewer sets are penalized, not flattered

A natural wine is ranked on the plain mean of its reviewers' z-scores, and the AI-adjusted percentile keeps only the share n / (n + 53) of that mean — so a wine rated by a handful of Vivino reviewers is held near the middle there until the sample earns its position. Every wine also publishes the 90% one-tailed lower bound around its mean as a diagnostic, so the uncertainty in a thin reviewer set is legible without being baked into a rank. A wine needs a real reviewer set to hold a high position, and the score sharpens as coverage grows. That is the same treatment any obscure conventional wine gets.

3. The crowd pool gets no natural-wine adjustment

Vivino reviewers are normalized identically whether the wine is natural or conventional. We don't boost the crowd signal for natural wines just because the community is more engaged there. That would be a hidden advocacy bias; we don't do advocacy biases.

4. Stylistic disagreement is surfaced, not hidden

Because the input is the Vivino reviewer pool rather than a critic panel, disagreement shows up inside the pool: a divisive natural wine produces a wide spread between its z-normalized score and its raw average, since the reviewers who rated it highly did so against their own baselines while the raw star average washes that out. Both numbers sit side by side on every wine page, so readers see the disagreement rather than a single figure that resolved it for them.

Natural wines aren't a rating problem — they're a transparency problem. Show readers when sources disagree and let them decide whose palate to trust.
— Rankquant editorial approach

What this means for the natural-wine reader

If you're specifically looking for natural wines on Rankquant:

Use filters, not separate category scores.Filter the category hub for wines tagged "natural" after sorting by normalized score. The wines that surface will have earned their score against mixed-style peer sets, which is a stronger signal than a natural-only peer set would provide.
Compare the two scores on the page.Where a wine's z-normalized score and its raw average diverge sharply, the wine divides its tasters — read individual reviews before buying. Neither number is misleading; the gap between them is the informative part.
Prefer wines with deep reviewer counts.The AI-adjusted percentile keeps only the share n / (n + 53) of a wine's measured distance from the corpus average, so a wine rated by a handful of people ranks below what its raw average alone would suggest. A large reviewer count is what lets a high score actually hold its position.

What we won't do

A few explicit non-choices that reflect our commitment to methodological rigor:

We won't apply a "natural bonus" to scores.Some retailers and curators do this implicitly. We don't — the per-reviewer z-score and the n / (n + 53) sample-size discount are the same for all wines.
We won't discount critics for penalizing natural styles.Nothing in the method could: no source and no reviewer is weighted above any other, so disagreeing with the natural-wine community costs a reviewer nothing here. Disagreement is a data point, not a failure.
We won't maintain a "natural-wine score" separate from the normalized score.That would be two scoring systems and therefore no scoring system.

Frequently asked questions

Does Rankquant penalize natural wines?+

No. Natural wines are normalized against their grape × region × vintage × price peer sets just like any other wine. If a natural wine scores lower than its peers, that reflects how its own reviewers rated it relative to their personal baselines — not any Rankquant bias.

How does Rankquant know which wines are natural?+

We tag wines using production-declaration data from producer websites (for small imports) and retailer category tags (for wines sold through specialty importers like Selection Massale, Louis/Dressner, or Jenny & Francois). "Natural" isn't a legal category so tagging is editorial, not algorithmic.

What about orange wines and other emerging styles?+

Orange wines (skin-contact whites), pét-nat sparkling, and other emerging styles are normalized the same way — peer sets are defined by style + variety, and we'll split out dedicated peer sets only when a category grows large enough to support n ≥ 30 independent wines in a given vintage × price slice.

Will Rankquant cover cloudy / unfiltered / high-VA wines fairly?+

We don't have the authority to say what's "fair" for a stylistic choice — professional critics disagree on exactly this question, and the method takes no side. What we commit to is transparency: every wine page shows its reviewer count, its z-normalized score and its raw average side by side, so a wine that splits its tasters is visible as a split rather than resolved into one number.