RankquantRQ

The 7 review sources that dominate every category (and how much to trust each)

The pattern across every category

Every consumer category falls into a similar review-source ecology:

Our answer to that ecology is not to rank the tiers against each other. It is to read whichever of them we can reach, normalize every reviewer against their own history — subtract their personal mean, divide by their personal standard deviation — and rank inside a peer cohort, with every qualifying reviewer counting equally. The method is written out at /methodology. The tiers in the tables below are a separate thing: an argument about which sources are worth your attention, not a coefficient anything gets multiplied by.

What Rankquant reads today

None of the professional publications in the tables that follow is ingested anywhere, in any category — so those tables are landscape analysis, not a description of a shipped computation. The live inputs are crowd platforms:

The actual review pool behind every live Rankquant score. No professional critic publication is ingested in any category.
WinesVivino reviewer pool.
Movies & TVIMDb, blended with Rotten Tomatoes on the subset of titles that carries it.
BooksGoodreads and Amazon.
HotelsTripAdvisor, Booking.com, Agoda, Expedia, Trip.com, Ostrovok.
CruisesCruise Critic.
ProductsAmazon customer reviews.
Reviewer weightingNone. Every qualifying reviewer counts exactly the same, in every category; the differentiation comes from per-reviewer normalization, not from ranking the platforms or the people inside them.
The actual review pool behind every live Rankquant score. No professional critic publication is ingested in any category.

So the rest of this article is an assessment of the review landscape: which sources repay your attention when you go read them yourself, and which do not. None of it feeds a Rankquant number.

Wines — who to trust

Wine review sources, ordered most- to least-trustworthy. Editorial opinion for readers, not a scoring input: none of these professional publications is ingested, and Rankquant does not rank its sources against each other.
Robert Parker / Wine Advocate / VinousOldest professional 100-pt scale; historically the highest credibility in the category.
Wine SpectatorInstitutional panel reviews; staff tastings.
Jancis Robinson MW20-pt scale with less inflation pressure; rigorous editorial.
DecanterInternational panel coverage.
James SucklingProlific; known to be slightly generous.
Jeb DunnuckGrowing credibility; narrower coverage.
CellarTracker (crowd, enthusiast)Deep for specific wines.
Vivino (crowd, general)Largest sample; lowest rigor per review.
Retailer reviews (Wine.com, Total Wine, etc.)Retail bias; the least informative of the set.
Wine review sources, ordered most- to least-trustworthy. Editorial opinion for readers, not a scoring input: none of these professional publications is ingested, and Rankquant does not rank its sources against each other.

Movies & TV — who to trust

Film and TV sources, ordered most- to least-trustworthy. The professional aggregate (Metacritic) outranks the binary aggregate (Rotten Tomatoes) on information-per-review. Of these, only IMDb and the Rotten Tomatoes score are read today.
MetacriticProfessional aggregate, pre-weighted before you see it — that weighting is Metacritic's, and it is undisclosed.
Letterboxd (crowd, cinephile)Enthusiast audience; cleaner than IMDb.
Rotten Tomatoes (critics)Binary fresh/rotten loses information.
IMDb weighted averageHuge sample; genre ballot-stuffing pressure.
Film and TV sources, ordered most- to least-trustworthy. The professional aggregate (Metacritic) outranks the binary aggregate (Rotten Tomatoes) on information-per-review. Of these, only IMDb and the Rotten Tomatoes score are read today.

Books — who to trust

Book review sources, ordered most- to least-trustworthy. Trade-press editorial (Kirkus, Publishers Weekly, NYT Book Review) outranks the crowd platforms on rigor. Of these, Rankquant reads Goodreads and Amazon — and reads every reviewer in them on equal terms.
NYT Book ReviewRigorous editorial; highest-profile professional reviews.
Kirkus ReviewsPre-publication professional reviews; industry standard.
Publishers WeeklyTrade-press professional editorial.
Booklist / LitHub / NPRProfessional critics; somewhat narrower coverage.
Goodreads (crowd)Massive sample; severe inflation.
Amazon book reviewsKnown manipulation; read the raw average with suspicion.
Book review sources, ordered most- to least-trustworthy. Trade-press editorial (Kirkus, Publishers Weekly, NYT Book Review) outranks the crowd platforms on rigor. Of these, Rankquant reads Goodreads and Amazon — and reads every reviewer in them on equal terms.

Amazon consumer products — who to trust

Consumer-goods sources, ordered most- to least-trustworthy. Lab-based professional testing outranks Amazon's own platform by a wide margin. Rankquant reads Amazon customer reviews only.
RTINGS / Wirecutter / Consumer ReportsLab-tested; professional editorial.
Amazon verified purchaseTransaction-verified but self-selection biased.
Reddit topical subreddits (aggregated)Noisy but directionally informative.
Consumer-goods sources, ordered most- to least-trustworthy. Lab-based professional testing outranks Amazon's own platform by a wide margin. Rankquant reads Amazon customer reviews only.

Hotels — who to trust

Hotel review sources, ordered most- to least-trustworthy. Michelin Keys and Forbes Travel Guide sit above the verified-stay aggregators on editorial rigor. Rankquant reads the aggregators — TripAdvisor, Booking.com, Agoda, Expedia, Trip.com, Ostrovok — with no platform counting for more than another.
Michelin Keys / Forbes TravelProfessional; rigorous standards.
Booking.com (verified stays)Verified but 8.4/10 average inflation.
TripAdvisorOlder, more gaming pressure.
Google HotelsLower verification strictness.
Hotel review sources, ordered most- to least-trustworthy. Michelin Keys and Forbes Travel Guide sit above the verified-stay aggregators on editorial rigor. Rankquant reads the aggregators — TripAdvisor, Booking.com, Agoda, Expedia, Trip.com, Ostrovok — with no platform counting for more than another.

Restaurants — who to trust

Restaurant review sources, ordered most- to least-trustworthy. Professional critics (Michelin, NYT) dominate; Yelp is the most-gamed surface in the category. Rankquant has no restaurants category — this list is landscape analysis only.
NYT / Michelin / professional criticsRigorous editorial.
Resy / OpenTable (verified diners)Verified but inflated.
Google ReviewsHigh volume; lower verification.
YelpMost-gamed platform in the category.
Restaurant review sources, ordered most- to least-trustworthy. Professional critics (Michelin, NYT) dominate; Yelp is the most-gamed surface in the category. Rankquant has no restaurants category — this list is landscape analysis only.

Why publish an opinion no score depends on

Every other review aggregator treats its weighting as proprietary secret sauce. Metacritic doesn't tell you why a New York Times review is weighted more than a Variety review. Rotten Tomatoes doesn't explain which reviewer gets which weight.

The opposite move is to have no secret sauce, because there is no weighting to keep secret. Every qualifying reviewer we read counts once, and the whole computation — admission rules, per-reviewer z-scores, the confidence correction and the percentile mapping — is written out at /methodology, so a reader can rerun it line by line. The tier lists above are an opinion you can disagree with at no cost to the arithmetic, precisely because no score reads them. The output is deterministic: same inputs, same arithmetic, same score.

The same transparency is how we answer accusations of bias on the scores that are live. If a brand claims Rankquant is under-ranking their product, we can point to the reviewer set, the raw ratings, each reviewer's own baseline, the published CI-floor diagnostic and the peer set — and demonstrate that the normalized output is mechanically correct given the inputs.

Frequently asked questions

How are these trust tiers decided?+
Three factors: (1) editorial rigor (professional staff > individual crowd reviewers), (2) manipulation resistance (verified purchase > anonymous), and (3) inflation pressure (lower-inflation scales rank higher). They are an editorial judgment about the sources, published so it can be argued with. They rank sources for a reader; nothing multiplies them into a Rankquant score.
Which of these sources actually feeds a Rankquant score?+
Vivino for wine, IMDb blended with Rotten Tomatoes where a title carries it, Goodreads and Amazon for books, TripAdvisor/Booking.com/Agoda/Expedia/Trip.com/Ostrovok for hotels, Cruise Critic for cruises, and Amazon customer reviews for products. Every one of those is a crowd platform, and every qualifying reviewer in them counts equally. The professional publications in the tables above are not ingested.
Can the tiers change?+
Yes. If a source gets acquired, loses its editorial staff, changes scoring methodology, or demonstrably starts being gamed, we revisit its tier and publish a changelog entry with the diff and the reasoning. No live score moves when that happens, because no live score reads the tiers in the first place.
What stops one biased review from moving a score?+
Two things. Reviewers are normalized against their own rating history, so a reviewer who rates everything highly contributes little signal by being positive again. Then the AI-adjusted percentile discounts the mean by n / (n + 53), so a thin reviewer set is pulled toward the corpus average rather than flattered by one loud voice. The 90% CI-floor is published alongside as an uncertainty diagnostic; it ranks nothing.
What if an item has almost no usable reviewers?+
It does not get a score. The catalog filter requires a minimum count of z-normalizable reviewers per item, and items below it are published with the platform's own figures and no Rankquant number. We would rather show nothing than a statistic built on three reviewers.

Related: The full methodology · Rating inflation explained · Bayesian averaging