The Awful Truth

Everyone seems to advertise they have them, though not everyone’s sure they need them. Publishers both embrace and revile them. Everyone things they can do one it seems, but few seem to get the job done right. No, I’m not talking about freelancers. I speak of reviews.

To be fair, I believe that a good review can do a lot to help customers decide if a particular product would make a good addition to their gaming library. Good reviews delve deep into the heart of the book, exposing its gems and its imperfections. They project what the book could be used for that goes beyond the limitations of its page count. They enlighten the reader as to the possibilities.

Unfortunately, a lot of reviews I see only regurgitate the table of contents. While knowing what’s in the book is a good thing, it doesn’t tell me anything that I couldn’t figure out if I opened the book on my own or went to the publisher’s website. Such reviews fail in my eyes because they fail to:

Take A Stand 
Show the Good 
Show the Bad 
Rate the Product 

It seems all too often these days that a reviewer wants to stay on the ‘good side’ of a company in order to make sure they’re still getting that free product for their future reviews. Their reviews read like an oatmeal recipe—a guarantee that the review will quickly pass from memory but without any savory afterthoughts.

Good reviews take a stand and hold their ground. They run hot or cold but never lukewarm. They compare the release versus others of similar nature and point out why this product is better or why it fails. 

There are some stances not worth taking, though. For instance, we’ve had reviews for a product where the review said things like: “If you hate spellcasters, there’s not much reason to buy this book,” and “If you think druids and alchemy are best left out of the game, this book won’t interest you that much.” If the concept of new spells and magic items for your game reviles you, then you’re probably going to steer clear from our Spells & Magic release. Some things are better left unsaid (because a review comes across as inane if it proclaims what’s obvious).

Taking a stand requires intelligence and integrity. It requires the review to look at things from the viewpoint that the book was created and to compare and contrast its strengths and weaknesses in an intelligent light. 

No matter how bad the product, there’s always something good to say about it. If a review doesn’t show those good things, it shouldn’t have been written. Fortunately, problems like this are rare these days.

A review should pull out the true gems in a product and let them shine for the entire world to see. They should be specific in their praise, like a skilled archer seeking the orc chieftain in the masses. Comparisons to similar products should be made, and issues in which the product shines over others should be highlighted.

But tracking down the gems isn’t something that’s inherently obvious to every reviewer. Reading something and realizing its true genius isn’t a realization that occurs to most; a lot of times a new rule or feature isn’t appreciated until it’s actually played in a game. 

Like showing the good things, those elements of a book that cloak the work in a darker glow should be specifically identified. Good reviews pick the worst element of the book that they can find, matching it up point-for-point against the good things. That doesn’t mean there needs to be as many bad things said about a product as there are good things, but it does mean that grievous errors, oversights, and omissions should be identified.

Good reviews shouldn’t go overboard pointing out bad things, though, especially when it comes to items that are extremely subjective (writing style and artwork, for example). Reading the phrase ‘gentle reader’ or seeing artwork by a particular artist shouldn’t warrant a death sentence on the product review. Reviewers should accept that some players really love a specific writing style and that others call that hated artist their great hero. 

That doesn’t mean reviews shouldn’t critique the artwork, writing style, or graphic design. It just means that reviewers should keep in mind subjective elements when handing down their condemnation.

Finally, make sure those errors and omissions that you point out are, in fact, errors and omissions. It seems obvious, but we recently had an Oathbound review on ENWorld that slammed part of Oathbound because of missing keys for maps—but the real problem was that the review hadn’t read through the adventure.

After all the writing is done, it’s time to apply some rules and assign the product some kind of grade or score. Applying fair standards that apply to all reviews is an important element, as it lets the readers understand why a product was marked down the way it was. Reading an amazing review and wondering why it only got 6 out of 10 doesn’t help anyone. Ratings should be as impartial as possible and based largely on the gems and flaws identified in the review.

To be fair, none of the publishers are going to be happy with ‘average’ reviews of their products. If all the reviews are in the top 10% of scoring, then one of two things is happening: Either the reviewer is cherry-picking only the top products to review, or the reviews are geared toward assuring more review copies.

Some of the best rating systems are those that we can immediately relate to—a d20 score, letter grades like we got in school, etc. It’s important to note on the review the reasons for lowering the score, though, as it allows readers (and publishers) to identify the specific troubles that led to the lowering of the score.

One of the issues facing publishers, retailers, and distributors these days is the ever-diminishing lifecycle of a product. It used to be that products had a very healthy sales cycle, selling for almost a full 90 days before being relegated to the backlist. That 90-day cycle is now down to around 30-45 days, meaning that retailers aren’t restocking titles as much as they used to.

What this means to a review is that timeliness is essential if you truly want to help gamers with their buying decisions. If it takes you 4-6 weeks to review a product, most everyone who wanted that product has already purchased it.

Writing a good review is an art form in itself. It requires a thorough understanding of the gaming marketplace as well as the heart of a true gamer. It requires being fair to the reader, explicitly identifying flaws and highlighting gems to truly showcase a new product. 

As a publisher, I both love and hate reviews. It’s nerve-wracking to see reviewers make mistakes when reviewing your products, and it’s heart-breaking to see something you thought was great be panned by someone. It’s a great day when I read a review from someone who actually understands the intent of the product and identifies the things I think are truly great about it. 

You can view all of the previous columns by clicking here.