Things Reviewers Always Get Wrong
I review gear. I also make mistakes. Some are self-inflicted. Some are just flukes. Others are endemic to the form. Let’s look at those last ones, because they occur most often and impact all of us.
Preferences as Standards
Go watch some Metal Complex reviews. He likes big, heavy folders. If you have read some of my reviews, you will see we have very different tastes. I like to think that my tastes are the right tastes, but everyone thinks this. I am sure that Metal thinks his preferences are right. The reality is, as a reviewer, you make your position clear by emphasizing the things you like and over time this makes you conflate your preferences with actual objective standards. And there is no real way out of this trap. The more you drill down, the harder it is to separate standards and preferences. The best way to fight against this is to watch multiple good reviewers, especially those that have preferences different than your own. As a reviewer, I try to do this all the time. I try to push against my own preferences in the hope that I can break the preferences=standards problem.
Steel Opinions
We need to be honest with ourselves—reviewers are TERRIBLE at reviewing steel. There are three reasons: lack of expertise, small sample sizes (which is a problem that will recur with all knife reviews), and conflating blade geometry and heat treat with steel performance.
The vast majority of reviewers do not know enough about steel to say one way or the other whether a heat treat is good or bad. We tend to homogenize our opinions over time (“M390 is good because it was good on this knife” slowly becomes “M390 is good”).
In addition to lacking the critical expertise, we also don’t have a relevant sample size. Steel makers produce metric tons of steel. Literally. And we see a very slim slice of that entire yield. The idea that we could get some meaningful handle on a steel over an entire batch with one knife is like digging in your backyard, not finding gold and declaring that gold doesn’t exist.
But the biggest flaw reviewers have with steel, especially newer reviewers is that they conflate geometry and heat treat for blade steel. The TRM knives have proven to me, over and over again, that geometry trumps everything. A good, thin, slicey knife like the Canal Street Cutlery Boys knife or more currently the Knafs Co Lander can get a lot done with comparatively subpar steels when the grind is thin. A good steel on a thin grind is even more remarkable, but not THAT much more remarkable. For those of us that have been banging around a while, this last problem is solved by experienced, but for newer reviewers that don’t have a lot of experience with knives before becoming reviewers (thanks YouTube!) this is a critical issue.
Fit and Finish
The other small sample size problem for reviewers is fit and finish. I have owned, in one configuration or another, 10 different Dragonfly 2s. The DF2 is my favorite production knife of all time and between the VG-10, ZDP-189, and M390 (with various handle scales), I crossed the double digits a few years ago. But even that is a drop in the bucket compared to the number of knives made. So if one has a bit of a loose pivot or an offset blade, is it really fair to ding the entire production run? Probably not. Often times the IKC becomes obsessed with a problem on a knife and others chime in with agreeing voices, not because there is some genuine flaw, but because the Internet’s format LOVES a pig pile. Remember the ZT Elmax “burnt edge” problem? Something tells me that a high percentage of the complainers are both not metallurgists AND were influenced by the opinions of others.
There are a few things that I think are not worth objecting to on the fit and finish front. If a blade has a loose pivot that’s not great, but if you can tighten it and it stays tight, then I don’t think that is a fair criticism in terms of fit and finish. Similarly if a knife is not scalpel sharp out of the box you shouldn’t complain all that much. In the woodworking world, people just assume that their chisels and planes need a little work. Why it would be different for knives has always been a mystery to me.
But there are other things that are fair to complain about even with a sample size of one. If the lock doesn’t work, that’s a fair criticism. It might be 1 out of a million, but a failure of the many safety component on a knife is not acceptable even with that small a sample size. We don’t let airplane engines or cars skate by when they are a 1 in a million crash because the outcomes are so bad. So too with knives. A failing lock bar is a failing knife. Similarly, if EVERYTHING is messed up, I recently reviewed a knife where literally every screw was messed up somehow, then I think it is fair to criticize fit and finish. Overall, though, I have become much more forgiving of small things and only hammer on fit and finish when there is something major, something that cannot be fixed with a small amount of effort, or when EVERYTHING is messed up.
Discussing Value
Value is such a hard thing. For a long time, I refused to evaluate value as I think that fundamentally, value is about finances and priorities and not the product itself. Think about this—if you were a billionaire (assuming, dear reader, that you aren’t) you have no use for reviews, especially for something as cheap as a knife. You could simply buy EVERYTHING, try out everything and pitch what you don’t like. Because value is ultimately tied into price, what counts as a fair value is largely dependent on how much money you have to spend. The problem is compounded when you are a reviewer and get sent stuff for free. What does value mean when the thing costs you nothing? That’s one reason why I have largely gone to a pay only model for reviews. I still get sent stuff, but most of my reviews recently have been for items I purchase. When that is not the case, I make sure to disclose that fact.
But value is REALLY hard to evaluate. If I had $100 to spend on a knife, my conception of value would change compared to if I had $30 to spend. The Gerber Paraframe, which is about as close to a replacement level knife as you will ever find, starts looking compelling at $30, while it would never come into the conversation at $100 (which, for the record, ends with the Civivi Lumi). Because it is so tied to personal factors, value is virtually impossible to evaluate. I changed my view of value, but not tying it to what I can afford but instead how an item compares to similarly priced items. Even with this hacked approach, it is still hard to evaluate value.
The Lure of the New
Oh let’s dispense with all of the “neutral objective evaluator” baloney. The release of something new and interesting will grab reviewers because, at the end of the day, reviewers are enthusiasts. The lure of the new means we also ignore old stuff or modest upgrades to old stuff. I got a Magnacut knife because, well, Magnacut. When given the choice of steels, I will almost always go with the newest steel. If something sports a new emitter or a steel I have used to before, I will be more likely to buy it. New stuff is cool. Cool stuff appeals to enthusiasts. And reviewers are enthusiasts. Hence the new turning the head of every reviewer out there. I think that the best antidote for the lure of the new, is having a cache of all time greats on hand to compare with the new stuff. Stacking up to the Sebenza is a sobering process for new stuff.
Reviewers aren’t perfect. But if you know what limitations we function under, maybe it will make our reviews better. In the end, though, the true soluction to all of these problems is simple—use multiple reviewers.
Amazon Links