In reply to Alan James - Rockfax:
> The problem we have here is that any review that says kit is good appears to be immediately judged by some people as being a 'puff piece'.
No doubt, but the fact that some people see puff pieces everywhere does not mean that you don't have puff pieces.
I think that, given that a statistically significant review is far beyond the technical and financial abilities of most reviewing entities, particular editorial care ought to be exercised about the nature of claims made, restricting them to things that have genuinely been confirmed by the reviewers, who then explain how they have confirmed their claims. Without such editorial oversight, the puff piece perception is never going to go away, and shouldn't.
To be more specific, one way to avoid the puff piece perception is to help reviewers understand the difference between actual experience and sheer speculation, especially when that sheer speculation parrots the manufacturer's advertising claims. The present DMM review has a lot of such speculation. For example,
> Straight away, these 'TripleGrip' lobes look like they'll be more effective in gripping the rock - and my experience to date bears this out.
Ok, this is a good start, but what "experience" bore this out? Weighting the gear? Falls on the gear? No mention of the "testing protocol." If either of these, how is it known that other gear wouldn't have been just as good?
> Extra metal has just been added on the part of the lobe intended to grip in an optimum placement, boosting the strength and the holding power of each unit...
How does the reviewer (or anyone else) know that the "extra metal" boosts strength and holding power? If the extra metal does boost either of these quantities, is the boost any any sense significant?
> The grip that comes with the new TripleGrip lobes actually feels better to me, and combining that with a constant camming angle of 13.75 degrees to maximise holding power really gives you confidence.
Glad to know it "feels better," but is it in fact any better, and better than what? And what exactly is the point of mentioning the precise cam angle?
> I suspect that these cams will probably have superior holding power in winter time too, when placements are compromised by powder snow and verglas. Good news for trad all rounders I hope. Time will tell.
The reviewer "suspects?" We are now in a realm that even the reviewer hasn't actually experienced. In this regard, it is hard not to notice that unsupported positive speculation is fair game, but there seems to be no place for unsupported negative speculation. For example, all that grooving creates a surface made of many individual raised portions, which might be subject to shear failures at lower loads than a more solid surface. Could this be a problem?
The obvious response to these observations is that neither the reviewer nor UKC has anything close to the ability to genuinely resolve any of the issues mentioned. Quite right, but then statements that appear to resolve such issues shouldn't be in the review. But it is also true that there are cases in which the reviewer might have tried to do something. For example, how about bounce-testing some actual placements, using the new cams and some other models?
One of the many problems of reviewing gear donated by manufacturers (a fact that should always be explicitly specified in the review, by the way), is that no one is donating equivalent gear from other sources for comparison purposes. So a reviewer might well be comparing the latest innovation to stuff he or she owns that is several years old---that is if they are doing any comparing at all. A typical current example is the Polartech Alpha pieces. Most reviews compare them to the less breathable types of insulation, which is certainly interesting, but there are a lot fewer reviews comparing different brands all using the newer breathable insulation materials.
A pervasive problem of gear reviews is that the reviewer takes the stuff out for a while and climbs with it. This might mean that the gear is never subjected to any kind of extremes, even though, as with safety gear like cams and belay devices, behavior in extreme situations is of critical interest. The reviews end up praising performance in very mild conditions without ever addressing what happens in much more severe cases. This isn't something the reviewing entities can really fix, but it certainly means that most reviews need to be taken with quite a few grains of salt.