I will be buying a lens soon, and out of curiosity I read a few reviews from what I consider to be reputable sources: sources publishing detailed tests based on optical and mechanical measurements, using consistent methodology, providing sample high resolution photos, and (to my best knowledge) not manufacturer paid.
Specific details (who, what, when) are irrelevant, and would make this note unnecessarily long and complex. I read about a dozen of detailed tests of the lens I am interested in, and now I am able quote a whole spectrum of opinions. Let us briefly compare Photozone and another site (named here: "reviewer B"). As a consumer I can only reach one conclusion: the test are useless. If you are happy with a lens, keep it, if not, sell it. But... what is the real purpose of publishing lens reviews? I do not know. Below a few extracted quotes for the same lens (yes, they refer to measurable, physical properties):
[photozone]: "regarding its optical characteristics it is nothing special neither in a positive nor negative sense", vs:
[reviewer B]: "stellar(...) it produces excellent, sharp photos at all focal lengths, even wide open, and has minimal distortion, vignetting and chromatic aberration".
(both opinions are based on series of measurements and real life tests)
and one more quote:
[photozone]: "The inner-most tube wobbles slightly and, to be honest, the lens feels a bit fragile when fully extended" vs:
[reviewer B]: "the whole lens feels very solid without any wiggle from the interior barrels".
(both opinions are based on a reviewer... trying to wiggle extended lens barrels, I guess? Nothing was measured here).
No, but SLRGear (imaging resource)is quite useless. At times (or often) they come up with the most questionable "measurements".
Another questionable source is DXO lens tests.
Usually quite reliable sources: Photozone, Lenstip, lensrentals.
Pick your favorite sites - three of them. Then average the results. That's our usual recommendation.
None of the review sites owns universal wisdom. Most of the sites have faith in what they are doing (but not all of them). However, we are all testing slightly different aspects, using slightly different parameters and usually on different cameras and we all have different standards. On digital devices, the results should be pretty much the same but lenses are "analog" (in the sense that they don't behave in a discrete way). Testing lenses is also tough because you have to work accurately and every once in a while things do go wrong. There are also surprising sample variations.
Above all it is important to relax. The worst thing that can happen are frequent update cycles. Know your stuff and be happy. Most of the lenses are actually quite decent. The manufacturers can't really afford to be "bad" these days. Of course, the occasional exception to the rule exists but common sense helps.
Let us look at another lens, and opinions from the places you consider to be reliable:
[photozone]: "Mechanical Quality: 3 1/2 out of 5" vs:
[Lenstip]: "you deal here with a very solid construction which build quality would be hard to blame. In fact it is really the most solid [in this class] lens we’ve ever tested".
Yes, they both talk about the same lens. So... is it mechanically: "a half way between good/average and very good", or is it "the most solid lens in its class"?
(Lensrentals publishes reports of limited range of lenses only, usually the lenses they rent).
Here, 3½ stars is actually not "rock bottom" but pretty good. "Solid" is also a description that doesn't describe everything: a rock is solid but using it as a tool may not be very convenient.
Klaus, I am relaxed. I am also an engineer. I know the difference between analogue, discrete, manufacturing and testing methodologies, etc, etc. Would you trust a device I designed if I told you that I asked three people to test it, averaged the results, and my recommendation is: relax, it works fine. When I see two testers, one saying: "solid, no wobble", and another one: "wobbles slightly, feels a bit fragile" there is only one conclusion I can reach: it is in the subject of this thread.
Nice talking to both of you, Vlad.
All I can say is I read most of the reviews now as a hobby, but PZ's reviews helped me enormously when I was buying Pentax lenses and cut down buying blind.
Sample variations are a given as is the bane of the optical world..... de-centered lenses.
Each site has it's way, PZ's has a very clever test scene which demonstrates "in one hit" the rendering of OOF fore and backgrounds as well as highlight rendering and bokeh balls. Also PZ does a separate review for FF and APSc bodies, Lenstip doesn't.
I couldn't buy a lens these days without reading some sort of review, especially if there is good choice of alternatives.
Excellent reply from you Klaus!
Reviews are like bikinis they show so many things but hide the essential.
You simply can't rely on just reviews for choosing or buying your lens, it's not only about what suits your needs best but also how important the differences are.
Let's suppose you are considering a standard lens for 7D MKII compare the reviews of Canon 15-85 and 18-135 STM here, when you check sharpness distortions , vignetting, etc you do have numbers, but how much are they useful for you ? can they help you make a choice ?
Consider now 15-85 vs 17-85, despite the focal length advantage you can see the catastrophic CAs and distortions on 15-85 and hence the review here is more helpful.
You can say it is because 15-85 and 17-85 are more comparable that's why the review helped, but can reviews help you decide between getting Canon 16-35f4 IS 16-35f2.8 MKII and 16-35f2.8 MKIII with extra cost ?
Quote:Which lens ?
Since the OP does not want to disclose this...
The Panasonic 14-14mm lens
http://www.lenstip.com/339.3-Lens_review...ation.html
So PZ says 3.5 stars for a kitlens. Very good. And Lenstip says: great for its class (kitlens).
I don't see how those two are in conflict, at all.
OP: context matters. If you really want to discuss things, you do have to provide enough context.