[Guest]

First of all, sorry for all the questions I've been posting lately. I'm trying to come to a solid understanding of the various lens-resources out there.



I've been looking into the Tokina 11-16mm, and I noticed that Photozone's review had very different results from Lenstip's review in certain key areas.



Namely, Photozone states that the tokina has excellent border and edge performance...at all focal lengths...even at max aperture.

Whereas, Lenstip states that the border performance is fairly dismal throughout the range at max aperture. 16mm isn't even "usable" by their terms until F8. 11mm and 13.5mm become usable for them at F4.0.



They state that the unusable range for this particular test is below 30 lpmm, and here's their "border performance" chart:



(sidenote: they did give the lens high marks in center performance)



Photozone on the other hand is much more optimistic:





Why the apparent contradiction? Is there a difference in testing that accounts for this? Is it just sample variation?

More importantly, who should I believe?

[Rainer]

[quote name='curriguy' timestamp='1298577646' post='6331']

Whereas, Lenstip states that the border performance is fairly dismal throughout the range at max aperture. 16mm isn't even "usable" by their terms until F8. 11mm and 13.5mm become usable for them at F4.0.

[/quote]



You might want to reread the lenstip test ... just to cite them:



Lenstip: "It’s worth adding that its border performance should be rated very high.

Stopping down to F/4 causes the values to reach, or even go beyond

the level 30 lpmm, so it enables us to get really crisp images. "



So ... Lenstip does say: "beyond 30lpmm you get really crisp images" ...

Lenstip does not say "below 30lpmm performance is dismal"



With that, your questions should already be answered.



Just my 1ct ... Rainer

[Guest]

[quote name='Rainer' timestamp='1298579287' post='6332']

You might want to reread the lenstip test ... just to cite them:



Lenstip: "It’s worth adding that its border performance should be rated very high.

Stopping down to F/4 causes the values to reach, or even go beyond

the level 30 lpmm, so it enables us to get really crisp images. "



So ... Lenstip does say: "beyond 30lpmm you get really crisp images" ...

Lenstip does not say "below 30lpmm performance is dismal"



With that, your questions should already be answered.



Just my 1ct ... Rainer

[/quote]



I'm sorry - to be clear I was paraphrasing. They don't call it dismal. That is not a quote.



But, nevertheless, they DO consider lenses tested on the Canon 20D that perform below 30lpmm "unusable".... and, in their test, the Tokina was tested on the Canon 20D.



From their FAQ:

"The oldest tests on the Optyczne.pl were performed on an 8-megapixel sensor of a Canon EOS 20D. In its case the best “primes” reach results near 45 lpmm and when you stop them down to f/16 the resolution decreases to the level a tad above 30 lpmm. That last value is deemed to be the decency borderline."



From their review of the Canon 18-55mm (also tested on the Canon 20D):

"at the maximum aperture, the Canon just exceeds the 30 lpmm decency"



Most tellingly, from their review of the Canon 15-85 (also tested on the Canon 20D):

"Usually, to acknowledge that the lens isn’t bad we require that it reach over 30 lpmm at the maximum aperture, and 40 lpmm at the sharpest apertures."



Edit: Just for the record, I hope this isn't something controversial. I'm just trying to rationalize the differences between the two review sites. Its clear to me that lens testing is complicated stuff - and I guess subtle changes in HOW you test can sometimes lead top opposite conclusions - both of which are based in truth. I guess I'm just looking for an expert's insight into all this. Thanks again for any help

[mst]

[quote name='curriguy' timestamp='1298579882' post='6333']

IEdit: Just for the record, I hope this isn't something controversial. I'm just trying to rationalize the differences between the two review sites. Its clear to me that lens testing is complicated stuff - and I guess subtle changes in HOW you test can sometimes lead top opposite conclusions - both of which are based in truth.

[/quote]



Yes, sort of. In fact they felt the need to explain why their results are somewhat different than on other sites here:



http://www.lenstip.com/127.1-article-Why_the_Lenstip_reviews_differ_from_others_.html



There is some truth in that article, some simple fact-stating and unfortunately some useless bashing, too. And also some factual errors.



Because of the tone of that rant we (or Klaus in particular) never felt the need to go down to the same level and "officially" reply. However, since their claims we're doing something wrong slowly seemed to creep into discussions about photozone results (and that lenstip article was often quoted or linked), I felt the need to reply on dpreview recently:



http://forums.dpreview.com/forums/read.asp?forum=1030&message=37747289



However, please see this as my personal point of view, only.



-- Markus

[Yakim]

[quote name='curriguy' timestamp='1298577646' post='6331']

More importantly, who should I believe?[/quote]



The site which you trust. The site which looks to you as the one with the best testing methodology. The site that you deem as the more reliable. And besides, how can any of us tell you that? It's a personal issue.



Example: Most Jews believe that the bible was written by god but some [url="http://en.wikipedia.org/wiki/Documentary_hypothesis"]think otherwise[/url]. Who should I believe?



Oh, and have you considered the possibility of sample difference?



And BTW, it's a bit odd to post such a question here.

[allanmb]

[quote name='curriguy' timestamp='1298579882' post='6333']

I'm sorry - to be clear I was paraphrasing. They don't call it dismal. That is not a quote.



But, nevertheless, they DO consider lenses tested on the Canon 20D that perform below 30lpmm "unusable".... and, in their test, the Tokina was tested on the Canon 20D.



From their FAQ:

"The oldest tests on the Optyczne.pl were performed on an 8-megapixel sensor of a Canon EOS 20D. In its case the best “primes” reach results near 45 lpmm and when you stop them down to f/16 the resolution decreases to the level a tad above 30 lpmm. That last value is deemed to be the decency borderline."



From their review of the Canon 18-55mm (also tested on the Canon 20D):

"at the maximum aperture, the Canon just exceeds the 30 lpmm decency"



Most tellingly, from their review of the Canon 15-85 (also tested on the Canon 20D):

"Usually, to acknowledge that the lens isn’t bad we require that it reach over 30 lpmm at the maximum aperture, and 40 lpmm at the sharpest apertures."



Edit: Just for the record, I hope this isn't something controversial. I'm just trying to rationalize the differences between the two review sites. Its clear to me that lens testing is complicated stuff - and I guess subtle changes in HOW you test can sometimes lead top opposite conclusions - both of which are based in truth. I guess I'm just looking for an expert's insight into all this. Thanks again for any help

[/quote]



I think you are misinterpreting what they mean. When they say they, "require that it reach over 30 lpmm", I think they mean in the centre of the lens. One expects the centre to be far sharper than the edges but having the edges close to acceptable centre quality is surely the mark of a very good lens!



Allan

[Guest]

[quote name='allanmb' timestamp='1299593625' post='6605']

I think you are misinterpreting what they mean. When they say they, "require that it reach over 30 lpmm", I think they mean in the centre of the lens. One expects the centre to be far sharper than the edges but having the edges close to acceptable centre quality is surely the mark of a very good lens!



Allan

[/quote]





I'm not misinterpreting them at all. They slam lenses in several different reviews for being below the "usable level" at the edges. The "usable level" changes depending on which camera body they chose to do the test on, but they're always disappointed when a lens performs below it at the edges.



That being said - maybe this is just one of those lenses that benefits greatly from a little sharpening in post. a lens test with no sharpening whatsoever isn't exactly a real world evaluation. *shrug*

[Guest]

[quote name='Yakim' timestamp='1299590064' post='6604']

The site which you trust. The site which looks to you as the one with the best testing methodology. The site that you deem as the more reliable. And besides, how can any of us tell you that? It's a personal issue.



Example: Most Jews believe that the bible was written by god but some [url="http://en.wikipedia.org/wiki/Documentary_hypothesis"]think otherwise[/url]. Who should I believe?



Oh, and have you considered the possibility of sample difference?



And BTW, it's a bit odd to post such a question here.

[/quote]



it could be sample difference for sure.



As far as your religious analogy goes, that's kinda ridiculous. That's a little misguided - I only posed that question here because I wanted clarification about the differing results. I honestly think both sites are reliable.



The big lesson here though: Don't compare reviews BETWEEN sites, just focus on reviews within each site's system -- and preferably even on the same body.

[Yakim]

[quote name='curriguy' timestamp='1300060722' post='6722']

it could be sample difference for sure.[/quote]



Great. We agree about at least one possible differentiating factor.



[quote name='curriguy' timestamp='1300060722' post='6722']

As far as your religious analogy goes, that's kinda ridiculous. [/quote]



It was a deliberate provocation. All I wanted is to emphasize that your question - as you phrased it - can only be answered by you.



[quote name='curriguy' timestamp='1300060722' post='6722']

That's a little misguided - I only posed that question here because I wanted clarification about the differing results. I honestly think both sites are reliable. [/quote]



Great. We agree about another thing. <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/smile.gif' class='bbc_emoticon' alt='' />



[quote name='curriguy' timestamp='1300060722' post='6722']

The big lesson here though: Don't compare reviews BETWEEN sites, just focus on reviews within each site's system -- and preferably even on the same body.[/quote]



I follow a different route. I think that variability has a strength of it's own. I read all the tests in all the the sites which I think are reliable e.g. DPR, SLRgear, LensTip, PZ, TDP etc. I also read as many as possible user reviews in various sites. Then, when I have a significant amount of info I feel more confident in my view.