Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Lens Reviews Test Setup
#1
Hi everybody!

I'm a new member despite I'm used to attend Photozone to read the great reviews on lenses for many years.



Because I've complaints with one of my lenses sharpness, I've started going more into the issues of lens testing by means of MTF tools. Despite results among test performed on different systems are not easily comparable, I think it's possible to find a correlation between results obtained with different methodologies.



In order for me to try to find out this correlation, it would be desirable to know the ways Photozone did the set up for lens testing in terms of:



- Used Test Charts (ISO 12233, USAF 1951, Custom, etc.)

- Lighting conditions

- Shooting distance

- MTF Software tool (It seems it's IMATEST)

- JPG vs RAW processing

- etc.



If anyone knows something and wants to share its knowledge, its help would really be appreciated!
#2
[quote name='emarm' timestamp='1317387434' post='11997']

..... If anyone knows something and wants to share its knowledge, its help would really be appreciated!

[/quote]

....can't answer your question about testing ... but i've learnt a thing or two about buying lenses over the years, and recently i bought a new lens - put the shop demo lens on the camera and it tested beautifully (looking at the files on my nice convenient laptop) - then - "very good sir, we have a lovely new one of these fresh in it's box, please take this one" - hmmm, well alright then .... if i can also test it first - so, put nice new lens on the camera, took the same shots with it outside the shop and sadly it looked like absolute s*** - so i said, you have a sale if i can buy the demo lens but no sale if i have to take the nice new lens in the box ... and the nice shop keeper sold me their demo lens (nice shop keeper) - and it's a beautiful lens - buyer beware
#3
[quote name='anyscreenamewilldo' timestamp='1317388712' post='11999']

....can't answer your question about testing ... but i've learnt a thing or two about buying lenses over the years, and recently i bought a new lens - put the shop demo lens on the camera and it tested beautifully (looking at the files on my nice convenient laptop) - then - "very good sir, we have a lovely new one of these fresh in it's box, please take this one" - hmmm, well alright then .... if i can also test it first - so, put nice new lens on the camera, took the same shots with it outside the shop and sadly it looked like absolute s*** - so i said, you have a sale if i can buy the demo lens but no sale if i have to take the nice new lens in the box ... and the nice shop keeper sold me their demo lens (nice shop keeper) - and it's a beautiful lens - buyer beware

[/quote]



Yo, this is the point.



Don't ever buy a lens from a shop without a proper return policy .. which is why I will never go for a "real" shop anyway.
#4
Such a setup is an overkill just for QC.



Start here:

[url="http://forum.photozone.de/index.php?/topic/830-do-it-yourself-centering-sanity-check"]http://forum.photozone.de/index.php?/topic/830-do-it-yourself-centering-sanity-check[/url]



Then take images at max. aperture and check the corners for symmetry.
#5
[quote name='anyscreenamewilldo' timestamp='1317388712' post='11999']

....can't answer your question about testing ... but i've learnt a thing or two about buying lenses over the years, and recently i bought a new lens - put the shop demo lens on the camera and it tested beautifully (looking at the files on my nice convenient laptop) - then - "very good sir, we have a lovely new one of these fresh in it's box, please take this one" - hmmm, well alright then .... if i can also test it first - so, put nice new lens on the camera, took the same shots with it outside the shop and sadly it looked like absolute s*** - so i said, you have a sale if i can buy the demo lens but no sale if i have to take the nice new lens in the box ... and the nice shop keeper sold me their demo lens (nice shop keeper) - and it's a beautiful lens - buyer beware

[/quote]

I like this, asw! <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/smile.gif' class='bbc_emoticon' alt='Smile' />

Thanks

Ian
#6
[quote name='anyscreenamewilldo' timestamp='1317388712' post='11999']

....can't answer your question about testing ... but i've learnt a thing or two about buying lenses over the years, and recently i bought a new lens - put the shop demo lens on the camera and it tested beautifully (looking at the files on my nice convenient laptop) - then - "very good sir, we have a lovely new one of these fresh in it's box, please take this one" - hmmm, well alright then .... if i can also test it first - so, put nice new lens on the camera, took the same shots with it outside the shop and sadly it looked like absolute s*** - so i said, you have a sale if i can buy the demo lens but no sale if i have to take the nice new lens in the box ... and the nice shop keeper sold me their demo lens (nice shop keeper) - and it's a beautiful lens - buyer beware

[/quote]





Many years ago, when a student, I worked in a camera shop in vacations.



Yes, it's just a 'business' like any other. Any empty camera boxes were stored, then 'mint' secondhand cameras were sold as new. That particular practice would be more difficult nowadays, but it just illustrates that what could happen probably does.



Actually, I suppose that's little worse than reselling the 'dog' lenses returned nowadays. They're not refurbished, they're not thrown away, either.
#7
Wow, I've obtained a 100% Off Topic Replies (not taking into account Klaus reply on the sanity check)!!!



Anyway, it seems nobody knows how testing is performed...
#8
[quote name='emarm' timestamp='1317550407' post='12019']

Wow, I've obtained a 100% Off Topic Replies (not taking into account Klaus reply on the sanity check)!!!



Anyway, it seems nobody knows how testing is performed...

[/quote]



see lens test FAQ
#9
[quote name='emarm' timestamp='1317550407' post='12019']

Wow, I've obtained a 100% Off Topic Replies (not taking into account Klaus reply on the sanity check)!!!

[/quote]



I don't see the replies as 100% OT.



[quote name='emarm' timestamp='1317550407' post='12019']

Anyway, it seems nobody knows how testing is performed...

[/quote]



There are two more valuable replies by Klaus. 1st: testing lenses as we do is overkill for your purposes. And even if you tried to you wouldn't be able to exactly match our results until we disclosed all details and parameters of our workflow (which for obvious reasons we won't do).



2nd: if you still insist to use a similar testing approach as we do, there's the lens test FAQ which explains some details of the procedure. To summarize: The chart we use is a slanted edge chart with quadrants only in the places we measure (not an ISO or USAF chart). Lighting is not static but depends on the focal length used. You need to make sure the chart is evenly lit and shows no reflections. We use daylight steady lights (no flashes) combined with reflectors to fill from additional angles. The measurement requires careful manual focus bracketing (don't rely on AF for lens testing) and a fixed software workflow (to get comparable results). Our MTF measurements are done with RAW files converted to JPGs which are then fed into Imatest.



Your biggest challenge will not be to get to know the workflow and software (especially Imatest), but to align the camera to the test chart. You will also need a very solid tripod and tripod head. Nonetheless use MLU and let the camera rest for a while after the mirror moved up. Weak camera support will render your MTF attempts useless right from the start.



-- Markus
Editor
opticallimits.com

#10
[quote name='mst' timestamp='1317585997' post='12027']

I don't see the replies as 100% OT.



There are two more valuable replies by Klaus.

[/quote]



Yes, you're right: I've specified in the "(not taking into account Klaus reply on the sanity check)" that Klaus replies were near to what I was looking for; two usefull contributes, in other words. Sorry if I wasn't clear enough...





[quote name='mst' timestamp='1317585997' post='12027']

2nd: if you still insist to use a similar testing approach as we do...

[/quote]



Great! Thank you for all the information you provided!

I know the procedure is really challenging, mailnly due to chart-camera alignement problems, but I still need to assess the sharpening focus issues of my lens from an objective standpoint. Subjective opinions lead to a "dispersion" of results that has no usefulness in judging acceptable limits for a lens behaviour.

I use Quick MTF tool mainly because of the price of the light version, more compatible with my target of private user. Furthermore, I think that Imatest and Quick MTF rely on the same public domain SFR routines...



The reason for my interest in your testing procedures was mainly due to the preliminary results I've obtained from my tests; the big difference with yours were asking me if I was missing the right approach.



So, thanks for you support and what's more for the great reviews you share with us!



Emanuele.
  


Forum Jump:


Users browsing this thread:
1 Guest(s)