I am trying to reconcile the differences in MTF50 values between PZ and dpreview.com site say for the Canon 70-200mm f/2.8L IS II on an APS-C format body. To what extent does the PZ results reflect postprocessing sharpening via Imatest?
genotypewriter
Unregistered
[quote name='clkirksey' timestamp='1299682020' post='6636']
I am trying to reconcile the differences in MTF50 values between PZ and dpreview.com site say for the Canon 70-200mm f/2.8L IS II on an APS-C format body. To what extent does the PZ results reflect postprocessing sharpening via Imatest?
[/quote]
I wouldn't go to do a cross-site comparison of reviews. They probably don't even use the same RAW converters, sharpening aside. Take a look at the effects of different RAW converters below:
http://www.rawtherapee.com/RAW_Compare/
Also, a lot of lens review sites are very cryptic about their methods (for example, some places say they address field curvatures and focus shifts but they never really explain how their test methodology achieves this). Dpreview is known for having good database of technical specs but they never really figured out how to do technical experiments... which is not a serious problem for them because their audience is mainly composed of soccermoms and soccerdads looking for a final rating such as "highly recommended", "recommended", etc. without wanting to understand the different facets of these complex things.
Personally speaking, I only look at the center performance when I'm looking at MTF results, unless they show the actual images and show the coming-in to and going-out of focus, so I can judge for myself.
GTW
03-10-2011, 06:55 AM
(This post was last modified: 03-10-2011, 07:33 AM by Klaus.)
[quote name='clkirksey' timestamp='1299682020' post='6636']
I am trying to reconcile the differences in MTF50 values between PZ and dpreview.com site say for the Canon 70-200mm f/2.8L IS II on an APS-C format body. To what extent does the PZ results reflect postprocessing sharpening via Imatest?
[/quote]
We are using a minor degree of post-sharpening simply to compensate the effect of the AA filter.
The AA filter as well as USM-type sharpening are both near linear in terms of sharpening.
So let's say that we've got a signal X. Assuming the AA filter dampens the signal by say 20% we try to use USM to equalize this to some degree. +25% would be perfect here.
e.g.
signal = 1 -> 1 * 0.8 (AA) = 0.8 -> 0.8 * 1.25 (USM) = 1
It is, of course, not possible to find the perfect compensation values which is why we're generally sticking relatively close to the default USM settings in the RAW converter. After all that's exactly what most users do.
If we didn't do some USM tuning the differences between the brands would be quite pronounced at times resulting in the illusion that the lenses of brand A are generally better than the lenses from brand B which is, of course, nonsense - they're all very close in reality. This issue is already a problem regarding the different megapixel base which is why I would actually prefer to go for an abstract scale altogether. Markus has kept me from doing so so far. :-)
Just to mention - the quality of the demosaicing engine (of the RAW converter) is also affecting the resolution. You may argue that you could use the same engine for all brands (e.g. dcraw or so) but these engines are never static. They are getting better (thus different) over time.. e.g. ACR has changed its engine at least 3 times as far as I can tell. So some sites may argue that they're sticking to the same engine but if they've a naive approach they'll get very different results after certain updates (Adobe did one change even in a minor version update for instance).
genotypewriter
Unregistered
[quote name='Klaus' timestamp='1299740143' post='6645']
So let's say that we've got a signal X. Assuming the AA filter dampens the signal by say 20% we try to use USM to equalize this to some degree. +25% would be perfect here.
e.g.
signal = 1 -> 1 * 0.8 (AA) = 0.8 -> 0.8 * 1.25 (USM) = 1
It is, of course, not possible to find the perfect compensation values which is why we're generally sticking relatively close to the default USM settings in the RAW converter. After all that's exactly what most users do.
[/quote]
Klaus, how do you determine the USM radius?
[quote name='Klaus' timestamp='1299740143' post='6645']
Just to mention - the quality of the demosaicing engine (of the RAW converter) is also affecting the resolution. You may argue that you could use the same engine for all brands (e.g. dcraw or so) but these engines are never static. They are getting better (thus different) over time.. e.g. ACR has changed its engine at least 3 times as far as I can tell. So some sites may argue that they're sticking to the same engine but if they've a naive approach they'll get very different results after certain updates (Adobe did one change even in a minor version update for instance).
[/quote]
Why not stick to a single version of something primitive (that doesn't apply brand-specific profiles) like dcraw and then convert all files to DNG v1 using the latest Adobe DNG converter?
GTW
[quote name='genotypewriter' timestamp='1299800615' post='6660']
Why not stick to a single version of something primitive (that doesn't apply brand-specific profiles) like dcraw and then convert all files to DNG v1 using the latest Adobe DNG converter?
[/quote]
The answer is already in the paragraph you quoted. RAW engines are never static. This is especially true for Adobe.
-- Markus
Editor
opticallimits.com
[quote name='genotypewriter' timestamp='1299800615' post='6660']
Klaus, how do you determine the USM radius?
GTW
[/quote]
There's no hard rule. Upon a start of a new test system we're analyzing the USM effect (with different parameters) on the center portion of an image taken at the sweet spot of a lens. Usually we stick to a smaller than average USM radius in order to avoid an unreasonable boost of the usually much weaker border/corner region.
Technically we're probably using too weak USMs. There're many lenses which are outperforming an AA-less sensor in the image center after all. However, it is, of course, reasonabe to stay below the max. resolution of a sensor after USM and I do actually refuse to sharpen aggressively (remember then Pentax K-5 discussions about this). While imatest can look a bit beyond Nyquist due to being able of making assumptions about the test target (an assumption-less Nyquist can't look beyond the barrier, of course) I don't really think that this is overly reliable anymore.
genotypewriter
Unregistered
[quote name='mst' timestamp='1299827188' post='6667']
The answer is already in the paragraph you quoted. RAW engines are never static. This is especially true for Adobe.
[/Quote]
that's why I suggested the use of a single version of a primitive raw converter and handling of new raw files using the latest dng converter but converting only to the oldest dng version.
[quote name='Klaus' timestamp='1299828632' post='6670']
There's no hard rule. Upon a start of a new test system we're analyzing the USM effect (with different parameters) on the center portion of an image taken at the sweet spot of a lens. Usually we stick to a smaller than average USM radius in order to avoid an unreasonable boost of the usually much weaker border/corner region.
Technically we're probably using too weak USMs. There're many lenses which are outperforming an AA-less sensor in the image center after all. However, it is, of course, reasonabe to stay below the max. resolution of a sensor after USM and I do actually refuse to sharpen aggressively (remember then Pentax K-5 discussions about this). While imatest can look a bit beyond Nyquist due to being able of making assumptions about the test target (an assumption-less Nyquist can't look beyond the barrier, of course) I don't really think that this is overly reliable anymore.
[/quote]
Thanks for your response.
GTW
[quote name='genotypewriter' timestamp='1299841507' post='6677']
that's why I suggested the use of a single version of a primitive raw converter and handling of new raw files using the latest dng converter but converting only to the oldest dng version.
[/quote]
That doesn't change anything, because both dcraw and the DNG converter evolve over time.
-- Markus
Editor
opticallimits.com
[quote name='mst' timestamp='1299841615' post='6678']
That doesn't change anything, because both dcraw and the DNG converter evolve over time.
-- Markus
[/quote]
Just to add - please remember that we do always mention that the test results are not cross-(test-)system comparable.
There're simply parameters which introduce variations into the final results:
- the style and layout of the test target
- AA filter
- sensor micro-lenses
- layout of the photodiodes
- bayer interpolation (demosaicing)
- AA filter-compensation sharpening
- quality of the analysis tool
As far as we are concerned a cross-system comparison can only be coarse at best and it's technically not valid.
Within a test system these individual parameters don't really matter - after all this is just a chain of multiplication factors - and whether you use 0% or 1000% for the USM or demosaicing engine A or B is, in the end, absolutely meaningless because it only affects the scale but not the quality of the numbers.
e.g.
Test parameter set A: center = 9, border = 5 (based on a presentation scale from 4 to 10)
Test parameter set B: center = 4.5, border = 2.5 (based on a presentation scale from 2 to 5)
The qualitative result of both test setups is identical. This is just an example for a linear factor but the same would be applicable for an exponential factor - as long as you remain within a single test environment it just does not matter on a qualitative level. The specific numbers are always a matter of interpretation - whether a dollar is little money or much money is dependent on your point of view.
genotypewriter
Unregistered
[quote name='mst' timestamp='1299841615' post='6678']
That doesn't change anything, because both dcraw and the DNG converter evolve over time.
-- Markus
[/quote]
Dcraw won't evolve by itself if you stick to a single version like I said and not install the new versions.
You also will only need to upgrade the dng converter when you're upgrading the camera used for the test, which is very infrequently. Either way, the quality of the raw files will be determined by the raw converter, draw, the dng converter won't change the quality.
GTW
