ilkka_nissila
Unregistered
[quote name='Klaus' timestamp='1293434625' post='5220']
Even if we followed a linear approach - the results wouldn't be pure anyway. The micro-lenses as well as the characteristic of the photo-diodes of the sensor "pollute" the results, of course. Ultra-wides don't vignette quite as bad as the data suggests but who cares how a lens behaves without a camera ...
[/quote]
It's not just about linearity - what you're presenting is an unknown nonlinear function of something that would be a valuable and meaningful quantity. Of course the sensor optics affect the results since any measurement is based on recorded data from the sensor, and that's how it should be.
Problems with your approach:
1) lack of cross-comparability across brands and across review sites
2) dependence of the results on exposure
3) no possibility to estimate the significance of the vignetting in a quantitative way. If you reported the vignetting on a calibrated EV scale, I could for example deduce how much the noise increases when vignetting is corrected in software. Now I have no such possibility. Basically 1 EV vignetting means that to get corner pixels to have the same luminosity that the center has, one needs to double the exposure. Since this is done in software, all artifacts, including noise, compression artifacts, CA etc. increase accordingly. Because your scale is an unknown nonlinear function of the true EV scale, there is no way of figuring the magnitude of these increases in artifacts without considerable additional information about the camera used, the settings used, and where the target's center was placed at on the luminosity scale.
If you measured the true EV value of the vignetting (preferably as a function of distance from the center of the frame) one could calculate everything about the effects of the vignetting on practical results. These numbers that you give mean nothing that could be easily interpreted or compared with.
In any quantitative image quality analysis, the crucial aspect is to present quantities that are relative to subject contrast. For example, if we have a 1 stop lighting contrast on a portrait, we could then look at the table that says 1 EV vignetting and say that ok, the vignetting corresponds to a lighting difference of 1 stop. If you use a higher contrast curve (everybody who cares about their images will alter the tone curve in post-processing on an individual image basis to optimize the effect and visual appearance of the image) it still remains 1 stop, whereas your numbers would be altered. Alter the exposure, again your numbers vary but real-world results stay at the same 1 stop.
I would highly recommend that you consult someone with a degree in optics, image processing, or physics before designing a laboursome experiment that turns out meaningless over time. It is a small amendment that could save a lot of work and help reduce the ghastly variability of results across sites and testers that exists today. Provided that you have maintained constant lighting in your setup and have done the experiments with exactly the same settings over the years, you can still fix this by calibrating your scale for each camera by using a gray patch series with known densities (that you can measure) and then fixing the reports. I understand that complete cross-comparability in results across sites and brands is not possible when it comes to some quantities but for vignetting there is no reason why it can't be done correctly.
Sorry but results from different cameras are always non-cross-comparable - neither here nor elsewhere. The difference is that we tell so whereas others don't publish this aspect or even offer invalid comparison tools.
The results are, of course, not comparable between different websites because the testing procedure is always different.
The vignetting data is obtained at the same gray point peak so the exposure variation is less than 1/6th of an f-stop. The vignetting analysis reduces the error margin even further. The primary aspect - the lens ranking - remains intact anyway. As such I fail to see your point.
[quote name='ilkka_nissila' timestamp='1293455557' post='5228']
It's not just about linearity - what you're presenting is an unknown nonlinear function of something that would be a valuable and meaningful quantity. Of course the sensor optics affect the results since any measurement is based on recorded data from the sensor, and that's how it should be.
Problems with your approach:
1) lack of cross-comparability across brands and across review sites
2) dependence of the results on exposure
3) no possibility to estimate the significance of the vignetting in a quantitative way. If you reported the vignetting on a calibrated EV scale, I could for example deduce how much the noise increases when vignetting is corrected in software. Now I have no such possibility. Basically 1 EV vignetting means that to get corner pixels to have the same luminosity that the center has, one needs to double the exposure. Since this is done in software, all artifacts, including noise, compression artifacts, CA etc. increase accordingly. Because your scale is an unknown nonlinear function of the true EV scale, there is no way of figuring the magnitude of these increases in artifacts without considerable additional information about the camera used, the settings used, and where the target's center was placed at on the luminosity scale.
If you measured the true EV value of the vignetting (preferably as a function of distance from the center of the frame) one could calculate everything about the effects of the vignetting on practical results. These numbers that you give mean nothing that could be easily interpreted or compared with.
In any quantitative image quality analysis, the crucial aspect is to present quantities that are relative to subject contrast. For example, if we have a 1 stop lighting contrast on a portrait, we could then look at the table that says 1 EV vignetting and say that ok, the vignetting corresponds to a lighting difference of 1 stop. If you use a higher contrast curve (everybody who cares about their images will alter the tone curve in post-processing on an individual image basis to optimize the effect and visual appearance of the image) it still remains 1 stop, whereas your numbers would be altered. Alter the exposure, again your numbers vary but real-world results stay at the same 1 stop.
I would highly recommend that you consult someone with a degree in optics, image processing, or physics before designing a laboursome experiment that turns out meaningless over time. It is a small amendment that could save a lot of work and help reduce the ghastly variability of results across sites and testers that exists today. Provided that you have maintained constant lighting in your setup and have done the experiments with exactly the same settings over the years, you can still fix this by calibrating your scale for each camera by using a gray patch series with known densities (that you can measure) and then fixing the reports. I understand that complete cross-comparability in results across sites and brands is not possible when it comes to some quantities but for vignetting there is no reason why it can't be done correctly.
[/quote]
finaldesignrb
Unregistered
In contrast (ha ha), I could claim that Klaus's current vignetting analysis, the EV numbers, is too much information. All I really want to know is a few words on whether or not, at a given aperture, most of my photos are going to have noticeable corner darkening. On the camera being tested. The reader would do well to look at a lot of different 'zone reports to get a feel for what is reasonably achievable.
The wisdom to extrapolate from the tested camera to any other particular camera, and a zillion other variables has, to come from the reader thinking about their particular work and hardware. Does the reader's camera indeed have microlenses with tuned orientation? Is the scene going to have overall contrast boosted? Accurately predicting the final significance would be so dependent on image content, post-processing and/or contrast settings. For example, every one of my own photos gets its contrast boosted an unknowable-beforehand amount by my Picture Window Pro's command to stretch the luminances to occupy the "full range" of possible brightnesses.
The Photozone vignetting reports, which always have a few words of discussion/perspective on the practical significance of the charts, strike a pretty good balance between thoroughness and conciseness. Am always happy to hear whether or not the reviewers think that compensation is happening in camera firmware.
Well, it is interesting when some other reviewers give a hint as to whether or not the transition to darkening is sudden or gradual, perhaps with a sample photo of a gray wall. But even that is only important when the absolute amount of corner darkening is high. Let us note that if you start down that road, you'd have to show a different chart/photo for each aperture (and perhaps for "each" zoom focal length). The benefit of taking up time and space on the topic is arguable.
In general, am not sure what other aspect of testing I would like the Photozone to cut corners on (?), in order to tweak a part of the work that is already nice.
[quote name='RussellB' timestamp='1293466402' post='5233']
In contrast (ha ha), I could claim that Klaus's current vignetting analysis, the EV numbers, is too much information. All I really want to know is a few words on whether or not, at a given aperture, most of my photos are going to have noticeable corner darkening. On the camera being tested. The reader would do well to look at a lot of different 'zone reports to get a feel for what is reasonably achievable.
The wisdom to extrapolate from the tested camera to any other particular camera, and a zillion other variables has, to come from the reader thinking about their particular work and hardware. Does the reader's camera indeed have microlenses with tuned orientation? Is the scene going to have overall contrast boosted? Accurately predicting the final significance would be so dependent on image content, post-processing and/or contrast settings. For example, every one of my own photos gets its contrast boosted an unknowable-beforehand amount by my Picture Window Pro's command to stretch the luminances to occupy the "full range" of possible brightnesses.
The Photozone vignetting reports, which always have a few words of discussion/perspective on the practical significance of the charts, strike a pretty good balance between thoroughness and conciseness. Am always happy to hear whether or not the reviewers think that compensation is happening in camera firmware.
Well, it is interesting when some other reviewers give a hint as to whether or not the transition to darkening is sudden or gradual, perhaps with a sample photo of a gray wall. But even that is only important when the absolute amount of corner darkening is high. Let us note that if you start down that road, you'd have to show a different chart/photo for each aperture (and perhaps for "each" zoom focal length). The benefit of taking up time and space on the topic is arguable.
In general, am not sure what other aspect of testing I would like the Photozone to cut corners on (?), in order to tweak a part of the work that is already nice.
[/quote]
I have a very simplistic opinion: I actually like vignetting. 80% of my photos have additional vignetting added to the natural vignetting of the lens. The rest is shot at low iso, where vignetting can be corrected in post without quality concerns.
ilkka_nissila
Unregistered
[quote name='Klaus' timestamp='1293457937' post='5230']
Sorry but results from different cameras are always non-cross-comparable - neither here nor elsewhere. The difference is that we tell so whereas others don't publish this aspect or even offer invalid comparison tools.
[/quote]
Not true.
If the vignetting is quantified by finding the point in the gray scale (of known density, positioned in the center of the image) which matches the luminosity of the corner of the image studied, then the resulting EV values are independent of the post-processing used in the camera and remain unaffected by any uniformly applied post-processing. Increase contrast? The vignetting gets stronger in visual appearance but with it increases the contrast of the subject. These always go hand in hand. Zeiss reports vignetting as a function of distance from the center, which is again a little more work but since how sharply the vignetting appears in the corners varies greatly, so one value is insufficient.
This procedure is a little harder to do, but it produces values that are immediately meaningful and comparable across sites and camera systems, when the experiment is correctly carried out.
Quote:The vignetting data is obtained at the same gray point peak so the exposure variation is less than 1/6th of an f-stop.
What gray point is that? Which settings (i.e. tone curve) do you use in the camera? What is 1 (pseudo-)EV in terms of values in the file? (EV refers to the physical domain not the digital values). The pseudo-EV values are difficult to interprete if this mapping between real (i.e. subject) EV and the values given in your reports is not documented. Also, how large a patch of the image (as % of the image area) is used to calculate the vignetting value?
12-29-2010, 07:07 AM
(This post was last modified: 12-29-2010, 07:09 AM by Klaus.)
As mentioned there're always dependencies from the micro-lenses and probably also the AA-filter.
We are using the default camera settings for the vignetting tests - this is what we consider as normal user setting here. Technically you may debate the non-linear "curve" but 99.99% of the users don't shoot this way. Consequently this is reflected in our tests. You may, of course, criticize that we don't use the same S-curve across the board. Till we started with the D3x tests this wasn't really an issue because the S-curve was fairly similar across the board. However, again you are only talking about quantitative aspects, not qualitative ones - technically it is even possible to apply a mathematical transformation to normalize the data. We won't do so though. As long as the MTF data is not cross comparable it is just pointless to start with other aspects (honestly I'm aware of a method for the MTFs but it is simply way too much effort).
Feel free to check out the documentation over at imatest.com how it is all done. The testing procedure is no secret after all (apart from some adjustments of course).
In all seriousness, is the previous 24-120 VR any worse? According to your tests, not really... Do we think the 28-300mm VR is going to be any better? Perhaps it was trying to reach the f/4.0 that really did it for the new 24-120?
[quote name='Lord Beau' timestamp='1294131173' post='5331']
In all seriousness, is the previous 24-120 VR any worse? According to your tests, not really... [/quote]
I can't tell you, yet. The review published here is a DX review, only. An FX review will follow, though.
[quote name='Lord Beau' timestamp='1294131173' post='5331']
Do we think the 28-300mm VR is going to be any better?
[/quote]
I can already tell you that it has even more pincushion distortion <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/wink.gif' class='bbc_emoticon' alt='  ' />
-- Markus
Editor
opticallimits.com
D'you know, I'm tempted to go over to the Canon 5D Mkii because at least the 24-105 commands SOME respect....
|
