Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How does Nikon do it?
#1
There's quite a misconception out there thinking massive low-noise advantages can come over night. Nikon somehow seem to imply that they defy physics that all other mortals have to obey. Interestingly enough, in the midst of these supposed noise improvements, Nikon cameras are the least used in astrophotography* applications for some reasons explained in the first link below. The other links are also related to the same topic of "cooking RAW files" or manipulating the data captured by the sensor before writing them to the RAW files.





Cooking RAW Files



Look for the line "this occurs even before the raw image is written to the file" in the Nikon section below:

http://www.astropix.com/HTML/I_ASTROP/NIK_CAN.HTM



Another article:

http://www.dxomark.com/index.php/en/Our-...cooked-RAW



Dark frame subtraction is also evident especially in the D3X "full SNR" compared to other sensors:

http://www.dxomark.com/index.php/en/Came.../Nikon/D3X





Noise vs. Resolution



My own comparison of 1Ds3 vs. D3 vs. a900 ISO1600 RAW noise w.r.t. resolution:

http://www.flickr.com/photos/genotypewriter/3012247230



DXOMark on the same (1Ds3 vs. D3) confirms this:

http://www.dxomark.com/index.php/en/Our-...omparisons





I know this post can be easily thought of as a N vs. C thing... but instead, I hope the readers here are able to see it as a "photographers vs. manufacturers" thing instead.



GTW





* Astrophotography as in taking pictures of stars and deep space objects through dedicated astro-telescopes. Taking cameras to space to take pictures of space-stations, Earth, etc. isn't what I'm referring to here.
#2
[quote name='genotypewriter' date='18 July 2010 - 08:57 AM' timestamp='1279439858' post='1086']

There's quite a misconception out there thinking massive low-noise advantages can come over night. Nikon somehow seem to imply that they defy physics that all other mortals have to obey. Interestingly enough, in the midst of these supposed noise improvements, Nikon cameras are the least used in astrophotography* applications for some reasons explained in the first link below. The other links are also related to the same topic of "cooking RAW files" or manipulating the data captured by the sensor before writing them to the RAW files.





Cooking RAW Files



Look for the line "this occurs even before the raw image is written to the file" in the Nikon section below:

http://www.astropix.com/HTML/I_ASTROP/NIK_CAN.HTM



Another article:

http://www.dxomark.com/index.php/en/Our-...cooked-RAW



Dark frame subtraction is also evident especially in the D3X "full SNR" compared to other sensors:

http://www.dxomark.com/index.php/en/Came.../Nikon/D3X





Noise vs. Resolution



My own comparison of 1Ds3 vs. D3 vs. a900 ISO1600 RAW noise w.r.t. resolution:

http://www.flickr.com/photos/genotypewriter/3012247230



DXOMark on the same (1Ds3 vs. D3) confirms this:

http://www.dxomark.com/index.php/en/Our-...omparisons





I know this post can be easily thought of as a N vs. C thing... but instead, I hope the readers here are able to see it as a "photographers vs. manufacturers" thing instead.



GTW





* Astrophotography as in taking pictures of stars and deep space objects through dedicated astro-telescopes. Taking cameras to space to take pictures of space-stations, Earth, etc. aren't what I'm referring to here.

[/quote]

Some side notes.

Nikon's D3 (and D200, D300/D3s probably, D700, probably all Nikon DSLRs since the D200) appear to apply some NR only (or extra?) when exposure time reaches a certain duration. It can vary from model to model (I have forgotten the exposure time values where you see it kick in with the different models).



With earlier Nikons you could force it to write RAW data by using long exposure NR (black frame subtraction) and then turning the camera off while it was taking the black frame. I wonder if that is still possible with the newer models like the D3 and D3s? That would mean that you can prove exactly which impact the NR on the RAW data has.



The Canon 5D mk II got some kind of pattern noise firmware fix. Not sure how that impacts the RAW data integrity.
#3
[quote name='Brightcolours' date='18 July 2010 - 02:29 PM' timestamp='1279456196' post='1092']

Some side notes.

Nikon's D3 (and D200, D300/D3s probably, D700, probably all Nikon DSLRs since the D200) appear to apply some NR only (or extra?) when exposure time reaches a certain duration. It can vary from model to model (I have forgotten the exposure time values where you see it kick in with the different models).[/quote]

You can actually switch this feature on and off with Canon, long exposure noise reduction, that is. However, this is not what affects astrophotography. It reduces heated sensor noise, by substracting the pattern noise from a sensor that is hot because of a long exposure. It is the only way to do this, and is different literally from camera to camera, as no individual sensor is the same in this regard. Furthermore, funnily enough, this is a technique borrowed from astrophotography in the first place <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />.

Quote:With earlier Nikons you could force it to write RAW data by using long exposure NR (black frame subtraction) and then turning the camera off while it was taking the black frame. I wonder if that is still possible with the newer models like the D3 and D3s? That would mean that you can prove exactly which impact the NR on the RAW data has. The Canon 5D mk II got some kind of pattern noise firmware fix. Not sure how that impacts the RAW data integrity.

With regard to the 5D II: it doesn't affect it. It was creating noise where there isn't any. It is in a way a similar issue to the black pixel syndrome, where in high iso shots with high contrast transitions, the neighbouring pixels got clipped by the A/D converter hardware to 0 values due to the preceived too big difference in eV values. It is a strange phenomenon in that neigbouring signal outputs are somehow interrelated, which is a physics thing as far as I understand (i am no expert on this, BTW, so this is my layman interpretation). The boundaries between two neighbouring well sites is not as clear cut, it seems, as we would expect.



However, Raw is still Raw with the 5D II, as all pixe values are still there. IOW, th eproblem lies in gettign out the real values, not so much trying to get rid of noise, with these issues anyway.



Kind regards, Wim
Gear: Canon EOS R with 3 primes and 2 zooms, 4 EF-R adapters, Canon EOS 5 (analog), 9 Canon EF primes, a lone Canon EF zoom, 2 extenders, 2 converters, tubes; Olympus OM-D 1 Mk II & Pen F with 12 primes, 6 zooms, and 3 Metabones EF-MFT adapters ....
#4
You can actually switch this feature on and off with Canon, long exposure noise reduction, that is. However, this is not what affects astrophotography. It reduces heated sensor noise, by substracting the pattern noise from a sensor that is hot because of a long exposure. It is the only way to do this, and is different literally from camera to camera, as no individual sensor is the same in this regard. Furthermore, funnily enough, this is a technique borrowed from astrophotography in the first place <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />.

[/quote]

But that is not what I am referring to, long exposure NR is with a black frame. And Nikon has the feature too (which you can switch on and off). The type of NR you can see Nikon apply is NOT that kind of NR, and it switches on at 3secs for some models to 0.5 sec or so on other models. You can find it by careful RAW data analysis.



Whether this is connected to the bump you see in dynamic range in DXOmark data for the D3s at ISO 25600 I do not know. The graphs of DXOmark do not show the used exposure times.



Anyway: My point is that you can see (extra?) NR kick in with Nikon models when they pass a certain exposure time, and this is NOT connected with "long exposure" black frame subtraction NR.

[quote name='wim' date='18 July 2010 - 02:44 PM' timestamp='1279460649' post='1093']

With regard to the 5D II: it doesn't affect it. It was creating noise where there isn't any. It is in a way a similar issue to the black pixel syndrome, where in high iso shots with high contrast transitions, the neighbouring pixels got clipped by the A/D converter hardware to 0 values due to the preceived too big difference in eV values. It is a strange phenomenon in that neigbouring signal outputs are somehow interrelated, which is a physics thing as far as I understand (i am no expert on this, BTW, so this is my layman interpretation). The boundaries between two neighbouring well sites is not as clear cut, it seems, as we would expect.



However, Raw is still Raw with the 5D II, as all pixe values are still there. IOW, th eproblem lies in gettign out the real values, not so much trying to get rid of noise, with these issues anyway.



Kind regards, Wim

[/quote]
#5
AFAIK, the noise issue has many aspects in digital photography. There's more behind the fact that most of us know; area of photosites, which effects the result in raw data. Of course it begins with the light sensitive area. Think of pixels as buckets to collect photons from rain of light. If the bucket is big and there are enough number of photons in it, you start the high ISO race with an advantage. But then the A/D converter must count them correctly and try to find the (almost) perfect RGB value for each pixel (interpolating / demosaicing). If the quality of count and finding correct values is not ok, then this usually leads to noise in the raw data. I believe that 14 bit A/D converters can be more helpful than the 12bits, but in the end most of the images are published as 8 bit JPEGs. So if you have high gamut monitor and you don't like sharing your images on the net, go for the high bits and see them in your terrific monitor.



The D3 and 5D MKII are the best examples regarding the facts mentioned above. Noise in the final raw data has many dependencies, and only some of them are known to us "regular users".



PS: In the astrophotography, Thom Hogan thinks that the Canons has the advantage because of the even distribution of noise values (like bell-shape around value zero) by the A/D converter. So the noise is distributed equally to either side of 0 value. According to him, this allows proper stacking of multiple images to extract the real image. OTOH, Nikon truncates the noise through its A/D converter, so that an uneven distribution is produced... I don't know why Nikon does it and how are the actual effects but this info comes from a Nikon speacialist.



Best regards,



Serkan
#6
[quote name='Brightcolours' date='18 July 2010 - 04:07 PM' timestamp='1279462047' post='1094']

But that is not what I am referring to, long exposure NR is with a black frame. And Nikon has the feature too (which you can switch on and off). The type of NR you can see Nikon apply is NOT that kind of NR, and it switches on at 3secs for some models to 0.5 sec or so on other models. You can find it by careful RAW data analysis.



Whether this is connected to the bump you see in dynamic range in DXOmark data for the D3s at ISO 25600 I do not know. The graphs of DXOmark do not show the used exposure times.



Anyway: My point is that you can see (extra?) NR kick in with Nikon models when they pass a certain exposure time, and this is NOT connected with "long exposure" black frame subtraction NR.

[/quote]

Ok, thanks for the additional explanation!



I do think it is connected to the, IMO, artificial DR bump that DxOmark reports <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />.



Kind regards, Wim
Gear: Canon EOS R with 3 primes and 2 zooms, 4 EF-R adapters, Canon EOS 5 (analog), 9 Canon EF primes, a lone Canon EF zoom, 2 extenders, 2 converters, tubes; Olympus OM-D 1 Mk II & Pen F with 12 primes, 6 zooms, and 3 Metabones EF-MFT adapters ....
#7
Hi Serkan,

[quote name='PuxaVida' date='18 July 2010 - 04:11 PM' timestamp='1279462319' post='1095']

AFAIK, the noise issue has many aspects in digital photography. There's more behind the fact that most of us know; area of photosites, which effects the result in raw data. Of course it begins with the light sensitive area. Think of pixels as buckets to collect photons from rain of light. [/quote]

Except that these buckets are leaky, as in, having holes on the sides, IOW, partly spilling into adjacent buckets. This is one of the areas of great headaches, I think. <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />

Quote:If the bucket is big and there are enough number of photons in it, you start the high ISO race with an advantage. But then the A/D converter must count them correctly and try to find the (almost) perfect RGB value for each pixel (interpolating / demosaicing). If the quality of count and finding correct values is not ok, then this usually leads to noise in the raw data. I believe that 14 bit A/D converters can be more helpful than the 12bits, but in the end most of the images are published as 8 bit JPEGs. So if you have high gamut monitor and you don't like sharing your images on the net, go for the high bits and see them in your terrific monitor.

This is where Zone-system type photography and/or HDR comes in. It does require a lot of learning and a lot of extra work, however <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />

Quote:The D3 and 5D MKII are the best examples regarding the facts mentioned above. Noise in the final raw data has many dependencies, and only some of them are known to us "regular users".

Agreed.

Quote:PS: In the astrophotography, Thom Hogan thinks that the Canons has the advantage because of the even distribution of noise values (like bell-shape around value zero) by the A/D converter. So the noise is distributed equally to either side of 0 value. According to him, this allows proper stacking of multiple images to extract the real image. OTOH, Nikon truncates the noise through its A/D converter, so that an uneven distribution is produced... I don't know why Nikon does it and how are the actual effects but this info comes from a Nikon speacialist.



Best regards,



Serkan

Well, it does look like the in-camera Nikon AD conversion process sets values below a certain threshold to zero, thus eliminating a lot of noise, and thus distorting the noise distribution curve, but also wiping out a lot of very fine astro-information in de case of astrophotography, and of course simultaneously increasing the DR in a fashion that to me looks rather artificial. Setting, e.g., the lowest two bits to black (zero value), provides a seemingly two stop noise and DR advantage immediately <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />. The exact process and how it is done, and what the actual algorithm used is (more complicated for sure than the simple form I mention here) I don't know either, obviously <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />. I think it is very clever, just a pity that it is done prior to saving the Raw file, rather than as a very effective noise reduction method for jpeg generation.



Kind regards, Wim
Gear: Canon EOS R with 3 primes and 2 zooms, 4 EF-R adapters, Canon EOS 5 (analog), 9 Canon EF primes, a lone Canon EF zoom, 2 extenders, 2 converters, tubes; Olympus OM-D 1 Mk II & Pen F with 12 primes, 6 zooms, and 3 Metabones EF-MFT adapters ....
#8
[quote name='wim' date='18 July 2010 - 05:32 PM' timestamp='1279463532' post='1098']

Hi Serkan,



Except that these buckets are leaky, as in, having holes on the sides, IOW, partly spilling into adjacent buckets. This is one of the areas of great headaches, I think. <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />

[/quote]



Thank for the additional info Wim... Do they "leak" or rather overflow to the nearest pixel? Because if these pixels have holes in their walls, then this is nothing but a sabotage of high quality tonal range by the manifacturer himself [Image: blink.gif]. And if it's an overflow, than I think the micro lenses must be able to take care of this issue... Am I asking too much from the manifacturers... it's just a special type of coating over the lens which gets light in but not out [Image: tongue.gif]...





[quote name='wim' date='18 July 2010 - 05:32 PM' timestamp='1279463532' post='1098']

This is where Zone-system type photography and/or HDR comes in. It does require a lot of learning and a lot of extra work, however <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />

[/quote]



Both the zone system and HDR require a recording medium with high dynamic/tonal range for best results. If all you have for the net is the 8 bit JPEGs, what can you do? Producing high quality images always come with a price: they're not easily portable (which is not an issue for me personally... most of the time it's enough for me to see them on my screen)...



King regards,



Serkan
#9
[quote name='PuxaVida' date='18 July 2010 - 07:02 PM' timestamp='1279472524' post='1103']

Thank for the additional info Wim... Do they "leak" or rather overflow to the nearest pixel? Because if these pixels have holes in their walls, then this is nothing but a sabotage of high quality tonal range by the manifacturer himself [Image: blink.gif]. And if it's an overflow, than I think the micro lenses must be able to take care of this issue... Am I asking too much from the manifacturers... it's just a special type of coating over the lens which gets light in but not out [Image: tongue.gif]...



[/quote]



It's tricky business. At that size, quantum effects start coming into play - photons will randomly decide to just jump into the next bin over, and of course the more there are, the more will do that. The bucket metaphor doesn't hold up to that level of strangeness.



And as for the special coating that lets light in but not out - I know of a lot of very powerful people who'd like to have such a thing.



-Lars
#10
Hi Serkan,

[quote name='PuxaVida' date='18 July 2010 - 07:02 PM' timestamp='1279472524' post='1103']

Thank for the additional info Wim... Do they "leak" or rather overflow to the nearest pixel? Because if these pixels have holes in their walls, then this is nothing but a sabotage of high quality tonal range by the manifacturer himself [Image: blink.gif]. And if it's an overflow, than I think the micro lenses must be able to take care of this issue... Am I asking too much from the manifacturers... it's just a special type of coating over the lens which gets light in but not out [Image: tongue.gif]...[/quote]

I am afraid you are asking too much <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />.



All this stuff is so incredibly small, that something has to give. This happens at the micro structure level of the chip if you like, and the A/D conversion is another place where things like this happen. This, BTW, is no different from the analog days, just that almost nobody noticed it <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />. Both the film, exposing it, and the development process contributed to noise and artefacts. However, the algorithms to get rid of it were less complex <img src='http://forum.photozone.de/public/style_emoticons/<#EMO_DIR#>/biggrin.gif' class='bbc_emoticon' alt='Big Grin' />.



Quote:Both the zone system and HDR require a recording medium with high dynamic/tonal range for best results. If all you have for the net is the 8 bit JPEGs, what can you do? Producing high quality images always come with a price: they're not easily portable (which is not an issue for me personally... most of the time it's enough for me to see them on my screen)...



King regards,



Serkan

Not necessarily. It really all is about the printing, or display medium, and what you want recorded there. Photographic paper only has a DR of about 5 or maybe 6 stops, the best inkjet papers maybe slightly more. It is really all about what you find most important and what you want to be visible with regard to DR on this limited medium.



This means that you really have to think beforehand what to do, how to approach it. That is what the Zone System is all about.. What has to go in the middle ranges, Zone V etc., and what in Zone 2, Zone 8, etc., and how do you deal with the rest. Essentially you compress the DR so that it fits in the tonal scale you are using to show the finished image.



Current slrs all more or less have a similar DR to film, if not slightly more, so that hasn't changed. With film you would expose for a specific film development process that you used to convert what you saw in the field to what you wanted to show. Overexposing and underdeveloping to compress the DR, underexposing and overdeveloping to stretch it, specifically for that which you wanted to show in the final photograph. It's slightly more complicated than that, but this was what it was about. You effectively manipulate the resulting tonal S-curve so that it fits that what you visualized and what you want printed on the lower DR medium.



This is no different from what you want to do with digital, just that digital doesn't take well to overexposing, and film doesn't do well to underexposing. Other than that, the principle is really the same, the way I see it. Just that the method to apply this differs - computer vs darkroom.



Kind regards, Wim
Gear: Canon EOS R with 3 primes and 2 zooms, 4 EF-R adapters, Canon EOS 5 (analog), 9 Canon EF primes, a lone Canon EF zoom, 2 extenders, 2 converters, tubes; Olympus OM-D 1 Mk II & Pen F with 12 primes, 6 zooms, and 3 Metabones EF-MFT adapters ....
  


Forum Jump:


Users browsing this thread:
1 Guest(s)