• Log in with Facebook Log in with Twitter
Photo

next PZ lens test report - Sigma 14mm f/1.8 DG HSM ART


  • Please log in to reply
58 replies to this topic

#21 JoJu

JoJu

    Advanced Member

  • Members
  • PipPipPip
  • 2,877 posts
  • LocationSwitzerland

Posted 21 November 2017 - 11:43 AM

BC, you also can imagine - like me - the radius or at least the shape of the "in-focus-field" can change depending of the distance?

 

I imagine a relief which changes between close and far distance it's levels.

 

Therefore there might be field curvature at testing distance, but a very different one at infinity. Speaking of infinity: Stars are not arranged in a focal pane... question just remains if in the curvature focus at infinity is possible or not.



#22 mst

mst

    Advanced Member

  • Moderators
  • 2,095 posts
  • LocationWesterwald, Germany

Posted 21 November 2017 - 02:42 PM

If I need sharp borders. I'll stop dwon anyway, but I really want to know a single case or subjectwhen you need high res corners wide open? In my book there's none of this subjects.Not even in theory.


"Theory?" - that's my keyword, right? ;)

Seriously: if you compose your images following the rule of thirds or golden rule, then your main subject is close to or (if you stretch the rule a little) almost at where we measure border performance. In any case closer to border than to the center.

Edit: Oh and by the very way: The Nikon got 4 stars at it's time. At 14 mm and f/2.8, it is in no respect better than the Sigma - not on 20 and especially not on 45 MP!


Besides the fact that we keep preaching "thou shalt not compare results across systems": are we looking at the same reviews? The Nikkor has excellent borders at f/2.8 already (on the D3x... on the D850, we will hopefully see in not too distant future).

It got "only" 4 stars because of the flare issue, otherwise the rating would have been a little higher.
Editor
photozone.de

#23 Brightcolours

Brightcolours

    Advanced Member

  • Members
  • PipPipPip
  • 4,850 posts
  • LocationThe Netherlands

Posted 21 November 2017 - 03:36 PM

PZ's sharpening definitely muddies the waters, trying to compare the 5DS-R and D3X results. And Lenstip's reviews do not clear things up a little, as the Nikkor 14-24mm f2.8 appears to be only tested on a D200?

 

So will have to wait till the Sigma 14mm f1.8 is tested on a D3X. 

 

And yes JoJu, I agree with the distance appears to have a big influence on the shape of the focus field, demonstrably so with certain lenses. 



#24 dave's clichés

dave's clichés

    Advanced Member

  • Members
  • PipPipPip
  • 1,769 posts

Posted 21 November 2017 - 04:29 PM

 If I need sharp borders. I'll stop down anyway, but I really want to know a single case or subject when you need high res corners wide open? In my book there's none of these subjects.Not even in theory.

 

 

     Strange JoJu you forgot the star-spangled night sky .........in practice and theory!

 

 

             ......the very reason I bought the Samyang 14mm.



#25 JoJu

JoJu

    Advanced Member

  • Members
  • PipPipPip
  • 2,877 posts
  • LocationSwitzerland

Posted 21 November 2017 - 04:51 PM

Markus,
 
I simply don't believe that the Nikkor @14mm and f/2.8 will be better in any aspect than the Sigma 14mm and f/2.8. Distortion is roughly the same - mind you, I'm talking about f/2.8. I think the Sigma will kick Nikkor's ass in any possible way.
 
But it got 4 stars - and the Sigma only 2 1/2 respectively 3 1/2. I already am a bit suspicious because at 21 MP the extreme borders are better and at 50 MP weaker - I don't think,. one can blame "field curvature" here?
 
Don't get me wrong, I don't want to lift up the rating of the Sigma. I just think, controlling the flares much better, offering 1 1/2 stop more light, being at least as sharp in the center as the Nikon wide open (not to speak about stopped down to f/2.8) this rating is a bit off my experience with the lens.
 
Anyway.
 
Oh, and tehre's a remark of a reader about "Available in native Canon and Nikon mount, it is possible to use Sigma's Mount Converter MC-11 for using the lens on Sony and Sigma mirrorless cameras."
 
I think the reader's right to criticize that part of the text.
 
MC11 is either to convert a Canon or Sigma lens mount to connect to a Sony E-mount. You cannot use the MC-11 to connect a Sigma lens mount on a Canon camera. Of course, who ever will use this brick with a Sigma camera, but you never know....

#26 JoJu

JoJu

    Advanced Member

  • Members
  • PipPipPip
  • 2,877 posts
  • LocationSwitzerland

Posted 21 November 2017 - 05:23 PM

Markus,

 

I simply don't believe that the Nikkor @14mm and f/2.8 will be better in any aspect than the Sigma 14mm and f/2.8. Distortion is roughly the same - mind you, I'm talking about f/2.8. I think the Sigma will kick Nikkor's ass in any possible way.

 

But it got 4 stars - and the Sigma only 2 1/2 respectively 3 1/2. I already am a bit suspicious because at 21 MP the extreme borders are better and at 50 MP weaker - I don't think,. one can blame "field curvature" here?

 

Don't get me wrong, I don't want to lift up the rating of the Sigma. I just think, controlling the flares much better, offering 1 1/2 stop more light, being at least as sharp in the center as the Nikon wide open (not to speak about stopped down to f/2.8) this rating is a bit off my experience with the lens.

 

Anyway.

 

Oh, and tehre's a remark of a reader about "Available in native Canon and Nikon mount, it is possible to use Sigma's Mount Converter MC-11 for using the lens on Sony and Sigma mirrorless cameras."

 

I think the reader's right to criticize that part of the text.

 

MC11 is either to convert a Canon or Sigma lens mount to connect to a Sony E-mount. You cannot use the MC-11 to connect a Sigma lens mount on a Canon camera. Of course, who ever will use this brick with a Sigma camera, but you never know....



#27 Klaus

Klaus

    Chief Editor

  • Moderators
  • 5,453 posts
  • LocationSydney, Australia

Posted 21 November 2017 - 10:21 PM

@Joju - how would you rate the Sigma based on these charts?

 

Honestly I was about to go lower just based on the MTFs but I thought that the bokeh is too nice for that.

I can add a "field" rating here - which is probably more like 3.5-4*.


Chief Editor
photozone.de

#28 JoJu

JoJu

    Advanced Member

  • Members
  • PipPipPip
  • 2,877 posts
  • LocationSwitzerland

Posted 22 November 2017 - 01:18 AM

Klaus, I don't know, honestly. On one hand it's consistent to base the ratings on MTFs. On the other hand the lens can do some tricks others can't, at least according to the shots I made with it so far.

 

I don't think it's 1 ½ ★ worse than the Nikon. The Canon 11-24/4(!) got "no rating at this stage". Meanwhile the Sigma "at this stage" got a rating at 50MP and if you check the MTF, the corners of the Canon at 14/4 are worse, the center and borders slightly better and no MTF at close center. Hmm.

 

Do what you like as a rating, it's no problem for me to keep the feeling this rating is not doing justice to a lens like that - I have no better suggestion. And besides of any ratings, the lens works the way I was hoping for - I am not Sigma, I can enjoy a "badly" rated lens without being afraid about bad sales.  :D

 

It is due to it's weight and size anyway no lens one just throws in the photo bag. It is highly debatable if an UWA zoom like the Nikkor or the Canon, even the Sigma 12-24/4 is not a better idea at the end, all are "more versatile". I bought it for low light and for bokeh at closer distances with more background than usual - there's no real alternative to it, but I also don't find it perfect. For landscapes in sunlight I'd prefer it to the Nikkor. For architectural "inside a room" views the Nikkor offers a better package.

 

But you see, there's a lot more than just MTFs which (might) matter for people interested in it. I think, the best recommendation is "rent one and see if it works for you".



#29 Brightcolours

Brightcolours

    Advanced Member

  • Members
  • PipPipPip
  • 4,850 posts
  • LocationThe Netherlands

Posted 22 November 2017 - 08:27 AM

It again is clear (to me at least) that the way you guys approach the MFT tests with the sharpening makes for problematic results that no one really can evaluate. 

The sharpening exaggerates sharpness, and will not touch less sharp results, exaggerating the difference in corners and center, and also skewing results twice when comparing for instance how that Nikkor 14-24mm f2.8 did on 24mp. 

I also wonder if the R-ness of the 5DS-R also brings its own part of the problem, here. Fake sharpness from "no" AA-filter skewing results up until the lens acts as AA-filter (corners).

 

Something worth investigating?

 

The 50mp versus 21mp results do show part of the sharpening issue(s). The 50mp extreme corners getting lower results than the 21mp extreme corners.



#30 JoJu

JoJu

    Advanced Member

  • Members
  • PipPipPip
  • 2,877 posts
  • LocationSwitzerland

Posted 22 November 2017 - 09:23 AM

MTF. Multilateral Trading Facility. Modulation transmission frequency. So "MTF" without "charts" makes as much sense as "gimme a 17er" without "wrench". People understand fi someone lying under a car yells that in a workshop.

 

MFT is abused for µ 4/3.  :rolleyes:

 

:D



#31 Klaus

Klaus

    Chief Editor

  • Moderators
  • 5,453 posts
  • LocationSydney, Australia

Posted 22 November 2017 - 03:17 PM

It again is clear (to me at least) that the way you guys approach the MFT tests with the sharpening makes for problematic results that no one really can evaluate. 

The sharpening exaggerates sharpness, and will not touch less sharp results, exaggerating the difference in corners and center, and also skewing results twice when comparing for instance how that Nikkor 14-24mm f2.8 did on 24mp. 

I also wonder if the R-ness of the 5DS-R also brings its own part of the problem, here. Fake sharpness from "no" AA-filter skewing results up until the lens acts as AA-filter (corners).

 

Something worth investigating?

 

The 50mp versus 21mp results do show part of the sharpening issue(s). The 50mp extreme corners getting lower results than the 21mp extreme corners.

 

In essence - you want us to process images in a way that no user would do out there ... sorry - won't happen.

If you think that the charts would "look" any different - hardly - we would simply adjust the scale accordingly.

Yes, the numbers would be different but the numbers ... well ...

 

I can only repeat again that we tune reference images to neutral sharpness which Imatest identifies are neither over- nor undersharpened. That amount is less than the default sharpening.


Chief Editor
photozone.de

#32 Brightcolours

Brightcolours

    Advanced Member

  • Members
  • PipPipPip
  • 4,850 posts
  • LocationThe Netherlands

Posted 22 November 2017 - 03:35 PM

In essence - you want us to process images in a way that no user would do out there ... sorry - won't happen.

If you think that the charts would "look" any different - hardly - we would simply adjust the scale accordingly.

Yes, the numbers would be different but the numbers ... well ...

 

I can only repeat again that we tune reference images to neutral sharpness which Imatest identifies are neither over- nor undersharpened. That amount is less than the default sharpening.

No, i want the lenses to be tested, and not some specific sensor or a specific idea of post processing or choice of RAW converter.

In case of the in this thread mentioned Canon 11-24mm, a user would convert the image with DPP4 and its lens profile, getting much better sharpness in the corners than your USM mask sharpening in your MFT PP.

The argument that you sharpen because a user would do that does not fly. 

 

Would the chart look different? Yes, of course it would. Like JoJu and I have pointed out, the 21mp and 50mp Sigma 14mm f1.8 "charts" do show the issue (the 21mp test gets a higher MFT score than the 50mp test, same lens...). Without the sharpening, the 21mp wide open corner would have a less high bar than it has now. And the difference between sharp figures and less sharp figures would be way less pronounced in the 50mp charts. You will see less of a hump in the charts.

 

Run a test run for this Sigma without sharpening, and put in next to the results you get now, just to get an idea of how the sharpening gets skewed twice, in regards to lower resolution tests.

 

And the Imatest software anyway can't distinguish between fake ("enhanced") sharpness from pixel edges, and and actual sharpness, can it? That then poses an issue that is not under your control.



#33 JoJu

JoJu

    Advanced Member

  • Members
  • PipPipPip
  • 2,877 posts
  • LocationSwitzerland

Posted 22 November 2017 - 04:29 PM

Changing a preparation method after years of testing would then mean retesting. Otherwise the results would be as comparable as today. I don't see the resources for a two-person test site. Usually I check at lensrentals or lenscore if I want to know about MTF charts - with the advantage of lensrentals' batch testing. But honestly, I didn't care about test results with the last dozen purchases.

 

Once I rented a lens (to find out the 85/1.4 Art is too massive), other times I visited a weekend event of Sigma or Fuji to try out. Often I trusted Sigma or Fuji (and occassionally ended up with a disappointment). To me, testing falls short in daily practice. What good is excellent center sharpness after purchase if after a while the zoom ring starts to wobble or I have to find out the VR has some weak spots?

 

Yesterday night I did some star shots with the 14/1.8. In the corners wide open I saw some butterfly-lights. At 100% they look awful, as an A3 print hardly noticeable. One shot I thought I focused at infinity - and some 3-4 meters distant trees were pretty okay. 

 

We are super spoilt by lenses which were literally impossible to produce only a decade ago.



#34 dave's clichés

dave's clichés

    Advanced Member

  • Members
  • PipPipPip
  • 1,769 posts

Posted 22 November 2017 - 05:45 PM

     Lenstip don't use sharpening.......within the review site the resolution scores take on a meaning by results of other lenses,   I very much like their method.

 

  Here sharpening is used, just like I use it (typically opens with 75%) Ol's graphics become familiar by use also, in spite of the sharpening, results on the 50 Mps sensor is a struggle...so apart from just upsetting the site's apple cart and with already two tested resolution sensors.........

 

  I would leave well alone!..

    

 

 

   ...besides Lentips's sample images almost all look soft, the top lenses, well just less soft.



#35 mst

mst

    Advanced Member

  • Moderators
  • 2,095 posts
  • LocationWesterwald, Germany

Posted 23 November 2017 - 09:17 AM

There is no solution to this, as long as any kind of camera is involved. Using a digital camera for MTF tests, the test becomes a system test that includes characteristics of both the test camera used (or its sensor, to be more precise) AND the raw converter used. There is no way around this, no matter if you apply default, adjusted or (as they claim) no sharpening. The effect is the same, you just scale the results in the end.

We tried, BC. In fact, we see the effects everytime we get a new test camera, since that camera is usually not compatible with any raw converter we have used in the past, sometimes has a new sensor technology or comes without an AA filter. If you use default sharpening, the MTF results usually go through the roof, with peak values way above the maximum resolution of the camera. So, in other words: your camera or your raw converter usually oversharpen quite a bit already.

We aim much lower with our sharpening. For the D7200 reviews, to give you an idea about the amount of work that is required, I went through no less than 15 iterations to find the best parameters. As Klaus pointed out several times in the past already, we do so only to counterbalance the softening that is a result of the Bayer pattern interpolation, so already an influence of the camera's sensor.

Even if you try to use the same parameters for all reviews (like "no sharpening"), you will not get comparable results. A camera without AA filter for example with deliver higher results than one without. And with different converter releases (which you HAVE to use, unless you want to stick to completely outdated cameras), algorithms, especially the demosaic algorithms, change, and that can have all kinds of effects. Klaus had to learn this quite a while ago when Adobe released ACR 3 (I think) and MTF results suddenly dropped. Another example: when calibrating the D7200 I was surprised to get CA values that are lower (!) than on the D7000. It turned out, that this was because of the RAW converter (the same I used for the D7000 reviews, just a newer version), which consistently gives lower CA results (with CA correction of course switched off).

So, no matter what approach you follow: as long as a camera sensor is involved, you're doing system tests, not pure lens tests. And because of the described influences, the results are not directly comparable, no matter what set of parameters you apply. Your only way out of this is the Cicala way, an optical bench. But then you get results that don't tell you anything about how a given lens performs on a given camera. Or, in a more generic way: on a certain level of sensor resolution.

Regarding the rating: to be honest, I don't like them (and I'm not sure if Klaus' opinion is any different). In fact, there were no rating for a few years in the beginning, but there were so many requests to introduce some kind of rating that we couldn't ignore it. I don't like the idea, because they oversimplify things and in some cases, for example the Sigma 14 Art or the Nikkor AF-D 85/1.4, can't do a lens full justice. That's why we introduced field ratings, at least when there is a significant difference between the objective performance figures and the (usual) intended usage of a lens.
I don't like the rating, because I often see that our reviews are reduced to the ratings. The Sigma is considered a "bad lens", because it doesn't have many stars. That's not an issue of the star rating, though, it's an issue of those readers that reduce their effort of review reading to just looking at the stars and are not willing to invest the time to read our 2- or 3-page (so, not really that long) reviews completely and thus get a much better overview of the performance of a tested lens. A star rating can't give you that, if that's all you look at. But it can help to put things into perspective (for those who don't narrow their perspective and view, intentionally or unintentionally).
Editor
photozone.de

#36 JoJu

JoJu

    Advanced Member

  • Members
  • PipPipPip
  • 2,877 posts
  • LocationSwitzerland

Posted 23 November 2017 - 12:04 PM

To my surprise, Capture One had a D850 in their list of supported cameras before I had the camera - and before it was available. But usually you're right, especially with exotic stuff. Or Fujifilm...

 

But no matter how many iterations you will go, the values on another system (body, lens, converter) will change again. Basically I would support the idea of no sharpening at all (what with noise reduction? Capture One uses a pretty strong on eby default) and I suspect, some companies are also fiddling with the RAW before they export it to an SD card.

 

About the ratings: If there's a transparent system for ratings, why not? In fact it is quicker than reading. At the same time I sometimes miss aspects which are important for me - but do not influence ratings. And when Klaus mentions to rate the Sigma even lower but went higher at the end because of the nice bokeh then I conclude: oh the ratings are more based on feeelings than on facts (which is normal, I'd say, as soon as one puts subjective points like handling or finish into the rating). Then the "state of the art" also appears to be an issue. Years ago, Klaus said, that lens would have been outstanding - so why not today? I'd like to read about the reasons.

 

I see it from a readers side: OL/PZ doesn't create comparisons between lenses (like "how good are the corners of this lens at this FL compared to another lens"). Reading through lens reviews and seeing some remarks about bad vignetting (don't care) bad flare behaviour (care more, but don't agree with Klaus' findings) leaves me a bit clueless. If lenscore rated the lens, they use one sensor for all, so the results of the lenses are comparable, even if the system has oter issues. Lensrentals MTF curves also allow comparisons - but MTF charts are not the only aspect.

 

Inner design of the lens is not an issue for ratings - and not important for most as long as it's working.

 

I don't care much about 1/2 or 1 stars difference. But I do care about 1 1/2 star diff beteeen a zoom getting 4, flaring like mad and a prime with 1 1/3 stop more speed and fully adjustable AF at 4 different distances - tis matters because sharp or not is more important than getting 10 lines more resolution in lab conditions. I don't want to create a big debate, but that lens is not worse at 14/2.8 than the 4 star Nikkor is. At least not (transfering stars into percent is ridiculous, I know) nearly 40% worse (4 stars 100%, 2.5 stars 62.5%).

 

Regular readers also know that you're testing only one lens. Measuring 10 and getting sample variations also doesn't help much to verifiy my own copy.

 

It all is about "the best lens for the cash" - and it's not static, so the idea of a static rating itself is highly questionable and in my eyes a systematic flaw. That's why I basically don't care. As already said, I do not mind how you rate my lens which I bought without waiting for anyone's verdict. Everything I ever bought has strong sides and weak ones, Use the first, avoid the latter. This was true decades ago and hasn't changed although many people think the best lens is the best in all aspects - as if Usain Bolt would be a quick swimmer and a fast bicycle driver....



#37 mst

mst

    Advanced Member

  • Moderators
  • 2,095 posts
  • LocationWesterwald, Germany

Posted 23 November 2017 - 01:40 PM

To my surprise, Capture One had a D850 in their list of supported cameras before I had the camera - and before it was available. But usually you're right, especially with exotic stuff. Or Fujifilm...


The RAW converters I use for the D3x and D7000 reviews were released in 2011... I don't have high hopes that they support the D850 ;) 
 

But no matter how many iterations you will go, the values on another system (body, lens, converter) will change again.


That was the intended message ;)
 

I suspect, some companies are also fiddling with the RAW before they export it to an SD card.


Sure. Leica does, for example. Not necessarily with any influence on MTF values, though.
 

About the ratings: If there's a transparent system for ratings, why not? In fact it is quicker than reading. At the same time I sometimes miss aspects which are important for me - but do not influence ratings.


And that's the issue, as explained above already: there is no way to rate and at the same time match everyone's priorities. That's why we write 2 or 3 pages of text and include all the graphs. If we honestly thought that a simple rating would do, we could summarize the whole site onto a single page with just the lens name and the rating... something that has been requested very often already and we so far declined, for the same reasons already mentioned when discussing review ratings in general.
 

And when Klaus mentions to rate the Sigma even lower but went higher at the end because of the nice bokeh then I conclude: oh the ratings are more based on feeelings than on facts (which is normal, I'd say, as soon as one puts subjective points like handling or finish into the rating).


Handling and finish have no influence on the optical rating. Numbers do, for the most part, and then some personal impressions or priorities, I can't rule that out. We're humans, in the end, and shutterbugs and gerhads on top. We try to be as neutral as possible, though.
 

I see it from a readers side: OL/PZ doesn't create comparisons between lenses (like "how good are the corners of this lens at this FL compared to another lens").


I'm baffled, seriously. What else are the MTF charts there for?
 

If lenscore rated the lens, they use one sensor for all, so the results of the lenses are comparable, even if the system has oter issues.


And what does this rating tell you if you want to know how the lens performs on a sensor that is very different?
 

I don't care much about 1/2 or 1 stars difference. But I do care about 1 1/2 star diff beteeen a zoom getting 4, flaring like mad and a prime with 1 1/3 stop more speed and fully adjustable AF at 4 different distances - tis matters because sharp or not is more important than getting 10 lines more resolution in lab conditions. I don't want to create a big debate, but that lens is not worse at 14/2.8 than the 4 star Nikkor is. At least not (transfering stars into percent is ridiculous, I know) nearly 40% worse (4 stars 100%, 2.5 stars 62.5%).


I'm not sure why you keep coming back to this single point. Each lens is tested and judged across the whole specification set it offers. The Nikkor starts at f/2.8, and it is fairly great at its largest aperture already.

The Sigma starts with a bigger aperture, and thus is rated across a larger aperture range. At f/1.8, it shows weaknesses at the borders and corners. Which, by the way, I think can even be seen in the full res sample you posted (the lady at the lake). I don't find the border anyhow impressive in there and think they match the findings quite well.

If a lens offers a large aperture, we expect it to be great across the frame at that setting to receive a great rating,, when it comes to MTF, it's as simple as that.

Beside: looking at the available data, I still have doubts the Sigma matches the Nikkor at 14/2.8
Editor
photozone.de

#38 mst

mst

    Advanced Member

  • Moderators
  • 2,095 posts
  • LocationWesterwald, Germany

Posted 23 November 2017 - 01:42 PM

Regular readers also know that you're testing only one lens. Measuring 10 and getting sample variations also doesn't help much to verifiy my own copy.


Regular readers know that quite often we don't ;) At least for my reviews there is often more than one review unit. It is not necessarily mentioned in the text, though.
 

It all is about "the best lens for the cash" - and it's not static, so the idea of a static rating itself is highly questionable


Right. Just what I said :)
 

As already said, I do not mind how you rate my lens


To be honest, seeing how much effort you put into arguing against the rating in this particular case, one might get a slightly different impression ;)
Editor
photozone.de

#39 toni-a

toni-a

    Advanced Member

  • Members
  • PipPipPip
  • 1,317 posts
  • LocationLebanon

Posted 23 November 2017 - 01:46 PM

https://photos.smugm...DSC00864-X3.jpg

 

the horizon seems tilted :)



#40 mst

mst

    Advanced Member

  • Moderators
  • 2,095 posts
  • LocationWesterwald, Germany

Posted 23 November 2017 - 02:03 PM

Naa, that's just how horizons are down under ;)
Editor
photozone.de




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users



© by photozone.de