[thxbb12]

It would be interesting to see results from other manufacturers for the same class of lenses (on crop sensors), for instance Panasonic 25 f1.4, Olympus 25 f1.8, Zeiss 32 f1.8, Fuji 35 f1.4

[Scythels]

 

I understood, even the worst Otus copy to be better than the best Canon 50/1.8 STM, no problem with that. But given most if not all manufacturers use the MTF method for testing, I expected the "best" lens not necessarily best manufactured, but tested the hardest way and with tightest tolerances.

Most of them don't.  The best of them have a pseudo-imatest like setup that is much faster but has lower resolution limit.  It's economical to test each lens when it takes 15 seconds.  Testing an Otus on the ImageMaster takes about 25 minutes, though that could be reduced if you used 3 field points instead of 10 per side and eliminated the focus confirmation between rotations.  TriOptics does make some mfg line MTF equipment that provides souces and microscopes for different field points simultaneously, but they are prohibitively expensive to be used on anything as cheap as a consumer product. 

 

Here are some charts:

 

[sharedmedia=gallery:images:1343]
 

[sharedmedia=gallery:images:1344]
 

Complexity is a bullshit weighted composite of groups, autofocus method (i.e number of cams), IS, tilt-shift, aperture, number of aspheres, and a focal length componenet.  There is a trend but I don't know how meaningful it is. 

 

thxbb12,

 

Micro four thirds lenses use an electromagnetic "always on" focus method so the focus group is loose when disconnected from the camera.  If you shake most of the lenses you can hear the focusing elements slapping around inside.  We need an electronic adapter to test them, which is a project we're working on but more aggressively with sony E mount.  We're just waiting for parts now. 

 

JoJu,

 

I never found a failure copy of the STM, which is a godsend since it is nonadjustable and lensrentals owns exactly 10 of them.  The otus I found one failure copy of, but it had rented something like 36 weeks which is a very high amount and had probably been dropped a bunch. 

 

LensRentals has 40-50 copies of the Otus between EF/F mount.  More expensive lenses are kept in higher stock than less expensive ones because they rent more.  There is a sweet spot around $2000 though.  There's something like 700 copies of the canon and nikon 24-70s and 70-200s combined.

 

Klaus,

 

None of these are grossly decentered.  All are somewhat, but it is not terribly too bad.

 

edit: regarding how easy it is to get a bad copy, the primes are not so bad.  I rejected maybe 8 out of the 650 or so I've tested.  Of course they are pre-screened, but the daily volume of these lenses that comes into repair for optical adjustment is, well, "meh."  Far more come in for dust issues.  2-5% are DoA, depends on the manufacture.


I'm starting zooms now with supposedly one of the best and, well, I fear for the rest.  These are truly a shitshow. 

[davidmanze]

In a nutshell: Samyang lenses are a lottery.

 I guess lotteries are all about the risk of winning nothing at all, for once the lottery came up trumps for me with the Samyang 8mm fisheye, I won  a good one, a perfect copy , one of the few lenses which didn't join the carousel of the postal system!

[Scythels]

Dave, how much would you bet you got a perfect copy of that fisheye?  I bet it has issues a camera sensor can't see! =) 


Seriously, I've measured 12 copies of the 14mm, 10 of the 24, 10 of the 35, 10 of the 50, and 10 of the 85 and I have not once seen a samyang without some problems.

[Klaus]

The slowest was the Zeiss 50/2 macro.

 

It is, of course, just a rule of the thumb.

If you limit this to the same manufacturer, it is "about right".

There are other factors than just speed as AirDiscus mentioned (e.g. age of the manufacturing line - thus the newer the better). 

[JJ_SO]

...

JoJu,

 

I never found a failure copy of the STM, which is a godsend since it is nonadjustable and lensrentals owns exactly 10 of them.  The otus I found one failure copy of, but it had rented something like 36 weeks which is a very high amount and had probably been dropped a bunch. 

 

LensRentals has 40-50 copies of the Otus between EF/F mount.  More expensive lenses are kept in higher stock than less expensive ones because they rent more.  There is a sweet spot around $2000 though.  There's something like 700 copies of the canon and nikon 24-70s and 70-200s combined.

...

 

So, there are correlations between rental time (=risk of damage increasing by different customers and their kind of use), design of the lens and number of samples? Of course, it's close to impossible to do this kind of mass-testing before all lenses went out for rentals because customers are waiting and the longer lensrentals keeps them waiting the more possible they try to get it elsewhere.

 

I don't want to talk this results down. But rented lenses have a harder live than the bought ones and I'm not talking about scratches. I know that lensrentals is checking each copy coming back from a customer, I don't know how far this check goes. And to be honest, I couldn't estimate it since i didn't study optics.

 

The tests are very interesting, however, I suspect in real life (customer buys one copy) results would change a bit from country to country (number of ordered copies and relevance for the manufacturer) and from dealer to dealer (repair-carousel). So, your testing tells me a lot about lensrentals' situation but I'm insecure to put this results 1:1 into my buying behavior. But the good news is: Since we talk about those issues and the "mint-conditions" the lenses arrive in, at least some manufacturers will put more care into their production.

 

What I see: At least Sigma already did and is not at the bad end of the results, given the highly competitive prices and redesigned mounts. So, for primes we customers do have real choices. If we're willing to wait  <_<

[Scythels]

I consider a rental week to be worth about a buyer's month in a lens' lifecycle, so the failed copy of the otus is quite old.  The otus lenses have a particular design flaw; the hood is still a normal bayonette, yet users insist on grabbing the considerable heft of the lenses by the hood because it is visually so well integrated, and the torque backs off a screw in the front barrel, allowing it to wobble.  The optics are all inside their own cell, filled with more cells inside, but this is the type of thing that requires repair very often (and is a 5 min fix with $0 of parts, vs $hundreds from a manufacture to do the same fix). 

 

Different buyers rent different kinds of equipment.  The cinema equipment is rented more in a cashflow sense, so e.g the C300s and true cine lenses may go to one customer for a month and then return.  This type of customer will take better care of the equipment than for example someone renting a supertele for 2 days to go on a particular trip.  The cine stuff also ships inside hard cases by and large, though I do not think that is of particular value since IMO shock damage in shipping is more important than crush, and the foam in most hard cases is too dense to cussion shocks. 

 

Roger did testing beforeand there isn't much difference between new in box copies and copies in lensrentals' rotation.  For your typical Canon, Nikon, sigma "A / S", very new Tamron, etc, lens a failure means something coming loose entirely and optically the lens will look terrible.  More budget manufactures and ones with a cheaper construcion style (looking at you, micro four thirds) will be more subject to "slipping" over time. 

 

In terms of LR's checking - a recieving tech shoots each and every copy of every lens at full aperture on an ISO 12233 chart when it is returned, after which it is put into the stock waiting to be pulled.  A good tech can do about 80-100 lenses an hour in this setup.

 

If I pull lenses from that stock to test, they don't get any further screening by me and they get run - unless the results are an absolute trainwreck I read that copy into the database and move on with my life.  If the copy looks bad I will test it on a camera.  If it looks bad on camera, it gets removed from the db, repaired, and re-run later in the day.  It takes quite a terrible failure for a lens to fail on camera - a 70-200 IS II properly aligned for 200mm looks something like this - https://www.dropbox.com/s/ycq6824ryd4n77...s.pdf?dl=0 yet even a copy as bad as this https://www.dropbox.com/s/d5letwr5n7htfl...s.pdf?dl=0 which should show a bad corner looks A-ok.  This isn't us having a loose tolerance - honestly that corner looks maybe a tad muddy but if you didn't have 4 good copies to compare to you may think its normal.  There's a full mechanical check too, but I don't do that stuff - just write test software, do testing, and build things as needed to facilitate testing. 


If I pull lenses from the incoming shelves (on which there are about 500 lenses today) I screen them on OLAF, which is perhaps 5x as sensitive as a camera and is faster - it takes 10 seconds per copy.  A copy I deem as bad gets camera checked and if it passes, it gets tested.  The ISO 12233 method we use is at least as sensitive as what most camera manufactures are using, so I do not think we are setting the bar artificially low for them. 

 

Regarding copy age again - I can tell you that on average the 50 STMs included in this grouping rented 6 weeks more than the 50/1.8Gs tested here.  In fact, 8 of those Gs were never rented before. 

 

Regarding regional differences - the manufactures do not ship different batches to different regions or any of that nonsense.  The biggest regional differences are the level of service offered and the skill of those technicians.  If you are in Australia, CPS will not have as good a service as on east coast USA, for example.  The customer base isn't there to support it. 


If you return a lens to a manufacture, they will test it more thoroughly than it was tested on the production line.  If it fails a quick service attempt is made, if that does not work it will be scrapped for parts, the parts checked to be working individially, and working parts cleaned and returned to production lines.  Everything else is trashed. 

 

It is far faster to test many copies of the same model at once (minimal if any re-alignment between lenses) so with future lens releases we should be pulling 4-10 copies on arrival to test.  It depends on the focal length greatly - an 85 takes about 5 minutes.  A 14 takes about 20.  One ties up the copies all day, one ties them up for an hour. 

 

At this point we're talking about what we're going to do for the rest of the summer.  Primes are basically as done as I care to finish them (about 675 copies - 50+ models in the database - all of the ones lensrentals carries good stock of in EF or F mount completed) and we can either start zooms, or start stopped down data collection.  The 70-200 IS IIs took about 2 days, but I worked on a number of other projects and did not test them as fast as I could have.  Something like a 16-35 f4 IS would take the entire week, and god is it maddening to take a distortion profile for each and every copy to facilitate measurement. 

 

We are also working with a partner to put up a proper database online of the charts with a comparison tool as well.  As of now we are working with the MTF charts, but variance should be done as well.  Later the 3D plots I linked above are something we may do too.  I hope so, it took quite a bit of effort to program them.  I think they may make the data more intuitive for people who don't speak MTF.

 

It is essentially this:

 

https://www.dropbox.com/s/3e0l67u59p4981..._.png?dl=0


vs this+this:

 

https://www.dropbox.com/s/8ziuhp3yuhnwg9...O.pdf?dl=0


https://www.dropbox.com/s/iwc5ovnobivu7r...O.pdf?dl=0


Or, flat


https://www.dropbox.com/s/6d899z4dukebln...p.pdf?dl=0

 

https://www.dropbox.com/s/q0tp4cvbyc5f3d...p.pdf?dl=0

 

While we won't stick our foots in our mouths and state it officially and explicitly, the 10lp/mm chart does show contrast while the 30lp/mm chart shows the highest resolution that can be recorded by most FF cameras.  40 or 50 lp/mm would be needed for something like M4/3.

[davidmanze]

Dave, how much would you bet you got a perfect copy of that fisheye?  I bet it has issues a camera sensor can't see! =) 


Seriously, I've measured 12 copies of the 14mm, 10 of the 24, 10 of the 35, 10 of the 50, and 10 of the 85 and I have not once seen a samyang without some problems.

  It's enough of a gamble in itself just buying a lens without laying out bets on top, Ladbrooks are very wise keeping their nose out of the lens industry!   

 

As it happens I bought t from England from a store destocking Pentax for £150 . I dare say it you delved into it you could surely find some glitch or another, but as maybe you've gathered I don't settle for decentered lenses and it that department I can't complain. It isn't a sharp lens to the edges till F8-11, but overall it's  a goodie. Also the Pentax mount conserves the automatisms from the body, no AF of course.