Camera Labs

Hi folks,

Here's a question I can't answer. The traditional Bayer sensor, as found in most DSLRs and compacts these days, has twice as many green photosites as red or blue.

My question: "Is it inevitably true that making each photosite as sensitive as possible produces the best image quality?"

What if each green photosite had half the sensitivity of the red and blue photosites? Well, some simple math (probably too simple) would suggest that the overall sensitivity of the sensor would be reduced by 25%. That's workable (c.f. Sony's translucent mirrors) but the benefit is that those green pixels would each take twice as long before they saturated as the full-well capacity wouldn't have changed. Some clever binning where adjacent green pixels could have their signal combined might help restore the green signal to noise ratio though the S/N ratio would still be worse than having green photosites with maximum sensitivity. I'm guessing that there would be a very slight loss in resolution in dimly lit areas of a photograph and a loss of colour information in the very bright areas although those bright areas would still have image detail that would have been lost had the green photosites been at full sensitivity and had thus saturated. Obviously some smart de-bayering would have to be employed to avoid colour tints in very bright or very dim areas of a scene but another guess on my part is that this is quite possible to do.

I generally find that if something as simple as my suggestion above hasn't been done there's a "gotcha" which means it can't be done. Maybe it has been done, or partially done, but nobody has admitted it? As to why one would want to do it, well, if the manufacturers are going to cram much more than, say, 20 MP into an APS-C sized sensor then dynamic range may suffer as there's less silicon available to store the electrons released during an exposure. This may be a way to retain more dynamic range as pixel densities continue to climb.

Kodak have gone the other way with some of their sensors and substituted clear lenses for half of the green ones in an effort to make their sensors more sensitive but has anyone tried the trick of making the green photosites less sensitive even if the reduction in sensitivity is less than the 50% I mentioned at the start?

Bob.

_________________
OM-D E-M5 + ED 12-50mm, H-PS14042E, H-F007014E, L-RS014150E, M.ZUIKO DIGITAL 45mm 1:1.8.
Gitzo GT1541T, Arca-Swiss Z1 DP ball-head.
Astrophotography: TEC 140 'scope, FLI ML16803 camera, ASA DDM60 Pro mount.

I'm not even remotely qualified to answer your question. But what the heck!

Does making the sensor as sensitive as possible improve the image quality? I don't know. I know jacking up the ISO sensitivity makes my 50D look like crap.

As far as the 50% of the bayer sensor being green. It kinda makes sense. I read somewhere that they use extrapolation of adjacent colored cells to reconstruct the full color image with bayer sensors. Well, since green light is a wavelength in between red and blue, maybe it makes green more ideal. I'm talking out of my arse here.

Bob, if I'm reading you right, you're just de-sensitising the green to give it more headroom? Why wouldn't you do the same to red/blue?

Taking that idea more generically, I have wondered what would happen if, for example, you made 50% of all photosites less sensitive. Then I remember Fuji done that arrangement already.

Personally I think high sensitivity is the way to go. It is easier to reduce exposure than increase it.

My variation on that would be to have a B+G, R+G, R+B matrix, which could be approximated as CMY. The differing sensitivities to colour could still be used to reconstruct RGB image. e.g. for R, assuming you have a trio of values, (R+G) + (R+B) - (B+G) = 2R. Depending on the layout you might need to apply a bit of scaling, but overall you've got roughly double the sensitivity of a RGB filter. I think the possible issue here is colour space conversions, as I think there is some non-exact mapping between CMY and RGB. But I think providing the working gamut is sufficient that shouldn't be an issue. In a quick search, apparently there is a CMGY filter used in the past. I don't even want to think about how you translate that to RGB!

_________________
Canon DSLRs: 7D, 5D2, 1D, 600D, 450D full spectrum, 300D IR mod
Lenses: EF 35/2, 50/1.8, 85/1.8, 135/2+SF, 28-80 V, 70-300L, 100-400L, TS-E 24/3.5L, MP-E 65, EF-S 15-85 IS
3rd party: Zeiss 2/50 makro, Samyang 8mm fisheye, Sigma 150 macro, 120-300 f/2.8 OS, Celestron 1325/13
Tinies: Sony HX9V.

Hi popo,

My reasoning behind desensitising just the green sites is that as there are twice as many of them as there are red or blue it would be reasonable to recover some of the lost S/N ratio by some clever binning. While you could do that with the red and/or blue sites they are a lot further away from each other so, intuitively, there would be a greater loss of spatial resolution. With the green sites every green site physically touches another green site, albeit only at the corners.

Desensitising the whole sensor is equivalent to underexposing. As we all know that recovers highlight detail at the expense of shadow noise. That's got me wondering why, unless memory is playing tricks on me, chroma noise in shadows is deficient in green while high ISO chroma noise in the mid-tones is often most evident in green objects. Might that be something to do with in camera noise reduction via selective binning of red, green and blue photosites prior to debayering???

I'll confess I'm really struggling with this and as I'm down for kitchen duties for the rest of the day I'll have to hand over the baton to other forum members...

Bob.

_________________
OM-D E-M5 + ED 12-50mm, H-PS14042E, H-F007014E, L-RS014150E, M.ZUIKO DIGITAL 45mm 1:1.8.
Gitzo GT1541T, Arca-Swiss Z1 DP ball-head.
Astrophotography: TEC 140 'scope, FLI ML16803 camera, ASA DDM60 Pro mount.

Hmm, not sure I fully understand...
If you have one color desensitized wouldn't that mean that clipping occurs 1 stop later than with the other colors? That would then lead to color shift in highlights, wouldn't it?
Come to think of it: Plus a color shift in the shadows where the greens go to zero when the other colors still register photons.

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Hi Thomas,

You are right and I sort of alluded to it in my opening post. I'm assuming that if red and blue have saturated but green hasn't then one assumes white and just then just use the extra green headroom as a luminance signal to provide detail. If the green sites were at full sensitivity all the highlights would have blown anyway in that scenario so the image would just show white with no detail. Shadows may also be difficult but binning adjacent green pixels would get most of the signal back (per pixel ISO boost might help in conjunction with binning) although I'd guess there would be slightly more noise in the green channel after best effort processing. Could that be dealt with in firmware? Maybe.

At the end if the day this is just talk, unless one of the forum members is a closet sensor designer, but I thought i'd open up a discussion to see what members thought.

Bob.

_________________
OM-D E-M5 + ED 12-50mm, H-PS14042E, H-F007014E, L-RS014150E, M.ZUIKO DIGITAL 45mm 1:1.8.
Gitzo GT1541T, Arca-Swiss Z1 DP ball-head.
Astrophotography: TEC 140 'scope, FLI ML16803 camera, ASA DDM60 Pro mount.

Is light not made up with more green and if reducing green capture with reduce the capture of light?

_________________
Mark Osborne
My life through an iPhone

Isn't this what fuji did with the S5, for instance? Reduce the sensitivity of some sites to capture more highlight detail.

Light is made up of everything, really. If you have "white"(meaning the whole spectrum) light and shine that through a prism, you will see equal amounts of red, green, blue etc.

The reason the sensor has more green photosites, is that the human eye is more sensitive to yellow-ish green (~500nm) than to other colors. This is also the reason why green laser pointers are easier to spot than red ones.

_________________
I take pictures so quickly, my highschool was "Continuous High".

.
Marijn, you're a star! From this Luminous Landscape review of the FujiFilm FinePix S3 Pro:

The Fuji S3's claim to fame is its enhanced dynamic range through the use of its proprietary Super CCD SR II sensor, with two separate photo-sites – one to record the normal exposure range, and the other just the highlights.

The camera can be set to record just using the S (normal) pixels, or also with the R (highlight) pixels as well. These are then combined in the Raw file, and you have the choice within the Fuji Hyper Utility II Raw conversion software, or via in-camera settings if generating JPG files, to blend these either automatically, or in varying degrees.

Now, these extra highlight sensors don't come without a price. The size of the Raw file is doubled, from some 13MB when just the normal S sensors are used, to 25.5MB when they are integrated. Just by way of comparison, an 8MP Canon 20D's Raw files are about 8 MB in size on disk, while those form the 16MP Canon 1Ds MKII are about 18 Megabytes.

Not quite the same scheme as I proposed but it does demonstrate that it is possible to reconstruct an image despite having photosites of differing sensitivities and that one can increase the dynamic range in the process.

Does anyone know if Fuji, or others, are still using this technology?

Bob.

_________________
OM-D E-M5 + ED 12-50mm, H-PS14042E, H-F007014E, L-RS014150E, M.ZUIKO DIGITAL 45mm 1:1.8.
Gitzo GT1541T, Arca-Swiss Z1 DP ball-head.
Astrophotography: TEC 140 'scope, FLI ML16803 camera, ASA DDM60 Pro mount.

well Fuji aren't producing cameras anymore &...they were the only ones with this brilliant sensor.

I agree to what Marijn said,I knew about how the eye's perception of green & yellow. I wonder if wouldn't it be easier just to redesign from scrath the sensor. One think escapes me...why did the Fuji sensors stop producing? I mean,they managed to found a great formula for details in higlights.

_________________
Gear: Nikon D80, 18-105mm, 35mm F/1.8, 85mm F/1.8, SB-600
Visit my work here

University told me that we see more green than any other colour of light hence sensors are built to accommodate our eyesight.

_________________
Mark Osborne
My life through an iPhone

Hi Mark,

I remember being taught the same but so far as camera response is concerned here are the quantum efficiencies for the EOS 40D and 50D, courtesy of Christian Buil:

While those cameras are very slightly more sensitive to green than blue there's very little difference though, by eye, the area under the curves differs rather more. Getting a natural colour balance in the viewable images is the pixie dust applied by firmware and/or post processing.

Of course if my scheme of lowering the green sensitivity were applied an extra helping of pixie dust would be needed to get the colour balance back on track.

Bob.

_________________
OM-D E-M5 + ED 12-50mm, H-PS14042E, H-F007014E, L-RS014150E, M.ZUIKO DIGITAL 45mm 1:1.8.
Gitzo GT1541T, Arca-Swiss Z1 DP ball-head.
Astrophotography: TEC 140 'scope, FLI ML16803 camera, ASA DDM60 Pro mount.

Why Fuji stopped?

Well nobody bought it, because they stuck their sensors in outdated bodies. I think the S5 was based on the Nikon D200, but released when the D300 came out.

_________________
I take pictures so quickly, my highschool was "Continuous High".

If say a Nikon D800 would have the same Fuji sensor tehnology,I would buy such a camera without second thoughts. That would be the ultimate full-frame camera & I wouldn't care of the aqsual space the RAW files require. Just think of the off-the-charts dynamic range it would produce.
I don't know why Nikon wouldn't allow the Fuji sensors in their cameras. Does a CMOS sensor like the D7000's have a better ISO performance than the Fuji dual CCD? And/or better color representation? It should be a strong reason why Nikon didn't aquire these type of sensors for their own cameras.

_________________
Gear: Nikon D80, 18-105mm, 35mm F/1.8, 85mm F/1.8, SB-600
Visit my work here

It'd be a great camera for landscape photographers, or perhaps wedding photographers. It wouldn't work for me as a concert photographer (needing a high framerate and good noise performance).

_________________
I take pictures so quickly, my highschool was "Continuous High".