How can I simulate a bayer filter (or just RGB channels) using photoshop layers?

Early color photography exposed three black & white plates (film) one to red, one to green, one to blue. The exposures were made via Wratten 47B blue – Wratten 58 green – Wratten 25 red. These are the three “Tri-Color” filters used to make color separations. We ended up with three negatives that represented the three primary colors. These were printed via an enlarger onto film or paper. These are “narrow cut filters. The filters overlaid on the photosites must be these or their cousins: Wratten 98 blue, Wratten 99 green, Wratten 25 red. These were used on the enlarger to make separation exposures.

The Wratten catalog numbers come from the master filter maker Frederick Wratten (England firm of Wratten and Wainwright, purchased by Kodak 1912. The Wratten cat # retained. Today these filters are obtainable, likely the name Wratten has been dropped.

Any way, you can photograph three images via these filter sets. The Bayer matrix likely uses these same filter values. On the Bayer matrix, the proportion of red to green to blue filtered photosites is based on the sensitivity of the chip. You should be able to accomplish your task by making three images using these filters values.

When talking about blending the layers, creating 3 layers with red/green/blue data and choosing a blend mode of Difference can achieve what you want.

How can I essentially combine pure red, green, and blue info in 3 layers to create full color?

You have to start with pure 'Red', pure 'Green', and pure 'Blue' color information. But that's not what you can get from a Bayer masked sensor, since the actual colors of each set of filters are not 'Red', 'Green", and 'Blue'.

It's not what we get from the cones in our retinas, either.

Keep in mind that there's no specific color intrinsic in any wavelength of visible light, or other wavelengths of electromagnetic radiation for that matter. The color we see in a light source at a specific wavelength is a product of our perception of it, not of the light source itself. A different species may well not perceive wavelengths included in the human defined visible spectrum, just as many species of bugs and insects can perceive light in near infrared wavelengths that do not produce a chemical response in human retinas.

Color is a construct of of how our eye-brain system perceives electromagnetic radiation at certain wavelengths.

Our Bayer masks mimic our retinal cones far more than they mimic our RGB output devices.

The actual colors to which each type of retinal cone is most sensitive:

Compare that to the typical sensitivity measurements of digital cameras (I've added vertical lines where our RGB - and sometimes RYGB - color reproduction systems output the strongest):

The Myth of "only" red, "only" green, and "only" blue

If we could create a sensor so that the "blue" filtered pixels were sensitive to only 420nm light, the "green" filtered pixels were sensitive to only 535nm light, and the "red" filtered pixels were sensitive to only 565nm light it would not produce an image that our eyes would recognize as anything resembling the world as we perceive it. To begin with, almost all of the energy of "white light" would be blocked from ever reaching the sensor, so it would be far less sensitive to light than our current cameras are. Any source of light that didn't emit or reflect light at one of the exact wavelengths listed above would not be measurable at all. So the vast majority of a scene would be very dark or black. It would also be impossible to differentiate between objects that reflect a LOT of light at, say, 490nm and none at 615nm from objects that reflect a LOT of 615nm light but none at 490nm if they both reflected the same amounts of light at 535nm and 565nm. It would be impossible to tell apart many of the distinct colors we perceive.

Even if we created a sensor so that the "blue" filtered pixels were only sensitive to light below about 480nm, the "green" filtered pixels were only sensitive to light between 480nm and 550nm, and the "red" filtered pixels were only sensitive to light above 550nm we would not be able to capture and reproduce an image that resembles what we see with our eyes. Although it would be more efficient than a sensor described above as sensitive to only 420nm, only 535nm, and only 565nm light, it would still be much less sensitive than the overlapping sensitivities provided by a Bayer masked sensor. The overlapping nature of the sensitivities of the cones in the human retina is what gives the brain the ability to perceive color from the differences in the responses of each type of cone to the same light. Without such overlapping sensitivities in a camera's sensor, we wouldn't be able to mimic the brain's response to the signals from our retinas. We would not be able to, for instance, discriminate at all between something reflecting 490nm light from something reflecting 540nm light. In much the same way that a monochromatic camera can not distinguish between any wavelengths of light, but only between intensities of light, we would not be able to discriminate the colors of anything that is emitting or reflecting only wavelengths that all fall within only one of the the three color channels.

Think of how it is when we are seeing under very limited spectrum red lighting. It is impossible to tell the difference between a red shirt and a white one. They both appear the same color to our eyes. Similarly, under limited spectrum red light anything that is blue in color will look very much like it is black because it isn't reflecting any of the red light shining on it and there is no blue light shining on it to be reflected.

The whole idea that red, green, and blue would be measured discreetly by a "perfect" color sensor is based on oft repeated misconceptions about how Bayer masked cameras reproduce color (The green filter only allows green light to pass, the red filter only allows red light to pass, etc.). It is also based on a misconception of what 'color' is.

How Bayer Masked Cameras Reproduce Color

Raw files don't really store any colors per pixel. They only store a single brightness value per pixel.

It is true that with a Bayer mask over each pixel the light is filtered with either a "Red", "Green", or "Blue" filter over each pixel well. But there's no hard cutoff where only green light gets through to a green filtered pixel or only red light gets through to a red filtered pixel. There's a lot of overlap.² A lot of red light and some blue light gets through the green filter. A lot of green light and even a bit of blue light makes it through the red filter, and some red and green light is recorded by the pixels that are filtered with blue. Since a raw file is a set of single luminance values for each pixel on the sensor there is no actual color information to a raw file. Color is derived by comparing adjoining pixels that are filtered for one of three colors with a Bayer mask.

Each photon vibrating at the corresponding frequency for a 'red' wavelength that makes it past the green filter is counted just the same as each photon vibrating at a frequency for a 'green' wavelength that makes it into the same pixel well.³

It is just like putting a red filter in front of the lens when shooting black and white film. It didn't result in a monochromatic red photo. It also doesn't result in a B&W photo where only red objects have any brightness at all. Rather, when photographed in B&W through a red filter, red objects appear a brighter shade of grey than green or blue objects that are the same brightness in the scene as the red object.

The Bayer mask in front of monochromatic pixels doesn't create color either. What it does is change the tonal value (how bright or how dark the luminance value of a particular wavelength of light is recorded) of various wavelengths by differing amounts. When the tonal values (gray intensities) of adjoining pixels filtered with the three different color filters used in the Bayer mask are compared then colors may be interpolated from that information. This is the process we refer to as demosaicing.

What Is 'Color'?

Equating certain wavelengths of light to the "color" humans perceive that specific wavelength is a bit of a false assumption. "Color" is very much a construct of the eye/brain system that perceives it and doesn't really exist at all in the portion of the range of electromagnetic radiation that we call "visible light." While it is the case that light that is only a discrete single wavelength may be perceived by us as a certain color, it is equally true that some of the colors we perceive are not possible to produce by light that contains only a single wavelength.

The only difference between "visible" light and other forms of EMR that our eyes don't see is that our eyes are chemically responsive to certain wavelengths of EMR while not being chemically responsive to other wavelengths. Bayer masked cameras work because their sensors mimic the trichromatic way our retinas respond to visible wavelengths of light and when they process the raw data from the sensor into a viewable image they also mimic the way our brains process the information gained from our retinas. But our color reproduction systems rarely, if ever, use three primary colors that match the three respective wavelengths of light to which the three types of cones in the human retina are most responsive.