If you often shop online, you must be familiar with one of the product descriptions: "Tip: Due to factors such as shooting light and display, it is unavoidable that there is a color difference in the product picture, please refer to the actual product you receive."

In the era of digitalization, pictures and videos are replacing text and becoming the main way for people to record and share; while screens are also replacing paper as the main medium for people to obtain information. As a result, people surrounded by digital images are increasingly intolerant of "color differences"

The design of the computer is blue, how does it look purple when it is replaced by a mobile phone? The gray pants you bought on the phone are green when you see them in your hands. 

Recently, the research team of Qiu Min, Chair Professor of Optical Engineering at Westlake University, published the latest research results entitled "Nonlinear Color Space Coded by Additive Digital Pulse" in the international optical journal Optica. They propose to use the color space based on human eye perception instead of the color space based on the physical device itself to solve the problem of color distortion in the process of preservation and reproduction.

That is to say, they endow the color with a new set of "coding" - not only can it solve the color difference between different monitors to the greatest extent, but also solve the "color difference" between the color seen by the human eye and the display of the machine. Do "seeing is believing".

Tang Ni, a 2019 joint Ph.D. student of Westlake University and Zhejiang University, is the first author of this paper, and Dr. Jiyong Wang and Professor Min Qiu, Chair professor from the school of Engineering of Westlake University, are the co-corresponding authors.

Let's see how they "eliminate" chromatic aberration.

A lamp's confusion

        At first, the research team members did not focus on color, they just wanted to develop a lamp.

It could be a red and blue light therapy lamp used for cosmetic purposes, or a sleep lamp used to regulate mood and rhythm ...... However, the team found that no matter what kind of lamp is made, the color of the light needs to be adjusted precisely, such as the need to ensure the illuminance, color coordinates, and color temperature.

        Blue is blue, not bluish; red is red, not reddish. Surprisingly, this seemingly basic core element is almost impossible to achieve under existing technical conditions.

        The problem lies in the color.

        Color is an old and fresh proposition. We know that color is the perception and cognition of the visible spectrum by the human visual system; however, the popularity of electronic devices has brought new troubles to the issue of color, the screen on the secondary presentation of color often "what you get is not what you see". How to record and reproduce a colorful world indiscriminately on imaging and display devices, has become a big problem.

        Taking the LED display screen as an example, the color that the human eye sees on the screen depends on the red, green, and blue light sources used by the LED. To allow the machine to "tune" as many colors as possible using these three light sources, someone invented a set of color "codes": red light for the X-axis, green light for the Y-axis, blue light for the Z-axis, we put this the coordinate system is called the "color space".

In this space, each color is defined as a digital coordinate composed of three numbers, representing a fixed mix of three light sources. For example, what we generally call red has its digital coordinates fixed at (255, 0, 0), which means that the red-light source is turned on to its brightest, while the blue and green light sources are turned off.

This digital color space is usually referred to as RGB. 256 levels of RGB are calculated to be able to mix about 16.78 million colors with red, green, and blue light sources, i.e., 256×256×256=16777216. We can think of it as a huge table, where each color has a fixed coordinate value that can be found somewhere in the table.

At present, most of the monitors used by people use the RGB color space. so where does the color difference come from?

To trace the root requires traceability. Due to technical means, production standards, and other restrictions, the red, green, and blue light sources used by different manufacturers have chromatic aberrations themselves, which also causes other colors based on them to show more or less "inaccurate". It is not difficult to imagine that the same picture in cell phones, tablets, computers, and outdoor advertising screen display, due to the chromatic aberration of the light source, the displayed picture must have chromatic aberration, for the cross-media adaptation of the design of inconvenience.

"Just like when we paint, we obviously mix the paints according to the standard proportion, but if the pigments used come from different batches, different origins and different brands, the colors themselves will be different and it is inevitably impossible to guarantee that the colors mixed out are the same every time." Qiu Min said.

So, starting from the brainstorming of a lamp, Qiu Min's group focused on how to fundamentally calibrate the color.

"Colour" from nature

How to calibrate it?

CIE is the abbreviation of the International Commission on Illumination, which gathered a group of color experts decades ago to develop a set of color "standards" that we call CIE. CIE is also a system of coordinates, with the special feature that it quantifies the colors seen by the human eye.

If RGB is a device-related digital color space, the colors it presents are directly related to the red, green, and blue light sources used by the device itself. Then CIE is different in that the colors it presents are not related to the device light source, but only to the human visual perception, and is a color space that integrates multiple dimensions such as color coordinates, color temperature, and luminance.

We can imagine CIE as the palette of nature, which restores as much as possible the colorful colors that the human eye sees in the natural world.

Because of this, even for monitors using RGB, to make the display effect as close as possible to the human eye perception, so that the "red" on the screen is as close as possible to the "red" that people see in the world outside the screen, it is necessary to be designed, calibrated, shipped and calibrated against CIE to correct the color.

As a result, the research team shifted its focus from RGB to CIE. Instead of calibrating back and forth between RGB and CIE, "why not skip the RGB, skip the huge table, and 'take the color' directly from the CIE?" Wang Jiyong said.

With such thinking, Qiu Min's group finally proposed a complete set of digital color coding and decoding algorithms based on CIE color space after rigorous theoretical derivation and experimental verification.

Let's take the red color as an example. According to the current conventional practice in the industry, red, green, and blue light sources are used to derive the final red color according to the mixing ratio of red (255,0,0) in RGB. The result is likely to be that the red of manufacturer A is a little redder than the red of manufacturer B (as shown in the figure below).

Switch to the algorithm researched by Qiu Min's team. We assume that the red, green, and blue light sources used remain the same, but instead of using the RGB table to "tune" the color, the team directly finds the red color perceived by the human eye in nature's color palette, the CIE coordinate system, we call it the "reference red". The role of the algorithm is to calculate a string of digital signals representing the "base red" in combination with the actual light source, which is equivalent to giving the "reference red" a "code". When the light source performs this encoding, it will appear as the red color perceived by the human eye.

Repeating the same steps, the research team located the "reference red", "reference green" and "reference blue" in the CIE color space and calculated the corresponding codes. With this set of "primary colors", the research team can encode arbitrary colors within the CIE space.

As long as the future display adopts this set of algorithms, regardless of the number of differences in the use of red, green, and blue light sources, the final display is taken from the CIE color space, and infinitely close to the perception of the human eye. In this way, not only will there be no chromatic aberration between different devices, but the world inside and outside the screen will no longer have chromatic aberration.

Of course, before this special display is developed and popularized, this algorithm can also be used on the display devices we are currently using. Whether combined with display hardware or design software, this algorithm can "back-propagate" and correct RGB coordinates based on the results of color extraction and calculations on the CIE.

From the user's point of view, if you are a designer and you decide to go with red, you have routinely selected (255, 0, 0), but inside the device, the coordinates may have been quietly moved to (254, 1, 1).

When you paint a landscape on the computer, the "straw" used to pick up the color can be directly from nature's red flowers, green leaves, and blue sky to pick up a touch of primary color, is not cool?

Possibilities other than "chromatic aberration"

The new algorithm developed by Qiu Min's group offers a new solution to the long-standing problem of "chromatic aberration". But what excites them is not the algorithm itself, but the possibilities beyond "chromatic aberration" when we can accurately grasp the light behind the color.

Let's look at them one by one.

Not long ago, Shenzhou 12 manned spacecraft carrying three astronauts flew to space, successfully staying in the Tianhe core module. We know that the biological clock is very critical to maintaining the physical and mental health of the human body, and light can affect the biological rhythm, which is one of the most important factors affecting the biological clock. Imagine if we can simulate the same light conditions as the Earth's life so that astronauts in space to enjoy the regular life of "sunrise and sunset," how wonderful it would be.

From the "heaven" back to the "underground". A system that precisely controls light can also make a difference in the planting industry. For example, bayberry, one of the main economic crops in Zhejiang Province, has its yield, size, and sweetness inseparable from light. Assuming that we obtain the light data of the harvest year of bayberry through research, and then use this system to simulate and "copy" it, we can taste delicious bayberry every year.

In addition, this algorithm is not only applicable to LED light sources, but also can be used for digitally controllable lasers. Lasers are widely used in lighting, display, and scientific research due to their high brightness and good monochromaticity, and the research team has developed some new applications based on laser properties.

For example, when the encoded laser passes through nanoparticles of different particle sizes, we can deduce the exact size of the nanoparticles by the subtle difference in color, with an accuracy of up to 10 nm. This provides a new idea for fast identification and ultra-sensitive sensing of nanoparticle particle size.

In addition, this research also has promising applications in laser communication, artificial vision, virtual/augmented reality, and multi-wavelength light-matter interaction. For example, the augmented reality glasses of Microsoft Hololens2 are using 3-wavelength laser color mixing to get a frame of color images. Using Qiu Min's group's algorithm, chromatic aberrations can be further eliminated and ultimately enhance the sense of reality.

"The greatest significance of this research is that it is both interesting and useful." Qiu Min painted a picture of the future for us with CIE as the keyword:

We will use a CIE photodetector-based camera or cell phone to record life moments, and the captured images will be automatically stored as a picture file with the CIE format suffix, so that we can later relive the recorded moments with a CIE cell phone or CIE monitor. The images initially captured with the eye are captured, imaged, and multi-screen displayed, and everything is still perfectly reproduced.

For both academia and industry, this has the potential to be transformative.

 Reprinted from: Westlake University