Visual Perception of Material

We can easily judge material properties of a surface such as color, glossiness, and translucency. It has long been a mystery how the brain achieves this job.

Lightness and Glossiness

We recently found that the human vsual system estimates the glossiness and lightness of natural surfaces using simple statistics of the 2D image that can be extraced by low-level neural mechanisms.

The image histogram of a glossy and dark surface tends to be skewed toward the brighter direction, but the historgram of a matte and light surface does not. The perceived glossiness and lightness of various natural surfaces are well correlated with, and controlled by, skewness in the image histogram. Skewness can be computed by low-level neural networks in the brain. A new visual illusion (glossiness aftereffect [demo movie]) demonstrates a link between the output of such mechanisms and the perceived glossiness and lightness.

Transparency and Translucency

Many natural surfaces such as those of foods and human skin have translucent / transparent properties. The perception of these complicated properties could also be determined by relatively simple image features.

Translucent surfaces (middle) tend to have shading patterns that are faint and blurred as compared to opaque ones (left). For more transparent surfaces, shading patterns are even reversed (right). In contrast, the pattern of highlights is constant. This mismatch between highlights and shading patterns can be used as a cue for the perceived translucency. In fact, one can alter an opaque surface into a transparent one by reducing or revsersing the contrast of shading patterns.

Image-based material control

Understanding the use of simple image cues in material perception enable us to control the apparent material by simple manipulations of image features.

The left object (*1) looks opaque like a stone, the middle translucent like jelly, and the right metallic like chrome. In fact, the middle and right objects were made from the left image through a simple manipulation. These demonstrate image-based control of apparent materials, and support the notion that the perception of various surface properties is largely determined by low-level neural representations.

*1 The left object was rendered using a 3D model by Stanford Computer Graphics Laboratory．

References

Motoyoshi, I.& Matoba, H. (2012). Variability in constancy of the perceived surface reflectance across different illumination statistics. Vision Research, 53, 30-39.

Yang, J., Otsuka, Y., Kanazawa, S., Yamaguchi, M.K. & Motoyoshi, I. (in press) Perception of surface glossiness in 5- to 8- month old infants. Perception, 40, 1491-1502.

Yoshida, K., Motoyoshi, I., Fukuda, K., Uchikawa, K. (2011). Visual perception of surfaces with transparent layers. Journal of Vision, 11(15): 65 [Optical Society of America Fall Vision Meeting].

Motoyoshi, I. (2010). Highlight-shading relationship as a cue for the perception of translucent and transparent materials. Journal of Vision, 10(9): 6, 1-11 [PDF]

Motoyoshi, I., Nishida, S., Sharan, L. & Adelson, E.H. (2007). Image statistics and the perception of surface qualities. Nature, 447, 206-209.

Motoyoshi, I., Nishizawa, T. & Uchikawa, K. (2007). Specular reflectance and the perception of metallic surfaces. Journal of Vision 7, 451a [Annual Meeting of Vision Sciences Society 2007]