There is ample research and articles on the roles and effects of each of the five human senses (vision, auditory-hearing, olfactory-smell, taste and touch) on perception and higher-level cognition, emotions, decision making etc. But the senses very often do not operate in isolation from each other — two or more senses can be activated together (e.g., sight and sound, sight and touch, smell and taste) in exposure to stimuli, and they create integrated sensorial and perceptual experiences. There is research also on joint effects of different senses, but the greater part of it is still concerned with a single stimulus (e.g., product item) at a time. A research question that receives too little attention is: What if feeling an object with one of the senses interacts with applying another sense to other objects, and affects the response to them (e.g., eliciting preferential bias and choice)?
Researchers Mathias Streicher and Zachary Estes (2016) investigated this possibility of joint effects of a couple of senses on different product items: How the touch of one product item interacts with the sight of other product items competing for consumer choice? They hypothesise that the haptic exposure (touch) to an object can facilitate the visual processing and choice of other seen products of the same shape and size.
Information from two sensory inputs can complement each other, and reinforce or enhance the perception of an object when the sensorial inputs add together. When applied to a particular product item, the sight (image) of that object — its size, shape and texture — can help an individual to predict how it would feel when holding and touching that item in one’s hand, before even approaching it physically. On haptic contact with the product item, the image information may enhance the haptic (touch) experience, or it could require correction and update of the object’s perception. Vice versa, however, when the sight of an item is impaired (e.g., temporarily due to bad lighting conditions, permanently due to visual disability or impairment), touching and handling an item may allow an individual to ‘build’ an imagery of how the item would look like.
Yet Streicher and Estes test something different as they separate between the sensorial inputs, that is of the product item one holds (without seeing it) and the product items he or she watches and responds to. They propose that a haptic exposure to a product item will positively prime the choice of another seen product item if the latter has greater similarity in shape and size to the product object touched.
- First, Streicher and Estes show that the visual fluency of observing a product package is improved when the package seen matches in its shape and size the package grasped by the hand (the options used were a can and a bottle, both of Fanta soft drink). When the haptic feeling of a container package activates a shape representation that aligns with visual shape representations of other objects, it can facilitate the visual processing (fluency) of the package image of the similar shape. Unlike in the visuo-haptic condition-group, when no object is given to grasp (visual-only), no significant difference was found between the containers (bottle and can) shown.
- Second, the container seen which matches the container grasped and touched (either can or bottle) was chosen more frequently (60%) than the other type of container package. That advantage was not found in the visual-only condition-group (note: the matching container was always shown to the right; in the visual-only group, the container shown on the right side was chosen by only 43% of participants).
- Furthermore, the researchers confirm that the indirect (positive) effect of haptic exposure to a product on the choice of another seen product is strongly mediated by visual fluency. In other words, the improved visual fluency of the product, that would seem to feel similar by touch to the one grasped, favours the choice of the seen product.
The first experiment performed by the authors leaves two caveats. First, the product packages used, as held by and shown to participants, were from the same brand; this was done to exclude the effect of possible differences in brand preference of the products visually displayed. Yet, in actual shopping situations, such as in a store, a consumer may be holding a product of any brand while looking at products of competing and moreover different brands on the shelf. We do not get a clear answer if the preference of a product can be biased merely since it seems to feel similar by touch as the product being held in hand, and thus overcome existing brand preferences. Second, the products visually displayed are shown as pictorial images on a computer screen. That may be regarded as an isolated, easy condition for viewing and choosing products compared with the more crowded shelf displays and ‘noisy’ environments that are frequently found in-store (e.g., supermarkets, department stores and other large stores).
We may find at least a partial answer on the issue of crowded displays in the second experiment. In that experiment the researchers tried displays of different density: (a) sparse display with two package items — a bar and a square of Milka chocolate snack; (b) a crowded display comprising an arrangement of 18 Milka bars and 18 Milka squares. Striecher and Estes posited (following optimal integration theory) that haptic and visual information contribute to object recognition in a compensatory manner: “As visual perception becomes less reliable, haptic perception assumes a greater role in the recognition of object shape“. Visual perception may be degraded, for instance, due to difficulty to discern details or when parts of the object are obscured in a crowded display; haptic information should then receive greater weight in figuring the object’s shape.
They found that a very high proportion of participants chose an item that matched a product item held in their hand from a crowded display (a bar or square of Milka), higher than when choosing from a sparse display, especially when participants had a high need for touch as a source of information (instrumental NFT) which is of greater relevance when encountering a crowded display. When a product of completely non-matching shape (a Kinder chocolate egg) was grasped in hand, there was little significance or no advantage to which package of Milka, a bar or square, was in the right position of the display (the position reserved when a matching object is given). The results show, Streicher and Estes conclude, how the effect of haptic information on product choice is accentuated in crowded displays, which may overload the visual system and thus require greater reliance on haptic (touch) information.
It should be noted that the ‘crowded display’ was nonetheless emulated in a rather isolated and ‘perfected’ setup, so it is an approximation of what consumers might experience in an actual in-store setting. Also note that participants did not see what product and brand they were holding, particularly those holding a non-matching product were not made aware it was of Kinder. In a real shopping situation, consumers may be holding a product they picked-up, and then continue looking at other items on shelf display while either looking again at the item in their hand (e.g., for comparison) or ignoring it. The interesting notion that arises from the research of Streicher and Estes is that even when shoppers may be ignoring what product of which brand they are grasping in their hand, their choice may still be unconsciously influenced by the touch feeling of shape and size of the product item kept in their hand — they would be biased to choose another product that visually appears similar in kind. Yet, we still cannot tell if this effect can overcome the identity of brands the consumers see in-store and their preferences for them.
The research relates to an issue of differentiation in product management: should a company (manufacturer) design a product or packaging form that is distinct or more similar to products of competing brands. The researchers suggest that later entrants in a category may gain more by making their product packaging more similar in aspects relevant to touch such as size and shape to existing product alternatives. However, we should acknowledge that confusion can happen both ways, and while it may at times “steal” choices from leading competitors, the product of the late entrant may be completely lost and left unnoticed in the display. The choice of consumers-shoppers remains dependent also, and perhaps first, on other properties and details of visual appearance (e.g., colour & labels), and on brand preference.
The research of Streicher and Estes brings to surface intriguing aspects of a visuo-haptic experience, especially about the interplay that can occur between haptic (touch) and visual information. They show the effect that touch of one product item can have on choice between products visually observed, where this effect may also occur unconsciously (i.e., the identity of the product consumers hold and touch is dissociated from the products they see). There are some important factors that need to be addressed with respect to real shopping environments on which we are left with unclear answers. Yet, this research highlights the core phenomenon, and can give directions where and how to continue investigating.
Reference:
Multisensory Interaction in Product Choice: Grasping a Product Affects Choice of Other Seen Products; Mathias C. Streicher and Zachary Estes, 2016; Journal of Consumer Psychology, 26 (4), pp. 558-565
Consumer / Shopper / Customer Behaviour 