Haptic - Visual

In the literature, most discussions on incongruence of sensory modalities in study and

recognition would deem insignificant or has lesser effect on recognition compared to

congruence. The low scores reflected that it is relatively better to study with congruent sensory

modalities. In an experiment, children made more correct responses on the haptic-haptic than on

the haptic-visual sequence with both kinds of objects, whether it is familiar or unfamiliar objects

(Bushnell and Baxt, 1999).

Bushnell and Baxt (1999) studied on the relationship of haptic and visual with object

recognition. With touching as a way to study and visual to test recognition, the evidence showed

that respondents scored lower. As in Experiment 1 of their study, children made more correct

responses on the haptic-haptic sequences than on the haptic-visual sequence, as mentioned.

Additionally, 4 of the respondents tied their worst performance on the haptic-visual sequence

with familiar objects.

There were more studies regarding haptic and visual study. When modalities were

congruent, there was less false recognition than when they were incongruent. After haptic study,

false recognition rates in the haptic test condition were lower than those in the visual test

condition (Nabeta & Kahawara, 2006). This specifically means that there will be better results if

there is congruency in sensory modalities, as emphasized. It has been suggested that presenting

study and test items in congruent modalities is a key variable that facilitates retrieval of the

perceptual cue (Morris, Bransford, & Franks, 1977; Tulving & Thomson, 1973).

Performance on the haptic-visual sequence with unfamiliar objects was significantly less

accurate than on the haptic-haptic sequence. This result reinforces the earlier conclusion that
cross-modal recognition is not merely constrained by imprecise haptic abilities. We have argued

that this drop-off in performance across modes may be due to a conflict of interest between

vision and touch, in which the properties of an object that are salient and therefore encoded

during haptic exploration are not necessarily the same as the properties that are salient during

visual exploration (Bushnell & Bast, 1999).

However, there was an additional discussion with Nebeta & Kahawara (2006). Since the

participants visually recognised haptically studied objects with very high accuracy (about 93%),

the transfer from haptic study to visual test indicates that they were able to identify the real

objects haptically during the study phase. If the participants had been unable to identify the real

objects haptically, they would have been unable to visually recognise the studied objects. These

results, therefore, validate the assumption that the participants were able to identify the objects

haptically (Nabeta & Kahawara, 2006). Familiarity of the object can become a mediator in

visually recognizing an object even if it is studied through touch only. This mean that there can

be different results to these studies considering different variables such as, as mentioned,

familiarity.
 Haptic – Haptic + visual

What aspect of haptic processing is the critical one? Is it the three-dimensional aspect of

the object, the characteristics of the objects, such as texture and color, or is it in some way

related to the functionality or affordance of the objects themselves (Gibson, 1966)? Many

researchers tried to answer this particular question and it aims in focusing in the modality effect

of touch.

The researches on haptic touch and its modality effect mostly focused on haptic touch

alone and its effect if congruent. There must be more involved in the superiority of haptic

processing than simply imaging, since the vision-and-haptic condition also was better than

vision-only. The motor component may indeed provide an added benefit as speculated by

Wippich (1991), as cited by Stadtlander et. al. (2014).

Previous researchers have demonstrated that recognition memory is better when both

study and test objects are presented haptically (Bushnell & Baxt, 1999; Reales & Ballesteros,

1999), instead of being presented in different modalities. For example, Bushnell and Baxt (1999)

showed that recognition was better when both presentation modalities of the study and test

stimuli were haptic than when they were different. The improvement in memory performance as

a result of haptic congruency may also apply to false recognition and, if so, false recognition

should be reduced more when study and test modalities are congruent (e.g., both haptic or both

auditory).

Additionally, Nabeta & Kahawara (2006) predicted that false recognition would be

reduced with haptic study, because haptic presentation provides distinctive object characteristics

(e.g., Klatzky, Lederman, & Metzger, 1985; Klatzky, Loomis, Lederman, Wake, & Fujita, 1993).
If distinctive features encoded through haptic presentation of objects help participants to

differentiate lures from studied items, the reduction in false recognition will be greater following

haptic study than following auditory study. To examine these possibilities, they tested haptics

alone.

This review of the literature shows that studies and experiment on sensory modalities and

its effect of object recognition highly suggest to present stimuli and test it with congruent

modalities. Haptically explored and visually and haptically recognized is a condition that is not

quite studied. However there is a hypothesis that since haptic is as superior as visual, recognizing

the object visually and haptically would result to higher performance on object recognition

(Ballesteros, 2008).

Haptic + Visual – Visual or Haptic

The haptic system was considered inadequate for identifying objects for a long time,

especially when compared to the visual system, for example, which acquires information through

multiple parallel channels, providing information about luminance, color, movement, and depth

(Ballesteros, 2008; Ballesteros, Reales, & Manga, 1999; Klatzky et al., 1985). Therefore, it was

neglected the fact that the tactile sensory system involves the acquisition of several different

information through several characteristics of the objects as weight, texture, temperature,

pressure, and the like (Martinovic, Lawson, & Craddock, 2012). Thus, it is noticed that haptic

perception is an independent system from visual perception, as well as it is neither secondary nor

inferior to it (Ballesteros, 2008). According to Révész (1950), the haptic system demonstrates
some independence from the visual system, being guided by its own principles (Gadelha et. al.,

2013).

The behavioral evidence, then, suggests that vision and haptics represent the shape of

objects in the same way. It is possible, therefore, that these two sensory systems could also share

a common neural substrate for representing the shape of objects. Three studies suggest that the

neural substrate underlying visual and haptic object recognition is found within extra-striate

cortex (James et. al, 2002).

Being said, if there is combination of visual and haptic as study phase, how would the

performance on test phase result? Sadly, there isn’t much literature also in combination of senses

in object recognition or false memory. Moreover, the researchers still would like to see results if

visual and haptic study phase would result to a better object recognition haptically or visually.

Sources:

Bushner and Baxt

Nebeta & Kahawara

Gadelha et, al

Stadtlander et. al,

Ballesteros, S. (2008). Implicit and explicit memory effects in haptic perception. In M.

Grunwald (Ed.), Human Haptic Perception: Basics and Applications (pp. 207-222). Boston:

Birkhäser Verlag.
 Ballesteros, S., Reales, J. M., & Manga, D. (1999). Implicit and explicit memory for

familiar and novel objects presented to touch. Psicothema, 11, 785-800.

Gibson,J.J.(1966). The senses considered as perceptual systems .Boston:Houghton

Mifflin.

James, T. W., Humphrey, G., Gati, J. S., Servos, P., Menon, R. S., & Goodale, M. A.

(2002). Haptic study of three-dimensional objects activates extrastriate visual areas.

Neuropsychologia, 40(10), 1706-1714. doi:10.1016/s0028-3932(02)00017-9

Klatzky, R. L., Lederman, S. J., & Metzger, V. (1985). Identifying objects by touch: An

‘expert system’. Perception & Psychophysics, 37, 299–302. doi:10.3758/BF03211351.

Loomis, J. M., Klatzky, R. L., & Lederman, S. J. (1991). Similarity of tactual and visual

picture recognition with limited field of view. Perception, 20, 167±177.

Martinovic J., Lawson, R., & Craddock, M. (2012). Time course of information

processing in visual and haptic object classification. Frontiers in Human Neuroscience, 6, 1-11.

doi: 10.3389/fnhum.2012.00049

Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus

transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16, 519±533.

Reales, J. M., & Ballesteros, S. (1999). Implicit and explicit memory for visual and

haptic objects: Crossmodal priming depends on structural descriptions. Journal of Experimental

Psychology: Learning, Memory, and Cognition, 25, 644±663.

Révész, G. (1950). Psychology and art of the blind. London: Longmans, Green.
 Wippich,W.(1991).Hapticinformationprocessingindirectandindirectmemory tests.

Psychological Research , 53, 162-168.