Hamza Naseer
Affiliation: Universita della Svizzera Italiana
Category: Philosophy
Keywords: Multimodality
Date: Tuesday 2nd of September
Time: 16:00
Location: GSSR Plenary Hall (268)
View the full session: Quantifiers, Plurals & Numbers
In the literature to date, amodal completion is presented as a phenomenon that occurs independently of sensory input. For example, when one sees a cat partially occluded by a fence, one still presents the cat as a whole object. In this case, there is no sensory input from the occluded bits - and yet we have a sense of a complete cat. If there is no sensory input contributing to this whole representation of a cat, then what is? One option is to argue that previously held beliefs participate in the act of amodal completion. For example, I think I see something that resembles a cat - and the previous cats I’ve seen are whole cats, so therefore, this must also be a whole cat.
This alternative view, which evokes previously held beliefs, faces a distinct challenge posed by Bence Nanay, namely that, “If amodal completion happens in the primary visual cortex, it is not happening on the level of beliefs/non-perceptual representations - it happens much earlier.” (Nanay, 2023, pg. 76) His argument is that amodal completion “in the early cortices happens within 100-200 milliseconds of retinal stimulation” (ibid) and “this is much shorter than the time that would be needed for perceptual processing to reach all the way up to beliefs or non-perceptual representations and then trickle all the way down again to the primary visual cortex.” (ibid).
However, Nanay’s proposal does not hold up, empirically speaking. Firstly, the time taken for amodal completion to occur in the early cortices can be up to 250 milliseconds, so it’s not limited to 100-200 milliseconds. That is not just a factual point, but also an important one in the context of perceptual processing - a mere 50 milliseconds could potentially allow for non-perceptual representations (or as I called them, previously held beliefs) to influence amodal completion in the primary visual cortex. Nonetheless, it turns out that previously held beliefs can influence amodal completion in as early as 150 ms! This is based on the Yun et al. 2018 experiment, which found that “knowledge has an influence as early as 150 ms after the onset of the prime.” (Yun et al., 2018, pg. 26). When subjects were presented with partly occluded objects (bananas, apples, etc.) that were well known, they amodally completed them in accordance with their previous knowledge of said objects. This is a clear indication of how there is a sensory component to amodal completion in the form of stored sensory experiences. A very simple example also makes this conceptually clearer: we are good at amodally completing words in the language we do know, as opposed to the ones we don’t; thereby indicating how previously held information can influence amodal completion.
However, there is also evidence that suggests that even during the act of amodal completion, one is directly interacting with the object in such a way so as to extract information from the unoccluded bit to fill in the occluded bits. This kind of amodal completion is said to be structure-driven and was also investigated by Yun et al., 2018. In this particular case, it is also more likely that working memory is involved, as suggested in §2.
All in all, there are definite sensory components to amodal completion, both in the form of previously held beliefs and in the form of working memory. What does this entail for the debate regarding representation of numbers highlighted in §3? If amodal completion has a sensory component, and if amodal completion is involved in the OTS, and the OTS is a part of the number sense, that necessarily implies that the number sense is based on a perceptual system.
But at this point, one may object that one component of the number sense is based on a perceptual system i.e. the OTS but another component i.e. the ANS still remains non-perceptual. There are two responses to this objection: (a) which suggests that Paz’s 2018 defense of an amodal approach towards the ANS is actually a defense of a multi-modal approach, meaning that the ANS, by virtue of being multi-modal, is still reducible to a sensory/perceptual system, and (b) as the OTS and ANS interact and do not work in a stand-alone manner, that means that the functioning of the ANS is based on processing done by the OTS. As we’ve seen, the OTS is a sensory/perceptual system. Therefore, if the ANS is using information provided by the OTS, that implies it too can be thought of as engaging in sensory processing i.e. being modal as opposed to amodal.
Let’s unpack (a) and (b) further.
For (a), Paz’s 2018 argument for the amodal nature of the ANS has been presented in §3. To briefly recap, he argues that the ANS represents numerosity irrespective of the triggering modality. However, this position falls prey to something Barsalou 2016 pointed out, regarding a misuse of the word “amodal.” Barsalou shows that the word amodal is used in the literature more and more to refer to what is actually “multimodal.” (Barsalou, 2016, pg. 1126) Paz 2018 makes the same mistake. He assumes that just because the representation of numerosity by the ANS can be done across two different modalities, such as the visual and auditory modality, that therefore implies that there is amodality. However, that is in fact, multimodality.
This multimodality does not necessarily imply that there is amodality. Specifically, researchers assume that there are amodal symbols in the brain when they see that the same brain areas respond to different forms of stimuli (for example, both visual and auditory areas are activated when a subjects see the image of a dog). However, it is possible that this activation is not indicative of amodal symbols. Instead, hearing the word “dog” and seeing a dog both activate those parts of the brain that share “modality-specific information” (Barsalou, 2016, pg. 1129) about the dog, such as its motion properties. This would mean the brain is retrieving information specific to the modalities involved rather than performing amodal representations.