Conceptual short-term memory: a new tool for understanding perception, cognition, and consciousness
The notion of memory, as used in ordinary language, may seem to have little to do with perception or conscious experience.
While perception informs us about the world as it is now, memory almost by definition tells us about the past. Similarly, whereas conscious experience seems like an ongoing, occurrent phenomenon, it’s natural to think of memory as being more like an inert store of information, accessible when we need it but capable of lying dormant for years at a time.
However, in contemporary cognitive science, memory is taken to include almost any psychological process that functions to store or maintain information, even if only for very brief durations (see also James, 1890). In this broader sense of the term, connections between memory, perception, and consciousness are apparent.
After all, some mechanism for the short-term retention of information will be required for almost any perceptual or cognitive process, such as recognition or inference, to take place: as one group of psychologists put it, “storage, in the sense of internal representation, is a prerequisite for processing” (Halford, Phillips, & Wilson, 2001). Assuming, then, as many theorists do, that perception consists at least partly in the processing of sensory information, short-term memory is likely to have an important role to play in a scientific theory of perception and perceptual experience.
In this latter sense of memory, two major forms of short-term store have been widely discussed in relation to perception and consciousness. The first of these is the various forms of sensory memory, and in particular iconic memory. Iconic memory was first described by George Sperling, who in 1960 demonstrated that large amounts of visually presented information were retained for brief intervals, far more than subjects were able to actually utilise for behaviour during the short window in which they were available (Figure 1).
This phenomenon, dubbed partial report superiority, was brought to the attention of philosophers of mind via the work of Fred Dretske (1981) and Ned Block (1995, 2007). Dretske suggested that the rich but incompletely accessible nature of information presented in Sperling’s paradigm was a marker of perceptual rather than cognitive processes.
Block similarly argued that sensory memory might be closely tied to perception, and further, suggested that such sensory forms of memory could serve as the basis for rich phenomenal consciousness that ‘overflowed’ the capacity for cognitive access.
A second form of short-term term that has been widely discussed by both psychologists and philosophers is working memory. Very roughly, working memory is a short-term informational store that is more robust than sensory memory but also more limited in capacity.
Unlike information in sensory memory, which must be cognitively accessed in order to be deployed for voluntary action, information in working memory is immediately poised for use in such behaviour, and is closely linked to notions such as cognition and cognitive access. For reasons such as these, Dretske seemed inclined to treat this kind of capacity-limited process as closely tied or even identical to thought, a suggestion followed by Block.[1]
Psychologists such as Nelson Cowan (2001: 91) and Alan Baddeley (2003: 836) take encoding in working memory to be a criterion of consciousness, while global workspace theorists such as Stanislas Dehaene (2014: 63) have regarded working memory as intimately connected – if not identical – with global broadcast.[2]
The foregoing summary is over-simplistic, but hopefully serves to motivate the claim that scientific work on short-term memory mechanisms may have important roles to play in understanding both the relation between perception and cognition and conscious experience.
With this idea in mind, I’ll now discuss some recent evidence for a third important short-term memory mechanism, namely Molly Potter’s proposed Conceptual Short-Term Memory. This is a form of short-term memory that serves to encode not merely the sensory properties of objects (like sensory memory), but also higher-level semantic information such as categorical identity.
Unlike sensory memory, it seems somewhat resistant to interference by the presentation of new sensory information; whereas iconic memory can be effaced by the presentation of new visual information, CSTM seems somewhat robust. In these respects, it is similar to working memory.
Unlike working memory, however, it seems to have both a high capacity and a brief duration; information in CSTM that is not rapidly accessed by working memory is lost after a second or two (for a more detailed discussion, see Potter 2012).
Evidence for CSTM comes from a range of paradigms, only two of which I discuss here (interested readers may wish to consult Potter, Staub, & O’Connor, 2004; Grill-Spector and Kanwisher, 2005; and Luck, Vogel, & Shapiro, 1996). The first particularly impressive demonstration is a 2014 experiment examining subjects’ ability to identify the presence of a given semantic target (such as “wedding” or “picnic”) in a series of rapidly presented images (see Figure 2).
A number of features of this experiment are worth emphasizing. First, subjects in some trials were cued to identify the presence of a target only after presentation of the images, suggesting that their performance did indeed rely on memory rather than merely, for example, effective search strategies.
Second, a relatively large number of images were displayed in quick succession, either 6 or 12, in both cases larger than the normal capacity of working memory. Subjects’ performance in the 12-item trials was not drastically worse than in the 6‑item trials, suggesting that they were not relying on normal capacity-limited working memory alone.
Third, because the images were displayed one after another in the same location in quick succession, it seems unlikely that they were relying on sensory memory alone; as noted earlier, sensory memory is vulnerable to overwriting effects.
Finally, the fact that subjects were able to identify not merely the presence of certain visual features but the presence or absence of specific semantic targets suggests that they were not merely encoding low-level sensory information about the images, but also their specific categorical identities, again telling against the idea that subjects’ performance relied on sensory memory alone.
Another relevant experiment for the CSTM hypothesis is that of Belke et al. (2008). In this experiment, subjects were presented with a single array of either 4 or 8 items, and asked whether a given category of picture (such as a motorbike) was present. In some trials in which the target was absent, a semantically related distractor (such as a motorbike helmet) was present instead.
The surprising result of this experiment, which involved an eye-tracking camera, was that subjects reliably fixated upon either targets or semantically related distractors with their initial eye movements, and were just as likely to do whether the arrays contained 4 or 8 items, and even when assigned a cognitive load task beforehand (see Figure 3).
Again, these results arguably point to the existence of some further memory mechanism beyond sensory memory and working memory: if subjects were relying on working memory to direct their eye movements, then one would expect such movements to be subject to interference from the cognitive load, whereas the hypothesis that subjects were relying on exclusively sensory mechanisms runs into the problem that such mechanisms do not seem to be sensitive to high-level semantic properties of stimuli such as their specific category identity, whereas in this trial, subjects’ eye movements were sensitive to just such semantic properties of the items in the array.[3]
Interpretation of experiments such as these is a tricky business, of course (for a more thorough discussion, see Shevlin 2017). However, let us proceed on the assumption that the CSTM hypothesis is at least worth taking seriously, and that there may be some high-capacity semantic buffer in addition to more widely accepted mechanisms such as iconic memory and working memory.
What relevance might this have for debates in philosophy and cognitive science? I will now briefly mention three such topics. Again, I will be oversimplifying somewhat, but my goal will be to outline some areas where the CSTM hypothesis might be of interest.
The first such debate concerns the nature of the contents of perception. Do we merely see colours, shapes, and so on, or do we perceive high-level kinds such as tables, cats, and Donald Trump (Siegel, 2010)? Taking our cue from the data on CSTM, we might suggest that this question can be reframed in terms of which forms of short-term memory are genuinely perceptual. If we take there to be good grounds for confining perceptual representation to the kinds of representations in sensory memory, then we might be inclined to take an austere view of the contents of experience.
By contrast, if the kind of processing involved in encoding in CSTM is taken to be a form of late-stage perception, then we might have evidence for the presence of high-level perceptual content. It might reasonably be objected that this move is merely ‘kicking the can down the road’ to questions about the perception-cognition boundary, and does not by itself resolve the debate about the contents of perception. However, more positively, this might provide a way of grounding largely phenomenological debates in the more concrete frameworks of memory research.
A second key debate where CSTM may play a role concerns the presence of top-down effects on perception. A copious amount of experimental data (dating back to early work by psychologists such as Perky, 1910, but proliferating especially in the last two decades) has been produced in support of the idea that there are indeed ‘top-down’ effects on perception, which in turn has been taken to suggest that our thoughts, beliefs, and desires can significantly affect how the world appears to us.
Such claims have been forcefully challenged by the likes of Firestone and Scholl (2015), who have suggested that the relevant effects can often be explained in terms of, for example, post-perceptual judgment rather than perception proper.
However, the CSTM hypothesis may again offer a third compromise position. By distinguishing core perceptual processes (namely those that rely on sensory buffers such as iconic memory) from the kind of later categorical processing performed by CSTM, there may be other positions available in the interpretation of alleged cases of top-down effects on perception.
For example, Firestone and Scholl claim that many such results fail to properly distinguish perception from judgment, suggesting that, in many cases, experimentalists’ results can be interpreted purely in terms of strictly cognitive effects rather than as involving effects on perceptual experience. However, if CSTM is a distinct psychological process operative between core perceptual processes and later central cognitive processes, then appeals to things such as ‘perceptual judgments’ may be better founded than Firestone and Scholl seem to think.
This would allow us to claim that at least some putative cases of top-down effects went beyond mere post-perceptual judgments while also respecting the hypothesis that early vision is encapsulated; see Pylyshyn, 1999).
A final debate in which CSTM may be of interest is the question of whether perceptual experience is richer than (or ‘overflows’) what is cognitively accessed. As noted earlier, Ned Block has argued that information in sensory forms of memory may be conscious even if it is not accessed – or even accessible – to working memory (Block, 2007).
This would explain phenomena such as the apparent ‘richness’ of experience; thus if we imagine standing in Times Square, surrounded by chaos and noise, it is phenomenologically tempting to think we can only focus on and access a tiny fraction of our ongoing experiences at any one moment. A common challenge to this kind of claim is that it threatens to divorce consciousness from personal level cognitive processing, leaving us open to extreme possibilities such as the ‘panpsychic disaster’ of perpetually inaccessible conscious experience in very early processing areas such as the LGN (Prinz, 2007).
Again, CSTM may offer a compromise position. As noted earlier, the capacity of CSTM does indeed seem to overflow the sparse resources of working memory. However, it also seems rely on personal level processing, such as an individual’s store of learned categories. Thus one new position, for example, might claim that information must at least reach the stage of CSTM to be conscious, thus allowing that perceptual experience may indeed overflow working memory while also ruling it out in early sensory areas.
These are all bold suggestions in need of extensive clarification and argument, but it is my hope that I have at least demonstrated to the reader how CSTM may be a hypothesis of interest not merely to psychologists of memory, but also those interested in broader issues of mental architecture and consciousness. And while I should also stress that CSTM remains a working hypothesis in the psychology of memory, it is one that I think is worth exploring on grounds of both scientific and philosophical interest.
[1] Note that Dretske does not use the term working memory in this context, but clearly has some such process in mind, as made clear by his reference to capacity-limited mechanisms for extracting information.
[2] A complicating factor in discussion of working memory comes from the recent emergence of variable resource models of working memory (Ma et al., 2014) and the discovery that some forms of working memory may be able to operate unconsciously (see, e.g., Bergström & Eriksson, 2014).
[3] Given that the arrays remained visible to subjects throughout the experiment, one might wonder why this experiment has relevance for our understanding of memory. However, as noted earlier, I take it that any short-term processing of information presumes some kind of underlying temporary encoding mechanism.
References
Baddeley, A.D. (2003). Working memory: Looking back and looking forward. Nature Reviews Neuroscience, 4(10), 829–839.
Belke, E., Humphreys, G., Watson, D., Meyer, A. and Telling, A., (2008). Top-down effects ofsemantic knowledge in visual search are modulated by cognitive but not perceptual load. Perception & Psychophysics, 70 8, 1444 – 1458.
Bergström, F., & Eriksson, J. (2014). Maintenance of non-consciously presented information engages the prefrontal cortex. Frontiers in Human Neuroscience 8:938.
Block, N. (2007). Consciousness, Accessibility, and the Mesh Between Psychology and Neuroscience, Behavioral and Brain Sciences 30, pp. 481–499.
Cowan, N., (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences 241, 87–114.
Dehaene, S. (2014). Consciousness and the Brain: Deciphering How the Brain Codes Our Thoughts. Viking Press, 2014.
Dretske, F. (1981). Knowledge and the Flow of Information. MIT Press.
Firestone, C., & Scholl, B.J. (2015). Cognition does not affect perception: Evaluating the evidence for ‘top-down’ effects. Behavioral and Brain Sciences:1–77.
Grill-Spector, K., & Kanwisher, N. (2005). Visual Recognition. Psychological Science, 16(2), 152–160.
Halford, G. S., Phillips, S., & Wilson, W. H. (2001). Processing capacity limits are not explained by storage limits. Behavioral and Brain Sciences 24 (1), 123–124.
James, W. (1890). The Principles of Psychology. Dover Publications.
Luck, S. J., Vogel, E. K., & Shapiro, K. L. (1996). Word meanings can be accessed but not reported during the attentional blink. Nature, 383(6601), 616–618.
Ma, W. J., Husain, M., & Bays, P. M. (2014). Changing concepts of working memory. Nature Neuroscience, 17(3), 347–356.
Potter, M. C. (2012). Conceptual Short Term Memory in Perception and Thought. Frontiers in Psychology, 3:113.
Potter, M. C., Staub, A., & O’Connor, D. H. (2004). Pictorial and conceptual representation of glimpsed pictures. Journal of Experimental Psychology: Human Perception and Performance, 30, 478–489.
Prinz, J. (2007). Accessed, accessible, and inaccessible: Where to draw the phenomenal line. Behavioral and Brain Sciences, 305–6.
Pylyshyn, Z. (1999). Is vision continuous with cognition?: The case for cognitive impenetrability of visual perception. Behavioral and Brain Sciences, 22(03).
Shevlin, H. (2017). Conceptual Short-Term Memory: A Missing Part of the Mind? Journal of Consciousness Studies, 24, No. 7–8.
Siegel, S. (2010). The Contents of Visual Experience. Oxford.
Sperling, G. (1960). The Information Available in Brief Visual Presentations, Psychological Monographs: General and Applied 74, pp. 1–29.