Cognit activation: a mechanism enabling temporal integration in working memory

From Specialized Modules to Large-scale Cognitive Networks

For most of the past century, cognitive neuroscience has been dominated by modular concepts of neural specialization in the cerebral cortex. Not only the contents of cognition--memory and knowledge--but also the cognitive functions that process them have been ascribed to specialized modules in one or another area of cortex.

Karl Lashley [1] was the first to empirically challenge that modular trend. From his failure to elicit deficits in discrimination behavior by assorted cortical lesions, he concluded “by default” that memory had to be distributed in the cortex. Around the same time, two other psychologists, Hebb [2] and Hayek [3]--the latter also a famous economist--came upon the concept of cortical networks by theorizing about principles of neural integration that guide the acquisition of percepts and memories from experience. The idea that large-scale brain networks underlie cognition, recently reviewed in these pages [4], has extended and refined the contributions of Lashley, Hebb, and Hayek. Given that large-scale cognitive networks are distributed across the entire cortex, their ascent has provided a powerful alternative to the antiquated notion of cognitive modularity in the cortex.

In the past five decades, two robust trends have strengthened the understanding of large-scale brain networks. Firstly, the principles of neural integration, upon which cognitive network formation is based, have been substantiated in invertebrates, in the mammalian neocortex, and in the hippocampus. Enormous progress has been made in elucidating the molecular and electrochemical mechanisms of synaptic physiology, which govern network integration and thus the association of different items of memory and knowledge. Secondly, the concept of large-scale brain network has gained considerable theoretical momentum in cognitive science from connectionist studies of parallel-distributed processing models.

In this article, we discuss the accumulating evidence that working memory (WM) depends on interactions within and between large-scale networks of the cerebral cortex. We argue that, whereas such networks encompass several cortical areas, network operations are irreducible to the operations of those areas [5]. We postulate that WM, like any other cognitive function, operates by a relational code, which determines the specific spatio-temporal patterns of inter-areal interaction underlying cognitive processing. In our view, the large-scale cortical network represents an “epistemological floor” for understanding WM, and any attempt to describe it at levels below this floor will inevitably fail. Therefore, we propose that to understand WM, the dynamics of large-scale cortical networks must be investigated at the “mesoscopic” level of analysis.

In recent years, one of us [6,7] has proposed a heuristic network model of memory that is amenable to that analysis. Essentially, the model posits that memory and knowledge (semantic memory) consist of large-scale cortical networks, which in turn comprise a large array of distributed, overlapping, and interactive networks named “cognits.” Cognits are made of dispersed neuronal assemblies associated synaptically by life experience. Each cognit represents an item of memory or knowledge.

The cognit, in our view, represents the critical conceptual linchpin for understanding the human connectome [8]. In their latent state, cognits embody the connection matrix of the cortex, which has been uniquely sculpted in each individual by experience. In their active state, they constitute the interacting components of the large functional networks at play in cognition. Thus, cognit activation is seen as an essential brain mechanism that allows stored memory and knowledge to be used in all cognitive functions.

WM is a cognitive function that serves as a critical component of the perception/action (PA) cycle, which essentially enables a subject to execute goal-directed behaviors and linguistic expressions. We propose here that WM in the PA cycle depends on interactions between activated posterior (perceptual) and frontal (executive) cognits, and that these interactions occur by way of reentrant circuits that allow information exchange between cognits. If a goal is novel or complex, and depends on the resolution of uncertainties or ambiguities, the interactions are between cognits at the highest levels of the perceptual and executive cortical hierarchies. When the cycle is interrupted by a physical constraint, by parsing of the task or of the discourse, or by experimental design, the resulting time lapse must be covered with WM. In that case, we predict that WM is able to bridge the time lapse by the continued activation of posterior and frontal cognits. This proposed mechanism has the benefit of explaining how perceptual and executive cognits accommodate information that is accumulated over multiple PA cycles.

In what follows, we first present a summary description of the cognit, pointing out the similarities and overlaps with other views of cognitive networks. Second, we deal with the cortical dynamics of WM, highlighting the most recent literature on computational models, electrophysiology of inter-areal relations, and cortical neuroimaging. Finally, we substantiate our prediction about the role of WM in the PA cycle, focusing on the cortical dynamics of WM as revealed by those three methodologies.

The Cognit

A cognit is a unit of memory or knowledge made of a network of synaptically associated cortical neurons. The neurons may be situated within a single cortical area, or dispersed across several contiguous or noncontiguous areas. The cognit’s dimensions can vary extensively, depending on such factors as the complexity or abstraction (i.e., categorical generality) of the information it contains and the strength of its associations with other cognits. Cognits profusely interconnect, overlap, and share nodes of common association. Therefore, the cortical borders of a cognit are diffuse and fade into a penumbra of weak associations. The strength and makeup of a cognit are constantly changing by virtue of changes in the weights of the synapses between its constituent neurons. Those changes are the product of new experience, learning, attrition by disuse or aging, and associative recombination with other cognits.

The cognit resembles Hebb’s “cell assembly” [2] in two important respects. Both assume that memory has a network structure. They also assume that change in the makeup and connectivity of memory networks is brought about by synaptic modulation of their component neurons as a result of learning and life experience. The cognit, however, differs from the “cell assembly” in several respects. First, unlike Hebb’s networks, cognits overlap with one another and share nodes of association that represent common features of perception, memory or knowledge. In this respect, the cognit is similar to a connectionist network [9,10]. Second, although a Hebbian cell assembly remains relatively static after it has been formed, the cognit is constantly changing as a result of spontaneous or induced alteration of its synaptic connectivity. Third, activated cognits undergo reentrant interactions, that is, loops of reciprocal dynamic connectivity between and within networks. Instead, Hebb proposed that cell assemblies undergo only feed-forward progression of activations, as they do in more recent Hebbian cell-assembly constructs [11]. (Hebb’s concept of “reverberation” involves a feed-forward sequence rather than reentrant feedback.) Finally, while cognit theory is consistent with the dense synaptic connectivity between a cortical neuron and many others, as well as the sparse connectivity between any pair of neurons, Hebb’s cell assemblies rely exclusively on neuron-to-neuron connectivity.

Despite these differences, the cognit adopts the now widely accepted Hebbian principles of synaptic modulation whereby the temporal coincidence of pre- and post-synaptic events underlies the network formation that is responsible for memory acquisition. In particular, Hebb’s lesser known principle of “sensori-sensory” association [2, p.70], emphasized by Hayek [3] for perception, is central to the cognit model (Box 1): it has been called the principle of “synchronous convergence” [6,12]. Unlike Hebb’s cell assembly, however, the cognit does not need repeated coincidence, except in skilled learning. One-trial learning is sufficient for cognit formation, and a pre-existent cognit may be modified by exposure to a single new input.

Cognits are essential for both perception and memory, two closely interdependent cognitive functions. We remember what we perceive and we perceive what we remember [13], because both perceiving and remembering are cognit-based. Cognits are formed in bottom-up fashion from sensory input in perception. Once formed, they function as memory to guide new perception and new memory formation in top-down fashion. Cognits not only serve the acquisition and storage of information, but also the utilization of that information in our interactions with the world. Established cognits, in latent state, are re-activated with the arrival of new information that bears some relation of content or similarity to them. Thereby, new information can expand an old cognit by new associations. Thus new memories become incorporated into old memory and remain nested within it. The hippocampus, through mechanisms that are still poorly understood, plays a decisive role in the formation and re-formation of cognits in the cerebral cortex [14], but the cortical distribution of cognits, and the retrieval of information from them, are largely independent of the hippocampus [15].

Cognits blend with one another as they share nodes of heavy association in smaller networks that represent common cognitive properties, such as a variety of sensory or motor features. The memory of these properties may be part of many interrelated cognits, including autobiographical memories and items of knowledge. The synaptic strength of the connections within a cognit varies widely as a function of rehearsal, recall, forgetting and practice. Cognits not only serve the storage of memory, but also provide the anatomical infrastructure for the neural computations that underlie all cognitive functions, including WM. Those cognitive functions rely on neural interactions within and between cognits, and the efficacy of those interactions closely depends on synaptic strength, which defines the makeup, relations, and boundaries of cognits.

In the adult, as in the child, cognitive development literally consists of cognit development. We assume that the prenatal development of cognit connectivity, shaped by genetic and hormonal influences, takes place primarily in the sensory and motor cortices. Thus, cognit structure is rudimentary at birth, but rapidly develops postnatally out of the innate synaptic connectivity of sensory and motor cortices (phyletic memory) as a result of life experience (Box 2). It is reasonable to assume that the onslaught of postnatal experience and cognitive learning is accompanied by an explosive growth of cortical connectivity that supports the formation and expansion of cognits via selective synaptic modification outside of the sensory and motor cortices.

As they develop, cognits are assumed to self-organize hierarchically, with simple sensory and motor cognits near primary sensory and motor areas, and complex cognits–categorical knowledge derived from multiple instantiations–in the higher cortical association areas [16]. Despite profuse nesting and overlap, two general hierarchies of cognits can be discerned (Figure 1): A perceptual hierarchy in posterior cortex for memory acquired through the senses, and an executive hierarchy in frontal cortex for memory acquired through inputs from motor systems (proprioception and efferent copies of movement signals). Nonetheless, both hierarchies interconnect with each other at every level. Above the primary cortical areas, most memory is heterarchical, made of interconnecting cognits from different hierarchies and hierarchical levels.

Recent computational network models of cognition exhibit to varying degrees some of the properties of the cognit. Among these common properties, the following stand out: (a) wide spatial extent of cognitive networks [17–20], (b) stochastic computation [21–23], (c) inherent tendency to oscillation [24–26], (d) nodes or hubs of association [27], and (e) hierarchical organization [27,28]. Many computational models, therefore, not only are compatible with the cognit, but also elaborate on its major characteristics, namely, nonlinearity, stochasticity (Bayesian probability), hierarchy, and sparse coding. The biological plausibility of these computational attributes becomes more apparent when we consider the cognit from the perspective of cognitive dynamics, as we do below for WM.

The cognit model has three distinctive features that set it apart from other network models of cortical cognition. Unlike them, our model assumes that: (1) Cognits encode long-term memory and knowledge by the synaptic association of cortical assemblies, whereas functional modulations within and between selectively activated cognits subserve all cognitive functions; (2) In their formation, cognits grow on a base of “phyletic” memory, that is, on the modular structure of primary sensory and motor areas (see Box 2); (3) Cognits develop from the interactions of the cortex with the environment; along phylogenetic, ontogenetic, and connective gradients, cognits develop into two hierarchies of associative areas–one hierarchy, perceptual, in posterior cortex and the other, executive, in frontal cortex.

Working Memory

Working memory (WM) is the temporary retention of an item of information for the prospective attainment of a goal. Thus, by definition, WM has two temporal limits, one in the recent past (the information to be retained) and the other in the proximate future (the consequent action). WM mediates the logical and behavioral cross-temporal contingency between the two. Thus WM has a retrospective function of retention and a prospective function of anticipating--and preparing for--the forthcoming action. That prospective function distinguishes WM from all other forms of short-term memory.

We propose that both the retrospective and prospective functions of WM are embodied in active cognits. The retrospective function consists in the activation of posterior perceptual cognits anchored in past experience; thus, that activation is the situation-specific implementation of information that has been stored in the form of latent cognits in long-term memory (LTM). Indeed, there is ample evidence that WM consists of LTM that is instantiated in a given situation [7,29–31]. Similarly, the prospective function of WM depends on the activation of goal-directed actions represented in the form of frontal executive cognits also anchored in past experience.

Clearly, the information content that the cortex must hold in WM is complex, requiring both perceptual and executive cognits, and this content extends beyond the sensory or linguistic cues that the subject must retain for a goal-directed action. Those cues are “new” memoranda that update the pre-existing cognit of the task, which embodies the constellation of associations in which the “new” cues are embedded. Those cues also activate a host of hierarchically organized and widely distributed cognits, some nested within others, but all part of the subject’s LTM. Those pre-existing cognits are in latent state until they are activated for cognitive function. Thus, the memoranda of WM essentially consist of activated memoranda of LTM with ad hoc task-specific memoranda embedded in them [16].

From these considerations, we postulate that each activated cognit binds cortical assemblies representing different sensory and/or motor features into its network. An extension to this postulate is that each activated cognit oscillates at a given “binding frequency.” As a consequence, the activation of a complex cognit in WM is predicted to result in a heterogeneous mixture of concomitant oscillatory rhythms. It is further predicted that cortical assemblies whose neurons participate in the same cognit in WM will show high degrees of coherence between them. Recent literature indicates that these predictions are generally correct [26,27,32–34]. But before further discussion, we must distinguish between spatial and temporal binding.

Traditionally, synchronized oscillatory rhythms, even though temporal by definition, have been postulated to substantiate the spatial binding of stimulus properties in perception [35]. In WM, however, we must address temporal binding, which is a more complex and unexplored phenomenon. Just as cognition includes spatial structures, e.g. gestalts, in visual perception, it also includes temporal structures or gestalts in the form of strategies, melodies, sentences, scripts, or trials in a WM task. The temporal nature of WM requires that it mediate cross-temporal perceptual and action-based contingencies, and that cognits bind the contents of WM across time until the behavioral goal is achieved.

A large body of studies reviewed in [32] and [36] exposes the oscillatory dynamics subserving WM. However, in order to study the role of temporal binding in WM, it is desirable to use tasks with delays long enough to segregate cognitive operations that may have long time constants. Several studies meet this criterion [31,37–41].

To summarize the results of those studies, electro-cortical activity reveals, in WM, a wide range of oscillatory frequencies, either concomitantly or seriatim. In the first case, different frequencies co-modulate each other in phase and/or amplitude; in the second, trains of oscillations of different frequency succeed one another. It can be reasonably assumed that the pattern of frequencies at which a cognit in WM oscillates depends on a number of factors, including: (a) the circulation times of reentrant interaction between the distributed assemblies of the cognit; (b) the time constants of local excitatory-inhibitory interaction within its assemblies; and (c) the strength of connectivity within it. Therefore, based on our assumption, each cognit activated during WM is predicted to produce an electro-cortical signature, that is, a unique pattern of oscillation frequencies --a spectral activation profile or fingerprint [42].

Several lines of evidence suggest a relation between oscillatory cognit frequencies and behavior. The gradual diminution of oscillatory activity in the course of the delay of a WM task is indirect evidence of cognit deactivation as a function of time [43,44]. Further evidence of the behavioral significance of cognit activation is the correlation--direct or inverse--between performance and oscillatory WM activity [36,45,46]. Yet another is the fragmentation of oscillatory frequency in multi-item WM [47,48].

Probably the most direct evidence of the relation between active cognits and oscillatory frequency comes from the assessment of oscillatory activity as a function of WM content. Hsieh et al. [49] tested subjects on two types of delay performance: in one, the subject had to retain the order of item presentation, in the other the items themselves. Oscillatory power in the theta (5–7 Hz) frequency range increased in prefrontal cortex when the memorandum was order, whereas power in the alpha (9–12 Hz) range increased in posterior cortex when the memorandum was item (Figure 2). These results are consistent with the well-known evidence that the lateral prefrontal cortex is critically involved in temporal order [16]. Further, they are consistent with the notion that cognits representing order should be more extensive, and thus oscillate at (lower) theta frequencies, than those representing discrete items and oscillating at (higher) alpha frequencies.

More compelling for our argument, and often ignored, is the prospective component of WM, which is accompanied by the activation of executive cognits in the preparation for a motor choice in accord with the memorandum. That prospective WM-component is evinced by numerous anticipatory phenomena, especially in the frontal lobe [16]. In monkeys, cells are found, there and elsewhere, that during the memory period of a delay task, anticipate the choice with accelerating firing. That acceleration increases as a function of the certainty with which the animal can predict the direction of rewarded choice at the end of a trial (Figure 3).

The biophysical relationship between cellular activity and electro-cortical oscillation is still only tentatively understood. It seems clear, however, that WM oscillations are not simply an epiphenomenon of cell discharge, but rather a manifestation of circuit interactions in a neuronal network. It has been demonstrated, in parietal cortex [50] and extrastriate visual cortex [51], that single-neuron spiking in WM is phase-locked to oscillatory activity. Furthermore, memory cells in prefrontal cortex discriminate different objects in WM by time-locking of neuronal spikes at different oscillatory phases [52]. From these findings, it is reasonable that the phase synchronization of oscillatory activity between two neuronal assemblies in WM serves to confine or facilitate the transmission of spikes between them, thus optimizing their communication, as proposed by Fries [53], as well as strengthening the synapses between them, as proposed by Fell and Axmacher [33].

Reentry at different spatial and temporal scales seems to be behind the oscillatory cortical activity observed in WM. In any case, reentry appears at the foundation of the cortical mechanisms of WM. Here the recent empirical and theoretical evidence bolsters the old belief that information is maintained in WM by the “reverberation” of excitatory activity between neurons of the same and different cortical areas. However, the modern understanding of this phenomenon includes inhibitory as well as excitatory feedback. Although WM mechanisms are still incompletely understood, they probably involve the reentry of excitation within and between posterior and frontal cortex [36,54–56]. Which cognits are activated in either cortex must depend on the nature of the perceptual and executive information in WM. Recently, Fuster and colleagues [57] published a computational model of WM supporting this opinion inasmuch as, by virtue of its reentrant architecture, the model predicts remarkably well the behavior of cortical neurons in WM. In essence, the model consists of a reentrant system of attractor-networks with dynamic synapses, which simulates a variety of real cell-firing patterns in the memory period of WM tasks. These patterns deviate considerably from those of the theoretical bi-stable cell stereotype of WM (Figure 4).

In summary, during WM cognits are actively sustained in posterior and frontal cortices whenever signals and contingencies need to be cross-temporally reconciled, as is the case in delay tasks, in deliberative decisions, and in the pursuit of elusive or deferred goals. This role of cognits in temporal integration can best be understood in the context of the PA cycle.

Temporal Binding in the Perception / Action Cycle

The concept of the PA cycle [6] is the extension to the cerebral cortex of a deeply rooted biological principle. That principle stipulates that the adaptive relationship of an organism with its environment is governed cybernetically by the continuous circular processing of information between the two [58]. Stimuli from the environment act on sensory organs to inform the organism, which acts adaptively on that environment through effector organs, thus generating changes in it that lead to new sensory input, which informs new adaptive action, and so on. In the mammalian brain, the external feedback in that cycle is supplemented by internal feedback that runs from action systems to sensory systems, anticipating the action and preparing the organism for its execution. Thus, pre-adaptive internal feedback becomes the physiological foundation for such essential future-oriented cognitive functions as preparatory set, selective motor attention, decision-making, and WM, all based on interactions between prefrontal (executive) and posterior (perceptual) cognits. Figure 5 illustrates, schematically, the PA cycle and the direction of computational interactions within the cortex in the course of sequential goal-directed behavior.

Inasmuch as WM is essential for the mediation of cross-temporal contingencies, the electro-cortical evidence of oscillations in WM supports their temporal integrative role in the PA cycle. This evidence naturally is consistent with the fact that cortical oscillatory activity is a temporally extended phenomenon and with our attribution to those oscillations of implicit content (i.e., memoranda). Oscillatory synchronization has been considered fundamental to feedback-guided learning–a critical part of the PA cycle that forms and updates what we call executive cognits [59].

In any form of sequential goal-directed behavior, the PA cycle must integrate perception with action at every step leading to the goal. Integration is iterative and memory contents are interleaved in time. Although electro-cortical analysis has proven useful for exposing active cognits in delay tasks, where each trial is equivalent to one phase of the PA cycle, that analysis has proven difficult to use for scrutinizing the complexities of the PA cycle in more intricate phenomena such as language, which typically involve multiple phases. Our argument for temporal integration, however, is also supported by studies of functional neuroimaging.

Functional neuroimaging provides a bird’s-eye view of the cortical PA cycle, especially the co-activation of posterior and frontal cognits in WM, which through reentrant circuits integrates perception with action in the temporal domain. By recurrent activation between areas, the cerebral cortex mediates cross-temporal contingencies. That activation of cognits in WM has now been substantiated by a long series of imaging studies during performance of WM tasks (Table 1). Almost without exception, those studies reveal that WM involves the co-activation of several non-contiguous cortical areas. Typically reported is the activation of a lateral prefrontal region and, concomitantly, a region of posterior cortex that varies with the sensory modality of the memorandum. If the memorandum is visual, that posterior region includes inferotemporal and parastriate cortex (Figure 6); if it is auditory, superior temporal cortex; if it is spatial, posterior parietal cortex.

Imaging studies also reveal the functional connectivity between cortical areas that constitutes the dynamic linkage of these areas in the PA cycle [72–75]. The functional importance for WM of dynamically linked active cognits is confirmed by evidence that their activation is proportional to memory load, in other words, to the novelty and complexity of the memorandum, as well as to the accuracy of the memorandum, as well as to the accuracy of the subject’s performance of the task [76–83].

Concluding Remarks

In conclusion, WM, the temporary activation of perceptual and executive cognits for goal-directed action, is a time bridging function that enables the PA cycle to reach its goal despite temporal discontinuities in the cycle. Through WM, the cerebral cortex mediates the cross-temporal contingencies of the cycle. The cortical mechanisms of WM are believed to include at their core the reentry of excitation within and between cognits. Thus, whereas cognits constitute a structural relational code of LTM in cortical space, we posit that their functional recruitment by WM makes them a dynamic spatial-temporal code at the service of goal-directed action. We further posit that the temporal integration that WM serves is essentially accomplished by reentrant circulation of nerve impulses between posterior (perceptual) and frontal (executive) cognits.