short term memory research paper

• Advanced search

Advanced Search

Short-Term Memory and the Human Hippocampus

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• Info & Metrics

Every undergraduate psychology student is taught that short-term memory, the ability to temporarily hold in mind information from the immediate past (e.g., a telephone number) involves different psychological processes and neural substrates from long-term memory (e.g., remembering what happened yesterday). This dichotomous account of memory is grounded on evidence of neuropsychological dissociations such as those shown by patients with damage to medial temporal lobe (MTL), who until now have been thought to exhibit impaired long-term memory but normal short-term memory ( Squire, 1992 ). In recent years, however, this viewpoint has faced considerable challenges, given accumulating evidence suggesting that short-term memory and long-term memory, rather than being qualitatively distinct, may in fact share similar underlying neural mechanisms (for review, see Jonides et al., 2008 ).

A key recent observation is that patients with MTL damage perform poorly not only on long-term memory tasks, but also on short-term memory tasks that involve remembering novel information across brief intervals. Whereas the perirhinal cortex appears to support short-term memory for novel object information ( Brown and Aggleton, 2001 ), neuropsychological evidence suggests that the hippocampus is critical when associative information is involved (for review, see Jonides et al., 2008 ), in line with its proposed function as a relational binder in long-term memory ( Cohen and Eichenbaum, 1993 ). For instance, in one recent study, patients with hippocampal amnesia were impaired at remembering the locations of novel objects, even across a delay of a few seconds ( Jonides et al., 2008 ).

We know from neuropsychological evidence, therefore, that the hippocampus is critical to short-term memory for associative information. What is not clear from the neuropsychological data, however, is how the hippocampus supports this function. Hannula and Ranganath (2008) use functional magnetic resonance imaging (fMRI) to address this important issue by characterizing brain activity during each phase of a short-term associative memory task and by linking such neural activity to behavioral performance. Whereas a subsequent memory approach has been widely used to study long-term recognition memory, this has not been possible in previous short-term memory experiments because of the near-ceiling performance typically achieved by subjects. To circumvent this problem, the authors chose a relatively difficult task to ensure that sufficient numbers of correct and incorrect trials would be generated.

The paradigm used shares similarities with a task known to be hippocampal-dependent based on previous neuropsychological data ( Hartley et al., 2007 ). During the sample phase of each trial, subjects viewed a novel scene consisting of four objects (out of a set of nine objects), each in one of nine possible locations in a 3 × 3 grid [ Hannula and Ranganath (2008) , their Fig. 1 ( http://www.jneurosci.org/cgi/content/full/28/1/116/F1 )]. To encourage use of a hippocampally mediated allocentric (or world-centered) strategy, rather than an egocentric (or viewer-centered) strategy thought to rely on parietal and prefrontal cortices, subjects were asked to form a mental image of the scene rotated 90° to the right of the original viewpoint. They were then required to maintain the rotated representation during the ensuing 11 s delay phase in anticipation of the test stimulus. During the test phase, subjects' memory for the positions of the objects was assessed. This was done by asking them to classify, by button press, the test stimulus according to whether it constituted (1) a “match” (i.e., the original scene rotated 90°); (2) “mismatch-position” (i.e., one object occupied a new location); (3) “mismatch-swap” (i.e., two objects had swapped locations“). Performance in all conditions was significantly greater than a chance level of 33% correct responses: 78, 65, and 60% on match, mismatch-position and, mismatch-swap displays, respectively.

The authors first performed a subsequent memory analysis by contrasting correct trials with incorrect trials. This revealed that hippocampal activity during the sample phase predicted successful recognition judgments in the test phase [ Hannula and Ranganath (2008) , their Fig. 2 ( http://www.jneurosci.org/cgi/content/full/28/1/116/F2 )[. Critically, a subsequent memory correlation was also observed in the hippocampus during the test phase [ Hannula and Ranganath (2008) , their Fig. 3 ( http://www.jneurosci.org/cgi/content/full/28/1/116/F3 )]. This finding rules out an otherwise problematic explanation that greater neural activity during the sample phase predicts subsequent success not through the encoding of object-location associative information, but rather the objects (e.g., drums, birdbath) themselves. Indeed, a subsequent memory correlation was observed in the perirhinal cortex selectively during the sample phase, in line with proposals that this neural region is critical for the encoding of item-specific information.

Interestingly, there were no significant differences in hippocampal activity as a function of accuracy during the delay period. Although caution is advised in interpreting such a null finding, this result does suggest that persistent neural firing in the hippocampus does not occur during the delay period of short-term memory tasks, as is thought to occur in the entorhinal cortex. One possibility is that the hippocampus supports short-term memory for associative information through transient changes in synaptic efficacy, rather than active maintenance ( Jonides et al., 2008 ). Alternatively, active maintenance may occur, but by a different mechanism not detectable by fMRI (e.g., involving theta/gamma oscillations).

These results suggest that the hippocampus plays an important role in the encoding and retrieval, but perhaps not the active maintenance, of novel associative information in short-term memory. But how does the hippocampus compute the novelty, or conversely familiarity, of the test stimulus such that a correct recognition judgment can be made? An influential theoretical proposal is that the hippocampus acts as a comparator (or match-mismatch detector), identifying discrepancies between previous predictions based on past experience and current sensory inputs ( Norman and O'Reilly, 2003 ) (for review, see Kumaran and Maguire, 2007 ). One strategy for assessing the validity of this hypothesis is to characterize how hippocampal activity varies as a function of the novelty or familiarity of the test stimulus. Empirical evidence consistent with predictions arising from a comparator model was provided by a recent study using this approach ( Kumaran and Maguire, 2006 ), with hippocampal activity observed specifically under conditions of match-mismatch, and not in response to the mere presence of novelty per se.

Hannula and Ranganath (2008) adopted a similar approach to probe the nature of hippocampal novelty/familiarity signals, by including three types of test trials that varied according to their similarity to the sample stimulus. The authors used a region-of-interest analysis to demonstrate that hippocampal activation during correct trials was greatest in relation to match displays (compared with mismatch-position and mismatch-swap displays). Interestingly, when the novelty/familiarity of associative information is incidental to the task at hand (i.e., subjects are not required to make explicit recognition memory judgments), a qualitatively different pattern of findings has been observed with hippocampal activation maximal under conditions of mismatch rather than match ( Kumaran and Maguire, 2006 ). Before turning to “interesting” explanations for this discrepancy, it is worth considering the influence of subjects' superior performance during match displays (compare mismatch displays) on the observed neural data. Although the authors carefully considered and discounted such an effect, without confidence rating data, it is difficult to entirely exclude the possibility that subjects may have been more confident in making (correct) match, as opposed to mismatch, judgments.

That said, the most likely explanation for the observed findings is that the amplitude of hippocampal responses to novel (or familiar) sensory inputs depends on the specific task being performed. As such, the findings observed by Hannula and Ranganath (2008) resemble the well known phenomenon of “match enhancement” observed in monkey inferotemporal/perirhinal cortex in relation to the arrival of an anticipated target stimulus that matches the current stimulus being held in mind ( Miller and Desimone, 1994 ) [ Hannula and Ranganath (2008) , their Fig. 3 ( http://www.jneurosci.org/cgi/content/full/28/1/116/F3 )]. In contrast, during the automatic detection of novelty within the environment, increased neural activity in the hippocampus may reflect the relatively “pure” signature of a comparator mechanism, free from modulation by top-down influences ( Kumaran and Maguire, 2006 ). An important avenue for future work, therefore, will be to explore the importance of reciprocal interactions between the hippocampus and prefrontal cortices that vary according to specific task requirements (e.g., explicit recognition memory task) and therefore determine the amplitude of observed hippocampal novelty (or familiarity) signals.

To summarize, the study by Hannula and Ranganath (2008) nicely complements existing neuropsychological data concerning the importance of the hippocampus to short-term associative memory. Moreover, the evidence provided yields new insights into the nature of the hippocampal contribution to short-term memory, suggesting that it participates primarily in encoding and retrieval, but perhaps not active maintenance of associative information. One important direction for future research will be to develop and empirically test formal computational models of recognition memory [e.g., those described by Norman and O'Reilly (2003) ] and automatic novelty processing using fMRI. Ideally, these models should include MTL components as well as task-specific modulatory interactions with higher regions (e.g., prefrontal cortex). In this way, it may be possible to achieve a precise understanding of how novelty and familiarity are computed in the MTL and how these signals are used by other brain regions to effect successful recognition memory judgments and to automatically detect novelty within our sensory environment.

Editor's Note: These short, critical reviews of recent papers in the Journal, written exclusively by graduate students or postdoctoral fellows, are intended to summarize the important findings of the paper and provide additional insight and commentary. For more information on the format and purpose of the Journal Club, please see http://www.jneurosci.org/misc/ifa_features.shtml .

• Correspondence should be addressed to Dr. Dharshan Kumaran, Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, 12 Queen Square, London WC1N 3BG, UK. d.kumaran{at}fil.ion.ucl.ac.uk
• Aggleton JP
• Eichenbaum H
• Hannula DE ,
• Ranganath C
• Hartley T ,
• Cipolotti L ,
• Vargha-Khadem F ,
• Jonides J ,
• Lustig CA ,
• Berman MG ,
• Kumaran D ,
• Miller EK ,
• Norman KA ,
• O'Reilly RC

In this issue

• Table of Contents
• Table of Contents (PDF)
• About the Cover
• Index by author

Thank you for sharing this Journal of Neuroscience article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager

• Tweet Widget
• Facebook Like
• Google Plus One

Jump to section

Responses to this article, jump to comment:, related articles, cited by..., more in this toc section.

• Enhancement of Haloperidol-Induced Catalepsy by GPR143, an l -DOPA Receptor, in Striatal Cholinergic Interneurons
• Environmental and Interpersonal Factors Impact the Developing Brain
• Neural Dynamics of Self-Referential Processing and the Insight for Decoding Self-Concepts

• A-Z Publications

Annual Review of Psychology

Volume 59, 2008, review article, the mind and brain of short-term memory.

• John Jonides 1 , Richard L. Lewis 1 , Derek Evan Nee 1 , Cindy A. Lustig 1 , Marc G. Berman 1 , and Katherine Sledge Moore 1
• View Affiliations Hide Affiliations Affiliations: Department of Psychology, University of Michigan, Ann Arbor, Michigan 48109; email: [email protected]
• Vol. 59:193-224 (Volume publication date January 2008) https://doi.org/10.1146/annurev.psych.59.103006.093615
• © Annual Reviews

The past 10 years have brought near-revolutionary changes in psychological theories about short-term memory, with similarly great advances in the neurosciences. Here, we critically examine the major psychological theories (the “mind”) of short-term memory and how they relate to evidence about underlying brain mechanisms. We focus on three features that must be addressed by any satisfactory theory of short-term memory. First, we examine the evidence for the architecture of short-term memory, with special attention to questions of capacity and how—or whether—short-term memory can be separated from long-term memory. Second, we ask how the components of that architecture enact processes of encoding, maintenance, and retrieval. Third, we describe the debate over the reason about forgetting from short-term memory, whether interference or decay is the cause. We close with a conceptual model tracing the representation of a single item through a short-term memory task, describing the biological mechanisms that might support psychological processes on a moment-by-moment basis as an item is encoded, maintained over a delay with some forgetting, and ultimately retrieved.

Article metrics loading...

Full text loading...

Data & Media loading...

• Article Type: Review Article

Most Read This Month

Most cited most cited rss feed, job burnout, executive functions, social cognitive theory: an agentic perspective, on happiness and human potentials: a review of research on hedonic and eudaimonic well-being, sources of method bias in social science research and recommendations on how to control it, mediation analysis, missing data analysis: making it work in the real world, grounded cognition, personality structure: emergence of the five-factor model, motivational beliefs, values, and goals.

Publication Date: 10 Jan 2008

Online Option

Sign in to access your institutional or personal subscription or get immediate access to your online copy - available in PDF and ePub formats

Grab your spot at the free arXiv Accessibility Forum

Help | Advanced Search

Computer Science > Neural and Evolutionary Computing

Title: understanding lstm -- a tutorial into long short-term memory recurrent neural networks.

Abstract: Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN) are one of the most powerful dynamic classifiers publicly known. The network itself and the related learning algorithms are reasonably well documented to get an idea how it works. This paper will shed more light into understanding how LSTM-RNNs evolved and why they work impressively well, focusing on the early, ground-breaking publications. We significantly improved documentation and fixed a number of errors and inconsistencies that accumulated in previous publications. To support understanding we as well revised and unified the notation used.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

1 blog link

Dblp - cs bibliography, bibtex formatted citation.

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

REVIEW article

Working memory from the psychological and neurosciences perspectives: a review.

• 1 Department of Neurosciences, School of Medical Sciences, Universiti Sains Malaysia, Kubang Kerian, Malaysia
• 2 Center for Neuroscience Services and Research, Universiti Sains Malaysia, Kubang Kerian, Malaysia

Since the concept of working memory was introduced over 50 years ago, different schools of thought have offered different definitions for working memory based on the various cognitive domains that it encompasses. The general consensus regarding working memory supports the idea that working memory is extensively involved in goal-directed behaviors in which information must be retained and manipulated to ensure successful task execution. Before the emergence of other competing models, the concept of working memory was described by the multicomponent working memory model proposed by Baddeley and Hitch. In the present article, the authors provide an overview of several working memory-relevant studies in order to harmonize the findings of working memory from the neurosciences and psychological standpoints, especially after citing evidence from past studies of healthy, aging, diseased, and/or lesioned brains. In particular, the theoretical framework behind working memory, in which the related domains that are considered to play a part in different frameworks (such as memory’s capacity limit and temporary storage) are presented and discussed. From the neuroscience perspective, it has been established that working memory activates the fronto-parietal brain regions, including the prefrontal, cingulate, and parietal cortices. Recent studies have subsequently implicated the roles of subcortical regions (such as the midbrain and cerebellum) in working memory. Aging also appears to have modulatory effects on working memory; age interactions with emotion, caffeine and hormones appear to affect working memory performances at the neurobiological level. Moreover, working memory deficits are apparent in older individuals, who are susceptible to cognitive deterioration. Another younger population with working memory impairment consists of those with mental, developmental, and/or neurological disorders such as major depressive disorder and others. A less coherent and organized neural pattern has been consistently reported in these disadvantaged groups. Working memory of patients with traumatic brain injury was similarly affected and shown to have unusual neural activity (hyper- or hypoactivation) as a general observation. Decoding the underlying neural mechanisms of working memory helps support the current theoretical understandings concerning working memory, and at the same time provides insights into rehabilitation programs that target working memory impairments from neurophysiological or psychological aspects.

Introduction

Working memory has fascinated scholars since its inception in the 1960’s ( Baddeley, 2010 ; D’Esposito and Postle, 2015 ). Indeed, more than a century of scientific studies revolving around memory in the fields of psychology, biology, or neuroscience have not completely agreed upon a unified categorization of memory, especially in terms of its functions and mechanisms ( Cowan, 2005 , 2008 ; Baddeley, 2010 ). From the coining of the term “memory” in the 1880’s by Hermann Ebbinghaus, to the distinction made between primary and secondary memory by William James in 1890, and to the now widely accepted and used categorizations of memory that include: short-term, long-term, and working memories, studies that have tried to decode and understand this abstract concept called memory have been extensive ( Cowan, 2005 , 2008 ). Short and long-term memory suggest that the difference between the two lies in the period that the encoded information is retained. Other than that, long-term memory has been unanimously understood as a huge reserve of knowledge about past events, and its existence in a functioning human being is without dispute ( Cowan, 2008 ). Further categorizations of long-term memory include several categories: (1) episodic; (2) semantic; (3) Pavlovian; and (4) procedural memory ( Humphreys et al., 1989 ). For example, understanding and using language in reading and writing demonstrates long-term storage of semantics. Meanwhile, short-term memory was defined as temporarily accessible information that has a limited storage time ( Cowan, 2008 ). Holding a string of meaningless numbers in the mind for brief delays reflects this short-term component of memory. Thus, the concept of working memory that shares similarities with short-term memory but attempts to address the oversimplification of short-term memory by introducing the role of information manipulation has emerged ( Baddeley, 2012 ). This article seeks to present an up-to-date introductory overview of the realm of working memory by outlining several working memory studies from the psychological and neurosciences perspectives in an effort to refine and unite the scientific knowledge concerning working memory.

The Multicomponent Working Memory Model

When one describes working memory, the multicomponent working memory model is undeniably one of the most prominent working memory models that is widely cited in literatures ( Baars and Franklin, 2003 ; Cowan, 2005 ; Chein et al., 2011 ; Ashkenazi et al., 2013 ; D’Esposito and Postle, 2015 ; Kim et al., 2015 ). Baddeley and Hitch (1974) proposed a working memory model that revolutionized the rigid and dichotomous view of memory as being short or long-term, although the term “working memory” was first introduced by Miller et al. (1960) . The working memory model posited that as opposed to the simplistic functions of short-term memory in providing short-term storage of information, working memory is a multicomponent system that manipulates information storage for greater and more complex cognitive utility ( Baddeley and Hitch, 1974 ; Baddeley, 1996 , 2000b ). The three subcomponents involved are phonological loop (or the verbal working memory), visuospatial sketchpad (the visual-spatial working memory), and the central executive which involves the attentional control system ( Baddeley and Hitch, 1974 ; Baddeley, 2000b ). It was not until 2000 that another component termed “episodic buffer” was introduced into this working memory model ( Baddeley, 2000a ). Episodic buffer was regarded as a temporary storage system that modulates and integrates different sensory information ( Baddeley, 2000a ). In short, the central executive functions as the “control center” that oversees manipulation, recall, and processing of information (non-verbal or verbal) for meaningful functions such as decision-making, problem-solving or even manuscript writing. In Baddeley and Hitch (1974) ’s well-cited paper, information received during the engagement of working memory can also be transferred to long-term storage. Instead of seeing working memory as merely an extension and a useful version of short-term memory, it appears to be more closely related to activated long-term memory, as suggested by Cowan (2005 , 2008 ), who emphasized the role of attention in working memory; his conjectures were later supported by Baddeley (2010) . Following this, the current development of the multicomponent working memory model could be retrieved from Baddeley’s article titled “Working Memory” published in Current Biology , in Figure 2 ( Baddeley, 2010 ).

An Embedded-Processes Model of Working Memory

Notwithstanding the widespread use of the multicomponent working memory model, Cowan (1999 , 2005 ) proposed the embedded-processes model that highlights the roles of long-term memory and attention in facilitating working memory functioning. Arguing that the Baddeley and Hitch (1974) model simplified perceptual processing of information presentation to the working memory store without considering the focus of attention to the stimuli presented, Cowan (2005 , 2010 ) stressed the pivotal and central roles of working memory capacity for understanding the working memory concept. According to Cowan (2008) , working memory can be conceptualized as a short-term storage component with a capacity limit that is heavily dependent on attention and other central executive processes that make use of stored information or that interact with long-term memory. The relationships between short-term, long-term, and working memory could be presented in a hierarchical manner whereby in the domain of long-term memory, there exists an intermediate subset of activated long-term memory (also the short-term storage component) and working memory belongs to the subset of activated long-term memory that is being attended to ( Cowan, 1999 , 2008 ). An illustration of Cowan’s theoretical framework on working memory can be traced back to Figure 1 in his paper titled “What are the differences between long-term, short-term, and working memory?” published in Progress in Brain Research ( Cowan, 2008 ).

Alternative Models

Cowan’s theoretical framework toward working memory is consistent with Engle (2002) ’s view, in which it was posited that working memory capacity is comparable to directed or held attention information inhibition. Indeed, in their classic study on reading span and reading comprehension, Daneman and Carpenter (1980) demonstrated that working memory capacity, which was believed to be reflected by the reading span task, strongly correlated with various comprehension tests. Surely, recent and continual growth in the memory field has also demonstrated the development of other models such as the time-based resource-sharing model proposed by several researchers ( Barrouillet et al., 2004 , 2009 ; Barrouillet and Camos, 2007 ). This model similarly demonstrated that cognitive load and working memory capacity that were so often discussed by working memory researchers were mainly a product of attention that one receives to allocate to tasks at hand ( Barrouillet et al., 2004 , 2009 ; Barrouillet and Camos, 2007 ). In fact, the allocated cognitive resources for a task (such as provided attention) and the duration of such allocation dictated the likelihood of success in performing the tasks ( Barrouillet et al., 2004 , 2009 ; Barrouillet and Camos, 2007 ). This further highlighted the significance of working memory in comparison with short-term memory in that, although information retained during working memory is not as long-lasting as long-term memory, it is not the same and deviates from short-term memory for it involves higher-order processing and executive cognitive controls that are not observed in short-term memory. A more detailed presentation of other relevant working memory models that shared similar foundations with Cowan’s and emphasized the roles of long-term memory can be found in the review article by ( D’Esposito and Postle, 2015 ).

In addition, in order to understand and compare similarities and disparities in different proposed models, about 20 years ago, Miyake and Shah (1999) suggested theoretical questions to authors of different models in their book on working memory models. The answers to these questions and presentations of models by these authors gave rise to a comprehensive definition of working memory proposed by Miyake and Shah (1999 , p. 450), “working memory is those mechanisms or processes that are involved in the control, regulation, and active maintenance of task-relevant information in the service of complex cognition, including novel as well as familiar, skilled tasks. It consists of a set of processes and mechanisms and is not a fixed ‘place’ or ‘box’ in the cognitive architecture. It is not a completely unitary system in the sense that it involves multiple representational codes and/or different subsystems. Its capacity limits reflect multiple factors and may even be an emergent property of the multiple processes and mechanisms involved. Working memory is closely linked to LTM, and its contents consist primarily of currently activated LTM representations, but can also extend to LTM representations that are closely linked to activated retrieval cues and, hence, can be quickly activated.” That said, in spite of the variability and differences that have been observed following the rapid expansion of working memory understanding and its range of models since the inception of the multicomponent working memory model, it is worth highlighting that the roles of executive processes involved in working memory are indisputable, irrespective of whether different components exist. Such notion is well-supported as Miyake and Shah, at the time of documenting the volume back in the 1990’s, similarly noted that the mechanisms of executive control were being heavily investigated and emphasized ( Miyake and Shah, 1999 ). In particular, several domains of working memory such as the focus of attention ( Cowan, 1999 , 2008 ), inhibitory controls ( Engle and Kane, 2004 ), maintenance, manipulation, and updating of information ( Baddeley, 2000a , 2010 ), capacity limits ( Cowan, 2005 ), and episodic buffer ( Baddeley, 2000a ) were executive processes that relied on executive control efficacy (see also Miyake and Shah, 1999 ; Barrouillet et al., 2004 ; D’Esposito and Postle, 2015 ).

The Neuroscience Perspective

Following such cognitive conceptualization of working memory developed more than four decades ago, numerous studies have intended to tackle this fascinating working memory using various means such as decoding its existence at the neuronal level and/or proposing different theoretical models in terms of neuronal activity or brain activation patterns. Table 1 offers the summarized findings of these literatures. From the cognitive neuroscientific standpoint, for example, the verbal and visual-spatial working memories were examined separately, and the distinction between the two forms was documented through studies of patients with overt impairment in short-term storage for different verbal or visual tasks ( Baddeley, 2000b ). Based on these findings, associations or dissociations with the different systems of working memory (such as phonological loops and visuospatial sketchpad) were then made ( Baddeley, 2000b ). It has been established that verbal and acoustic information activates Broca’s and Wernicke’s areas while visuospatial information is represented in the right hemisphere ( Baddeley, 2000b ). Not surprisingly, many supporting research studies have pointed to the fronto-parietal network involving the dorsolateral prefrontal cortex (DLPFC), the anterior cingulate cortex (ACC), and the parietal cortex (PAR) as the working memory neural network ( Osaka et al., 2003 ; Owen et al., 2005 ; Chein et al., 2011 ; Kim et al., 2015 ). More precisely, the DLPFC has been largely implicated in tasks demanding executive control such as those requiring integration of information for decision-making ( Kim et al., 2015 ; Jimura et al., 2017 ), maintenance and manipulation/retrieval of stored information or relating to taxing loads (such as capacity limit) ( Osaka et al., 2003 ; Moore et al., 2013 ; Vartanian et al., 2013 ; Rodriguez Merzagora et al., 2014 ), and information updating ( Murty et al., 2011 ). Meanwhile, the ACC has been shown to act as an “attention controller” that evaluates the needs for adjustment and adaptation of received information based on task demands ( Osaka et al., 2003 ), and the PAR has been regarded as the “workspace” for sensory or perceptual processing ( Owen et al., 2005 ; Andersen and Cui, 2009 ). Figure 1 attempted to translate the theoretical formulation of the multicomponent working memory model ( Baddeley, 2010 ) to specific regions in the human brain. It is, however, to be acknowledged that the current neuroscientific understanding on working memory adopted that working memory, like other cognitive systems, involves the functional integration of the brain as a whole; and to clearly delineate its roles into multiple components with only a few regions serving as specific buffers was deemed impractical ( D’Esposito and Postle, 2015 ). Nonetheless, depicting the multicomponent working memory model in the brain offers a glimpse into the functional segregation of working memory.

TABLE 1. Working memory (WM) studies in the healthy brain.

FIGURE 1. A simplified depiction (adapted from the multicomponent working memory model by Baddeley, 2010 ) as implicated in the brain, in which the central executive assumes the role to exert control and oversee the manipulation of incoming information for intended execution. ACC, Anterior cingulate cortex.

Further investigation has recently revealed that other than the generally informed cortical structures involved in verbal working memory, basal ganglia, which lies in the subcortical layer, plays a role too ( Moore et al., 2013 ). Particularly, the caudate and thalamus were activated during task encoding, and the medial thalamus during the maintenance phase, while recorded activity in the fronto-parietal network, which includes the DLPFC and the parietal lobules, was observed only during retrieval ( Moore et al., 2013 ). These findings support the notion that the basal ganglia functions to enhance focusing on a target while at the same time suppressing irrelevant distractors during verbal working memory tasks, which is especially crucial at the encoding phase ( Moore et al., 2013 ). Besides, a study conducted on mice yielded a similar conclusion in which the mediodorsal thalamus aided the medial prefrontal cortex in the maintenance of working memory ( Bolkan et al., 2017 ). In another study by Murty et al. (2011) in which information updating, which is one of the important aspects of working memory, was investigated, the midbrain including the substantia nigra/ventral tegmental area and caudate was activated together with DLPFC and other parietal regions. Taken together, these studies indicated that brain activation of working memory are not only limited to the cortical layer ( Murty et al., 2011 ; Moore et al., 2013 ). In fact, studies on cerebellar lesions subsequently discovered that patients suffered from impairments in attention-related working memory or executive functions, suggesting that in spite of the motor functions widely attributed to the cerebellum, the cerebellum is also involved in higher-order cognitive functions including working memory ( Gottwald et al., 2004 ; Ziemus et al., 2007 ).

Shifting the attention to the neuronal network involved in working memory, effective connectivity analysis during engagement of a working memory task reinforced the idea that the DLPFC, PAR and ACC belong to the working memory circuitry, and bidirectional endogenous connections between all these regions were observed in which the left and right PAR were the modeled input regions ( Dima et al., 2014 ) (refer to Supplementary Figure 1 in Dima et al., 2014 ). Effective connectivity describes the attempt to model causal influence of neuronal connections in order to better understand the hidden neuronal states underlying detected neuronal responses ( Friston et al., 2013 ). Another similar study of working memory using an effective connectivity analysis that involved more brain regions, including the bilateral middle frontal gyrus (MFG), ACC, inferior frontal cortex (IFC), and posterior parietal cortex (PPC) established the modulatory effect of working memory load in this fronto-parietal network with memory delay as the driving input to the bilateral PPC ( Ma et al., 2012 ) (refer to Figure 1 in Ma et al., 2012 ).

Moving away from brain regions activated but toward the in-depth neurobiological side of working memory, it has long been understood that the limited capacity of working memory and its transient nature, which are considered two of the defining characteristics of working memory, indicate the role of persistent neuronal firing (see Review Article by D’Esposito and Postle, 2015 ; Zylberberg and Strowbridge, 2017 ; see also Silvanto, 2017 ), that is, continuous action potentials are generated in neurons along the neural network. However, this view was challenged when activity-silent synaptic mechanisms were found to also be involved ( Mongillo et al., 2008 ; Rose et al., 2016 ; see also Silvanto, 2017 ). Instead of holding relevant information through heightened and persistent neuronal firing, residual calcium at the presynaptic terminals was suggested to have mediated the working memory process ( Mongillo et al., 2008 ). This synaptic theory was further supported when TMS application produced a reactivation effect of past information that was not needed or attended at the conscious level, hence the TMS application facilitated working memory efficacy ( Rose et al., 2016 ). As it happens, this provided evidence from the neurobiological viewpoint to support Cowan’s theorized idea of “activated long-term memory” being a feature of working memory as non-cued past items in working memory that were assumed to be no longer accessible were actually stored in a latent state and could be brought back into consciousness. However, the researchers cautioned the use of the term “activated long-term memory” and opted for “prioritized long-term memory” because these unattended items maintained in working memory seemed to employ a different mechanism than items that were dropped from working memory ( Rose et al., 2016 ). Other than the synaptic theory, the spiking working memory model proposed by Fiebig and Lansner (2017) that borrowed the concept from fast Hebbian plasticity similarly disagreed with persistent neuronal activity and demonstrated that working memory processes were instead manifested in discrete oscillatory bursts.

Age and Working Memory

Nevertheless, having established a clear working memory circuitry in the brain, differences in brain activations, neural patterns or working memory performances are still apparent in different study groups, especially in those with diseased or aging brains. For a start, it is well understood that working memory declines with age ( Hedden and Gabrieli, 2004 ; Ziaei et al., 2017 ). Hence, older participants are expected to perform poorer on a working memory task when making comparison with relatively younger task takers. In fact, it was reported that decreases in cortical surface area in the frontal lobe of the right hemisphere was associated with poorer performers ( Nissim et al., 2017 ). In their study, healthy (those without mild cognitive impairments [MCI] or neurodegenerative diseases such as dementia or Alzheimer’s) elderly people with an average age of 70 took the n-back working memory task while magnetic resonance imaging (MRI) scans were obtained from them ( Nissim et al., 2017 ). The outcomes exhibited that a decrease in cortical surface areas in the superior frontal gyrus, pars opercularis of the inferior frontal gyrus, and medial orbital frontal gyrus that was lateralized to the right hemisphere, was significantly detected among low performers, implying an association between loss of brain structural integrity and working memory performance ( Nissim et al., 2017 ). There was no observed significant decline in cortical thickness of the studied brains, which is assumed to implicate neurodegenerative tissue loss ( Nissim et al., 2017 ).

Moreover, another extensive study that examined cognitive functions of participants across the lifespan using functional magnetic resonance imaging (fMRI) reported that the right lateralized fronto-parietal regions in addition to the ventromedial prefrontal cortex (VMPFC), posterior cingulate cortex, and left angular and middle frontal gyri (the default mode regions) in older adults showed reduced modulation of task difficulty, which was reflective of poorer task performance ( Rieck et al., 2017 ). In particular, older-age adults (55–69 years) exhibited diminished brain activations (positive modulation) as compared to middle-age adults (35–54 years) with increasing task difficulty, whereas lesser deactivation (negative modulation) was observed between the transition from younger adults (20–34 years) to middle-age adults ( Rieck et al., 2017 ). This provided insights on cognitive function differences during an individual’s lifespan at the neurobiological level, which hinted at the reduced ability or efficacy of the brain to modulate functional regions to increased difficulty as one grows old ( Rieck et al., 2017 ). As a matter of fact, such an opinion was in line with the Compensation-Related Utilization of Neural Circuits Hypothesis (CRUNCH) proposed by Reuter-Lorenz and Cappell (2008) . The CRUNCH likewise agreed upon reduced neural efficiency in older adults and contended that age-associated cognitive decline brought over-activation as a compensatory mechanism; yet, a shift would occur as task loads increase and under-activation would then be expected because older adults with relatively lesser cognitive resources would max out their ‘cognitive reserve’ sooner than younger adults ( Reuter-Lorenz and Park, 2010 ; Schneider-Garces et al., 2010 ).

In addition to those findings, emotional distractors presented during a working memory task were shown to alter or affect task performance in older adults ( Oren et al., 2017 ; Ziaei et al., 2017 ). Based on the study by Oren et al. (2017) who utilized the n-back task paired with emotional distractors with neutral or negative valence in the background, negative distractors with low load (such as 1-back) resulted in shorter response time (RT) in the older participants ( M age = 71.8), although their responses were not significantly more accurate when neutral distractors were shown. Also, lesser activations in the bilateral MFG, VMPFC, and left PAR were reported in the old-age group during negative low load condition. This finding subsequently demonstrated the results of emotional effects on working memory performance in older adults ( Oren et al., 2017 ). Further functional connectivity analyses revealed that the amygdala, the region well-known to be involved in emotional processing, was deactivated and displayed similar strength in functional connectivity regardless of emotional or load conditions in the old-age group ( Oren et al., 2017 ). This finding went in the opposite direction of that observed in the younger group in which the amygdala was strongly activated with less functional connections to the bilateral MFG and left PAR ( Oren et al., 2017 ). This might explain the shorter reported RT, which was an indication of improved working memory performance, during the emotional working memory task in the older adults as their amygdala activation was suppressed as compared to the younger adults ( Oren et al., 2017 ).

Interestingly, a contrasting neural connection outcome was reported in the study by Ziaei et al. (2017) in which differential functional networks relating to emotional working memory task were employed by the two studied groups: (1) younger ( M age = 22.6) and (2) older ( M age = 68.2) adults. In the study, emotional distractors with positive, neutral, and negative valence were presented during a visual working memory task and older adults were reported to adopt two distinct networks involving the VMPFC to encode and process positive and negative distractors while younger adults engaged only one neural pathway ( Ziaei et al., 2017 ). The role of amygdala engagement in processing only negative items in the younger adults, but both negative and positive distractors in the older adults, could be reflective of the older adults’ better ability at regulating negative emotions which might subsequently provide a better platform for monitoring working memory performance and efficacy as compared to their younger counterparts ( Ziaei et al., 2017 ). This study’s findings contradict those by Oren et al. (2017) in which the amygdala was found to play a bigger role in emotional working memory tasks among older participants as opposed to being suppressed as reported by Oren et al. (2017) . Nonetheless, after overlooking the underlying neural mechanism relating to emotional distractors, it was still agreed that effective emotional processing sustained working memory performance among older/elderly people ( Oren et al., 2017 ; Ziaei et al., 2017 ).

Aside from the interaction effect between emotion and aging on working memory, the impact of caffeine was also investigated among elders susceptible to age-related cognitive decline; and those reporting subtle cognitive deterioration 18-months after baseline measurement showed less marked effects of caffeine in the right hemisphere, unlike those with either intact cognitive ability or MCI ( Haller et al., 2017 ). It was concluded that while caffeine’s effects were more pronounced in MCI participants, elders in the early stages of cognitive decline displayed diminished sensitivity to caffeine after being tested with the n-back task during fMRI acquisition ( Haller et al., 2017 ). It is, however, to be noted that the working memory performance of those displaying minimal cognitive deterioration was maintained even though their brain imaging uncovered weaker brain activation in a more restricted area ( Haller et al., 2017 ). Of great interest, such results might present a useful brain-based marker that can be used to identify possible age-related cognitive decline.

Similar findings that demonstrated more pronounced effects of caffeine on elderly participants were reported in an older study, whereas older participants in the age range of 50–65 years old exhibited better working memory performance that offset the cognitive decline observed in those with no caffeine consumption, in addition to displaying shorter reaction times and better motor speeds than observed in those without caffeine ( Rees et al., 1999 ). Animal studies using mice showed replication of these results in mutated mice models of Alzheimer’s disease or older albino mice, both possibly due to the reported results of reduced amyloid production or brain-derived neurotrophic factor and tyrosine-kinase receptor. These mice performed significantly better after caffeine treatment in tasks that supposedly tapped into working memory or cognitive functions ( Arendash et al., 2006 ). Such direct effects of caffeine on working memory in relation to age was further supported by neuroimaging studies ( Haller et al., 2013 ; Klaassen et al., 2013 ). fMRI uncovered increased brain activation in regions or networks of working memory, including the fronto-parietal network or the prefrontal cortex in old-aged ( Haller et al., 2013 ) or middle-aged adults ( Klaassen et al., 2013 ), even though the behavioral measures of working memory did not differ. Taken together, these outcomes offered insight at the neurobiological level in which caffeine acts as a psychoactive agent that introduces changes and alters the aging brain’s biological environment that explicit behavioral testing might fail to capture due to performance maintenance ( Haller et al., 2013 , 2017 ; Klaassen et al., 2013 ).

With respect to physiological effects on cognitive functions (such as effects of caffeine on brain physiology), estradiol, the primary female sex hormone that regulates menstrual cycles, was found to also modulate working memory by engaging different brain activity patterns during different phases of the menstrual cycle ( Joseph et al., 2012 ). The late follicular (LF) phase of the menstrual cycle, characterized by high estradiol levels, was shown to recruit more of the right hemisphere that was associated with improved working memory performance than did the early follicular (EF) phase, which has lower estradiol levels although overall, the direct association between estradiol levels and working memory was inconclusive ( Joseph et al., 2012 ). The finding that estradiol levels modified brain recruitment patterns at the neurobiological level, which could indirectly affect working memory performance, presents implications that working memory impairment reported in post-menopausal women (older aged women) could indicate a link with estradiol loss ( Joseph et al., 2012 ). In 2000, post-menopausal women undergoing hormone replacement therapy, specifically estrogen, were found to have better working memory performance in comparison with women who took estrogen and progestin or women who did not receive the therapy ( Duff and Hampson, 2000 ). Yet, interestingly, a study by Janowsky et al. (2000) showed that testosterone supplementation counteracted age-related working memory decline in older males, but a similar effect was not detected in older females who were supplemented with estrogen. A relatively recent paper might have provided the explanation to such contradicting outcomes ( Schöning et al., 2007 ). As demonstrated in the study using fMRI, the nature of the task (such as verbal or visual-spatial) might have played a role as a higher level of testosterone (in males) correlated with activations of the left inferior parietal cortex, which was deemed a key region in spatial processing that subsequently brought on better performance in a mental-rotation task. In contrast, significant correlation between estradiol and other cortical activations in females in the midluteal phase, who had higher estradiol levels, did not result in better performance of the task compared to women in the EF phase or men ( Schöning et al., 2007 ). Nonetheless, it remains premature to conclude that age-related cognitive decline was a result of hormonal (estradiol or testosterone) fluctuations although hormones might have modulated the effect of aging on working memory.

Other than the presented interaction effects of age and emotions, caffeine, and hormones, other studies looked at working memory training in the older population in order to investigate working memory malleability in the aging brain. Findings of improved performance for the same working memory task after training were consistent across studies ( Dahlin et al., 2008 ; Borella et al., 2017 ; Guye and von Bastian, 2017 ; Heinzel et al., 2017 ). Such positive results demonstrated effective training gains regardless of age difference that could even be maintained until 18 months later ( Dahlin et al., 2008 ) even though the transfer effects of such training to other working memory tasks need to be further elucidated as strong evidence of transfer with medium to large effect size is lacking ( Dahlin et al., 2008 ; Guye and von Bastian, 2017 ; Heinzel et al., 2017 ; see also Karbach and Verhaeghen, 2014 ). The studies showcasing the effectiveness of working memory training presented a useful cognitive intervention that could partially stall or delay cognitive decline. Table 2 presents an overview of the age-related working memory studies.

TABLE 2. Working memory (WM) studies in relation to age.

The Diseased Brain and Working Memory

Age is not the only factor influencing working memory. In recent studies, working memory deficits in populations with mental or neurological disorders were also being investigated (see Table 3 ). Having identified that the working memory circuitry involves the fronto-parietal region, especially the prefrontal and parietal cortices, in a healthy functioning brain, targeting these areas in order to understand how working memory is affected in a diseased brain might provide an explanation for the underlying deficits observed at the behavioral level. For example, it was found that individuals with generalized or social anxiety disorder exhibited reduced DLPFC activation that translated to poorer n-back task performance in terms of accuracy and RT when compared with the controls ( Balderston et al., 2017 ). Also, VMPFC and ACC, representing the default mode network (DMN), were less inhibited in these individuals, indicating that cognitive resources might have been divided and resulted in working memory deficits due to the failure to disengage attention from persistent anxiety-related thoughts ( Balderston et al., 2017 ). Similar speculation can be made about individuals with schizophrenia. Observed working memory deficits might be traced back to impairments in the neural networks that govern attentional-control and information manipulation and maintenance ( Grot et al., 2017 ). The participants performed a working memory binding task, whereby they had to make sure that the word-ellipse pairs presented during the retrieval phase were identical to those in the encoding phase in terms of location and verbal information; results concluded that participants with schizophrenia had an overall poorer performance compared to healthy controls when they were asked to actively bind verbal and spatial information ( Grot et al., 2017 ). This was reflected in the diminished activation in the schizophrenia group’s ventrolateral prefrontal cortex and the PPC that were said to play a role in manipulation and reorganization of information during encoding and maintenance of information after encoding ( Grot et al., 2017 ).

TABLE 3. Working memory (WM) studies in the diseased brain.

In addition, patients with major depressive disorder (MDD) displayed weaker performance in the working memory updating domain in which information manipulation was needed when completing a visual working memory task ( Le et al., 2017 ). The working memory task employed in the study was a delayed recognition task that required participants to remember and recognize the faces or scenes as informed after stimuli presentation while undergoing fMRI scan ( Le et al., 2017 ). Subsequent functional connectivity analyses revealed that the fusiform face area (FFA), parahippocampal place area (PPA), and left MFG showed aberrant activity in the MDD group as compared to the control group ( Le et al., 2017 ). These brain regions are known to be the visual association area and the control center of working memory and have been implicated in visual working memory updating in healthy adults ( Le et al., 2017 ). Therefore, altered visual cortical functions and load-related activation in the prefrontal cortex in the MDD group implied that the cognitive control for visual information processing and updating might be impaired at the input or control level, which could have ultimately played a part in the depressive symptoms ( Le et al., 2017 ).

Similarly, during a verbal delayed match to sample task that asked participants to sub-articulatorly rehearse presented target letters for subsequent letter-matching, individuals with bipolar affective disorder displayed aberrant neural interactions between the right amygdala, which is part of the limbic system implicated in emotional processing as previously described, and ipsilateral cortical regions often concerned with verbal working memory, pointing out that the cortico-amygdalar connectivity was disrupted, which led to verbal working memory deficits ( Stegmayer et al., 2015 ). As an attempt to gather insights into previously reported hyperactivation in the amygdala in bipolar affective disorder during an articulatory working memory task, functional connectivity analyses revealed that negative functional interactions seen in healthy controls were not replicated in patients with bipolar affective disorder ( Stegmayer et al., 2015 ). Consistent with the previously described study about emotional processing effects on working memory in older adults, this reported outcome was suggestive of the brain’s failed attempts to suppress pathological amygdalar activation during a verbal working memory task ( Stegmayer et al., 2015 ).

Another affected group with working memory deficits that has been the subject of research interest was children with developmental disorders such as attention deficit/hyperactivity disorder (ADHD), developmental dyscalculia, and reading difficulties ( Rotzer et al., 2009 ; Ashkenazi et al., 2013 ; Wang and Gathercole, 2013 ; Maehler and Schuchardt, 2016 ). For instance, looking into the different working memory subsystems based on Baddeley’s multicomponent working memory model in children with dyslexia and/or ADHD and children with dyscalculia and/or ADHD through a series of tests, it was reported that distinctive working memory deficits by groups could be detected such that phonological loop (e.g., digit span) impairment was observed in the dyslexia group, visuospatial sketchpad (e.g., Corsi block tasks) deficits in the dyscalculia group, while central executive (e.g., complex counting span) deficits in children with ADHD ( Maehler and Schuchardt, 2016 ). Meanwhile, examination of working memory impairment in a delayed match-to-sample visual task that put emphasis on the maintenance phase of working memory by examining the brainwaves of adults with ADHD using electroencephalography (EEG) also revealed a marginally significantly lower alpha band power in the posterior regions as compared to healthy individuals, and such an observation was not significantly improved after working memory training (Cogmed working memory training, CWMT Program) ( Liu et al., 2016 ). The alpha power was considered important in the maintenance of working memory items; and lower working memory accuracy paired with lower alpha band power was indeed observed in the ADHD group ( Liu et al., 2016 ).

Not dismissing the above compiled results, children encountering disabilities in mathematical operations likewise indicated deficits in the working memory domain that were traceable to unusual brain activities at the neurobiological level ( Rotzer et al., 2009 ; Ashkenazi et al., 2013 ). It was speculated that visuospatial working memory plays a vital role when arithmetic problem-solving is involved in order to ensure intact mental representations of the numerical information ( Rotzer et al., 2009 ). Indeed, Ashkenazi et al. (2013) revealed that Block Recall, a variant of the Corsi Block Tapping test and a subtest of the Working Memory Test Battery for Children (WMTB-C) that explored visuospatial sketchpad ability, was significantly predictive of math abilities. In relation to this, studies investigating brain activation patterns and performance of visuospatial working memory task in children with mathematical disabilities identified the intraparietal sulcus (IPS), in conjunction with other regions in the prefrontal and parietal cortices, to have less activation when visuospatial working memory was deemed involved (during an adapted form of Corsi Block Tapping test made suitable for fMRI [ Rotzer et al., 2009 ]); in contrast the control group demonstrated correlations of the IPS in addition to the fronto-parietal cortical activation with the task ( Rotzer et al., 2009 ; Ashkenazi et al., 2013 ). These brain activity variations that translated to differences in overt performances between healthily developing individuals and those with atypical development highlighted the need for intervention and attention for the disadvantaged groups.

Traumatic Brain Injury and Working Memory

Physical injuries impacting the frontal or parietal lobes would reasonably be damaging to one’s working memory. This is supported in studies employing neuropsychological testing to assess cognitive impairments in patients with traumatic brain injury; and poorer cognitive performances especially involving the working memory domains were reported (see Review Articles by Dikmen et al., 2009 ; Dunning et al., 2016 ; Phillips et al., 2017 ). Research on cognitive deficits in traumatic brain injury has been extensive due to the debilitating conditions brought upon an individual daily life after the injury. Traumatic brain injuries (TBI) refer to accidental damage to the brain after being hit by an object or following rapid acceleration or deceleration ( Farrer, 2017 ). These accidents include falls, assaults, or automobile accidents and patients with TBI can be then categorized into three groups; (1) mild TBI with GCS – Glasgow Coma Scale – score of 13–15; (2) moderate TBI with GCS score of 9–12; and (3) severe TBI with GCS score of 3–8 ( Farrer, 2017 ). In a recently published meta-analysis that specifically looked at working memory impairments in patients with moderate to severe TBI, patients displayed reduced cognitive functions in verbal short-term memory in addition to verbal and visuospatial working memory in comparison to control groups ( Dunning et al., 2016 ). It was also understood from the analysis that the time lapse since injury and age of injury were deciding factors that influenced these cognitive deficits in which longer time post-injury or older age during injury were associated with greater cognitive decline ( Dunning et al., 2016 ).

Nonetheless, it is to be noted that such findings relating to age of injury could not be generalized to the child population since results from the pediatric TBI cases showed that damage could negatively impact developmental skills that could indicate a greater lag in cognitive competency as the child’s frontal lobe had yet to mature ( Anderson and Catroppa, 2007 ; Mandalis et al., 2007 ; Nadebaum et al., 2007 ; Gorman et al., 2012 ). These studies all reported working memory impairment of different domains such as attentional control, executive functions, or verbal and visuospatial working memory in the TBI group, especially for children with severe TBI ( Mandalis et al., 2007 ; Nadebaum et al., 2007 ; Gorman et al., 2012 ). Investigation of whether working memory deficits are domain-specific or -general or involve one or more mechanisms, has yielded inconsistent results. For example, Perlstein et al. (2004) found that working memory was impaired in the TBI group only when complex manipulation such as sequential coding of information is required and not accounted for by processing speed or maintenance of information, but two teams of researchers ( Perbal et al., 2003 ; Gorman et al., 2012 ) suggested otherwise. From their study on timing judgments, Perbal et al. (2003) concluded that deficits were not related to time estimation but more on generalized attentional control, working memory and processing speed problems; while Gorman et al. (2012) also attributed the lack of attentional focus to impairments observed during the working memory task. In fact, in a later study by Gorman et al. (2016) , it was shown that processing speed mediated TBI effects on working memory even though the mediation was partial. On the other hand, Vallat-Azouvi et al. (2007) reported impairments in the working memory updating domain that came with high executive demands for TBI patients. Also, Mandalis et al. (2007) similarly highlighted potential problems with attention and taxing cognitive demands in the TBI group.

From the neuroscientific perspective, hyper-activation or -connectivity in the working memory circuitry was reported in TBI patients in comparison with healthy controls when both groups engaged in working memory tasks, suggesting that the brain attempted to compensate for or re-establish lost connections upon the injury ( Dobryakova et al., 2015 ; Hsu et al., 2015 ; Wylie et al., 2015 ). For a start, it was observed that participants with mild TBI displayed increased activation in the right prefrontal cortex during a working memory task when comparing to controls ( Wylie et al., 2015 ). Interestingly, this activation pattern only occurred in patients who did not experience a complete recovery 1 week after the injury ( Wylie et al., 2015 ). Besides, low activation in the DMN was observed in mild TBI patients without cognitive recovery, and such results seemed to be useful in predicting recovery in patients in which the patients did not recover when hypoactivation (low activation) was reported, and vice versa ( Wylie et al., 2015 ). This might be suggestive of the potential of cognitive recovery simply by looking at the intensity of brain activation of the DMN, for an increase in activation of the DMN seemed to be superseded before cognitive recovery was present ( Wylie et al., 2015 ).

In fact, several studies lent support to the speculation mentioned above as hyperactivation or hypoactivation in comparison with healthy participants was similarly identified. When sex differences were being examined in working memory functional activity in mild TBI patients, hyperactivation was reported in male patients when comparing to the male control group, suggesting that the hyperactivation pattern might be the brain’s attempt at recovering impaired functions; even though hypoactivation was shown in female patients as compared to the female control group ( Hsu et al., 2015 ). The researchers from the study further explained that such hyperactivation after the trauma acted as a neural compensatory mechanism so that task performance could be maintained while hypoactivation with a poorer performance could have been the result of a more severe injury ( Hsu et al., 2015 ). Therefore, the decrease in activation in female patients, in addition to the observed worse performance, was speculated to be due to a more serious injury sustained by the female patients group ( Hsu et al., 2015 ).

In addition, investigation of the effective connectivity of moderate and severe TBI participants during a working memory task revealed that the VMPFC influenced the ACC in these TBI participants when the opposite was observed in healthy subjects ( Dobryakova et al., 2015 ). Moreover, increased inter-hemispheric transfer due to an increased number of connections between the left and right hemispheres (hyper-connectivity) without clear directionality of information flow (redundant connectivity) was also reported in the TBI participants ( Dobryakova et al., 2015 ). This study was suggestive of location-specific changes in the neural network connectivity following TBI depending on the cognitive functions at work, other than providing another support to the neural compensatory hypothesis due to the observed hyper-connectivity ( Dobryakova et al., 2015 ).

Nevertheless, inconsistent findings should not be neglected. In a study that also focused on brain connectivity analysis among patients with mild TBI by Hillary et al. (2011) , elevated task-related connectivity in the right hemisphere, in particular the prefrontal cortex, was consistently demonstrated during a working memory task while the control group showed greater left hemispheric activation. This further supported the right lateralization of the brain to reallocate cognitive resources of TBI patients post-injury. Meanwhile, the study did not manage to obtain the expected outcome in terms of greater clustering of whole-brain connections in TBI participants as hypothesized ( Hillary et al., 2011 ). That said, no significant loss or gain of connections due to the injury could be concluded from the study, as opposed to the hyper- or hypoactivation or hyper-connectivity frequently highlighted in other similar researches ( Hillary et al., 2011 ). Furthermore, a study by Chen et al. (2012) also failed to establish the same results of increased brain activation. Instead, with every increase of the working memory load, increase in brain activation, as expected to occur and as demonstrated in the control group, was unable to be detected in the TBI group ( Chen et al., 2012 ).

Taken all the insightful studies together, another aspect not to be neglected is the neuroimaging techniques employed in contributing to the literature on TBI. Modalities other than fMRI, which focuses on localization of brain activities, show other sides of the story of working memory impairments in TBI to offer a more holistic understanding. Studies adopting electroencephalography (EEG) or diffusor tensor imaging (DTI) reported atypical brainwaves coherence or white matter integrity in patients with TBI ( Treble et al., 2013 ; Ellis et al., 2016 ; Bailey et al., 2017 ; Owens et al., 2017 ). Investigating the supero-lateral medial forebrain bundle (MFB) that innervates and consequently terminates at the prefrontal cortex, microstructural white matter damage at the said area was indicated in participants with moderate to severe TBI by comparing its integrity with the control group ( Owens et al., 2017 ). Such observation was backed up by evidence showing that the patients performed more poorly on attention-loaded cognitive tasks of factors relating to slow processing speed than the healthy participants, although a direct association between MFB and impaired attentional system was not found ( Owens et al., 2017 ).

Correspondingly, DTI study of the corpus callosum (CC), which described to hold a vital role in connecting and coordinating both hemispheres to ensure competent cognitive functions, also found compromised microstructure of the CC with low fractional anisotropy and high mean diffusivity, both of which are indications of reduced white matter integrity ( Treble et al., 2013 ). This reported observation was also found to be predictive of poorer verbal or visuospatial working memory performance in callosal subregions connecting the parietal and temporal cortices ( Treble et al., 2013 ). Adding on to these results, using EEG to examine the functional consequences of CC damage revealed that interhemispheric transfer time (IHTT) of the CC was slower in the TBI group than the control group, suggesting an inefficient communication between the two hemispheres ( Ellis et al., 2016 ). In addition, the TBI group with slow IHTT as well exhibited poorer neurocognitive functioning including working memory than the healthy controls ( Ellis et al., 2016 ).

Furthermore, comparing the working memory between TBI, MDD, TBI-MDD, and healthy participants discovered that groups with MDD and TBI-MDD performed poorer on the Sternberg working memory task but functional connectivity on the other hand, showed that increased inter-hemispheric working memory gamma connectivity was observed in the TBI and TBI-MDD groups ( Bailey et al., 2017 ). Speculation provided for the findings of such neuronal state that was not reflected in the explicit working memory performance was that the deficits might not be detected or tested by the utilized Sternberg task ( Bailey et al., 2017 ). Another explanation attempting to answer the increase in gamma connectivity in these groups was the involvement of the neural compensatory mechanism after TBI to improve performance ( Bailey et al., 2017 ). Nevertheless, such outcome implies that behavioral performances or neuropsychological outcomes might not always be reflective of the functional changes happening in the brain.

Yet, bearing in mind that TBI consequences can be vast and crippling, cognitive improvement or recovery, though complicated due to the injury severity-dependent nature, is not impossible (see Review Article by Anderson and Catroppa, 2007 ; Nadebaum et al., 2007 ; Dikmen et al., 2009 ; Chen et al., 2012 ). As reported by Wylie et al. (2015) , cognitive improvement together with functional changes in the brain could be detected in individuals with mild TBI. Increased activation in the brain during 6-week follow-up was also observed in the mild TBI participants, implicating the regaining of connections in the brain ( Chen et al., 2012 ). Administration of certain cognitively enhancing drugs such as methylphenidate was reported to be helpful in improving working memory performance too ( Manktelow et al., 2017 ). Methylphenidate as a dopamine reuptake inhibitor was found to have modulated the neural activity in the left cerebellum which subsequently correlated with improved working memory performance ( Manktelow et al., 2017 ). A simplified summary of recent studies on working memory and TBI is tabulated in Table 4 .

TABLE 4. Working memory (WM) studies in the TBI group.

General Discussion and Future Direction

In practice, all of the aforementioned studies contribute to the working memory puzzle by addressing the topic from different perspectives and employing various methodologies to study it. Several theoretical models of working memory that conceptualized different working memory mechanisms or domains (such as focus of attention, inhibitory controls, maintenance and manipulation of information, updating and integration of information, capacity limits, evaluative and executive controls, and episodic buffer) have been proposed. Coupled with the working memory tasks of various means that cover a broad range (such as Sternberg task, n-back task, Corsi block-tapping test, Wechsler’s Memory Scale [WMS], and working memory subtests in the Wechsler Adult Intelligence Scale [WAIS] – Digit Span, Letter Number Sequencing), it has been difficult, if not highly improbable, for working memory studies to reach an agreement upon a consistent study protocol that is acceptable for generalization of results due to the constraints bound by the nature of the study. Various data acquisition and neuroimaging techniques that come with inconsistent validity such as paper-and-pen neuropsychological measures, fMRI, EEG, DTI, and functional near-infrared spectroscopy (fNIRS), or even animal studies can also be added to the list. This poses further challenges to quantitatively measure working memory as only a single entity. For example, when studying the neural patterns of working memory based on Cowan’s processes-embedded model using fMRI, one has to ensure that the working memory task selected is fMRI-compatible, and demands executive control of attention directed at activated long-term memory (domain-specific). That said, on the one hand, there are tasks that rely heavily on the information maintenance such as the Sternberg task; on the other hand, there are also tasks that look into the information manipulation updating such as the n-back or arithmetic task. Meanwhile, the digit span task in WAIS investigates working memory capacity, although it can be argued that it also encompasses the domain on information maintenance and updating-. Another consideration involves the different natures (verbal/phonological and visuospatial) of the working memory tasks as verbal or visuospatial information is believed to engage differing sensory mechanisms that might influence comparison of working memory performance between tasks of different nature ( Baddeley and Hitch, 1974 ; Cowan, 1999 ). For instance, though both are n-back tasks that includes the same working memory domains, the auditory n-back differs than the visual n-back as the information is presented in different forms. This feature is especially crucial with regards to the study populations as it differentiates between verbal and visuospatial working memory competence within individuals, which are assumed to be domain-specific as demonstrated by vast studies (such as Nadler and Archibald, 2014 ; Pham and Hasson, 2014 ; Nakagawa et al., 2016 ). These test variations undeniably present further difficulties in selecting an appropriate task. Nevertheless, the adoption of different modalities yielded diverging outcomes and knowledge such as behavioral performances, functional segregation and integration in the brain, white matter integrity, brainwave coherence, and oxy- and deoxyhaemoglobin concentrations that are undeniably useful in application to different fields of study.

In theory, the neural efficiency hypothesis explains that increased efficiency of the neural processes recruit fewer cerebral resources in addition to displaying lower activation in the involved neural network ( Vartanian et al., 2013 ; Rodriguez Merzagora et al., 2014 ). This is in contrast with the neural compensatory hypothesis in which it attempted to understand diminished activation that is generally reported in participants with TBI ( Hillary et al., 2011 ; Dobryakova et al., 2015 ; Hsu et al., 2015 ; Wylie et al., 2015 ; Bailey et al., 2017 ). In the diseased brain, low activation has often been associated with impaired cognitive function ( Chen et al., 2012 ; Dobryakova et al., 2015 ; Wylie et al., 2015 ). Opportunely, the CRUNCH model proposed within the field of aging might be translated and integrated the two hypotheses here as it suitably resolved the disparity of cerebral hypo- and hyper-activation observed in weaker, less efficient brains as compared to healthy, adept brains ( Reuter-Lorenz and Park, 2010 ; Schneider-Garces et al., 2010 ). Moreover, other factors such as the relationship between fluid intelligence and working memory might complicate the current understanding of working memory as a single, isolated construct since working memory is often implied in measurements of the intelligence quotient ( Cowan, 2008 ; Vartanian et al., 2013 ). Indeed, the process overlap theory of intelligence proposed by Kovacs and Conway (2016) in which the constructs of intelligence were heavily scrutinized (such as general intelligence factors, g and its smaller counterparts, fluid intelligence or reasoning, crystallized intelligence, perceptual speed, and visual-spatial ability), and fittingly connected working memory capacity with fluid reasoning. Cognitive tests such as Raven’s Progressive Matrices or other similar intelligence tests that demand complex cognition and were reported in the paper had been found to correlate strongly with tests of working memory ( Kovacs and Conway, 2016 ). Furthermore, in accordance with such views, in the same paper, neuroimaging studies found intelligence tests also activated the same fronto-parietal network observed in working memory ( Kovacs and Conway, 2016 ).

On the other hand, even though the roles of the prefrontal cortex in working memory have been widely established, region specificity and localization in the prefrontal cortex in relation to the different working memory domains such as manipulation or delayed retention of information remain at the premature stage (see Review Article by D’Esposito and Postle, 2015 ). It has been postulated that the neural mechanisms involved in working memory are of high-dimensionality and could not always be directly captured and investigated using neurophysiological techniques such as fMRI, EEG, or patch clamp recordings even when comparing with lesion data ( D’Esposito and Postle, 2015 ). According to D’Esposito and Postle (2015) , human fMRI studies have demonstrated that a rostral-caudal functional gradient related to level of abstraction required of working memory along the frontal cortex (in which different regions in the prefrontal cortex [from rostral to caudal] might be associated with different abstraction levels) might exist. Other functional gradients relating to different aspects of working memory were similarly unraveled ( D’Esposito and Postle, 2015 ). These proposed mechanisms with different empirical evidence point to the fact that conclusive understanding regarding working memory could not yet be achieved before the inconsistent views are reconciled.

Not surprisingly, with so many aspects of working memory yet to be understood and its growing complexity, the cognitive neuroscience basis of working memory requires constant research before an exhaustive account can be gathered. From the psychological conceptualization of working memory as attempted in the multicomponent working memory model ( Baddeley and Hitch, 1974 ), to the neural representations of working memory in the brain, especially in the frontal regions ( D’Esposito and Postle, 2015 ), one important implication derives from the present review of the literatures is that working memory as a psychological construct or a neuroscientific mechanism cannot be investigated as an isolated event. The need for psychology and neuroscience to interact with each other in an active feedback cycle exists in which this cognitive system called working memory can be dissected at the biological level and refined both empirically, and theoretically.

In summary, the present article offers an account of working memory from the psychological and neuroscientific perspectives, in which theoretical models of working memory are presented, and neural patterns and brain regions engaging in working memory are discussed among healthy and diseased brains. It is believed that working memory lays the foundation for many other cognitive controls in humans, and decoding the working memory mechanisms would be the first step in facilitating understanding toward other aspects of human cognition such as perceptual or emotional processing. Subsequently, the interactions between working memory and other cognitive systems could reasonably be examined.

Author Contributions

WC wrote the manuscript with critical feedback and consultation from AAH. WC and AAH contributed to the final version of the manuscript. JA supervised the process and proofread the manuscript.

This work was supported by the Transdisciplinary Research Grant Scheme (TRGS) 203/CNEURO/6768003 and the USAINS Research Grant 2016.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer EB and handling Editor declared their shared affiliation.

Andersen, R. A., and Cui, H. (2009). Review intention, action planning, and decision making in parietal-frontal circuits. Neuron 63, 568–583. doi: 10.1016/j.neuron.2009.08.028

PubMed Abstract | CrossRef Full Text | Google Scholar

Anderson, V., and Catroppa, C. (2007). Memory outcome at 5 years post-childhood traumatic brain injury. Brain Inj. 21, 1399–1409. doi: 10.1080/02699050701785070

Arendash, G. W., Schleif, W., Rezai-Zadeh, K., Jackson, E. K., Zacharia, L. C., Cracchiolo, J. R., et al. (2006). Caffeine protects Alzheimer’s mice against cognitive impairment and reduces brain β-amyloid production. Neuroscience 142, 941–952. doi: 10.1016/j.neuroscience.2006.07.021

Ashkenazi, S., Rosenberg-lee, M., Metcalfe, A. W. S., Swigart, A. G., and Menon, V. (2013). Neuropsychologia visuo – spatial working memory is an important source of domain-general vulnerability in the development of arithmetic cognition. Neuropsychologia 51, 2305–2317. doi: 10.1016/j.neuropsychologia.2013.06.031

Baars, B. J., and Franklin, S. (2003). How conscious experience and working memory interact. Trends Cogn. Sci. 7, 166–172. doi: 10.1016/S1364-6613(03)00056-1

CrossRef Full Text | Google Scholar

Baddeley, A. (1996). Exploring the central executive. Q. J. Exp. Psychol. A 49, 5–28. doi: 10.1080/713755608

Baddeley, A. (2010). Working memory. Curr. Biol. 20, R136–R140. doi: 10.1016/j.cub.2009.12.014

Baddeley, A. (2012). Working memory: theories, models, and controversies. Annu. Rev. Psychol. 63, 1–29. doi: 10.1146/annurev-psych-120710-100422

Baddeley, A., and Hitch, G. (1974). Working memory. Psychol. Learn. Motiv. 8, 47–89. doi: 10.1016/j.cub.2009.12.014

Baddeley, A. D. (2000a). The episodic buffer : a new component of working memory? Trends Cogn. Sci. 4, 417–423. doi: 10.1016/S1364-6613(00)01538-2

Baddeley, A. D. (2000b). Short-Term and Working Memory. The Oxford Handbook of Memory. Oxford: Oxford University Press.

Google Scholar

Bailey, N. W., Rogasch, N. C., Hoy, K. E., Maller, J. J., Segrave, R. A., Sullivan, C. M., et al. (2017). Increased gamma connectivity during working memory retention following traumatic brain injury. Brain Inj. 31, 379–389. doi: 10.1080/02699052.2016.1239273

Balderston, N. L., Vytal, K. E., O’Connell, K., Torrisi, S., Letkiewicz, A., Ernst, M., et al. (2017). Anxiety patients show reduced working memory related dlPFC activation during safety and threat. Depress. Anxiety 34, 25–36. doi: 10.1002/da.22518

Barrouillet, P., Bernardin, S., and Camos, V. (2004). Time constraints and resource sharing in adults’ working memory spans. J. Exp. Psychol. Gen. 133, 83–100. doi: 10.1037/0096-3445.133.1.83

Barrouillet, P., and Camos, V. (2007). “The time-based resource-sharing model of working memory,” in The Cognitive Neuroscience of Working Memory , ed. N. Osaka (Oxford: Oxford University Press), 59–80. doi: 10.1093/acprof:oso/9780198570394.003.0004

Barrouillet, P., Gavens, N., Vergauwe, E., Gaillard, V., and Camos, V. (2009). Working memory span development: a time-based resource-sharing model account. Dev. Psychol. 45, 477–490. doi: 10.1037/a0014615

Bolkan, S. S., Stujenske, J. M., Parnaudeau, S., Spellman, T. J., Rauffenbart, C., Abbas, A. I., et al. (2017). Thalamic projections sustain prefrontal activity during working memory maintenance. Nat. Neurosci. 20, 987–996. doi: 10.1038/nn.4568

Borella, E., Carretti, B., Sciore, R., Capotosto, E., Taconnat, L., Cornoldi, C., et al. (2017). Training working memory in older adults: is there an advantage of using strategies? Psychol. Aging 32, 178–191. doi: 10.1037/pag0000155

Chein, J. M., Moore, A. B., and Conway, A. R. A. (2011). NeuroImage domain-general mechanisms of complex working memory span. Neuroimage 54, 550–559. doi: 10.1016/j.neuroimage.2010.07.067

Chen, C. J., Wu, C. H., Liao, Y. P., Hsu, H. L., Tseng, Y. C., Liu, H. L., et al. (2012). Working memory in patients with mild traumatic brain injury: functional MR imaging analysis. Radiology 264, 844–851. doi: 10.1148/radiol.12112154

Cowan, N. (1999). “An embedded-processes model of working memory,” in Models of Working Memory: Mechanisms of Active Maintenance and Executive Control , eds A. Miyake and P. Shah (Cambridge: Cambridge University Press). doi: 10.1017/S0140525X01003922

Cowan, N. (2005). Working memory capacity. Exp. Psychol. 54, 245–246. doi: 10.1027/1618-3169.54.3.245

Cowan, N. (2008). What are the differences between long-term, short-term, and working memory? Prog. Brain Res. 169, 323–338. doi: 10.1016/S0079-6123(07)00020-9

Cowan, N. (2010). The magical mystery four. Curr. Dir. Psychol. Sci. 19, 51–57. doi: 10.1177/0963721409359277

Dahlin, E., Nyberg, L., Bäckman, L., and Neely, A. S. (2008). Plasticity of executive functioning in young and older adults: immediate training gains, transfer, and long-term maintenance. Psychol. Aging 23, 720–730. doi: 10.1037/a0014296

Daneman, M., and Carpenter, P. A. (1980). Individual differences in working memory and reading. J. Verbal Learn. Verbal Behav. 19, 450–466. doi: 10.1016/S0022-5371(80)90312-6

D’Esposito, M., and Postle, B. R. (2015). The cognitive neuroscience of working memory. Annu. Rev. Psychol. 66, 115–142. doi: 10.1146/annurev-psych-010814-015031

Dikmen, S. S., Corrigan, J. D., Levin, H. S., Machamer, J., Stiers, W., and Weisskopf, M. G. (2009). Cognitive outcome following traumatic brain injury. J. Head Trauma Rehabil. 24, 430–438. doi: 10.1097/HTR.0b013e3181c133e9

Dima, D., Jogia, J., and Frangou, S. (2014). Dynamic causal modeling of load-dependent modulation of effective connectivity within the verbal working memory network. Hum. Brain Mapp. 35, 3025–3035. doi: 10.1002/hbm.22382

Dobryakova, E., Boukrina, O., and Wylie, G. R. (2015). Investigation of information flow during a novel working memory task in individuals with traumatic brain injury. Brain Connect. 5, 433–441. doi: 10.1089/brain.2014.0283

Duff, S. J., and Hampson, E. (2000). A beneficial effect of estrogen on working memory in postmenopausal women taking hormone replacement therapy. Horm. Behav. 38, 262–276. doi: 10.1006/hbeh.2000.1625

Dunning, D. L., Westgate, B., and Adlam, A.-L. R. (2016). A meta-analysis of working memory impairments in survivors of moderate-to-severe traumatic brain injury. Neuropsychology 30, 811–819. doi: 10.1037/neu0000285

Ellis, M. U., DeBoard Marion, S., McArthur, D. L., Babikian, T., Giza, C., Kernan, C. L., et al. (2016). The UCLA study of children with moderate-to-severe traumatic brain injury: event-related potential measure of interhemispheric transfer time. J. Neurotrauma 33, 990–996. doi: 10.1089/neu.2015.4023

Engle, R. W. (2002). Working memory capacity as executive attention. Curr. Dir. Psychol. Sci. 11, 19–23. doi: 10.1111/1467-8721.00160

Engle, R. W., and Kane, M. J. (2004). “Executive attention, working memory capacity, and a two-factor theory of cognitive control,” in The Psychology of Learning and Motivation: Advances in Research and Theory , ed. B. H. Ross (New York, NY: Elsevier), 145–199. doi: 10.1016/S0079-7421(03)44005-X

Farrer, T. J. (2017). Encyclopedia of Geropsychology , ed. N. A. Pachana. Singapore: Springer. doi: 10.1007/978-981-287-080-3

CrossRef Full Text

Fiebig, F., and Lansner, A. (2017). A spiking working memory model based on hebbian short-term potentiation. J. Neurosci. 37, 83–96. doi: 10.1523/JNEUROSCI.1989-16.2016

Friston, K., Moran, R., and Seth, A. K. (2013). Analysing connectivity with granger causality and dynamic causal modelling. Curr. Opin. Neurobiol. 23, 172–178. doi: 10.1016/j.conb.2012.11.010

Gorman, S., Barnes, M. A., Swank, P. R., Prasad, M., Cox, C. S., and Ewing-Cobbs, L. (2016). Does processing speed mediate the effect of pediatric traumatic brain injury on working memory? Neuropsychology 30, 263–273. doi: 10.1037/neu0000214

Gorman, S., Barnes, M. A., Swank, P. R., Prasad, M., and Ewing-Cobbs, L. (2012). The effects of pediatric traumatic brain injury on verbal and visual-spatial working memory. J. Int. Neuropsychol. Soc. 18, 29–38. doi: 10.1017/S1355617711001251

Gottwald, B., Wilde, B., Mihajlovic, Z., and Mehdorn, H. M. (2004). Evidence for distinct cognitive deficits after focal cerebellar lesions. J. Neurol. Neurosurg. Psychiatry 75, 1524–1531. doi: 10.1136/jnnp.2003.018093

Grot, S., Légaré, V. P., Lipp, O., Soulières, I., Dolcos, F., and Luck, D. (2017). Abnormal prefrontal and parietal activity linked to deficient active binding in working memory in schizophrenia. Schizophr. Res. 188, 68–74. doi: 10.1016/j.schres.2017.01.021

Guye, S., and von Bastian, C. C. (2017). Working memory training in older adults: bayesian evidence supporting the absence of transfer. Psychol. Aging 32, 732–746. doi: 10.1037/pag0000206

Haller, S., Montandon, M.-L., Rodriguez, C., Moser, D., Toma, S., Hofmeister, J., et al. (2017). Caffeine impact on working memory-related network activation patterns in early stages of cognitive decline. Neuroradiology 59, 387–395. doi: 10.1007/s00234-017-1803-5

Haller, S., Rodriguez, C., Moser, D., Toma, S., Hofmeister, J., Sinanaj, I., et al. (2013). Acute caffeine administration impact on working memory-related brain activation and functional connectivity in the elderly: a BOLD and perfusion MRI study. Neuroscience 250, 364–371. doi: 10.1016/j.neuroscience.2013.07.021

Hedden, T., and Gabrieli, J. D. E. (2004). Insights into the ageing mind: a view from cognitive neuroscience. Nat. Rev. Neurosci. 5, 87–96. doi: 10.1038/nrn1323

Heinzel, S., Rimpel, J., Stelzel, C., and Rapp, M. A. (2017). Transfer effects to a multimodal dual-task after working memory training and associated neural correlates in older adults – a pilot study. Front. Hum. Neurosci. 11:85. doi: 10.3389/fnhum.2017.00085

Hillary, F. G., Medaglia, J. D., Gates, K., Molenaar, P. C., Slocomb, J., Peechatka, A., et al. (2011). Examining working memory task acquisition in a disrupted neural network. Brain 134, 1555–1570. doi: 10.1093/brain/awr043

Hsu, H.-L., Chen, D. Y.-T., Tseng, Y.-C., Kuo, Y.-S., Huang, Y.-L., Chiu, W.-T., et al. (2015). Sex differences in working memory after mild traumatic brain injury: a functional MR imaging study. Radiology 276, 828–835. doi: 10.1148/radiol.2015142549

Humphreys, M. S., Bain, J. D., and Pike, R. (1989). Different ways to cue a coherent memory system: a theory for episodic, semantic, and procedural tasks. Psychol. Rev. 96, 208–233. doi: 10.1037/0033-295X.96.2.208

Janowsky, J. S., Chavez, B., and Orwoll, E. (2000). Sex steroids modify working memory. J. Cogn. Neurosci. 12, 407–414. doi: 10.1162/089892900562228

Jimura, K., Chushak, M. S., Westbrook, A., and Braver, T. S. (2017). Intertemporal decision-making involves prefrontal control mechanisms associated with working memory. Cereb. Cortex doi: 10.1093/cercor/bhx015 [Epub ahead of print].

Joseph, J. E., Swearingen, J. E., Corbly, C. R., Curry, T. E., and Kelly, T. H. (2012). Influence of estradiol on functional brain organization for working memory. Neuroimage 59, 2923–2931. doi: 10.1016/j.neuroimage.2011.09.067

Karbach, J., and Verhaeghen, P. (2014). Making working memory work: a meta-analysis of executive control and working memory training in younger and older adults. Psychol. Sci. 25, 2027–2037. doi: 10.1177/0956797614548725

Kim, C., Kroger, J. K., Calhoun, V. D., and Clark, V. P. (2015). The role of the frontopolar cortex in manipulation of integrated information in working memory. Neurosci. Lett. 595, 25–29. doi: 10.1016/j.neulet.2015.03.044

Klaassen, E. B., De Groot, R. H. M., Evers, E. A. T., Snel, J., Veerman, E. C. I., Ligtenberg, A. J. M., et al. (2013). The effect of caffeine on working memory load-related brain activation in middle-aged males. Neuropharmacology 64, 160–167. doi: 10.1016/j.neuropharm.2012.06.026

Kovacs, K., and Conway, A. R. A. (2016). Process overlap theory: a unified account of the general factor of intelligence. Psychol. Inq. 27, 151–177. doi: 10.1080/1047840X.2016.1153946

Le, T. M., Borghi, J. A., Kujawa, A. J., Klein, D. N., and Leung, H.-C. (2017). Alterations in visual cortical activation and connectivity with prefrontal cortex during working memory updating in major depressive disorder. Neuroimage 14, 43–53. doi: 10.1016/j.nicl.2017.01.004

Liu, Z.-X., Glizer, D., Tannock, R., and Woltering, S. (2016). EEG alpha power during maintenance of information in working memory in adults with ADHD and its plasticity due to working memory training: a randomized controlled trial. Clin. Neurophysiol. 127, 1307–1320. doi: 10.1016/j.clinph.2015.10.032

Ma, L., Steinberg, J. L., Hasan, K. M., Narayana, P. A., Kramer, L. A., and Moeller, F. G. (2012). Working memory load modulation of parieto-frontal connections: evidence from dynamic causal modeling. Hum. Brain Mapp. 33, 1850–1867. doi: 10.1002/hbm.21329

Maehler, C., and Schuchardt, K. (2016). Working memory in children with specific learning disorders and/or attention deficits. Learn. Individ. Differ. 49, 341–347. doi: 10.1016/j.lindif.2016.05.007

Mandalis, A., Kinsella, G., Ong, B., and Anderson, V. (2007). Working memory and new learning following pediatric traumatic brain injury. Dev. Neuropsychol. 32, 683–701. doi: 10.1080/87565640701376045

Manktelow, A. E., Menon, D. K., Sahakian, B. J., and Stamatakis, E. A. (2017). Working memory after traumatic brain injury: the neural basis of improved performance with methylphenidate. Front. Behav. Neurosci. 11:58. doi: 10.3389/fnbeh.2017.00058

Miller, G. A., Galanter, E., and Pribram, K. H. (1960). Plans and the Structure of Behavior. New York, NY: Henry Holt and Company. doi: 10.1037/10039-000

Miyake, A., and Shah, P. (eds). (1999). Models of Working Memory: Mechanisms of Active Maintenance and Executive Control. New York, NY: Cambridge University Press. doi: 10.1017/CBO9781139174909

Mongillo, G., Barak, O., and Tsodyks, M. (2008). Synaptic theory of working memory. Science 319, 1543–1546. doi: 10.1126/science.1150769

Moore, A. B., Li, Z., Tyner, C. E., Hu, X., and Crosson, B. (2013). Bilateral basal ganglia activity in verbal working memory. Brain Lang. 125, 316–323. doi: 10.1016/j.bandl.2012.05.003

Murty, V. P., Sambataro, F., Radulescu, E., Altamura, M., Iudicello, J., Zoltick, B., et al. (2011). Selective updating of working memory content modulates meso-cortico-striatal activity. Neuroimage 57, 1264–1272. doi: 10.1016/j.neuroimage.2011.05.006

Nadebaum, C., Anderson, V., and Catroppa, C. (2007). Executive function outcomes following traumatic brain injury in young children: a five year follow-up. Dev. Neuropsychol. 32, 703–728. doi: 10.1080/87565640701376086

Nadler, R. T., and Archibald, L. M. D. (2014). The assessment of verbal and visuospatial working memory with school age canadian children. Can. J. Speech Lang. Pathol. Audiol. 38, 262–279.

Nakagawa, S., Takeuchi, H., Taki, Y., Nouchi, R., Sekiguchi, A., Kotozaki, Y., et al. (2016). Sex-related differences in the effects of sleep habits on verbal and visuospatial working memory. Front. Psychol. 7:1128. doi: 10.3389/fpsyg.2016.01128

Nissim, N. R., O’Shea, A. M., Bryant, V., Porges, E. C., Cohen, R., and Woods, A. J. (2017). Frontal structural neural correlates of working memory performance in older adults. Front. Aging Neurosci. 8:328. doi: 10.3389/fnagi.2016.00328

Oren, N., Ash, E. L., Tarrasch, R., Hendler, T., Giladi, N., and Shapira-Lichter, I. (2017). Neural patterns underlying the effect of negative distractors on working memory in older adults. Neurobiol. Aging 53, 93–102. doi: 10.1016/j.neurobiolaging.2017.01.020

Osaka, M., Osaka, N., Kondo, H., Morishita, M., Fukuyama, H., Aso, T., et al. (2003). The neural basis of individual differences in working memory capacity: an fMRI study. Neuroimage 18, 789–797. doi: 10.1016/S1053-8119(02)00032-0

Owen, A. M., McMillan, K. M., Laird, A. R., and Bullmore, E. (2005). N-back working memory paradigm: a meta-analysis of normative functional neuroimaging studies. Hum. Brain Mapp. 25, 46–59. doi: 10.1002/hbm.20131

Owens, J. A., Spitz, G., Ponsford, J. L., Dymowski, A. R., Ferris, N., and Willmott, C. (2017). White matter integrity of the medial forebrain bundle and attention and working memory deficits following traumatic brain injury. Brain Behav. 7:e00608. doi: 10.1002/brb3.608

Perbal, S., Couillet, J., Azouvi, P., and Pouthas, V. (2003). Relationships between time estimation, memory, attention, and processing speed in patients with severe traumatic brain injury. Neuropsychologia 41, 1599–1610. doi: 10.1016/S0028-3932(03)00110-6

Perlstein, W. M., Cole, M. A., Demery, J. A., Seignourel, P. J., Dixit, N. K., Larson, M. J., et al. (2004). Parametric manipulation of working memory load in traumatic brain injury: behavioral and neural correlates. J. Int. Neuropsychol. Soc. 10, 724–741. doi: 10.1017/S1355617704105110

Pham, A. V., and Hasson, R. M. (2014). Verbal and visuospatial working memory as predictors of children’s reading ability. Arch. Clin. Neuropsychol. 29, 467–477. doi: 10.1093/arclin/acu024

Phillips, N. L., Parry, L., Mandalis, A., and Lah, S. (2017). Working memory outcomes following traumatic brain injury in children: a systematic review with meta-analysis. Child Neuropsychol. 23, 26–66. doi: 10.1080/09297049.2015.1085500

Rees, K., Allen, D., and Lader, M. (1999). The influences of age and caffeine on psychomotor and cognitive function. Psychopharmacology 145, 181–188. doi: 10.1007/s002130051047

Reuter-Lorenz, P. A., and Cappell, K. A. (2008). Neurocognitive ageing and the compensation hypothesis. Curr. Dir. Psychol. Sci. 17, 177–182. doi: 10.1111/j.1467-8721.2008.00570.x

Reuter-Lorenz, P. A., and Park, D. C. (2010). Human neuroscience and the aging mind : a new look at old problems. J. Gerontol. Psychol. Sci. 65, 405–415. doi: 10.1093/geronb/gbq035

Rieck, J. R., Rodrigue, K. M., Boylan, M. A., and Kennedy, K. M. (2017). Age-related reduction of BOLD modulation to cognitive difficulty predicts poorer task accuracy and poorer fluid reasoning ability. Neuroimage 147, 262–271. doi: 10.1016/j.neuroimage.2016.12.022

Rodriguez Merzagora, A. C., Izzetoglu, M., Onaral, B., and Schultheis, M. T. (2014). Verbal working memory impairments following traumatic brain injury: an fNIRS investigation. Brain Imaging Behav. 8, 446–459. doi: 10.1007/s11682-013-9258-8

Rose, N. S., LaRocque, J. J., Riggall, A. C., Gosseries, O., Starrett, M. J., Meyering, E. E., et al. (2016). Reactivation of latent working memories with transcranial magnetic stimulation. Science 354, 1136–1139. doi: 10.1126/science.aah7011

Rotzer, S., Loenneker, T., Kucian, K., Martin, E., Klaver, P., and von Aster, M. (2009). Dysfunctional neural network of spatial working memory contributes to developmental dyscalculia. Neuropsychologia 47, 2859–2865. doi: 10.1016/j.neuropsychologia.2009.06.009

Schneider-Garces, N. J., Gordon, B. A., Brumback-Peltz, C. R., Shin, E., Lee, Y., Sutton, B. P., et al. (2010). Span, CRUNCH, and beyond: working memory capacity and the aging brain. J. Cogn. Neurosci. 22, 655–669. doi: 10.1162/jocn.2009.21230

Schöning, S., Engelien, A., Kugel, H., Schäfer, S., Schiffbauer, H., Zwitserlood, P., et al. (2007). Functional anatomy of visuo-spatial working memory during mental rotation is influenced by sex, menstrual cycle, and sex steroid hormones. Neuropsychologia 45, 3203–3214. doi: 10.1016/j.neuropsychologia.2007.06.011

Silvanto, J. (2017). Working memory maintenance: sustained firing or synaptic mechanisms? Trends Cogn. Sci. 21, 152–154. doi: 10.1016/j.tics.2017.01.009

Stegmayer, K., Usher, J., Trost, S., Henseler, I., Tost, H., Rietschel, M., et al. (2015). Disturbed cortico–amygdalar functional connectivity as pathophysiological correlate of working memory deficits in bipolar affective disorder. Eur. Arch. Psychiatry Clin. Neurosci. 265, 303–311. doi: 10.1007/s00406-014-0517-5

Treble, A., Hasan, K. M., Iftikhar, A., Stuebing, K. K., Kramer, L. A., Cox, C. S., et al. (2013). Working memory and corpus callosum microstructural integrity after pediatric traumatic brain injury: a diffusion tensor tractography study. J. Neurotrauma 30, 1609–1619. doi: 10.1089/neu.2013.2934

Vallat-Azouvi, C., Weber, T., Legrand, L., and Azouvi, P. (2007). Working memory after severe traumatic brain injury. J. Int. Neuropsychol. Soc. 13, 770–780. doi: 10.1017/S1355617707070993

Vartanian, O., Jobidon, M.-E., Bouak, F., Nakashima, A., Smith, I., Lam, Q., et al. (2013). Working memory training is associated with lower prefrontal cortex activation in a divergent thinking task. Neuroscience 236, 186–194. doi: 10.1016/j.neuroscience.2012.12.060

Wang, S., and Gathercole, S. E. (2013). Working memory deficits in children with reading difficulties: memory span and dual task coordination. J. Exp. Child Psychol. 115, 188–197. doi: 10.1016/j.jecp.2012.11.015

Wylie, G. R., Freeman, K., Thomas, A., Shpaner, M., OKeefe, M., Watts, R., et al. (2015). Cognitive improvement after mild traumatic brain injury measured with functional neuroimaging during the acute period. PLoS One 10:e0126110. doi: 10.1371/journal.pone.0126110

Ziaei, M., Salami, A., and Persson, J. (2017). Age-related alterations in functional connectivity patterns during working memory encoding of emotional items. Neuropsychologia 94, 1–12. doi: 10.1016/j.neuropsychologia.2016.11.012

Ziemus, B., Baumann, O., Luerding, R., Schlosser, R., Schuierer, G., Bogdahn, U., et al. (2007). Impaired working-memory after cerebellar infarcts paralleled by changes in bold signal of a cortico-cerebellar circuit. Neuropsychologia 45, 2016–2024. doi: 10.1016/j.neuropsychologia.2007.02.012

Zylberberg, J., and Strowbridge, B. W. (2017). Mechanisms of persistent activity in cortical circuits: possible neural substrates for working memory. Annu. Rev. Neurosci. 40, 603–627. doi: 10.1146/annurev-neuro-070815-014006

Keywords : working memory, neuroscience, psychology, cognition, brain, central executive, prefrontal cortex, review

Citation: Chai WJ, Abd Hamid AI and Abdullah JM (2018) Working Memory From the Psychological and Neurosciences Perspectives: A Review. Front. Psychol. 9:401. doi: 10.3389/fpsyg.2018.00401

Received: 24 November 2017; Accepted: 09 March 2018; Published: 27 March 2018.

Reviewed by:

Copyright © 2018 Chai, Abd Hamid and Abdullah. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Aini Ismafairus Abd Hamid, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals

Working memory articles from across Nature Portfolio

Working memory is the active and robust retention of multiple bits of information over the time-scale of a few seconds. It is distinguished from short-term memory by the involvement of executive or attentional control that makes the information flexible yet resistant to interference.

Latest Research and Reviews

Developing a fair and interpretable representation of the clock drawing test for mitigating low education and racial bias

• Jiaqing Zhang
• Sabyasachi Bandyopadhyay
• Parisa Rashidi

Timescales of learning in prefrontal cortex

The prefrontal cortex is critical for working memory, over a timescale of seconds. In this Review, Miller and Constantinidis examine how the prefrontal cortex facilitates the integration of memory systems across other timescales as well. In this framework of prefrontal learning, short-term memory and long-term memory interact to serve goal-directed behaviour.

• Jacob A. Miller
• Christos Constantinidis

Discriminating orientation information with phase consistency in alpha and low-gamma frequency bands: an EEG study

• Alireza Khadir
• Shamim Sasani Ghamsari
• Borhan Beigzadeh

Neuronal activation sequences in lateral prefrontal cortex encode visuospatial working memory during virtual navigation

The neural codes underlying working memory are not fully understood. Here the authors recorded neurons in the lateral prefrontal cortex of male macaque monkeys, during a working memory task, and identify activation sequences that encode target locations in the task.

• Alexandra Busch
• Megan Roussy
• Julio C. Martinez-Trujillo

Volatile working memory representations crystallize with practice

Delay- and choice-related activities that are essential for working-memory performance drift during learning and stabilize only after several days of expert performance.

• Arash Bellafard
• Ghazal Namvar
• Peyman Golshani

Children exhibit superior memory for attended but outdated information compared to adults

Children typically exhibit weaker memory than adults. Here, the authors report a developmental reversal-like phenomenon that children show better memory for attended but outdated information, suggesting underdeveloped memory selection in children.

News and Comment

Reply to ‘efficiency and capacity mechanisms can coexist in cognitive training’.

• Claudia C. von Bastian
• Sylvie Belleville
• Tilo Strobach

Efficiency and capacity mechanisms can coexist in cognitive training

• Da-Wei Zhang
• Bruno Sauce

Labels aid visual working memory

• Teresa Schubert

A frequency location to remember

Neuromodulation with specific frequencies at specific brain locations selectively enhances either working memory or long-term memory in older adult humans.

• Jake Rogers

Revisiting mixture models of memory

Probabilistic mixture models have contributed significantly to advancements in visual working memory research in recent decades. In a new paper, Schurgin and colleagues revisit the basic assumptions of mixture models and suggest that we cannot understand memory without first considering perception.

• Blaire Dube
• Julie D. Golomb

Random connections in memory

• Anne-Marike Schiffer

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Wind power forecasting using a GRU attention model for efficient energy management systems

• Original Paper
• Published: 15 August 2024

Cite this article

• Lakhdar Nadjib Boucetta 1 ,
• Youssouf Amrane 1 &
• Saliha Arezki 1  

Modern energy management systems play a crucial role in integrating multiple renewable energy sources into electricity grids, enabling a balanced supply–demand relationship while promoting eco-friendly energy consumption. Among these renewables, wind energy, with its environmental and economic advantages, poses challenges due to its inherent variability, demanding accurate prediction models for seamless integration. This paper presents an innovative hybrid deep learning model that integrates a gated recurrent unit (GRU)-based attention mechanism neural network for wind power generation forecast. The developed model’s performance is compared against six other models, comprising four deep learning approaches—long short-term memory (LSTM), 1D convolutional neural network, convolutional neural short-term memory (CNN-LSTM), and convolutional long short-term memory (ConvLSTM)—as well as two machine learning models—random forest and support vector regression. The proposed GRU-based attention model demonstrates superior performance, particularly in 1-step to 3-step ahead predictions, with mean absolute error values of 59.45, 114.95, and 176.06, root mean square error values of 109.03, 201.83, and 296.55, normalized root mean square error values of 0.080, 0.148, and 0.218, and coefficient of determination (R2) values of 0.992, 0.975, and 0.948, for forecast horizons of 10, 20, and 30 min, respectively. These results underscore the robust predictive capability of the proposed algorithm. Significantly, this research constitutes the first application of the hybrid GRU-based attention model to the Yalova wind turbine dataset, achieving better accuracy when compared to prior studies utilizing the same data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Explore related subjects

• Artificial Intelligence

Meliani M, Barkany AE, Abbassi IE et al (2021) Energy management in the smart grid: state-of-the-art and future trends. Int J Eng Bus Manag 13:18479790211032920. https://doi.org/10.1177/18479790211032920

Article   Google Scholar  

Wang H, Lei Z, Zhang X, Zhou B, Peng J (2019) A review of deep learning for renewable energy forecasting. Energy Convers Manag 198:111799. https://doi.org/10.1016/j.enconman.2019.111799

Hu J, Heng J, Wen J, Zhao W (2020) Deterministic and probabilistic wind speed forecasting with de-noising-reconstruction strategy and quantile regression-based algorithm. Renew Energy 162:1208–1226. https://doi.org/10.1016/j.renene.2020.08.077

Han Q, Meng F, Hu T, Chu F (2017) Non-parametric hybrid models for wind speed forecasting. Energy Convers Manag 148:554–568. https://doi.org/10.1016/j.enconman.2017.06.021

Yunus K, Thiringer T, Chen P (2016) ARIMA-based frequency-decomposed modeling of wind speed time series. IEEE Trans Power Syst 31:2546–2556. https://doi.org/10.1109/TPWRS.2015.2468586

Ait Maatallah O, Achuthan A, Janoyan K, Marzocca P (2015) Recursive wind speed forecasting based on Hammerstein Auto-Regressive model. Appl Energy 145:191–197. https://doi.org/10.1016/j.apenergy.2015.02.032

Demolli H, Dokuz AS, Ecemis A, Gokcek M (2019) Wind power forecasting based on daily wind speed data using machine learning algorithms. Energy Convers Manag 198:111823. https://doi.org/10.1016/j.enconman.2019.111823

Li L-L, Zhao X, Tseng M-L, Tan RR (2020) Short-term wind power forecasting based on support vector machine with improved dragonfly algorithm. J Clean Prod 242:118447. https://doi.org/10.1016/j.jclepro.2019.118447

Ding M, Zhou H, Xie H et al (2021) A time series model based on hybrid-kernel least-squares support vector machine for short-term wind power forecasting. ISA Trans 108:58–68. https://doi.org/10.1016/j.isatra.2020.09.002

Garg S, Krishnamurthi R (2023) A CNN encoder decoder LSTM model for sustainable wind power predictive analytics. Sustain Comput Inf Syst 38:100869. https://doi.org/10.1016/j.suscom.2023.100869

Huang J, Niu G, Guan H, Song S (2023) Ultra-short-term wind power prediction based on LSTM with loss shrinkage adam. Energies 16:3789. https://doi.org/10.3390/en16093789

Shahid F, Zameer A, Muneeb M (2021) A novel genetic LSTM model for wind power forecast. Energy 223:120069. https://doi.org/10.1016/j.energy.2021.120069

Zhao Z, Yun S, Jia L et al (2023) Hybrid VMD-CNN-GRU-based model for short-term forecasting of wind power considering spatio-temporal features. Eng Appl Artif Intell 121:105982. https://doi.org/10.1016/j.engappai.2023.105982

Yildiz C, Acikgoz H, Korkmaz D, Budak U (2021) An improved residual-based convolutional neural network for very short-term wind power forecasting. Energy Convers Manag 228:113731. https://doi.org/10.1016/j.enconman.2020.113731

Sun Z, Zhao M (2020) Short-term wind power forecasting based on VMD decomposition, ConvLSTM networks and error analysis. IEEE Access 8:134422–134434. https://doi.org/10.1109/ACCESS.2020.3011060

Xiang L, Liu J, Yang X et al (2022) Ultra-short term wind power prediction applying a novel model named SATCN-LSTM. Energy Convers Manag 252:115036. https://doi.org/10.1016/j.enconman.2021.115036

Xiong B, Lou L, Meng X et al (2022) Short-term wind power forecasting based on attention mechanism and deep learning. Electr Power Syst Res 206:107776. https://doi.org/10.1016/j.epsr.2022.107776

Bentsen LØ, Warakagoda ND, Stenbro R, Engelstad P (2023) Spatio-temporal wind speed forecasting using graph networks and novel Transformer architectures. Appl Energy 333:120565. https://doi.org/10.1016/j.apenergy.2022.120565

Yu G, Liu C, Tang B et al (2022) Short term wind power prediction for regional wind farms based on spatial-temporal characteristic distribution. Renew Energy 199:599–612. https://doi.org/10.1016/j.renene.2022.08.142

Zhang J, Liu D, Li Z et al (2021) Power prediction of a wind farm cluster based on spatiotemporal correlations. Appl Energy 302:117568. https://doi.org/10.1016/j.apenergy.2021.117568

Gong M, Li W, Yan C, Liu Y, Li S, Zhao Z, Xu W (2023) Wind power forecasting based on SCINet, reversible instance normalization, and knowledge distillation. J Renew Sustain Energy 15:053306. https://doi.org/10.1063/5.0166061

Abou Houran M, Salman Bukhari SM, Zafar MH et al (2023) COA-CNN-LSTM: Coati optimization algorithm-based hybrid deep learning model for PV/wind power forecasting in smart grid applications. Appl Energy 349:121638. https://doi.org/10.1016/j.apenergy.2023.121638

Zheng J, Du J, Wang B et al (2023) A hybrid framework for forecasting power generation of multiple renewable energy sources. Renew Sustain Energy Rev 172:113046. https://doi.org/10.1016/j.rser.2022.113046

Wang H-K, Song K, Cheng Y (2022) A hybrid forecasting model based on CNN and informer for short-term wind power. Front Energy Res. https://doi.org/10.3389/fenrg.2021.788320

Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. In: arXiv.org. https://arxiv.org/abs/1412.3555v1

Boucetta LN, Amrane Y, Arezki S (2023) Comparative analysis of LSTM, GRU, and MLP neural networks for short-term solar power forecasting. In: 2023 international conference on electrical engineering and advanced technology (ICEEAT), pp 1–6

Niu Z, Zhong G, Yu H (2021) A review on the Attention mechanism of deep learning. Neurocomputing 452:48–62. https://doi.org/10.1016/j.neucom.2021.03.091

Guo M-H, Xu T-X, Liu J-J, Liu Z-N, Jiang P-T, Mu T-J, Zhang S-H, Martin RR, Cheng M-M, Hu S-M (2022) Attention mechanisms in computer vision: a survey. Comput Visual Media 8:331–368. https://doi.org/10.1007/s41095-022-0271-y

Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. In: arXiv.org. https://arxiv.org/abs/1409.0473v7

Erisen, B. Wind Turbine Scada Dataset (2018) Available online: http:// www.kaggle.com/berkerisen/wind-turbine-scada-dataset . Accessed on 23 April 2023

Rahimi N, Park S, Choi W et al (2023) A comprehensive review on ensemble solar power forecasting algorithms. J Electr Eng Technol 18:719–733. https://doi.org/10.1007/s42835-023-01378-2

Chicco D, Warrens MJ, Jurman G (2021) The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation. PeerJ Comput Sci 7:e623. https://doi.org/10.7717/peerj-cs.623

Zhao Y, Jia L (2020) A short-term hybrid wind power prediction model based on singular spectrum analysis and temporal convolutional networks. J Renew Sustain Energy 12:056101. https://doi.org/10.1063/5.0007003

Delgado I, Fahim M (2021) Wind turbine data analysis and LSTM-based prediction in SCADA system. Energies 14:125. https://doi.org/10.3390/en14010125

Wei H, Wang W, Kao X (2023) A novel approach to ultra-short-term wind power prediction based on feature engineering and informer. Energy Rep 9:1236–1250. https://doi.org/10.1016/j.egyr.2022.12.062

Download references

Author information

Authors and affiliations.

LSEI Laboratory, Department of Electrical Engineering, Faculty of Electronics and Computer Science, University of Science and Technology Houari Boumediene, Algiers, Algeria

Lakhdar Nadjib Boucetta, Youssouf Amrane & Saliha Arezki

You can also search for this author in PubMed   Google Scholar

Contributions

BOUCETTA Lakhdar Nadjib carried out the conceptualization, methodology, and original draft preparation and writing. Youssouf Amrane and Saliha Arezki participated with BOUCETTA Lakhdar Nadjib in investigation, the three authors collaborated on editing, and visualization. All authors reviewed the manuscript.

Corresponding author

Correspondence to Lakhdar Nadjib Boucetta .

Ethics declarations

Conflict of interest.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Boucetta, L.N., Amrane, Y. & Arezki, S. Wind power forecasting using a GRU attention model for efficient energy management systems. Electr Eng (2024). https://doi.org/10.1007/s00202-024-02590-7

Download citation

Received : 20 January 2024

Accepted : 01 July 2024

Published : 15 August 2024

DOI : https://doi.org/10.1007/s00202-024-02590-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Wind energy
• Energy management system (EMS)
• Wind power forecasting
• Deep learning
• GRU-based attention mechanism
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

The use of standardised short-term and working memory tests in aphasia research: a systematic review

Laura murray.

a Department of Speech & Hearing Sciences, Indiana University, Bloomington, IN, USA

Christos Salis

b Speech & Language Sciences, Newcastle University, Newcastle upon Tyne, UK

Nadine Martin

c Department of Communication Sciences & Disorders, Temple University, Philadelphia, PA, USA

Jenny Dralle

d Department of Neurology, Brandenburgklinik, Bernau bei Berlin, Germany

Impairments of short-term and working memory (STM, WM), both verbal and non-verbal, are ubiquitous in aphasia. Increasing interest in assessing STM and WM in aphasia research and clinical practice as well as a growing evidence base of STM/WM treatments for aphasia warrant an understanding of the range of standardised STM/WM measures that have been utilised in aphasia. To date, however, no previous systematic review has focused on aphasia. Accordingly, the goals of this systematic review were: (1) to identify standardised tests of STM and WM utilised in the aphasia literature, (2) to evaluate critically the psychometric strength of these tests, and (3) to appraise critically the quality of the investigations utilising these tests. Results revealed that a very limited number of standardised tests, in the verbal and non-verbal domains, had robust psychometric properties. Standardisation samples to elicit normative data were often small, and most measures exhibited poor validity and reliability properties. Studies using these tests inconsistently documented demographic and aphasia variables essential to interpreting STM/WM test outcomes. In light of these findings, recommendations are provided to foster, in the future, consistency across aphasia studies and confidence in STM/WM tests as assessment and treatment outcome measures.

Introduction

The presence of short-term and working memory impairments in aphasia is ubiquitous ( Martin & Gupta, 2004 ; Murray, 2012a ; Schuell, Jenkins, & Jimenez-Pabon, 1964 ). Short-term memory (STM) involves storage of information for a brief period of time, usually a few seconds, in a relatively unprocessed state ( Baddeley, 2012 ; Cowan, 2010 ). This information could be auditory or visual and, within each of these modalities, verbal or non-verbal. When information, while being temporarily stored, is mentally manipulated to achieve a particular goal or plan, the manipulation is attributed to working memory (WM). Both STM and WM are considered capacity-limited systems indicating that a limited amount of information can be retained for a finite period of time ( Cowan, 2010 ; Logie, 2011 ). A distinctive feature of STM is that of recall or recognition of information (often serially) in a relatively unprocessed state, whereas the emphasis in WM is deliberate manipulation, which draws on processes related to attention and goal execution. Therefore, assessments designed to measure STM and WM share some features (e.g., temporary maintenance of information); WM tests, however, include additional task demands such as updating or manipulating the information while it is being briefly retained.

To determine whether or not the integrity of STM and WM following brain damage is within normal limits, there is a need to rely on measurement instruments (or tests) that would help ascertain the presence and severity of the impairment, be it STM and/or WM, for rehabilitation planning, advising patients and caregivers, as well as documenting treatment outcomes. However, a construct can be measured with a range of tests, each placing different demands on STM and WM and bringing its own perspective on the nature of the impairment and its behavioural manifestation, the so-called mono-method bias ( Coolican, 2014 ). The related issue of task impurity is also relevant because each task that measures an allegedly specific construct would rely upon a range of related or unrelated corollaries (cf., Miyake & Friedman, 2012 ). For example, in the context of aphasia, WM tests are inherently complex in terms of understanding task demands, and rely on understanding verbal instructions and examples. Consequently, it is especially important that the validity and reliability of STM and WM tests be of the highest quality. Indeed, a test with a higher quality in terms of psychometric properties would be associated with greater clinical confidence for accurate evaluation.

This review aims to identify and appraise standardised tests of STM and WM used in peer-reviewed studies of aphasia resulting from acquired and non-progressive neurological conditions affecting the language dominant hemisphere. We define aphasia as a range of impairments that affect a person’s ability to produce and often understand linguistic units, that is, words, sentences, or discourse ( Edwards, Salis, & Meteyard, 2015 ; Murray & Clark, 2015 ). In contrast to a circumscribed language problem related to a relatively isolated linguistic or perceptual issue (e.g., pure alexia; pure word deafness), aphasia is a complex disorder, with the majority of individuals with aphasia displaying a combination of spoken and written language production and comprehension symptoms. To our knowledge, this is the first systematic review of studies of aphasia involving standardised STM and WM tests.

Both verbal and non-verbal STM/WM deficits in auditory and visual modalities may co-occur in aphasia (e.g., De Renzi & Nichelli, 1975 ; Lang & Quitz, 2012 ). Such STM/WM deficits have been evoked as contributory and sometimes explanatory constructs in relation to several language abilities in aphasia. These range from broader language variables, such as aphasia severity ( Crocket, Clark, Spreen, & Klonoff, 1981 ), potential for aphasia recovery ( Seniów, Litwin, & Leśniak, 2009 ), and prognosis for linguistic treatments ( Harnish & Lundine, 2015 ), to more discrete linguistic levels, such as lexical processing ( Martin & Ayala, 2004 ), aspects of sentence processing, as well as spoken and written discourse comprehension ( Caspari, Parkinson, LaPointe, & Katz, 1998 ; Leff et al., 2009 ; Lehman & Tompkins, 1998 ; Martin & Allen, 2008 ; Sung et al., 2009 ). Furthermore, Sulleman and Kim (2015) have recently argued that WM limitations may negatively affect the ability of people with aphasia to make well-informed decisions about aspects of their rehabilitation. The clinical implication suggested by these studies, albeit not always explicitly, is that STM/WM abilities, both verbal and non-verbal, need to be assessed and, consequently, incorporated into the clinical decision making process to understand a person’s difficulties and strengths. Finally, a recent trend in the experimental rehabilitation literature has been the development and examination of STM and WM treatment protocols not only to remediate memory impairments but also concurrently to improve language and, in some cases, psychosocial functioning (see reviews by Murray, 2012a ; Salis, Kelly, & Code, 2015 ). If such treatments are to be replicated, refined, and ultimately implemented in clinical practice, there would be a need for psychometrically sound STM/WM measurement instruments to establish a diagnosis, explicate the nature and severity of the impairments, implement and monitor treatment, and measure the outcome ( Turkstra, Coelho, & Ylvisaker, 2005 ; de Vet, Terwee, Mokkink, & Knol, 2011 ).

Nonetheless, several issues augur investigation of the tests used to qualify and quantify STM/WM abilities in people with aphasia ( Mayer & Murray, 2012 ; Wright & Fergadiotis, 2012 ). A plethora of STM/WM measures, both standardised and experimental, have been utilised in the empirical aphasia literature, in part a reflection of the different theoretical conceptualisations and the multidimensional nature of these memory constructs. However, such diversity in measures poses challenges. First, it confounds resolving discrepant findings regarding the presence and/or strength of relationship between these memory skills and specific linguistic processes (e.g., Martin, 2009 vs. Majerus, Attout, Artielle, & Van der Kaa, 2015 ). Second, it muddles the search for appropriate STM/WM assessment tools by both researchers and clinicians. Third, it remains challenging to find research documenting the extent to which standardised tests and experimental tasks represent valid and reliable measures of STM/WM in the aphasic population. Such research is essential when using STM/WM measures to prognosticate and/or evaluate aphasia treatment outcomes. Another challenge, particularly pertinent in evaluating auditory-verbal STM and WM in aphasia, is that the response modality of many STM and WM tests involves the very same modalities that are impaired in aphasia. For example, repetition and word retrieval difficulties are impaired in aphasia and may confound STM and WM measurement, which often draws upon repetition and word retrieval (cf., Howard & Franklin, 1990 ). Likewise, motor speech skills can also be impaired (i.e., apraxia of speech, dysarthria), even in cases of mild aphasia ( Basilakos, Rorden, Bonilha, Moser, & Fridriksson, 2015 ; Bose & van Lieshout, 2008 ). It is also of interest to examine whether recent advances in cognitive testing (e.g., computerised test delivery) have been incorporated into the evaluation of STM and WM abilities in individuals with aphasia.

Consequently, the applied goal of the present systematic review is to put forward recommendations for clinicians, researchers, and other stakeholders regarding the suitability of tests when identifying or monitoring STM/WM, both verbal and non-verbal, in individuals with aphasia. To our knowledge, such a comprehensive analysis of tests has not been attempted previously. Specific aims of the present review are as follows:

• To identify standardised tests of STM and WM (verbal and non-verbal) utilised in the adult, acquired, non-progressive aphasia literature from 2000 to 2015. We focused on standardised tests as opposed to experimental tasks because the former category is likely to have more robust psychometric properties and wider availability, and thus more suitable appeal to evidence-based clinical practice. The time period reflects our goal to offer the most current assessment recommendations, and thus identify tests based on contemporary conceptions of STM/WM with recently documented normative data and a recently established evidence base.
• To evaluate critically the psychometric strength of these tests. This aim is embedded in key principles of evidence-based practice in that tests with stronger psychometric profiles would be preferable to those with weaker profiles ( Greenhalgh, 2014 ). This critical appraisal is also essential to identifying tests worthy of recommendation for future use in research and clinical practice.
• To evaluate critically the quality of the investigations utilising these tests. In a similar vein to evidence-based practice, ceteris paribus, if a study has utilised tests with stronger psychometric properties, its findings would be more robust. Likewise, if an investigation in which a test was developed and/or utilised has a strong study design and reporting features, its findings would be more robust. In contrast, when an investigation is poorly designed and lacks methodological detail, it cannot be replicated and such procedural issues confound confident interpretation and future application of the test results and study outcomes. Recommendations regarding STM and WM tests suitable for individuals with aphasia, therefore, should be developed in consideration of not only what tests have been used in the aphasia literature, but also the quality of studies using such tests.

Procedures adhered to previously established methods for performing and describing systematic reviews ( Khan, Kunz, Kleijnen, & Antes, 2003 ; Moher, Liberati, Tetzlaff, Altman, & The PRISMA Group, 2009 ; Schlosser, Wendt, & Sigafoos, 2007 ). This included developing beforehand our systematic review protocol for the literature search, including eligibility criteria and methods to gather and assess the quality of the data of interest.

Search strategy

A comprehensive list of previously established search terms was developed and operationalised into three subcategories: construct related, population related and topic related (see Table 1 ). Using the terms in Table 1 , the following electronic databases were searched: Cumulative Index to Nursing and Allied Health (CINAHL), Linguistics and Language Behaviour Abstracts (LLBA), Medline, and PsychINFO. Search terms within a subcategory were combined with the operator “OR” and across subcategories with the operator “AND” to derive a final list of citations. In addition, the on-line search functions of Science Direct and Taylor & Francis were also searched through the advanced search option using a simpler, two-step search strategy with the following terms: “short-term memory” AND “aphasia”, “working memory” AND “aphasia”. The final list of citations from all databases and Science Direct was exported into EndNote ™ reference management software, which removed duplicate citations. A subsequent hand search of the eligible citations removed further duplicate papers that EndNote ™ did not identify. These digital searches were supplemented by searching other sources. These were as follows: (1) a hand search of all papers published in the journal Aphasiology was also carried out to identify relevant papers; (2) a search of reference lists in STM/WM review papers that appeared in a special issue of Aphasiology on short-term memory and aphasia ( Murray, 2012a ); (3) for commercial tests the websites of Pearson and Psychology Press were reviewed; and, (4) contacting authors for difficult to obtain studies (i.e., Rey Complex Figure Test, version by Meyers & Meyers, 1995 ) or additional information about tests (i.e., Friedmann & Gvion, 2002 ). Duplicate citations that had been generated from the electronic searches were noted and excluded. In all searches, the timescale was from January 2000 until 15 April 2015.

Search terms.

Inclusion and exclusion criteria

To be included in the review, a study had to meet the following inclusion criteria:

• Study participants included adults (i.e., 18 years or older) with non-progressive, acquired aphasia due to any aetiology (e.g., stroke, traumatic brain injury, tumour, infection); we did not apply restrictions of aetiology although we were mindful that in some aetiologies, particularly traumatic brain injury and communication disorders associated with right hemisphere damage, the term aphasia per se may used (e.g., Myers, 2001 ; Sarno, 1980 ). Unless the term aphasia was used to identify participants, such studies were not included.
• When mixed participant groups were utilised (e.g., participants with and without aphasia within an acquired brain injury group), it was possible to identify the STM/WM assessment outcomes for the participants with aphasia (separate from those participants without aphasia).
• STM and/or WM were assessed via a standardised test with norms clearly identified and/or referenced in the study; in this review, a standardised test was defined as a test with clearly defined procedures for administration and scoring that includes norms with reference to scores from a normative sample ( Anastasi & Urbina, 2009 ; Turkstra et al., 2005 ). In addition, the duration of the information (either auditory or visual) that had to be retained or manipulated by participants following exposure of stimuli should not exceed 30 seconds ( Peterson & Peterson, 1959 ).
• The study was peer-reviewed or was a non-peer reviewed standardised test manual.
• The study or test manual was published in English.

Studies were excluded if they did not meet one or more of the above criteria, did not include original data (e.g., meta-analysis, review paper), and/or were unpublished dissertations or conference presentations. Studies were also excluded if they used STM or WM experimental tasks but failed to provide the stimuli and/or a description of the standardisation process, either within the same study and/or a citation for such information. Finally, we excluded studies in which it was clear that the participants with aphasia had been duplicated (i.e., the same participants with aphasia were included in more than one study). In such cases, studies that included the largest number of participants with relevant measures were included, whereas studies that reported subsets of such participants were excluded. This decision is reflected in Figure 1 . The purpose of this final exclusionary procedure was to maintain accuracy about the number and breadth of participants with aphasia involved in the literature base pertaining to STM/WM assessment.

Flow diagram of the identification-inclusion process.

Screening and eligibility

After removing duplicates, study titles and abstracts from the searches were screened against the eligibility criteria. In cases in which neither the title nor abstract indicated eligibility, the full text was screened, recording the reasons if these studies were subsequently excluded. Although all authors participated in screening studies for inclusion, the involvement of each author varied at different points in the screening and eligibility process. To ensure inclusion of studies, any queries regarding the eligibility of individual papers were addressed by consulting at least one additional independent rater from the research team. Any disagreements were discussed and resolved jointly. Additionally, two authors (CS and LM) independently screened a randomly selected sample of 100 studies; inter-rater agreement was 93%, with discrepancies resolved via discussion.

Data extraction

For each study meeting all eligibility criteria, data pertaining to the following were extracted: (1) study aims/objectives; (2) participant sample information including sample size, presence and type of comorbid conditions (e.g., hearing/vision screening; hemiparesis), age, education, gender, native language, aetiology, and aphasia type and severity profiles; (3) assessment setting (e.g., location at which testing took place, qualifications of assessor); and (4) STM/WM test(s) information, including the test name, which aspects of STM/WM were assessed (e.g., visual STM), type of test scores recorded (e.g., raw, scaled), and psychometric characteristics (e.g., inter- and intra-rater reliability, test construct validity). Data relating to other cognitive deficits (i.e., beyond aphasia and STM/WM abilities) were also gathered, but because of the inconsistency of these data across studies, this information is not reported. As in the screening and eligibility stage, all authors participated in data extraction of included studies, although the amount of involvement of each author varied at different points.

Quality appraisal

Each study that underwent data extraction was evaluated for quality using an assessment tool adapted from the Guidance for Undertaking Reviews in Healthcare ( Centre for Reviews and Dissemination, 2008 ), systematic review guidelines proposed by Khan et al. (2003) , and checklists from the STARD ( Bossuyt et al., 2003 ) and COSMIN ( Mokkink et al., 2009 , 2010 ) (see Appendix 1 ). An adapted rating tool was necessary given that existing quality appraisal scales were not suitable for the variety of study designs and/or participant sample characteristics and issues encountered in the aphasia literature. Our adapted tool appraised study quality in terms of five categories: study design, control for confounding factors, specification of aphasia and assessment variables, and STM/WM test score(s) interpretation. Ratings of high, moderate, or low were assigned for each quality category as well as the study as a whole. For a given study to receive an overall high quality rating, four of the five categories had to achieve a high rating with no category receiving a low rating; a study with an overall moderate rating could also not have any category receiving a low rating. Two authors (LM and JD) completed the study quality ratings. Inter-rater agreement was examined for 31 papers (out of 73 extracted papers; see Figure 1 ), and yielded 83% agreement across all items, with 99% agreement for each paper’s overall quality rating. All discrepant ratings were resolved via discussion.

In studies in which a standardised STM/WM test(s) was only used to characterise the aphasia participant sample (versus examine the STM/WM test for use with the aphasia population), the test manual or reference paper cited within the given study was reviewed to identify the test’s psychometric properties. Only the provided reference was analysed to describe and appraise psychometric strengths and weaknesses of the test as this was seen as the original source by the study authors. Different data extraction forms were developed for these test papers and manuals which included items on the following: (1) normative sample variables including whether or not adults with acquired, non-progressive aphasia were included in the test’s standardisation process and the appropriateness of the standardisation sample (e.g., age and education appropriate) given the aphasic participant(s) characteristics in the eligible paper which cited the test; (2) test administration characteristics (e.g., information on the assessment environment); (3) validity (i.e., construct, content/face, criterion-related, and discriminant validity); (4) reliability (i.e., test-retest, split-half/internal consistency, and inter-rater); and (5) measurement error.

Each STM/WM test utilised in the set of included studies was also appraised to rate the quality of its psychometric properties. Currently, however, there is no widely used existing “gold standard” for assessing STM/WM in aphasia (cf., DeDe et al., 2014 ). Accordingly, an appraisal tool (see Appendix 2 ) was developed by adapting the COSMIN checklist, which has empirical support of its reliability ( Mokkink et al., 2010 ) and validity ( Mokkink et al., 2009 ), in concert with the criteria for test reliability and validity established by the Agency for Healthcare Research and Quality Evidence-Based Practice Program ( Biddle et al., 2002 ). These criteria have been previously used by the Academy of Neurological Communication Disorders and Sciences to develop practice guidelines (e.g., Turkstra et al., 2005 ). As an example of how the COSMIN checklist was adapted, given that most STM/WM measures are subtests within a test battery (e.g., Digit Span of the Wechsler Memory Scale–Revised; Wechsler, 1987 ), COSMIN checklist items pertaining to internal consistency across test (sub)scales were not applicable and, thus, not included in our appraisal tool. Two authors (CS and JD) completed the quality ratings of the STM/WM tests. Inter-rater agreement was examined for 57.5% of the tests (i.e., 19 of 33 STM/WM tests). There was 94% agreement across all rated items, with 100% agreement for each test’s overall quality rating. Discussion was utilised to resolve any discrepant ratings.

Final study selection

Given the large number of studies that underwent data extraction, a post-hoc decision was made to categorise eligible studies into either (1) those in which the study purpose directly related to describing, assessing, or treating STM/WM abilities in aphasia, or (2) those in which STM/WM assessment was ancillary to the study purpose (e.g., the study focus was a word retrieval intervention and STM/WM assessment was completed as part of a comprehensive assessment of aphasic participants).

Primary reasons for study exclusion were: (1) the study listed aphasia as an exclusionary criterion; (2) there was no specification that acquired brain injury participants had aphasia (this was a particularly common basis for excluding traumatic brain injury sequelae or treatment studies); (3) when individuals with aphasia were included, their STM/WM test results were not separated from those of the individuals without aphasia (this was a particularly common basis for excluding stroke sequelae or treatment studies); (4) no standardised STM/WM test was used; and (5) the citations provided for standardised tests were wrong.

The search results across databases and other searches are shown in Figure 1 , together with the results from the screening and eligibility processes as well as the post-hoc final study selection. Of the 7299 studies screened, only 73 were deemed eligible. The 36 studies that became the main focus of the review are shown in Table 2 . On the basis of these studies, the STM/WM tests that were used within them were critically appraised and are shown in Tables 3 – 8 . The studies that used STM/WM tests but the main purpose of these studies was not STM/WM are shown in Table 3 . These studies will not be discussed further.

Summary of main studies (in alphabetical order).

RHD = right hemisphere damage; LHD = left hemisphere damage; N = total number of participants; n = number of participants in sub-samples; NR = not reported; TSA = transcortical sensory aphasia; TMA = transcortical motor aphasia;

CAT = Comprehensive Aphasia Test; CLQT = Cognitive Linguistic Quick Test; TEA = Test of Everyday Attention; WAIS = Wechsler Adult Intelligence Scale; WAIS-R = Wechsler Adult Intelligence Scale – Revised; WAIS-III = Wechsler Adult Intelligence Scale 3rd Edition; WMS = Wechsler Memory Scale; WMS-R = Wechsler Memory Scale – Revised; WMS-III = Wechsler Memory Scale 3rd Edition.

STM/WM as background testing: standardised auditory-verbal STM tests (listed alphabetically by test type).

WAIS-R = Wechsler Adult Intelligence Scale – Revised; WMS = Wechsler Memory Scale; WMS-R = Wechsler Memory Scale – Revised; CTOPP = Comprehensive Test of Phonological Processing; WAIS-III = Wechsler Adult Intelligence Scale 3rd Edition; EPLA = Evaluación del Procesamiento Lingüísticos en la Afasia.

Standardised visuo-spatial STM and WM tests (listed alphabetically by test type).

CLQT = Cognitive Linguistic Quick Test; WMS-III = Wechsler Memory Scale 3rd Edition; WMS-R = Wechsler Memory Scale – Revised; TEA = Test of Everyday Attention.

Participants in the final set of eligible studies

There were 898 participants with aphasia ( Table 2 ). In terms of aphasia characteristics, neither aphasia type nor severity was consistently reported (e.g., Allen, Martin, & Martin, 2012 ; Lang & Quitz, 2012 ). For instance, severity of aphasia was specified in only 16 (of 36, or 44%) studies. When aphasia type was noted, a variety of aphasia classification systems was used: Some studies more broadly only noted whether participants had fluent versus nonfluent aphasia (e.g., Carragher, Sage, & Conroy, 2013 ), whereas other studies used a more complex system such as the Boston classification system (e.g., DeDe et al., 2014 ). Participants with anomic aphasia (144) and/or a mild severity of aphasia (73) were the most common when authors reported these variables. In contrast, among studies specifying aphasia type and/or severity, individuals with Wernicke’s (38) or severe aphasia (13) were under-represented in the participant samples compared to the other aphasia types and severities, respectively. Across studies, participants with aphasia representing a range of education levels and ages were included. In several studies, however, education level information was either not provided (e.g., Galling et al., 2014 ; Sinotte & Coelho, 2007 ) or described in general terms (e.g., Lang & Quitz, 2012 who described education level in terms of less or more than nine years of formal education). Additionally, across studies, there were relatively few participants with aphasia over the age of 70 compared to those younger than age 70.

Standardised STM/WM tests used in final set of eligible studies

The auditory-verbal STM tests and WM tests utilised in the final set of extracted studies are displayed in Tables 6 and ​ and7, 7 , respectively; Table 8 provides a summary of the visuo-spatial STM and WM tests used. Quality appraisal ratings of these tests are displayed in Tables 9 – 11 .

Standardised auditory-verbal STM tests, listed alphabetically by test type.

CAT = Comprehensive Aphasia Test; WAIS-R = Wechsler Adult Intelligence Scale – Revised; WMS-R = Wechsler Memory Scale – Revised.

Standardised auditory-verbal and visual-verbal WM tests (listed alphabetically by test type).

TEA = Test of Everyday Attention.

Evaluation of auditory-verbal STM tests, listed alphabetically by test type.

CAT = Comprehensive Aphasia Test; WAIS-III = Wechsler Adult Intelligence Scale 3rd Edition; ; WMS-R = Wechsler Memory Scale – Revised.

Evaluation of standardised visuo-spatial STM and WM tests, listed alphabetically by test type.

CLQT = Cognitive Linguistic Quick Test; TEA = Test of Everyday Attention; WMS-R = Wechsler Memory Scale – Revised; WMS-III = Wechsler Memory Scale 3rd Edition.

Auditory-verbal STM tests

Review of Table 6 indicates that across studies, serial recall was the most frequently used task to assess auditory-verbal STM. Digit Span 1 , albeit from several different standardised tests and administered in a number of different languages, appeared the most popular auditory-verbal STM task being used with 272 (out of 898) or 30.2% of the participants with aphasia. The second most popular test was the Immediate Free Recall condition of the Rey Auditory Verbal Learning Test (RAVLT; Rey, 1964 ), which had been used with 34 (out of 898) or 3.7% participants with aphasia, albeit in only one study. The least popular test, used with only one participant with aphasia, was the Revised Token Test (RTT; McNeil & Prescott, 1978 ). Among the 20 different tests (or subtests of larger batteries) listed in Table 6 , 17 emphasise serial recall (either forward or backward) or recognition. In contrast, only two tests focus on free recall 2 (RAVLT; Word Span Probe test of Friedmann & Gvion, 2002 ). In terms of the response demands, the majority of auditory-verbal STM tests (12 out of 20) require spoken output; instead of a spoken response, in the remaining eight tests, examinees indicate recalled information via either a pointing response or a recognition judgement (e.g., yes/no response).

Auditory-verbal and visual-verbal WM tests

Compared to the number of auditory-verbal STM tests just reviewed, fewer auditory-verbal working memory tests (i.e., six) were used within the eligible studies ( Table 7 ). Complex span measures (i.e., Listening Span tests, Eye Movement WM task), which place demands on the shifting component of WM ( Miyake, Friedman, Emerson, Witzki, & Howerter, 2000 ), were most common, being administered to 156 (out of 898, 17.3%) of the participants with aphasia. The other less frequently used tests (i.e., TEA subtests, n -back of DeDe et al., 2014 ) place greater demands on updating functions within WM ( Morris & Jones, 1990 ); these tests had been administered to only 88 (out of 898; 9.7%) of the participants with aphasia. There was an even representation of auditory-verbal WM tests requiring a spoken response versus a response in another modality (i.e., pointing or eye gaze).

Visuo-spatial STM and WM tests

In contrast to the great variety of tests used to measure auditory-verbal and visual-verbal STM or WM abilities, only a limited number of visuo-spatial STM and WM tests were identified in the literature; accordingly, both visuo-spatial STM and WM test findings are described here and collapsed into Table 8 . As in assessment of auditory-verbal STM, serial recall tasks were the most popular for evaluating visuo-spatial STM abilities. That is, of the nine different tests (or subtests from larger test batteries), five were used to measure visuo-spatial span or serial recall. With 365 (out of 898; 40.6%) of the participants with aphasia completing a visuo-spatial span test, such tests represent the most frequently used STM measure among the eligible studies. In contrast, immediate recall of complex designs was rarely used to evaluate visuo-spatial STM, with administration to only 1 (out of 898) participant with aphasia. The TEA Visual Elevator was the only standardised visuo-spatial test encountered among the eligible studies to place substantial demands on shifting and updating components of WM. The most frequent mode of response among the visuo-spatial STM and WM tests was pointing. The WMS-R Visual Reproduction subtest requires a drawing response and the TEA Visual Elevator subtest requires a spoken response (i.e., counting).

Quality appraisal of standardised STM/WM tests

Each standardised STM/WM test encountered in the eligible studies (except for tests with psychometric data published in unobtainable manuals or studies) was evaluated in terms of its psychometric properties (see Appendix 2 for the test appraisal tool). Quality ratings for the auditory-verbal STM tests are displayed in Table 9 . In terms of validity, every auditory-verbal STM test except for the RTT appropriately documented discriminant validity. Across these tests, however, other types of validity were either not reported or received fair or poor ratings. Only five of the 16 auditory-verbal STM tests received an excellent rating for construct validity, and only the WAIS-III Digit Span received an excellent rating for content/face validity. Concurrent validity was rated as poor in 13 tests, with the remaining three receiving a fair rating. The tests fared poorly in terms of all types of reliability examined, with no excellent ratings. Only three tests were rated as having fair test-retest reliability, whereas only the WMS-R Digit Span received a fair rating for split-half reliability.

As Table 10 shows, there were issues with the psychometric characteristics of the auditory-verbal WM tests. Among the nine tests, seven received an excellent rating for their construct validity. Among the other types of validity, however, the only excellent rating was for the concurrent validity of the English version of the Eye Movement WM Span test ( Ivanova & Hallowell, 2014 ). Only the Listening Span task of Tompkins et al. (1994) provided evidence of predictive validity. Discriminant validity was documented in six (out of nine) of these tests. All types of reliability and measurement error were either rated as poor or not reported.

Evaluation of auditory-verbal and visual-verbal WM tests, listed alphabetically by test type.

Appraisal of the STM and WM visuo-spatial tests indicated that each had excellent construct validity (see Table 11 ). Other types of validity received less positive ratings, with only the WMS-III visual tapping test receiving an excellent rating for content/face validity and only the Block Tapping of Kessels et al. (2008) receiving an excellent rating for concurrent validity. Both predictive and discriminant validity were reported inconsistently across the eight visuo-spatial STM tests that were quality appraised: three provided evidence of predictive validity and six provided evidence of discriminant validity. In terms of reliability, the only excellent ratings were for the test-retest reliability of the WMS-R visual reproduction test and the split-half reliability of the WMS-R visual tapping test. The WMS-III visual tapping test received a fair rating for its test-retest and split-half reliability and also was the only test to include inter-rater reliability information. All other reliability quality ratings were poor. Measurement error was rated as poor except for the WMS-R and WMS-III visual tapping tests and the WMS-R visual reproduction test, all of which received an excellent rating.

Quality appraisal of final set of eligible studies

Table 12 lists the quality ratings for each of the 36 studies in the areas of design, control for confounds, aphasia variables, assessment variables, STM/WM score interpretation, and overall study quality. The majority of studies received a high quality rating in the areas of design (23/36) and STM/WM score interpretation (25/36). Of concern were the majority of low quality ratings in the area of assessment variables (24/36), with only a few studies stating in what environment participants were evaluated and/or who administered the test(s) and their professional qualifications. Few studies received a high quality rating in the area of control for confounds (7/36), with more than half of the studies (20/36) failing to indicate whether the effects of age and education on test performance were controlled or considered (i.e., a low rating). Accordingly, keeping in mind that a low quality rating in any category resulted in a low overall study quality rating, only three studies ( Chiou & Kennedy, 2009 ; Fucetola et al., 2009 ; Ivanova & Hallowell, 2014 ) received a high overall study quality rating, and three studies ( DeDe et al., 2014 ; Kalbe et al., 2005 ; Meteyard et al., 2015 ) received a moderate overall study quality rating.

Study quality ratings: high; moderate low (see Appendix 2 for a description of these rating categories).

The purpose of this systematic review was to comprehensively analyse standardised tests of STM and WM, both verbal and non-verbal, used in the contemporary aphasia literature. Our review involved not only identifying STM and WM tests, but also critically appraising both the psychometric properties of these tests as well as the quality of the aphasia investigations in which the tests were used. Overall, although a wide variety of standardised tests have been used to characterise STM and WM in individuals with aphasia, those that measure serial recall appeared most common, and substantial issues with respect to the psychometric strength of the STM/WM tests as well as the quality of studies were identified. Below is a more detailed discussion of the quality appraisal. This is followed with recommendations for improving assessment of STM and WM in aphasia, in both research and clinical practice.

Standardised STM and WM tests

Quality appraisal of auditory-verbal stm tests.

Auditory-verbal STM tests were the most popular. Within this broad category, the most popular task was Digit Span that required spoken recall. It was used with 272 persons with aphasia across 15 different studies. Such popularity may reflect that, historically, Digit Span, as a measure of STM ability, was one of the very first to be included in intelligence testing scales, dating back to the late 19th century ( Richardson, 2007 ). Since the late 1930s, it has been incorporated into the test batteries of Wechsler and is still present in their recent versions. Digit Span has also had a long history of use in aphasiology ( Eling, 2015 ; Schuell et al., 1964 ). Indeed, the Digit Span subtest of the Wechsler batteries was the most popular version in the current systematic review compared to other, more recent, versions (CAT; Computerised Neurocognitive Test).

The four versions of Digit Span with documentation available for our quality appraisal were rated as having excellent construct validity and all had documentation of predictive and discriminant validity ( Table 9 ). However, a mixed profile of quality was found for other aspects of validity and for reliability. For example, content/face validity was deemed excellent only in the WAIS-III while poor in the other three versions (CAT, WAIS-R, WMS-R). Measurement error was poor only in the CAT. The relatively low levels of test-retest reliability for Digit Span have been known for some time, making it customary to combine scores from its forward and backward recall versions to improve reliability ( Richardson, 2007 ). There is evidence to suggest that reliability coefficients improve when scores from different tasks that relate to a particular psychometric property are combined ( DeDe et al., 2014 ; Swinburn et al., 2004 ; Waters & Caplan, 2003 ). For example, Waters and Caplan (2003) showed that in non-brain-damaged adults (younger and older), test-retest reliability was acceptable (≥ .70) when individual memory test scores were combined.

One study ( Caza et al., 2002 ) used versions of Digit Span with old normative data based on 1957 and 1969 editions of the Wechsler tests. The so-called Flynn effect refers to the increment of IQ scores as time progresses ( Flynn, 1984 , 2009 ). Accordingly, older normative data as reference points may jeopardise discriminant and predictive validity. The Flynn effect has been evident in Digit Span data ( Wicherts et al., 2004 ) and could also operate in other STM and WM tests that use historical normative data. Loring and Bauer (2010) noted that what makes a test outdated is not necessarily the publication of a more recent version of it, but rather empirical evidence the new edition is more valid and reliable, always with reference to the clinical population for which the test is intended. To our knowledge, such empirical research for clinical use of Digit Span (not only the Wechsler but also other versions) with persons with aphasia, does not exist.

There were two additional versions of Digit Span that did not require speech production: the pointing and matching span versions of the FriGvi ( Friedmann & Gvion, 2002 ), a test developed in Israel for speakers of Hebrew. Both tests had fair construct validity and did display discriminant validity in differentiating STM performance in people with aphasia. However, both tests were poor in other aspects of validity, reliability, and measurement error. We should note that unlike some Digit Span tests requiring spoken recall (Wechsler versions, CAT), which present only two trials per span length, the FriGvi pointing version presents five trials per list length. As Woods et al. (2011) noted, the two trial paradigm assumes that a person’s true maximum length span can be assessed by only four list presentations: two at the maximum length and two above. However, this method may seriously underestimate the maximum length of persons who are distracted or encounter idiosyncratically difficult digit strings (e.g., permutations of their telephone area code) at a particular length.

Relying on Digit Span for assessing auditory-verbal STM in aphasia presents with other possible limitations. Numerical skills are often impaired in aphasia, so interpretation of Digit Span performance on its own may not truly reveal the integrity or decrement of STM ( DeDe et al., 2014 ). Furthermore, in aphasia, STM has been found to be sensitive to the lexical processing characteristics of the words within the STM test (e. g., lexicality, frequency) (e.g., Howard & Nickels, 2005 ; Martin & Ayala, 2004 ; Martin, Saffran, & Dell, 1996 ). For example, because the lexical frequency of digits is high in comparison to other words ( Martin, Lesch, & Bartha, 1999 ), relying only on digits to evaluate STM may yield inaccurate results.

Only the non-word span and the probe word span of the FriGvi ( Friedmann & Gvion, 2002 ) explicitly assess the influence of lexical variables in STM. Both tests were used in two studies, with a total of 17 participants with aphasia completing each test. Knowing if lexicality and other lexical variables influence STM has diagnostic and treatment implications. Studies have shown that the nature of auditory-verbal STM deficits in aphasia can vary along the phonological-semantic dichotomy and in some individuals can be differentially spared or impaired (e.g., Martin & Allen, 2008 ; Martin & Ayala, 2004 ).

Only two tests did not tap into serial aspects of STM, the immediate condition of the Rey Auditory Verbal Learning Test (RAVLT; Rey, 1964 ) and the probe word span of the FriGvi. The ability to process language effectively relies heavily on the ability to process information serially, and this may explain the popularity of serial STM tests. Regarding the RAVLT, we only included studies that reported results for the immediate recall condition, which assesses STM. Subsequent recall conditions rely on long-term memory. We were unable to obtain the 1964 version of the RAVLT used by Vukovic et al. (2008) , so are not in a position to appraise it. Whereas we are not aware of studies on the Flynn effect in relation to the RAVLT, Baxendale (2010) found that among healthy adults from the UK, verbal learning ability as measured by a test similar to the RAVLT was relatively stable across time with no significant differences between the scores in the majority of age ranges, apart from the 31–45 years age group. However, it should also be noted that Vukovic et al. (2008) administered the RAVLT in Serbian and used the test materials but not the norms.

Quality appraisal of auditory-verbal and visual-verbal WM tests

Compared to auditory-verbal STM, a more limited number of standardised tests have been used to evaluate auditory-verbal or visual-verbal WM in individuals with aphasia ( Table 7 ). Half of these WM tests were complex span tasks (Eye Movement WM task, Listening Span by spoken recall or written recognition), which place heavy demands on the WM submechanisms of rehearsal and shifting; the other half (i.e., TEA Elevator Counting with Distraction and with Reversal, n -back) evaluate WM more in terms of its monitoring and updating submechanisms ( Conway et al., 2005 ; Kearney-Ramos et al., 2014 ; Salis et al., 2015 ; Wright & Fergadiotis, 2012 ). The complex span tasks were more popular in that they were used in a larger number of studies and with a larger number of participants with aphasia.

The auditory-verbal and visual-verbal WM tests also varied in terms of whether they did (e.g., Elevator Counting with Reversal) or did not require a verbal response (e.g., n -back). Within the group of tests not involving a verbal response, a variety of nonverbal response modalities was used (i.e., pointing, computer key press, eye movement). Regardless of response modality, the complex span tasks had greater language demands (i.e., all required sentence processing) compared to the updating tasks. In fact, Tompkins et al. (1994) warned that their complex listening span test was likely unsuitable for individuals with severe aphasia.

Research in healthy as well as other patient populations indicates cognitive demand differences between complex span versus updating WM tests (e.g., Jaeggi, Buschkuehl, Perrig, & Meier, 2010 ; Kane, Conway, Hambrick, & Engle, 2007 ), which in turn may lead examinees to use different strategies when completing such tests ( Logie, 2011 ). Despite these findings, both types of tasks were used in only three studies (i.e., DeDe et al., 2014 ; Mayer & Murray, 2002 , 2012b ). Whereas debate persists concerning the theoretical architecture of WM, multidimensionality is a common feature ( Logie, 2011 ; Wright & Fergadiotis, 2012 ), thus suggesting that a test examining a limited set of WM submechanisms may not fully characterise WM abilities. Consequently, as we and others ( Conway et al., 2005 ; DeDe et al., 2014 ) have noted, until a more comprehensive verbal WM measure is developed, the practice of utilising just one test to characterise WM abilities should be avoided.

Quality appraisal findings further supported the conclusion that reliance on only one auditory-verbal or visual-verbal WM test is inadequate. Despite an excellent rating for construct validity across most of the verbal WM tests used in the eligible studies ( Table 10 ), ratings for other aspects of validity indicated substantial problems. For example, all of the tests received poor ratings for content/face validity, and only one test (Listening Span of Tompkins et al., 1994 ) had evidence of predictive validity. Reliability and measurement error were uniformly problematic for all of these WM tests. The most common issues leading to less desirable quality ratings included insufficient description of procedures used to examine validity or reliability (e.g., stating a correlation was calculated, but not specifying if it was an intra-class, Pearson, or Spearman), failure to include information regarding certain psychometric properties (e.g., split-half reliability and measurement error were rarely mentioned), and restricted sample sizes (which compromise certain aspects of reliability). Thus, although complex span tests were found to be used more frequently in the aphasia literature, there does not appear to be a psychometric rationale for their popularity compared to the other types of WM measures (i.e., n -back; TEA subtests). More generally, as previous authors have noted ( Salis et al., 2015 ; Wright & Fergadiotis, 2012 ), currently available auditory-verbal and visual-verbal WM tests require further empirical development (e.g., modifications to support performance of those with severe language difficulties) and evaluation to determine if their use with individuals with aphasia can yield psychometrically sound data.

Quality appraisal of visuo-spatial STM and WM tests

Visuo-spatial span tests were the most popular type of test for assessing visuo-spatial STM and WM, with a total of 365 aphasic participants tested, and half of the eligible studies including one or more visuo-spatial STM or WM test ( Table 8 ). Such popularity in the aphasia literature was expected given the relatively reduced language demands of visuo-spatial STM/WM tests compared to their auditory-verbal or visual-verbal counterparts. Among the types of visuo-spatial STM tests identified in the appraised literature, serial recall tasks were most prevalent. Of the four visuo-spatial serial recall tests reviewed, the WMS-III visual tapping subtest received the strongest quality ratings, although evidence of its discriminant validity was lacking ( Table 11 ). Notably, the newer visual tapping (WMS-III) did represent an improved version of the older WMS-R visual tapping in several psychometric domains. Reliability and measurement error were areas of significant concern for the visuo-spatial STM tests developed by Kessels et al. (2008) , a version of a Corsi block tapping task, and DeDe et al. (2014) : Both tests received poor quality ratings for these psychometric properties and neither reported inter-rater reliability. We should note that several studies (e.g., Berthier et al., 2011 ) used block tapping tests but were not included in this review because the wrong citations were provided. Milner (1971) was one of these erroneous citations: Milner (1971) referred to Corsi’s doctoral research (i.e., Corsi, 1972 ), which involved block tapping as a Hebbian learning task rather than visuo-spatial STM span test per se. Another problematic citation for the block tapping test was that of De Renzi and Nichelli (1975) who referred to their block tapping task as a “spatial span task” (p. 344), but provided insufficient description of how the task was implemented. In contrast, Kessels et al. (2008) included the actual sequences for their block tapping test. Regardless, our quality appraisal findings suggest that the WMS-III visual tapping appeared to be the most appropriate choice when looking for a measure of visuo-spatial serial recall.

Three visuo-spatial STM tests did not require serial recall: Two involved the immediate recognition of complex designs via a pointing response (i.e., Helm-Estabrooks, 2001 ; Kalbe et al., 2005 ) and one involved the recall of designs via a drawing response (i.e., WMS-R Visual Reproduction I). Of the two involving immediate recognition of complex designs, the version by Kalbe and colleagues received a stronger validity appraisal; however, both of these tests received poor ratings in measurement error and across all types of reliability. Consequently, neither test would be appropriate for monitoring recovery or treatment effects. Compared to these recognition tests, the WMS-R Visual Reproduction I had stronger psychometric characteristics, despite concerns with certain types of validity and reliability. Among the eligible studies, this visuo-spatial STM test was used in only one study with one participant (i.e., Murray et al., 2006 ). It is possible that this test was used infrequently because drawing abilities in individuals with aphasia may be confounded by a number of concomitant conditions (e.g., dominant hand paresis; constructional apraxia; visual neglect; Murray & Clark, 2015 ).

The only standardised visuo-spatial WM test encountered in the eligible studies was the TEA Visual Elevator subtest, which evaluates updating submechanisms of WM ( Kearney-Ramos et al., 2014 ). Our quality appraisal highlighted several psychometric concerns with this test including poor ratings of content and concurrent validity, measurement error, and test-retest and split-half reliability. Given that only one standardised test was identified, additional research is warranted to examine the visuo-spatial WM performance patterns of individuals with aphasia on other updating tests (e.g., n -back measures) as well as tests designed to evaluate shifting processes (e.g., complex span measures).

Quality appraisal of studies

Our systematic review and quality appraisal identified only six studies with high ( Chiou & Kennedy, 2009 ; Fucetola et al., 2009 ; Ivanova & Hallowell, 2014 ) or moderate ( DeDe et al., 2014 ; Kalbe et al., 2005 ; Meteyard et al., 2015 ) overall study quality ratings, and thus revealed a number of concerns regarding the description, use, and interpretation of STM and WM tests in the aphasia literature ( Table 12 ). Whereas study design was rated as high or moderate in the vast majority of the papers, issues arose in terms of the other appraisal categories. Inadequate description of aphasia variables (i.e., low rating) was encountered in several studies. That is, in these studies, the presence of aphasia was mentioned but with nominal description and/or documentation of the aphasia profile (e.g., no information concerning aphasia severity). Failure to include aphasia profile information subverts determining to which segment of the aphasia population the STM/WM test(s) findings apply. Approximately half of the studies adequately described the language profiles of the participants with aphasia but included a restricted range of profiles; in some cases this was related to the small sample size (e.g., Francis et al., 2003 ) whereas in others, the study was designed to focus on a particular aphasia profile (e.g., Gvion & Friedmann, 2012 ). A restricted range of profiles limits the extent to which STM/WM test findings can be generalised to the broad aphasia population and may result in a lack of evidence for certain segments of that population. Indeed, individuals with severe aphasia or a Wernicke’s aphasia type were under-represented in the studies reviewed.

With respect to the use and interpretation of the STM/WM tests, most studies failed to describe the assessment conditions, with only three studies specifying the characteristics of both the testing environment and the test administrator. Description of assessment variables is necessary to (1) allow replication of STM/WM test administration procedures not only in future research but also in clinical settings, and (2) aid in interpreting the test findings (e.g., different STM test scores at time point 1 and 2 could reflect administration differences versus a change in memory performance). Another major concern was the small number of investigations (i.e., 6 out of 36) in which age and education in concert with at least one other confounding factor were taken into account when administering and interpreting the STM/WM tests. Consideration of such factors is essential given the extensive literature documenting the substantial influence of demographic variables such as age, education, and ethnocultural background on cognitive test performances (e.g., Casaletto et al., 2015 ; Norman et al., 2011 ). Accordingly, STM/WM test outcomes become difficult to interpret when such factors have not been reported at all in a study or have been disregarded when scoring STM/WM tests or comparing patient and control groups. Relatedly, whereas most studies included the reference standard for the STM/WM test scores, close to 30% failed to do so. In these latter studies, whether the STM/WM test results indicate the presence or absence of impairment cannot be vetted.

Recommendations

Based on our review of the standardised STM/WM tests and the studies utilising such tests, we recommend the following in future endeavours related to the evaluation of STM or WM in aphasia:

• There is a need to obtain standardisation information from larger sample sizes to increase the power of STM/WM tests’ psychometric properties. This would provide confidence to researchers and clinicians in adopting specific tests. With some notable exceptions ( Kalbe et al., 2005 ; Swinburn et al., 2004 ), normative and validation sample sizes for individuals with aphasia were small. At the very least, age and education information must also be included in the normative and validation data, given the well-documented influence of such demographic variables on cognitive test performance (e.g., Casaletto et al., 2015 ). Description of aphasia variables and testing environments is also recommended to allow determination of the range of patient profiles and administration settings in which the test can detect STM/WM difficulties or changes in STM/WM abilities.
• There is a need to expand theoretical paradigms and study the psychometric properties of their tasks in both STM and WM. In auditory-verbal STM, there is a need to go beyond Digit Span and include tests that systematically manipulate word types and lexical variables (cf., Friedmann & Gvion, 2002 ). Such tests are needed to further delineate the role of phonological and semantic STM abilities in aphasia as well the role of item versus order deficits in STM (cf., Majerus et al., 2015 ; Martin, 2009 ). This issue has been addressed in experimental tasks that manipulate linguistic variables in verbal STM and WM tasks, but these experimental tasks have not yet undergone sufficient psychometric evaluation (e.g., Christensen & Wright, 2010 ; Martin, Kohen, & Kalinyak-Fliszar, 2010 ).
• Several relatively new standardised cognitive test batteries have STM and WM subt-ests (e.g., WMS-IV Symbol Span; Wechsler, 2009 ), but have yet to be utilised in the aphasia literature (at least as of April, 2015).
• Albeit one reviewed study solely used computerised STM tests (i.e., Lee & Pyun, 2014 ), expansion of computerised delivery of STM/WM tests appears an area in need of further exploration. Computerised tests afford timing precision, improve consistency of delivery, and minimise variability of presentation between different human assessors, ultimately improving testing ( Noyes & Garland, 2008 ; Woods et al., 2011 ). However, practical limitations in terms of computer portability and availability could be prohibiting factors.
• Investigations of staircase methods of presentation as opposed to the dominant “ascending” or “incremental” method of testing (i.e., from lists or blocks with fewer stimuli to lists or blocks with more stimuli) (cf., Ehrenstein & Ehrenstein, 1999 ) are needed. Although in the more traditional ascending testing method difficulty increases gradually, and thus possibly engages examinees in the testing process (because initial items are not too difficult), proactive interference also increases ( May, Hasher, & Kane, 1999 ). May et al. showed that, particularly in older adults, the traditional ascending testing method can produce smaller WM scores because of increased proactive interference. Staircase methods could diminish such proactive interference. Computerised tests would allow automated adjustment of list presentations in terms of the staircase method, highlighting the need for more frequent collaboration between human computer interaction specialists and aphasiologists ( Molero Martin, Laird, Hwang, & Salis, 2013 ; Salis & Hwang, 2016 ).
• There should be more research on the possible influence of response modality in STM and WM testing (e.g., spoken response versus recognition; drawing versus recognition), in both non-brain-damaged adults as well as those with aphasia. For example, comparisons of matching span versus spoken recall tasks have revealed a distinction between (a) encoding and storage associated with language input versus (b) retrieval associated with language output processes (e.g., Allport, 1984 ; Jacquemot, Dupoux, & Bachoud-Lévi, 2011 ; Romani, 1992 ). Nonetheless, issues of whether or not STM/WM tests that differ in response modality can be used interchangeably or should perhaps be used in concert to bolster the validity and reliability of STM/WM test results have not been systematically investigated.
• As clinical researchers we recognise that research needs are different from clinical needs. To ensure that research findings make an impact on clinical practice, there should be more dialogue between stakeholders, that is, researchers, clinicians, and people with aphasia, to achieve the design of STM and WM tests that are psychometrically sound and discriminating, as well as appealing to clinicians who have limited time to derive a differential diagnosis before treatment or to measure improvements following treatment.

Limitations

Some limitations must be acknowledged with respect to the current systematic review. First, only journal studies and test manuals in English were reviewed and appraised. Thus, selection bias is possible and our findings may not apply to STM/WM tests in other languages. Second, our ratings of the psychometric properties of tests were based on the sources available to us and the way those were reported. It may well be the case that if different statistical analyses were reported, the quality ratings might too have been different.

Conclusions

The present systematic review identified use of a number of standardised auditory-verbal and visuo-spatial STM and WM tests in the contemporary aphasia literature. Further research is warranted, however, given problems in terms of these tests’ validity, reliability, and measurement error, and in terms of how researchers documented their use and interpretation of such tests. That is, in concert with previous reviews (e.g., Salis et al., 2015 ), no gold standard for evaluating STM/WM abilities in people with aphasia was identified. Until such a gold standard STM/WM assessment tool has been ratified, reliance on just one test to characterise STM or WM in a given individual with aphasia appears an inadequate practice given (1) the psychometric concerns among the standardised tests currently being used in the aphasia literature, and (2) the multi-faceted nature of STM and WM specified in theoretical models of these memory constructs. Also, given the growing literature suggesting a crucial role for non-linguistic cognitive functions in mediating aphasia symptoms and outcomes (e.g., Nguyen et al., 2015 ; Nicholas, Hunsaker, & Guarino, 2015 ), our findings underscore the need not only to extend aphasia research regarding STM/WM standardised test development and validation, but also to review systematically standardised tests regularly being used in research and clinical practice to characterise other domains of cognitive functioning in individuals with aphasia.

STM/WM as background testing: standardised auditory-verbal WM tests.

STM/WM as background testing: standardised visuo-spatial STM and WM tests (listed alphabetically by test type).

CNS = central nervous system; DMS-48 = Delayed Matching-to-Sample-48; CLQT = Cognitive Linguistic Quick Test; WMS = Wechsler Memory Scale; WMS-R = Wechsler Memory Scale – Revised; WMS-III = Wechsler Memory Scale 3rd Edition.

Acknowledgments

We would like to thank Helen Kelly (University College Cork, Ireland) for her advice and assistance with the literature searches and Paul Conroy (University of Manchester, UK) for providing us with information about the Rey Complex Figure Test.

This work was supported in part by the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health under award number [R01DC013196]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Appendix 1. Adapted Study Quality Rating Tool

This Study Quality Rating Tool is based on information in NIHR York University Guidelines and Criteria for Appraising Diagnostic Test Studies; Khan et al. (2003) , STARD and COSMIN checklists. A study must score high in 4 out of 5 categories for an overall High rating (with no low rating); an overall moderate rating for a study cannot include any low rating.

Appendix 2. Test Psychometric Properties Quality Rating Tool

Scoring note: For any variable/construct with items rated on excellent to fair scale (i.e., from COSMIN checklist), if even one item is rated as POOR, the score for that variable/construct is POOR.

A. Validity

Excellent = If there is a statistical correlation/regression of any sort, even if simple, between target instrument and other instruments

Fair = If there is no statistical analysis but just a discussion

Low = If there is no discussion whatsoever

Excellent = Assessed if all items refer to relevant aspects of the construct to be measured

Fair = Aspects of the construct to be measured poorly described AND this was not taken into consideration

Poor = NOT assessed if all items refer to relevant aspects of the construct to be measured

Excellent = Assessed if all items are relevant for the study population in adequate sample size (≥ 10)

Good = Assessed if all items are relevant for the study population in moderate sample size (5–9)

Fair = Assessed if all items are relevant for the study population in small sample size (< 5)

Poor = NOT assessed if all items are relevant for the study population OR target population not involved

Excellent = Adequate description of the constructs measured by the comparator instrument(s) Good = Adequate description of most of the constructs measured by the comparator instrument(s)

Fair = Poor description of the constructs measured by the comparator instrument(s)

Poor = NO description of the constructs measured by the comparator instrument(s)

Excellent = Adequate measurement properties of the comparator instrument(s) in a population similar to the study population

Good = Adequate measurement properties of the comparator instrument(s) but not sure if these apply to the study population

Fair = Some information on measurement properties (or a reference to a study on measurement properties) of the comparator instrument(s) in any study population

Poor = No information on the measurement properties of the comparator instrument(s)

Excellent = Statistical methods applied appropriate

Good = Assumable that statistical methods were appropriate, e.g., Pearson correlations applied, but distribution of scores or mean (SD) not presented

Fair = Statistical methods applied NOT optimal

Poor = Statistical methods applied NOT appropriate OR statistical methods not reported Or no correlation between two different measures of memory

The test should predict performance on other measures/contexts to which the results will be generalised; does STM/WM test predict performance on other measures beyond the construct of STM/WM?

The STM/WM test has been shown to discriminate those with and without typical STM/WM abilities

B. Reliability

Excellent = Adequate sample size (≥ 100)

Good = Good sample size (50–99)

Fair = Moderate sample size (30–49)

Poor = Small sample size (< 30) OR sample size not reported OR test-retest reliability not reported

Excellent = at least 2 measurements

Poor = only one measurement

Excellent = Independent measurements

Good = Assumable that the measurements were independent

Fair = Doubtful whether the measurements were independent

Poor = measurements NOT independent OR not reported

Excellent = Time interval stated

Fair = time interval NOT stated

Excellent = Patients were stable (evidence provided)

Good = Assumable that patients were stable

Fair = Unclear if patients were stable

Poor = Patients were NOT stable OR patient status not reported

Excellent = Time interval appropriate

Fair = Doubtful whether time interval was appropriate

Poor = Time interval NOT appropriate

Excellent = Test conditions were similar (evidence provided)

Good = Assumable that test conditions were similar

Fair = Unclear if test conditions were similar

Poor = Test conditions were NOT similar OR no reported

Excellent = ICC calculated and model or formula of the ICC is described

Good = ICC calculated but model or formula of the ICC not described or not optimal

Fair = Pearson or Spearman correlation coefficient calculated WITHOUT evidence provided that no systematic change has occurred or WITH evidence that systematic change has occurred (i.e., strict simple r = or > .90; relaxed .80)

Poor = No ICC or Pearson or Spearman correlations calculated OR method not reported

Excellent = Kappa calculated and reported

Fair = other statistical analysis calculated and reported

Poor = Only percentage agreement calculated OR method not reported

Poor = Small sample size (< 30) OR not reported

Excellent = Yes

Fair = only item-total correlations calculated

Poor = no Cronbach’s or item-total correlations OR method not reported

• If Cronbach’s was reported does it meet criterion (strict Cronbach alpha = or > .90; relaxed .80)? Report criterion (i.e., need to extract from manual or article)
• Split half procedure not applicable

Strict criteria: simple correlation ( r ) for 2 ratings = or > .90; .80 for Kappa Relaxed criteria: r .80; Kappa .70

C. Measurement Error

Excellent = SEM, SDC, or LoA calculated

Good = Possible to calculate LoA from the data presented

Poor = SEM calculated based on Cronbach’s alpha, or on SD from another population OR not reported

1 The combination of the two versions of the Digit Span (forward and backward recall) in Table 6 and visuo-spatial correlates in Table 8 (e.g., forward and backward WMS-R visuo-spatial span subtests), does not imply that the two versions reflect similar processes. Although backward recall is often regarded as a WM (as opposed to an STM) task, it does differ from the other WM tasks we came across in terms of its complexity. Serial recall is not an explicit feature of WM tasks. Furthermore, we did not encounter separate exploration or description of the psychometric properties of standardised forward versus backward recall subtests.

2 Only a limited number of RTT subtests are non-serial tasks.

Disclosure statement

No potential conflict of interest was reported by the authors.

• Abou El Ella M, Alloush T, El-Shobary A, El-Dien Hafez N, El-Halim A, El-Rouby I. Modification and standardization of Arabic version of the comprehensive aphasia test. Aphasiology. 2013; 27 :599–614. [ Google Scholar ]
• Adrover-Roig D, Galparsoro-Izagirre N, Marcotte K, Ferréé P, Wilson MA, Inéés Ansaldo A. Impaired L1 and executive control after left basal ganglia damage in a bilingual Basque-Spanish person with aphasia. Clinical Linguistics & Phonetics. 2011; 25 :480–498. [ PubMed ] [ Google Scholar ]
• Allen CM, Martin RC, Martin N. Relations between short-term memory deficits, semantic processing, and executive function. Aphasiology. 2012; 26 (3–4):428–461. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Allport DA. Auditory-verbal short-term memory and conduction aphasia. In: Bouma H, Bouwhuis DG, editors. Attention and performance X: Control of language processes. Hove, UK: Lawrence Erlbaum; 1984. pp. 313–326. [ Google Scholar ]
• Anastasi A, Urbina S. Psychological testing. 7. Upper Saddle River, NJ: Prentice Hall; 2009. [ Google Scholar ]
• Ardila A, Concha M, Rosselli M. Angular gyrus syndrome revisited: Acalculia, finger agnosia, right-left disorientation and semantic aphasia. Aphasiology. 2000; 14 :743–754. [ Google Scholar ]
• Attard MC, Rose ML, Lanyon L. The comparative effects of multi-modality aphasia therapy and constraint-induced aphasia therapy-plus for severe chronic Broca’s aphasia: An in-depth pilot study. Aphasiology. 2013; 27 :80–111. [ Google Scholar ]
• Baddeley A. Working memory: Theories, models, and controversies. Annual Review of Psychology. 2012; 63 :1–29. [ PubMed ] [ Google Scholar ]
• Barbeau E, Didic M, Tramoni E, Felician O, Joubert S, Sontheimer A, … Poncet M. Evaluation of visual recognition memory in MCI patients. Neurology. 2004; 62 :1317–1322. [ PubMed ] [ Google Scholar ]
• Basilakos A, Rorden C, Bonilha L, Moser D, Fridriksson J. Patterns of poststroke brain damage that predict speech production errors in apraxia of speech and aphasia dissociate. Stroke. 2015; 46 :1561–1566. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Baxendale S. The Flynn effect and memory function. Journal of Clinical and Experimental Neuropsychology. 2010; 32 :699–703. [ PubMed ] [ Google Scholar ]
• Beeson PM, Rising K, Volk J. Writing treatment for severe aphasia: Who benefits? Journal of Speech Language and Hearing Research. 2003; 46 :1038–1060. [ PubMed ] [ Google Scholar ]
• Berthier ML, Davila G, Garcia-Casares N, Green C, Juarez R, Ruiz-Cruces R, … Barbancho MA. Atypical conduction aphasia and the right hemisphere: Cross-hemispheric plasticity of phonology in a developmentally dyslexic and dysgraphic patient with early left frontal damage. Neurocase. 2011; 17 :93–111. [ PubMed ] [ Google Scholar ]
• Biddle A, Watson L, Hooper C. Using evidence-based research in speech-language pathology. Presentation at the Academy of Neurologic Communication Disorders and Sciences Annual Meeting; Atlanta, GA. 2002. [ Google Scholar ]
• Biran M, Fisher S. Structured treatment can improve predicate argument structure impairment. Aphasiology. 2015; 29 :29–56. [ Google Scholar ]
• Bose A, van Lieshout P. Effects of utterance length on lip kinematics in aphasia. Brain and Language. 2008; 106 :4–14. [ PubMed ] [ Google Scholar ]
• Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, … De Vet HC. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. Clinical Chemistry and Laboratory Medicine. 2003; 41 :68–73. [ PubMed ] [ Google Scholar ]
• Butler RA, Ralph MAL, Woollams AM. Capturing multidimensionality in stroke aphasia: Mapping principal behavioural components to neural structures. Brain. 2014; 137 :3248–3266. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Campanella F, Crescentini C, Mussoni A, Skrap M. Refractory semantic access dysphasia resulting from resection of a left frontal Glioma. Neurocase. 2013; 19 :27–35. [ PubMed ] [ Google Scholar ]
• Cappelletti M, Freeman ED, Cipolotti L. Numbers and time doubly dissociate. Neuropsychologia. 2011; 49 :3078–3092. [ PubMed ] [ Google Scholar ]
• Carragher M, Sage K, Conroy P. The effects of verb retrieval therapy for people with non-fluent aphasia: Evidence from assessment tasks and conversation. Neuropsychological Rehabilitation. 2013; 23 :846–887. [ PubMed ] [ Google Scholar ]
• Casaletto KB, Umlauf A, Beaumont J, Gershon R, Slotkin J, Akshoomoff N, Heaton RK. Demographically corrected normative standards for the English version of the NIH toolbox cognition battery. Journal of the International Neuropsychological Society. 2015; 21 :378–391. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Caspari I, Parkinson SR, LaPointe LL, Katz RC. Working memory and aphasia. Brain and Cognition. 1998; 37 :205–223. [ PubMed ] [ Google Scholar ]
• Caza N, Belleville S, Gilbert B. How loss of meaning with preservation of phonological word form affects immediate serial recall performance: A linguistic account. Neurocase. 2002; 8 :255–273. [ PubMed ] [ Google Scholar ]
• Centre for Reviews and Dissemination. Systematic reviews: CRD’s guidance for undertaking reviews in health care. York: Centre for Reviews and Dissemination, University of York; 2008. [ Google Scholar ]
• Chiou HS, Kennedy MRT. Switching in adults with aphasia. Aphasiology. 2009; 23 :1065–1075. [ Google Scholar ]
• Christensen SC, Wright HH. Verbal and non-verbal working memory in aphasia: What three n-back tasks reveal. Aphasiology. 2010; 24 :752–762. [ Google Scholar ]
• Coelho C. Direct attention training as a treatment for reading impairment in mild aphasia. Aphasiology. 2005; 19 :275–283. [ Google Scholar ]
• Conroy P, Sage K, Lambon Ralph MA. Errorless and errorful therapy for verb and noun naming in aphasia. Aphasiology. 2009; 23 :1311–1337. [ Google Scholar ]
• Conway AR, Kane MJ, Bunting MF, Hambrick DZ, Wilhelm O, Engle RW. Working memory span tasks: A methodological review and user’s guide. Psychological Bulletin Review. 2005; 12 :769–786. [ PubMed ] [ Google Scholar ]
• Coolican H. Research methods and statistics in psychology. 6. Hove, UK: Psychology Press; 2014. [ Google Scholar ]
• Corsi PM. Unpublished PhD dissertation. McGill University; Canada: 1972. Memory and medial temporal region of the brain. [ Google Scholar ]
• Corsten S, Mende M, Cholewa J, Huber W. Treatment of input and output phonology in aphasia: A single case study. Aphasiology. 2007; 21 :587–603. [ Google Scholar ]
• Cowan N. Multiple concurrent thoughts: the meaning and developmental neuropsychology of working memory. Developmental Neuropsychology. 2010; 35 :447–474. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Crescentini C, Lunardelli A, Mussoni A, Zadini A, Shallice T. A left basal ganglia case of dynamic aphasia or impairment of extra-language cognitive processes? Neurocase. 2008; 14 :184–203. [ PubMed ] [ Google Scholar ]
• Crocket D, Clark C, Spreen S, Klonoff H. Severity of impairment or specific types of aphasia: An empirical investigation. Cortex. 1981; 17 :83–95. [ PubMed ] [ Google Scholar ]
• DeDe G, Ricca M, Knilans J, Trubl B. Construct validity and reliability of working memory tasks for people with aphasia. Aphasiology. 2014; 28 :692–712. [ Google Scholar ]
• De Renzi E, Nichelli P. Verbal and non-verbal short-term memory impairment following hemispheric damage. Cortex. 1975; 11 :341–354. [ PubMed ] [ Google Scholar ]
• De-Torres I, Dávila G, Berthier ML, Walsh SF, Moreno-Torres I, Ruiz-Cruces R. Repeating with the right hemisphere: Reduced interactions between phonological and lexical-semantic systems in crossed aphasia? Frontiers in Human Neuroscience. 2013; 7 doi: 10.3389/fnhum.2013.00675. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Dewar BK, Patterson K, Wilson BA, Graham KS. Re-acquisition of person knowledge in semantic memory disorders. Neuropsychological Rehabilitation. 2009; 19 :383–421. [ PubMed ] [ Google Scholar ]
• Dotan D, Friedmann N. Steps towards understanding the phonological output buffer and its role in the production of numbers, morphemes, and function words. Cortex. 2015; 63 :317–351. [ PubMed ] [ Google Scholar ]
• Ehrenstein WH, Ehrenstein A. Psychophysical methods. In: Windhorst U, Johansson H, editors. Modern techniques in neuroscience research. Berlin: Springer; 1999. pp. 1211–1241. [ Google Scholar ]
• Edwards S, Salis C, Meteyard L. Aphasia. In: Aronoff M, editor. Oxford bibliographies in linguistics. 2. New York: Oxford University Press; 2015. [ CrossRef ] [ Google Scholar ]
• Eling P. Kurt Goldstein’s test battery. Cortex. 2015; 63 :16–26. [ PubMed ] [ Google Scholar ]
• Fastenau PS, Denburg NL, Hufford BJ. Adult norms for the Rey-Osterrieth Complex Figure test and for supplemental recognition and matching trials from the extended Complex Figure test. The Clinical Neuropsychologist (Neuropsychology, Development and Cognition: Section D) 1999; 13 :30–47. [ PubMed ] [ Google Scholar ]
• Fillingham JK, Sage K, Lambon Ralph MA. The treatment of anomia using errorless learning. Neuropsychological Rehabilitation. 2006; 16 (2):129–154. [ PubMed ] [ Google Scholar ]
• Flynn JR. The mean IQ of Americans: Massive gains 1932–1978. Psychological Bulletin. 1984; 95 :29–51. [ Google Scholar ]
• Flynn JR. The WAIS-III and WAIS-IV: Daubert motions favor the certainly false over the approximately true. Applied Neuropsychology. 2009; 16 :98–104. [ PubMed ] [ Google Scholar ]
• Francis D, Clark N, Humphreys G. The treatment of an auditory working memory deficit and the implications for sentence comprehension abilities in mild receptive aphasia. Aphasiology. 2003; 17 :723–750. [ Google Scholar ]
• Francis DR, Clark N, Humphreys GW. Circumlocution-induced naming (CIN): A treatment for effecting generalisation in anomia? Aphasiology. 2002; 16 :243–259. [ Google Scholar ]
• Friedmann N, Gvion A. FriGvi: Friedmann Gvion battery for assessment of phonological working memory. Israel: Tel Aviv University; 2002. [ Google Scholar ]
• Friedmann N, Gvion A. As far as individuals with conduction aphasia understood these sentences were ungrammatical: Garden path in conduction aphasia. Aphasiology. 2007; 21 :570–586. [ Google Scholar ]
• Fucetola R, Connor LT, Perry J, Leo P. Aphasia, severity, semantics, and depression predict functional communication in acquired aphasia. Aphasiology. 2006; 20 :449–461. [ Google Scholar ]
• Fucetola R, Connor LT, Strube MJ, Corbetta M. Unravelling nonverbal cognitive performance in acquired aphasia. Aphasiology. 2009; 23 :1418–1426. [ Google Scholar ]
• Galling MA, Goorah N, Berthier ML, Sage K. A clinical study of the combined use of bromocriptine and speech and language therapy in the treatment of a person with aphasia. Aphasiology. 2014; 28 :171–187. [ Google Scholar ]
• Greenhalgh T. How to read a paper: Papers that report diagnostic or screening tests. BMJ Clinical Research. 2014; 315 :540–543. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Grindrod CM, Baum SR. Hemispheric contributions to lexical ambiguity resolution in a discourse context: Evidence from individuals with unilateral left and right hemisphere lesions. Brain and Cognition. 2005; 57 :70–83. [ PubMed ] [ Google Scholar ]
• Gualtieri CT, Johnson LG. Reliability and validity of a computerized neurocognitive test battery, CNS Vital Signs. Archives of Clinical Neuropsychology. 2006; 21 :623–643. [ PubMed ] [ Google Scholar ]
• Gvion A, Friedmann N. Does phonological working memory impairment affect sentence comprehension? A study of conduction aphasia. Aphasiology. 2012; 26 :494–535. [ Google Scholar ]
• Harnish SM, Lundine J. Nonverbal working memory as a predictor of anomia treatment success. American Journal of Speech-Language Pathology. 2015; 24 (4):S880–S894. [ PubMed ] [ Google Scholar ]
• Härting C, Markowitsch H-J, Neufeld H, Calabrese P, Deisinger K, Kessler J. Wechsler memory scale – Revised Edition, German Edition. Bern: Huber; 2000. [ Google Scholar ]
• Helm-Estabrooks N. Cognitive linguistic quick test. San Antonio, TX: The Psychological Corporation; 2001. [ Google Scholar ]
• Helm-Estabrooks N. Cognition and aphasia: A discussion and a study. Journal of Communication Disorders. 2002; 35 :171–186. [ PubMed ] [ Google Scholar ]
• Hendricks CT, Nicholas ML, Zipse L. Effects of phonological neighbourhood on the treatment of naming in aphasia. Aphasiology. 2014; 28 :338–358. [ Google Scholar ]
• Hoffman P, Jefferies E, Haffey A, Littlejohns T, Lambon Ralph MA. Domain-specific control of semantic cognition: A dissociation within patients with semantic working memory deficits. Aphasiology. 2013; 27 :740–764. [ Google Scholar ]
• Howard D, Franklin S. Memory without rehearsal. In: Vallar G, Shallice T, editors. Neuropsychological impairments of short-term memory. Cambridge: Cambridge University Press; 1990. pp. 287–318. [ Google Scholar ]
• Howard D, Nickels L. Separating input and output phonology: Semantic, phonological, and orthographic effects in short-term memory impairment. Cognitive Neuropsychology. 2005; 22 :42–77. [ PubMed ] [ Google Scholar ]
• Ivanova M, Hallowell B. A new modified listening span task to enhance validity of working memory assessment for people with and without aphasia. Journal of Communication Disorders. 2014; 52 :78–98. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Ivanova MV, Dragoy OV, Kuptsova SV, Ulicheva AS, Laurinavichyute AK. The contribution of working memory to language comprehension: Differential effect of aphasia type. Aphasiology. 2015; 29 :645–664. [ Google Scholar ]
• Jacquemot C, Dupoux E, Bachoud-Lévi AC. Is the word-length effect linked to subvocal rehearsal? Cortex. 2011; 47 :484–493. [ PubMed ] [ Google Scholar ]
• Jaeggi SM, Buschkuehl M, Perrig WJ, Meier B. The concurrent validity of the N-back task as a working memory measure. Memory. 2010; 18 (4):394–412. [ PubMed ] [ Google Scholar ]
• Kalbe E, Reinhold N, Brand M, Markowitsch HJ, Kessler J. A new test battery to assess aphasic disturbances and associated cognitive dysfunctions – German normative data on the aphasia check list. Journal of Clinical and Experimental Neuropsychology. 2005; 27 :779–794. [ PubMed ] [ Google Scholar ]
• Kambanaros M, Weekes BS. Phonological dysgraphia in bilingual aphasia: Evidence from a case study of Greek and English. Aphasiology. 2013; 27 :59–79. [ Google Scholar ]
• Kane MJ, Conway ARA, Hambrick DZ, Engle RW. Variation in working memory capacity as variation in executive attention and control. In: Conway ARA, Jarrold C, Kane MJ, Miyake A, Towse JN, editors. Variation in working memory. New York: Oxford University Press; 2007. pp. 21–48. [ Google Scholar ]
• Kasselimis DS, Panagiotis GS, Economou A, Peppas C, Evdokimidis I, Potagas C. Are memory deficits dependent on the presence of aphasia in left brain damaged patients? Neuropsychologia. 2013; 51 :1773–1776. [ PubMed ] [ Google Scholar ]
• Kay J, Lesser R, Coltheart M. Psycholinguistic assessments of language processing in Aphasia. Hove, UK: Psychology Press; 1992. [ Google Scholar ]
• Kearney-Ramos TE, Fausett JS, Gess JL, Reno A, Peraza J, Kilts CD, James GA. Merging clinical neuropsychology and functional neuroimaging to evaluate the construct validity and neural network engagement of the n -back task. Journal of the International Neuropsychological Society. 2014; 20 :736–750. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kendall D, Conway T, Rosenbek J, Gonzalez-Rothi L. Case study of phonological rehabilitation of acquired phonologic alexia. Aphasiology. 2003; 17 :1073–1095. [ Google Scholar ]
• Kendall DL, Nadeau SE, Conway T, Fuller RH, Riestra A, Gonzalez Rothi LJ. Treatability of different components of aphasia – Insights from a case study. The Journal of Rehabilitation Research and Development. 2006; 43 :323–335. [ PubMed ] [ Google Scholar ]
• Kessels RP, van den Berg E, Ruis C, Brands AM. The backward span of the Corsi block-tapping task and its association with the WAIS-III digit span. Assessment. 2008; 15 :426–434. [ PubMed ] [ Google Scholar ]
• Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. Journal of the Royal Society of Medicine. 2003; 96 :118–121. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lang CJG, Quitz A. Verbal and nonverbal memory impairment in aphasia. Journal of Neurology. 2012; 259 :1655–1661. [ PubMed ] [ Google Scholar ]
• Larsen J, Baynes K, Swick D. Right hemisphere reading mechanisms in a global alexic patient. Neuropsychologia. 2004; 42 :1459–1476. [ PubMed ] [ Google Scholar ]
• Lavoie M, Routhier S, Légaré A, Macoir J. Treatment of verb anomia in aphasia: Efficacy of self-administered therapy using a smart tablet. Neurocase. 2016; 22 (1):109–118. [ PubMed ] [ Google Scholar ]
• Law SP, Wong W, Chiu KMY. Preserved reading aloud with semantic deficits: Evidence for a non-semantic lexical route for reading Chinese. Neurocase. 2005; 11 :167–175. [ PubMed ] [ Google Scholar ]
• Lee B, Pyun SB. Characteristics of cognitive impairment in patients with post-stroke aphasia. Annals of Rehabilitation Medicine. 2014; 38 (6):759–765. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lee JB, Sohlberg MM. Evaluation of attention training and metacognitive facilitation to improve reading comprehension in aphasia. American Journal of Speech-Language Pathology. 2013; 22 :318–333. [ PubMed ] [ Google Scholar ]
• Lee TM, Yuen KS, Chan CC. Normative data for neuropsychological measures of fluency, attention, and memory measures for Hong Kong Chinese. Journal of Clinical and Experimental Neuropsychology (Neuropsychology, Development and Cognition: Section A) 2002; 24 :615–632. [ PubMed ] [ Google Scholar ]
• Leff AP, Schofield TM, Crinion JT, Seghier ML, Grogan A, Green DW, Price CJ. The left superior temporal Gyrus is a shared substrate for auditory short-term memory and speech comprehension: Evidence from 210 patients with stroke. Brain. 2009; 132 :3401–3410. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lehman MT, Tompkins CA. Reliability and validity of an auditory working memory measure: Data from elderly and right-hemisphere damaged adults. Aphasiology. 1998; 12 :771–785. [ Google Scholar ]
• Léonard B, de Partz MP, Grandin C, Pillon A. Domain-specific reorganization of semantic processing after extensive damage to the left temporal lobe. NeuroImage. 2009; 45 :572–586. [ PubMed ] [ Google Scholar ]
• Léonard B, Pillon A, de Partz M. Reacquisition of semantic knowledge by errorless learning in a patient with a semantic deficit and anterograde amnesia. Aphasiology. 2008; 22 :447–488. [ Google Scholar ]
• Lidzba K, Staudt M, Zieske F, Schwilling E, Ackermann H. Prestroke/poststroke fMRI in aphasia: Perilesional hemodynamic activation and language recovery. Neurology. 2012; 78 :289–291. [ PubMed ] [ Google Scholar ]
• Loring DW, Bauer RM. Testing the limits: Cautions and concerns regarding the new Wechsler IQ and Memory scales. Neurology. 2010; 74 :685–690. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lott SN, Sperling AJ, Watson NL, Friedman RB. Repetition priming in oral text reading: a therapeutic strategy for phonologic text alexia. Aphasiology. 2009; 23 :659–675. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Logie RH. The functional organization and capacity limits of working memory. Current Directions in Psychological Science. 2011; 20 :240–245. [ Google Scholar ]
• Majerus S, Attout L, Artielle M, Van der Kaa M. The heterogeneity of verbal short-term memory impairment in aphasia. Neuropsychologia. 2015; 77 :165–176. [ PubMed ] [ Google Scholar ]
• Martin N. The role of semantic processing in short-term memory and learning: Evidence from aphasia. In: Thorn A, Page M, editors. Interactions between short-term and long-term memory in the verbal domain. Hove, UK: Psychology Press; 2009. pp. 220–243. [ Google Scholar ]
• Martin N, Ayala J. Measurements of auditory-verbal STM span in aphasia: Effects of item, task, and lexical impairment. Brain and Language. 2004; 89 :464–483. [ PubMed ] [ Google Scholar ]
• Martin N, Gupta P. Exploring the relationship between word processing and verbal short-term memory: Evidence from association and dissociations. Cognitive Neuropsychology. 2004; 21 :213–228. [ PubMed ] [ Google Scholar ]
• Martin N, Kohen FP, Kalinyak-Fliszar M. A processing approach to the assessment of language and verbal short-term memory abilities in aphasia. Paper presented at the Clinical Aphasiology Conference.2010. [ Google Scholar ]
• Martin N, Saffran EM, Dell GS. Recovery in deep dysphasia: Evidence for a relation between auditory-verbal STM capacity and lexical errors in repetition. Brain and Language. 1996; 52 :83–113. [ PubMed ] [ Google Scholar ]
• Martin RC, Allen CM. A disorder of executive function and its role in language processing. Seminars in Speech and Language. 2008; 29 :201–210. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Martin RC, Lesch MF, Bartha MC. Independence of input and output phonology in word processing and short-term memory. Journal of Memory and Language. 1999; 41 :3–29. [ Google Scholar ]
• May CP, Hasher L, Kane MJ. The role of interference in memory span. Memory & Cognition. 1999; 27 :759–767. [ PubMed ] [ Google Scholar ]
• Mayer J, Murray L. Approaches to the treatment of alexia in chronic aphasia. Aphasiology. 2002; 16 :727–743. [ Google Scholar ]
• Mayer JF, Murray LL. Measuring working memory deficits in aphasia. Journal of Communication Disorders. 2012; 45 (5):325–339. [ PubMed ] [ Google Scholar ]
• McNeil MR, Prescott TE. The revised token test. Baltimore, MD: University Park Press; 1978. [ Google Scholar ]
• Meteyard L, Bruce C, Edmundson A, Oakhill J. Profiling text comprehension impairments in aphasia. Aphasiology. 2015; 29 :1–28. [ Google Scholar ]
• Meyers JE, Meyers KR. Rey complex figure test and recognition trial: Professional manual. Odessa, FL: Psychological Assessment Resources; 1995. [ Google Scholar ]
• Milner B. Interhemispheric differences in the localization of psychological processes in man. British Medical Bulletin. 1971; 27 :272–277. [ PubMed ] [ Google Scholar ]
• Miyake A, Friedman NP. The nature and organization of individual differences in executive functions: Four general conclusions. Current Directions in Psychological Science. 2012; 21 :8–14. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Miyake A, Friedman NP, Emerson MJ, Witzki AH, Howerter A. The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology. 2000; 41 :49–100. [ PubMed ] [ Google Scholar ]
• Moher D, Liberati A, Tetzlaff J, Altman DG The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009; 6 (6):e1000097. doi: 10.1371/journal.pmed1000097. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, … De Vet HC. The COSMIN checklist manual. Amsterdam: VU University Medical Centre; 2009. [ Google Scholar ]
• Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, … de Vet HC. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: An international Delphi study. Quality of Life Research. 2010; 19 :539–549. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Molero Martin D, Laird R, Hwang F, Salis C. Computerized short-term memory treatment for older adults with aphasia: Feedback from clinicians. Proceedings of the 15th international ACM SIGACCESS Conference on Computers and Accessibility; Bellevue, Washington. New York, NY: ACM; 2013. pp. 44:1–44:2. [ Google Scholar ]
• Morris N, Jones DM. Memory updating in working memory: The role of the central executive. British Journal of Psychology. 1990; 81 :111–121. [ Google Scholar ]
• Murray LL. Direct and indirect treatment approaches for addressing short-term or working memory deficits in aphasia. Aphasiology. 2012a; 26 :317–337. [ Google Scholar ]
• Murray LL. Attention and other cognitive deficits in aphasia: Presence and relation to language and communication measures. American Journal of Speech-Language Pathology. 2012b; 21 :51–64. [ PubMed ] [ Google Scholar ]
• Murray L, Ballard K, Karcher L. Linguistic specific treatment: Just for Broca’s aphasia? Aphasiology. 2004; 18 :785–809. [ Google Scholar ]
• Murray LL, Clark HM. Neurogenic disorders of language and cognition: Evidence-based clinical practice. Austin, TX: Pro-Ed; 2015. [ Google Scholar ]
• Murray LL, Keeton RJ, Karcher L. Treating attention in mild aphasia: Evaluation of attention process training-II. Journal of Communication Disorders. 2006; 39 :37–61. [ PubMed ] [ Google Scholar ]
• Murray L, Timberlake A, Eberle R. Treatment of underlying forms in a discourse context. Aphasiology. 2007; 21 :139–163. [ Google Scholar ]
• Myers PS. Toward a definition of RHD syndrome. Aphasiology. 2001; 15 :913–918. [ Google Scholar ]
• Nicholas M, Hunsaker E, Guarino AJ. Aphasiology. 2015. The relation between language, non-verbal cognition and quality of life in people with aphasia; pp. 1–14. ahead-of-print. [ CrossRef ] [ Google Scholar ]
• Nicholas M, Sinotte M, Helm-Estabrooks N. Using a computer to communicate: Effect of executive function impairments in people with severe aphasia. Aphasiology. 2005; 19 :1052–1065. [ Google Scholar ]
• Nguyen VQ, PrvuBettger J, Guerrier T, Hirsch MA, Thomas JG, Pugh T, Rhoads C. Factors associated with discharge to home versus discharge to institutional care after inpatient stroke rehabilitation. Archives of Physical Medicine and Rehabilitation. 2015; 96 :1297–303. [ PubMed ] [ Google Scholar ]
• Norman MA, Moore DJ, Taylor M, Franklin D, Jr, Cysique L, Ake C HNRC Group. Demographically corrected norms for African Americans and caucasians on the hopkins verbal learning test-revised, brief visuospatial memory test-revised, Stroop color and Word test, and Wisconsin card sorting test 64-card version. Journal of Clinical and Experimental Neuropsychology. 2011; 33 :793–804. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Noyes JM, Garland KJ. Computer- versus paper-based tasks: Are they equivalent? Ergonomics. 2008; 51 :1352–1375. [ PubMed ] [ Google Scholar ]
• Orsini A, Capitani E, Laiacona M, Papagno C, Vallar G. Verbal and spatial immediate memory span: Normative data from 1355 adults and 1112 children. The Italian Journal of Neurological Sciences. 1987; 8 :537–548. [ PubMed ] [ Google Scholar ]
• Otsuka Y, Suzuki K, Fujii T, Miura R, Endo K, Kondo H, Yamadori A. Proper name anomia after left temporal subcortical hemorrhage. Cortex. 2005; 41 :39–47. [ PubMed ] [ Google Scholar ]
• Peterson LR, Peterson MJ. Short-term retention of individual verbal items. Journal of Experimental Psychology. 1959; 58 :193–198. [ PubMed ] [ Google Scholar ]
• Plaza M, Gatignol P, Leroy M, Duffau H. Speaking without Broca’s area after tumor resection. Neurocase. 2009; 15 :294–310. [ PubMed ] [ Google Scholar ]
• Renvall R, Laine M, Laakso M, Nadine Martin N. Anomia treatment with contextual priming: A case study. Aphasiology. 2003; 17 :305–328. [ Google Scholar ]
• Rey A. L’examen clinique en psychologie. Paris: Presses Universitaires de France; 1964. [ Google Scholar ]
• Richardson JTE. Measures of short-term memory: A historical review. Cortex. 2007; 43 :635–650. [ PubMed ] [ Google Scholar ]
• Robertson IH, Ward T, Ridgeway V, Nimmo-Smith I. The Test of Everyday Attention (TEA) Bury St Edmunds, UK: Thames Valley Test Company; 1994. [ Google Scholar ]
• Robson H, Sage K, Lambon Ralph MA. Wernicke’s aphasia reflects a combination of acoustic-phonological and semantic control deficits: A case-series comparison of Wernicke’s aphasia, semantic dementia and semantic aphasia. Neuropsychologia. 2012; 50 :266–275. [ PubMed ] [ Google Scholar ]
• Romani C. Are there distinct input and output buffers? Evidence from an aphasic patient with an impaired output buffer. Language and Cognitive Processes. 1992; 7 :131–162. [ Google Scholar ]
• Sage K, Snell C, Lambon Ralph MA. How intensive does anomia therapy for people with aphasia need to be? Neuropsychological Rehabilitation. 2011; 21 (1):26–41. [ PubMed ] [ Google Scholar ]
• Salis C. Short-term memory treatment: Patterns of learning and generalisation to sentence comprehension in a person with aphasia. Neuropsychological Rehabilitation. 2012; 22 :428–448. [ PubMed ] [ Google Scholar ]
• Salis C, Hwang F. Digital technology and aphasia. Aphasiology. 2016; 30 :109–111. [ Google Scholar ]
• Salis C, Kelly H, Code C. Assessment and treatment of short-term and working memory impairments in stroke aphasia: A practical tutorial. International Journal of Language and Communication Disorders. 2015; 50 :721–736. [ PubMed ] [ Google Scholar ]
• Sarno MT. The nature of verbal impairment after closed head injury. The Journal of Nervous and Mental Disease. 1980; 168 :685–692. [ PubMed ] [ Google Scholar ]
• Schellig D. Block-tapping test. Lisse: Swets & Zeitlinger; 1997. [ Google Scholar ]
• Schlosser RW, Wendt O, Sigafoos J. Not all systematic reviews are created equal: Considerations for appraisal. Evidence-Based Communication Assessment and Intervention. 2007; 1 :138–150. [ Google Scholar ]
• Schuell H, Jenkins JJ, Jimenez-Pabon E. Aphasia in adults: Diagnosis, prognosis and treatment. London: Harper Row; 1964. [ Google Scholar ]
• Seniów J, Litwin M, Leśniak M. The relationship between non-linguistic cognitive deficits and language recovery in patients with aphasia. Journal of the Neurological Sciences. 2009; 283 :91–94. [ PubMed ] [ Google Scholar ]
• Sidiropoulos K, Ackermann H, Wannke M, Hertrich I. Temporal processing capabilities in repetition conduction aphasia. Brain and Cognition. 2010; 73 :194–202. [ PubMed ] [ Google Scholar ]
• Sidiropoulos K, de Bleser R, Ablinger I, Ackermann H. The relationship between verbal and nonverbal auditory signal processing in conduction aphasia: Behavioral and anatomical evidence for common decoding mechanisms. Neurocase. 2015; 21 :377–393. [ PubMed ] [ Google Scholar ]
• Sidiropoulos K, de Bleser R, Ackermann H, Preilowski B. Pre-lexical disorders in repetition conduction aphasia. Neuropsychologia. 2008; 46 :3225–3238. [ PubMed ] [ Google Scholar ]
• Sierpowska J, Gabarrós A, Ripollés P, Juncadella M, Castañer S, Camins Á, … Rodríguez-Fornells A. Intraoperative electrical stimulation of language switching in two bilingual patients. Neuropsychologia. 2013; 51 :2882–2892. [ PubMed ] [ Google Scholar ]
• Sinotte MP, Coelho CA. Attention training for reading impairment in mild aphasia: A follow-up study. NeuroRehabilitation. 2007; 22 :303–310. [ PubMed ] [ Google Scholar ]
• Solcà M, Di Pietro M, Schnider A, Leemann B. Impairment of semantic memory after basal forebrain and fornix lesion. Neurocase. 2015; 21 :198–205. [ PubMed ] [ Google Scholar ]
• Sulleman S, Kim E. Decision-making, cognition, and aphasia: Developing a foundation for future discussions and inquiry. Aphasiology. 2015; 29 :1409–1425. [ Google Scholar ]
• Sung JE, McNeil MR, Pratt SR, Dickey MW, Hula WD, Szuminsky NJ, Doyle PJ. Verbal working memory and its relationship to sentence-level reading and listening comprehension in persons with aphasia. Aphasiology. 2009; 23 :1040–1052. [ Google Scholar ]
• Swinburn K, Porter G, Howard D. Comprehensive aphasia test. Hove, UK: Psychology Press; 2004. [ Google Scholar ]
• Thompson HE, Jefferies E. Semantic control and modality: An input processing deficit in aphasia leading to deregulated semantic cognition in a single modality. Neuropsychologia. 2013; 51 :1998–2015. [ PubMed ] [ Google Scholar ]
• Tompkins CA, Bloise CGR, Timko ML, Baumgaertner A. Working memory and inference revision in brain damaged and normally aging adults. Journal of Speech Language and Hearing Research. 1994; 37 :896–896. [ PubMed ] [ Google Scholar ]
• Turkstra LS, Coelho C, Ylvisaker M. The use of standardized tests for individuals with cognitive-communication disorders. Seminars in Speech and Language. 2005; 26 :215–222. [ PubMed ] [ Google Scholar ]
• Valle F, Cuetos F. EPLA: Evaluación del Procesamiento Lingüísticos en la Afasia. Hove, UK: Lawrence Erlbaum; 1995. [ Google Scholar ]
• de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in medicine: A practical guide. Cambridge: Cambridge University Press; 2011. [ Google Scholar ]
• Vukovic M, Vuksanovic J, Yukovic I. Comparison of the recovery patterns of language and cognitive functions in patients with post-traumatic language processing deficits and in patients with aphasia following a stroke. Journal of Communication Disorders. 2008; 41 :531–552. [ PubMed ] [ Google Scholar ]
• Wagner R, Torgesen J, Rashotte C. Comprehensive test of phonological processing. Austin, TX: Pro-Ed; 1999. [ Google Scholar ]
• Waters GS, Caplan D. The reliability and stability of verbal working memory measures. Behavior Research Methods, Instruments, & Computers. 2003; 35 :550–564. [ PubMed ] [ Google Scholar ]
• Wechsler D. A standardized memory scale for clinical use. The Journal of Psychology. 1945; 19 :87–95. [ Google Scholar ]
• Wechsler D. Échelle d’Intelligence Ottawa-Wechsler. Ottawa: Institut de pschychologie de l’Université d’Ottawa; 1957. [ Google Scholar ]
• Wechsler D. Échelle Clinique de Wechsler. Paris: Les Éditions de Psychologie Appliquée; 1969. [ Google Scholar ]
• Wechsler D. Wechsler adult intelligence scale – Revised. San Antonio, TX: Psychological Corporation; 1981. [ Google Scholar ]
• Wechsler D. Wechsler memory scale – Revised. San Antonio, TX: Psychological Corporation; 1987. [ Google Scholar ]
• Wechsler D. Wechsler memory scale – III. San Antonio, TX: Psychological Corporation; 1997. [ Google Scholar ]
• Wechsler D. WAIS-III: Manuel de l’ echelle d’ intelligence de Wechsler pour adultes. Paris: Centre de Psychologie Appliquée; 2000. [ Google Scholar ]
• Wechsler D. Wechsler memory scale – IV. San Antonio, TX: Pearson; 2009. [ Google Scholar ]
• Wicherts JM, Dolan CV, Hessen DJ, Oosterveld P, van Baal GCM, Boomsma DI, Span MM. Are intelligence tests measurement invariant over time? Investigating the nature of the Flynn effect. Intelligence. 2004; 32 :509–537. [ Google Scholar ]
• Woods D, Kishiyama MM, Yund EW, Herron TJ, Edwards B, Poliva O, Hink RF, Reed B. Improving digit span assessment of short-term verbal memory. Journal of Clinical and Experimental Neuropsychology. 2011; 33 :101–111. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Wright HH, Fergadiotis G. Conceptualizing and measuring working memory and its relationship to aphasia. Aphasiology. 2012; 26 :258–278. [ PMC free article ] [ PubMed ] [ Google Scholar ]