literature review on internet

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Lydon and O'Leary libraries will be closing at 2:00pm on Wednesday, July 3rd , and will be closed on Thursday, July 4th . If you have any questions, please contact [email protected] .

• University of Massachusetts Lowell
• University Libraries

Literature Review Step by Step

• Internet Sources
• Refining Your Understanding
• Parts of a Literature Review
• Choosing a Topic
• Search Terms
• Peer Review
• Social Media Sources
• Information Landscape

What Tests to Apply When Evaluating Sources

If you cannot access the above video, you can watch it here .

•Is the  author  of the information named? ​• What are the author's  qualifications  and expertise?  •Who  sponsors  the website? •Is the sponsor of the page  reputable ? How reputable? •Is there a link to  information about the author  and/or the sponsor and/or a way to contact them? •Are there links to any other organization(s) that supplies information to the website? •If the page includes neither a signature nor indicates a sponsor, is there a way to determine its origin ?

Why is this important?

• Anyone can publish  anything  on the web.
• Qualifications  can bolster confidence in the information presented.
• Authorship  and/or sponsorship can be an  indication of bias  or limited viewpoint.
• Is the page dated? 
• If so, when was the last update? 
• How current are the links? 
• Have some expired or moved?

Why is this important? When dates are not provided,  you cannot tell how "stale" the information is  and whether newer information exists that's not covered on the website. Bad links are an indication of  a site that might be abandoned  or neglected. Even if  a date  is provided, it  may have various meanings . For example,

• It may indicate when the material was first written.
• It may indicate when the material was first placed on the Web.
• It may indicate when the material was last revised.

Determining Authorship

►Is the document signed? ►Can I get more information on the author by linking from this page to other documents? ►Was there information about the author on the page from which I linked to this one?

If you can answer "Yes" to the second or third question, it's possible that you will have enough information for evaluative purposes, or at least enough information to help you find the author's telephone number or e-mail address so that you may contact him or her with questions.

If you can answer "Yes" to the first question only, you may need to find further information on the author. Start by Googling the name. This process will require some judgement on your part, but if you are quickly lead to a dead end, this should be a red flag.

Who is the Publisher?

Does the page have the following:

•  A header or footer that shows its affiliation as part of a larger web site.
•  A watermark or wallpaper that fulfills the same function.
•  A link at the bottom or top of the page that allows you to go to the home page of the web site where the document lives.
•  A link that allows you to email the site

When evaluating a website there are several things to take into consideration, one of the first things to look at is the URL (Uniform Resource Locator: a protocol for specifying addresses on the Internet) this can often tell you several things about the website, the creator, the audience, the purpose and sometimes even the country of origin.  The URL is the address you type in to get to a web site, for example, https://www.uml.edu/library/ (our library’s address) or https://www.google.se/ (Sweden’s Google search).

A domain name is like a website’s proper name (the part after the www.), businesses and organizations often have a domain name that is their corporate name (for example Microsoft’s domain name is Microsoft.com).  The domain suffix is the end of the domain name (the .com part) and can offer insight into the type of organization the site is linked to.  For example, any commercial enterprise or corporation that has a web site will have a domain suffix of .com, which means it is a commercial entity. Popular domain suffixes include ".com," ".net," ".gov," and ".org," but there are dozens of domain suffixes.  However, since any entity can register domain names with these suffixes, the domain suffix does not always represent the type of website that uses the domain name. For example, many individuals and organizations register ".com" domain names for non-commercial purposes, since the ".com" domain is the most recognized.

The domain suffix might also give you a clue about the geographic origin of a web site, each country also has a unique domain suffix that is meant to be used for websites within the country. For example, Brazilian websites may use the ".br" domain suffix, Chinese websites may use the ".cn" suffix, and Australian websites may use the ".au" suffix. These country-based TLDs, sometimes referred to as "country codes," are also used to specify different versions of an international website. For example, the German home page for Google is "www.google.de" instead of "www.google.com."

http://agriculture. sc . gov /divisions/agency-services/state-farmers-markets/

From the web address above, you can tell it is a Government site (.gov) and it is for the state of South Carolina (.sc) the domain name is for the Department of Agriculture. The page is for a listing of state farmers markets, you can see the path to the filename after each backslash (/).

Recognizing a Personal Website

A personal web page is one published by an individual who may or may not be affiliated with a larger institution. Although the URL address of the page may have a variety of endings (e.g. .com, .edu, etc.), a tilde (~) is frequently embedded somewhere in the URL

Consider the following:

• Is it clear what individual is responsible for the contents of the page?
• Does the individual responsible for the page indicate his or her qualifications for writing on this topic?
• Is there a way of verifying the legitimacy of this individual? (Because it is difficult to verify the legitimacy of an individual, personal home pages may be a useful source for personal opinion but use extreme caution when using them as a source for factual information.)
• << Previous: Peer Review
• Next: Social Media Sources >>
• Last Updated: Jan 22, 2024 2:05 PM
• URL: https://libguides.uml.edu/lit_review_general

To read this content please select one of the options below:

Please note you do not have access to teaching notes, fake news on the internet: a literature review, synthesis and directions for future research.

Internet Research

ISSN : 1066-2243

Article publication date: 29 March 2022

Issue publication date: 7 September 2022

The extensive distribution of fake news on the internet (FNI) has significantly affected many lives. Although numerous studies have recently been conducted on this topic, few have helped us to systematically understand the antecedents and consequences of FNI. This study contributes to the understanding of FNI and guides future research.

Design/methodology/approach

Drawing on the input–process–output framework, this study reviews 202 relevant articles to examine the extent to which the antecedents and consequences of FNI have been investigated. It proposes a conceptual framework and poses future research questions.

First, it examines the “what”, “why”, “who”, “when”, “where” and “how” of creating FNI. Second, it analyses the spread features of FNI and the factors that affect the spread of FNI. Third, it investigates the consequences of FNI in the political, social, scientific, health, business, media and journalism fields.

Originality/value

The extant reviews on FNI mainly focus on the interventions or detection of FNI, and a few analyse the antecedents and consequences of FNI in specific fields. This study helps readers to synthetically understand the antecedents and consequences of FNI in all fields. This study is among the first to summarise the conceptual framework for FNI research, including the basic relevant theoretical foundations, research methodologies and public datasets.

• Literature review
• Input–process–output framework
• Antecedents and consequences

Acknowledgements

The authors thank the editor and three anonymous reviewers for their constructive comments, criticisms and help in improving the paper. Yuanyuan Wu was supported in part by Joint PhD programmes (PolyU-HIT) leading to Dual Awards. Pengkun Wu was supported in part by National Natural Science Foundation of China (62001314), MOE (Ministry of Education in China) Project of Humanity and Social Science (20YJC630159), Foundational Research Funds of the Central Universities (YJ202008, SXYPY202106), From 0 to 1 Project of Sichuan University (2021CXC19), and International Visiting Program for Excellent Young Scholars of Sichuan University. Chong Wu was supported in part by National Natural Science Foundation of China (72131005).

Wu, Y. , Ngai, E.W.T. , Wu, P. and Wu, C. (2022), "Fake news on the internet: a literature review, synthesis and directions for future research", Internet Research , Vol. 32 No. 5, pp. 1662-1699. https://doi.org/10.1108/INTR-05-2021-0294

Emerald Publishing Limited

Copyright © 2022, Emerald Publishing Limited

Related articles

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

A Systematic Literature Review on Internet of Things in Education: Benefits and Challenges

• November 2019
• Journal of Computer Assisted Learning 36(2):115-127
• 36(2):115-127

• Pennsylvania State University

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations
• Andreza Ferreira

• Rafael De Amorim Silva

• Purniadi Putra
• Dwi Septiwiharti
• Abdul Hafiz

• Euclides Carlos Pinto Neto

• Mohammed Eishorafa

• İhsan Pehlivan
• Cüneyt Bayılmış
• Xiaorui Wang
• Zhiyuan Teng
• Xingxing Zhang
• Gilang Kartika Hanum
• Ruly Nadian Sari
• Lismaryanti Lismaryanti
• Mohammad Yusup
• Aidil Ahmad
• Mardhiah Masril

• Nasser N. Albogami

• Abinaya Inbamani

• Viswak Sena.T

• Jeffrey Voas
• Qingyun Dai
• Meilin Wang
• Minglong Xiao
• Toshiki Ueda
• Yoshikazu Ikeda
• Zhongyun Jiang
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Advertisement

Bridging Digital Divides: a Literature Review and Research Agenda for Information Systems Research

• Published: 06 January 2021
• Volume 25 , pages 955–969, ( 2023 )

Cite this article

• Polyxeni Vassilakopoulou   ORCID: orcid.org/0000-0002-5947-4070 1 &
• Eli Hustad   ORCID: orcid.org/0000-0002-1150-1850 1  

49k Accesses

107 Citations

12 Altmetric

Explore all metrics

Extant literature has increased our understanding of the multifaceted nature of the digital divide, showing that it entails more than access to information and communication resources. Research indicates that digital inequality mirrors to a significant extent offline inequality related to socioeconomic resources. Bridging digital divides is critical for sustainable digitalized societies. Ιn this paper, we present a literature review of Information Systems research on the digital divide within settings with advanced technological infrastructures and economies over the last decade (2010–2020). The review results are organized in a concept matrix mapping contributing factors and measures for crossing the divides. Building on the results, we elaborate a research agenda that proposes [1] extending established models of digital inequalities with new variables and use of theory, [2] critically examining the effects of digital divide interventions, and [3] better linking digital divide research with research on sustainability.

Similar content being viewed by others

An Upward Spiral Model: Bridging and Deepening Digital Divide

Understanding Digital Inequalities in the Global South

Revitalizing the Quantitative Understanding of the Digital Divide: An Uptake on the Digital Divide Indicators

Explore related subjects.

• Artificial Intelligence

Avoid common mistakes on your manuscript.

1 Introduction

Digital inequalities have emerged as a growing concern in modern societies. These inequalities relate to disparities in access, actual use and use efficacy of digital resources. Digital resources including transformative technologies, such as business analytics, big data and artificial intelligence are key for the transition of societies towards sustainability (Pappas et al. 2018 ; United Nations 2018 ). Reducing digital inequalities is critical for sustainable digitalized societies. At a high level, all types of digital inequalities are encompassed in the term digital divide . One of the first uses of the term is traced back in a US government report published in 1999 referring to the divide between those with access to new technologies and those without (NTIA 1999 ). The term was soon broadened to signify the “gap between those who can effectively use new information and communication tools, such as the Internet, and those who cannot” (Gunkel 2003 ). Overall, the term digital divide includes digital inequalities between individuals, households, businesses or geographic areas (Pick and Sarkar 2016 ; OECD 2001 ). The conceptual broadness of the term aims to capture a multifaceted economic and civil rights issue in an era of continuous efforts to digitalize society. The ongoing digitalization poses a challenge for individuals who are not fully capable of using digital resources and may feel partially excluded or completely left out of the society.

Extant research has contributed insights on the different aspects of the digital divide phenomenon. In the past, the digital divide literature was mostly driven by policy-oriented reports that focused on access. Nevertheless, scientific research expanded to digital inequalities beyond access. Researchers foregrounded digital inequalities related to knowledge, economic and social resources, attributes of technology such as performance and reliability, and utility realization (DiMaggio et al. 2004 ; Van Dijk 2006 ; Van Deursen and Helsper 2015 ). In technologically and economically advanced settings, digital divides seem to be closing in terms of access, but inequalities that affect people’s ability to make good use of digital resources persist (Lameijer et al. 2017 ; Hsieh et al. 2011 ; Bucea et al. 2020 ). As digitalization becomes increasingly pervasive in work and everyday life, concerns are rising about continuing inequalities within societies that are at the digital forefront. At the same time, in low-resource settings there are still significant access issues. For instance, in the least developed countries (as defined by the United Nations) only 19 per cent of individuals had online access in 2019 while in developed countries, close to 87 per cent of individuals access the internet (Int.Telecom.Union 2019 ). Beyond big differences across settings in terms of access, low-resource settings are tormented by particular political, economic and social conditions inflicting digital divides (Venkatesh et al. 2014 ; Srivastava and Shainesh 2015 ; Luo and Chea 2018 ). Overall, prior research has shown that the modalities of digital inequalities are context-specific and it is important to be explicit about the context when researching the digital divide (Barzilai-Nahon 2006 ). This work is focused on digital divide research within settings with advanced technological infrastructures and economies.

The digital divide is an exemplary sociotechnical phenomenon and has attracted the interest of Information System (IS) researchers. IS research examines more than technologies or social phenomena, or even the two side by side; it investigates emergent sociotechnical phenomena (Lee 2001 ). Hence, IS researchers are well-positioned to study the digital divide phenomenon and have been producing a significant volume of related research. Nevertheless, no systematic review of the IS body of literature on the digital divide exists. Our study identifies, analyses, and integrates a critical mass of recent IS research on the digital divide focused on settings where the technological infrastructures and economies are advanced. To ensure a robust result, we performed a systematic literature review (Kitchenham 2004 ) guided by the following question: What are the key findings identified in extant IS research related to the digital divide in contemporary technologically and economically advanced settings?

Our contribution is threefold. First, we identify recurring digital divide factors for population groups threatened by digital inequalities. The factors identified indicate that digital inequalities frequently mirror offline inequalities (for instance, in terms of socioeconomic resources, knowledge and physical abilities). Second, we present measures proposed in the literature and organize them in three key intervention domains that can contribute to closing the gap (related to policies, training initiatives and tailored design). Finally, as a third contribution, we identify areas for future research providing a research agenda.

The remainder of the paper is organized as follows. First, we present the method used for selecting and analyzing the articles for this review. Then, we offer a synthesis of our findings related to digital divide factors and related measures and present them in a concise concept matrix. We continue by discussing the implications for further research and we end with overall concluding remarks.

The literature review is conceptual providing a synthesis of prior research and identifying areas for future research (Ortiz de Guinea and Paré 2017 ; Schryen et al. 2015 ). It includes research published during the last decade (2010–2020). The approach followed is based on the three-step structured literature review process proposed by Kitchenham ( 2004 ). Specifically, the three-step process includes: (a) planning the review, where a detailed protocol containing specific search terms and inclusion/exclusion criteria is developed, (b) conducting the review, where the identification, selection, quality appraisal, examination and synthesis of prior published research is performed and (c) reporting the review, where the write-up is prepared. We used these steps as our methodological framework. In addition, we utilized principles suggested by Webster and Watson ( 2002 ) for sorting the articles included in the review. Following these principles, we identified key concepts and created a concept-centric matrix that provides an overview of the literature reviewed.

To identify articles to be reviewed, we searched for “Digital” and “Divide” in the abstract, title or keywords within published Information Systems research. Inclusion and exclusion criteria were established to reduce selection bias, guarantee the quality of the papers selected and increase the review validity. Peer-reviewed, empirical papers, written in English were included. Conceptual papers that lack empirical evidence and papers focusing on the digital divide in developing countries were excluded. Figure 1 provides an overview of the selection process. To ensure a good coverage of Information Systems research we searched within the eight top journals in the field i.e. the basket of eight (AIS 2019 ). The journals included in the basket are: European Journal of Information Systems, Information Systems Journal, Information Systems Research, Journal of AIS, Journal of Information Technology, Journal of MIS, Journal of Strategic Information Systems and MIS Quarterly. Additionally, we searched within the journal Communications of the Association for Information Systems (CAIS) which has a key role within the IS research community communicating swiftly novel, original research. We also included in our search the journal Information Technology (IT) & People because it focuses on IS research that explores the interplay between technology individuals and society and the journal Information Systems Frontiers because it covers behavioural perspectives on IS research. Both journals are high quality IS outlets especially relevant for research on the digital divide. Furthermore, we included in our search the conferences of the Association of Information Systems (ICIS, ECIS; AMCIS; PACIS) and the Hawaiian International Conference on System Sciences (HICSS). We utilized Scopus as our search engine.

The literature selection process

In Scopus, we searched for papers from the selected journals and conferences excluding books, book chapters, commentaries, letters and short surveys. For the journal article search, the ISSNs of the selected journals were used for filtering the search results in Scopus. In total, 45 journal papers were identified. For the conference article search, the conference names were used in Scopus and 91 conference papers were identified. Overall, the search yielded 136 unique articles in total. The next step was to read the titles and abstracts of the articles identified checking their relevance to the research question. For this step, the exclusion criteria were used. Specifically, we excluded papers that only casually mentioned the digital divide but had a different focus, literature reviews and conceptual papers and papers focused on developing countries. After this step, 79 articles were shortlisted. The full text of each of the shortlisted articles was assessed for relevance applying the inclusion-exclusion criteria to the full content. Additionally, the quality of the research reported was assessed. For the quality assessment, each article´s method description was first checked. At this stage, conference papers reporting early stages of ongoing research were removed. In several cases of conference papers that were removed, we found that more mature and extensive results from the same studies were reported in journal articles that were already included in our shortlist and were published after the conference papers. After this step, a final corpus of 33 articles was defined (Table 1 ). A detailed overview of the reviewed articles is included in an electronic supplementary file that can be accessed in the journal´s web site (see Online Resource 1 ).

After selecting the papers, we analyzed their content. We started with extracting meta-data of the papers such as type of study, year of study, study context, research method and theoretical framework applied. In addition, we identified the study subjects for each paper distinguishing between papers that engage with the general population, or specific groups of people including the elderly and marginalized population groups (e.g. refugees, migrants). We continued with an intra-analysis of the content of the papers by looking for core themes in each paper. The themes that were identified for each paper were registered, and as a next step, we performed an inter-analysis and comparison across papers. Based on the comparison, recurring themes and patterns across the papers were discovered and further categorized. The outcomes of the papers´analysis are presented in the " Results " section that follows.

This section presents the key findings from the literature reviewed. First, we present the theoretical premises and the methodological approaches of extant publications on the Digital Divide within IS research and their evolution from 2010 to 2020. Table 2 provides an overview of the theories and concepts, methods and data sources in the literature reviewed. Then, recurring digital divide factors are presented for population segments that are particularly digitally challenged (the elderly and marginalized population groups) and also, for the general population. Finally, measures for addressing the digital divide are presented and organized in three key intervention domains (policy measures, education/training and design tailoring). The section also includes a concept matrix which provides an overview of digital divide factors and related measures identified in the literature reviewed (Table 3 ).

3.1 Trends, Methods and Theoretical Frames in IS Research on the Digital Divide

The work of Information Systems´ researchers on the digital divide has been influenced by policy-oriented reports that tend to be based on macro-level analyses. This influence is clear in the first half of the 2010–2020 period while in the second half, research extends towards a more complex and contextualized picture of digital divides. Newer papers tend to ask a wider range of questions related to access and use of information technologies and investigate a greater variety of factors. For instance, skill related factors are explored in about half of both earlier and later studies, but, newer studies tend to additionally explore motivation and personality aspects (about half of the newer studies include such aspects). Interestingly, several of the newer papers only focus on technology use. In these papers, researchers explore the second order digital divide and the extent of inclusion or involuntary exclusion of those that already have access to technologies. Furthermore, most earlier papers tend to investigate the general population while the majority of newer studies focus on specific population groups.

Overall, most of the studies employ quantitative research methods utilizing well-established survey instruments adapted for studying digital inequalities for certain groups (e.g. older adults) or re-using existing data sets from organizations like the International Telecommunication Union, the World Bank and the United Nations. A few studies use a mixed-method approach combining interviews with survey data, while the rest employ qualitative approaches. Well-known technology acceptance models such as TAM (Technology Acceptance Model), UTAUT (Unified Theory of Acceptance and Use of Technology) and MATH (Model of Adoption of Technology in Households) and theories on motivation and human behavior have been used to explore the digital divide. Typical variables included in the investigations are self-efficacy, performance and effort expectancy. Furthermore, social cognitive theories, social support theories and social capital conceptualizations have been used while some of the papers utilize selectively digital divide conceptualizations combined with constructs from social, sociotechnical or economic research.

3.2 Factors Contributing to the Digital Divide

The digital divide is often characterized as a digital divide cascade which is nuanced into different types of inequalities including unequal capabilities, engagement, and use outcomes in addition to inequalities of access and use. This points to the importance of identifying and aiming to remedy inequalities in what people are actually able to do and achieve with digital technologies (Burtch and Chan 2019 ; Díaz Andrade and Doolin 2016 ). In settings with advanced infrastructures and economy, physical access is not a key source of digital inequalities and IS studies that examine issues of unequal access show that access gaps are closing with the exception of marginalized population groups. Nevertheless, there is still a stark difference between access (first-order divide) and actual use (second-order divide) (Bucea et al. 2020 ). The latter relates to differences in digital skills, autonomy, social support and the aims of digital technology use (Rockmann et al. 2018 ). Going beyond socioeconomic demographics, additional personal contributing factors have been identified in the literature related to: (a) motivation, (b) personality traits (e.g. openness, extraversion, conscientiousness), (c) digital skills. Many of the studies reviewed focus on the elderly who are also referred to as “digital immigrants” (as opposed to digital natives that have been interacting with digital technology since childhood). Additionally, several studies focus on marginalized population groups. In the paragraphs that follow, we present research findings organizing them according to the different groups studied.

Elderly Population

Although digital technologies have been around for several decades, some of the elderly members of society have difficulties familiarizing with and adopting digital tools and services. Nevertheless, although a decade ago age-related underutilization of IT was significant (Niehaves and Plattfaut 2010 ), over the years, information and communication technologies (ICTs) have been gradually better integrated in the lives of elderly adults. A recent study on the digital divide related to mobile phone use among old adults in UK found that more than 70% have adopted smartphones (Choudrie et al. 2018 ). Specifically, research findings indicate that older adults frequently use internet-related smartphone features such as emailing and browsing although only very few use smartphones to access public services such as the National Health Service. One potential reason for the limited use of specialized web-based services among the elderly despite the wide adoption of smartphones, is that their former workplaces may have been characterized by low IT intensity causing a lower exploratory IT behavior when seniors are retiring (Rockmann et al. 2018 ). Niehaves and Plattfaut ( 2014 ) used the unified theory of acceptance and use of technology (UTAUT) and the model of adoption of technology in households (MATH) to explain internet acceptance and usage by the elderly. Performance expectancy was found to be the main use driver among senior citizens. These models were able to predict how the elderly could be encouraged to learn to use digital technologies.

When asked, the elderly themselves identified several key impeding factors for their digital involvement: fear and anxiety of using digital technology and services, negative attitude, a sense of feeling too old for learning, lack of knowledge, difficulties understanding digital terminology (Holgersson and Söderström 2019 ). Family support is key for developing mobile internet skill literacy and mobile internet information literacy among older adults (Xiong and Zuo 2019 ). Seniors become better positioned to take advantage of digital resources when they have cognitive and emotional support. Cognitive support from family facilitates learning and digital skills´ development, and also, the development of skills for judging, analyzing and selecting information (Xiong and Zuo 2019 ). Emotional support based on patience, praise, encouragement and comfort can help the elderly avoid computer anxiety and stress (Xiong and Zuo 2019 ). Emotional support is important because unwillingness to adopt advanced digital services by the elderly was found to stem from mistrust, high-risk perceptions, and privacy concerns (Fox and Connolly 2018 ).

Overall, older people are a heterogeneous group, and it is important not to overlook their differences in digital skills and digital practice. Klier and colleagues conducted a survey on older unemployed individulas in Germany and showed that they can be grouped into four different types of digital media users ranging from very active users (digital contributors) to sceptics with limited or no use (digital sceptics) characterised by their negative attitude towards digital media (Klier et al. 2020 ). Digitalization efforts should take into account “the various shades of grey in older adults’ ability to draw on IT-based innovations” (Lameijer et al. 2017 , p. 6).

Marginalized Population Groups

Language barriers as for instance, in the case of refugees and immigrants, and practical resource limitations as in the case of distressed urban areas and remote rural areas can cause social exclusion and hinder the process of digital technologies´ assimilation throughout society. Several researchers have studied specifically issues related to the digital divide within marginalized population groups. Alam and Imram ( 2015 ) found in their research that although refugees and immigrants in the US are motivated to learn about new technology, many are not able to do so because of unaffordable cost, language barriers and lack of skills. Refugees and immigrants realize that technology is helpful for finding new jobs or facilitating social engagement. Digital technologies are of particular value to refugees for multiple reasons: to participate in an information society; to communicate effectively; to understand a new society; to be socially connected; to express their cultural identities (Díaz Andrade and Doolin 2016 ). A study on mobile communications by labor migrants (Aricat 2015 ) showed that mobile phones may also facilitate the development of ghettos and the lack of integration in the new countries by easing communications between the migrants and their home countries. The study identified a visible divide in the framing of the prospects and potentialities of mobile phones related to acculturation.

Enhancing the relationship between citizens and government through digital services requires reaching out to individuals and communities on the unfortunate side of the divide. Digital technology access and use in the context of e-government services were explored within one of the most distressed cities in the US (Sipior et al. 2011 ). This study showed that socioeconomic characteristics (educational level and household income) have significant impact on access barriers, but they also found that employment plays a critical role and is associated both with perceived access barriers and with perceived ease of use. A study conducted among governmental participants representing rural communities in Australia suggests that rural digital exclusion can result from three intertwined layers: availability (elements of infrastructure and connectivity), adoption, and digital engagement (Park et al. 2015 ). Among these layers, availability is probably not as important as one could expect. Similarly, one large household study conducted across the US found that the availability of Internet Supply Providers (ISP) had little impact on Internet adoption, and that Internet adoption can almost exclusively be attached to differences in household attributes and not to ISP availability (Ma and Huang 2015 ).

As access gaps are closing in settings with advanced infrastructures and economy, those who do not have access are easily overlooked (Davis et al. 2020 ). Nevertheless, the first-level digital divide still requires attention for marginalized population groups. Furthermore, socioeconomic factors that were found to affect uptake more than two decades ago (for instance, education level and income) are still relevant in today’s context for particular segments of our societies. Contrary to traditional views, the availability of digital solutions does not always facilitate the resolution of long-standing problems for those that are less well-off in our societies (for instance, immigrants or financially troubled individuals). What people are actually able to do and achieve with digital technologies relates to their greater positioning in society (Burtch and Chan 2019 ) and affects their potential for improvement. As digital technologies are becoming indispensable for participating in the economy and engaging in society, sustained digital divides amplify marginalization.

General Population

A study by Pick and colleagues ( 2018 ) showed the positive influence of managerial/science/arts occupations, innovation, and social capital on the use of digital technologies (Pick et al. 2018 ). Nevertheless, unreasonably high expectations are found to have a negative impact on ICT acceptance (Ebermann et al. 2016 ). Findings from a study conducted within White and Hispanic-owned SMEs in the US (Middleton and Chambers 2010 ) indicate some level of inequality related to ethnicity and age (younger white SME owners being better positioned). Davis and colleagues (Davis et al.  2020 ) analyzed the influence of income, income distribution, education levels, and ethnicity on levels of access to Internet in the US. The findings show that low levels of education and levels of income below the poverty line still tend to lead to higher proportion of people with no Internet access (Davis et al. 2020 ). Even when individuals do have equal access to digital technologies, differences in skills can lead to digital inequalities (Burtch and Chan 2019 ). Taking a differentiated view on skills is needed to understand technology use and no-use (Reinartz et al. 2018 ). Physical skills matter; users with disabilities can be digitally disadvantaged and despite the benefits promised by specialized assistive technologies their adoption rate falls short of expectations (Pethig and Kroenung 2019 ).

Some groups may be challenged because they are too far embedded in older systems, which makes it difficult for them to adopt newer ICTs (Abdelfattah 2012 ). Social capital can trigger ICT awareness changing individual dispositions, thus converting social capital into cultural capital (Reinartz et al. 2018 ). An interesting study on crowdfunding showed that the benefits of medical crowdfunding accrue systematically less to racial minorities and less educated population segments (Burtch and Chan 2019 ). One of the reasons for this is the communication-rich nature of the context: less educated persons are not always capable of producing polished, persuasive pitches to solicit funds. Furthermore, digital inequality manifests on the efficacy of using crowdfunding platforms, due to a lack of critical mass in the number of potential transaction partners (donors). The results show the importance of looking beyond access or connectivity to investigate efficacy (in this case, expressed as success in fundraising), and how it associates with different population segments (Burtch and Chan 2019 ).

At the country level, a number of studies examined socio-economic influences on access and use of particular forms of technologies as for instance, personal computers and broadband internet (Zhao et al. 2014 ; Pick and Azari 2011 ; Dewan et al. 2010 ). A world-wide study found complementarities in the diffusion of PCs and the Internet leading to narrower digital divides (Dewan et al. 2010 ). These findings challenge the dominant understanding of characteristics such as country wealth, education levels and telecommunications infrastructure leading to the widening of the digital divide. Country-level studies are based on the analysis of data from census surveys, national statistics, and datasets from organizations like UNDP and ITO. The use of such datasets is helpful for performing comparisons across countries but due to the generic nature of data the purpose of digital technology use has been scarcely examined in country-level studies. This may be attributed to the fact that comparable data on specific online activities are not easy to collect across countries (Zhao et al. 2014 ). A study conducted by Bucea and colleagues ( 2020 ), is an exception to this. The study assessed specifically the use of e-Services and Social Networks within the 28 member-states of the European Union analyzing four socio-demographic factors (age, education, gender, and income). The findings showed that for e-Services, disparities relate mostly to education while for Social Networks age is the most important factor (Bucea et al. 2020 ). Overall, country level studies are important for assessing disparities across countries and can lead to the identification of factors reinforcing inequalities. At the same time, macro studies can not bring insights about digital inequalities across different population segments within countries.

3.3 Overcoming Digital Divides

Policy-making is considered instrumental for closing the digital gap and a mix of policy measures has been suggested in prior research. In general, policy initiatives can include subsidies targeting specific digitally disadvantaged segments as for instance rural populations (Talukdar and Gauri 2011 ). For instance, governments can apply strong intervention policies to provide equitable ICT access also in rural areas (Park et al. 2015 ). Furthermore, digital divides may be addressed at scale by crafting policies to equip underprivileged groups with better communication skills (reading, writing, and software use) enabling meaningful engagement with digital platforms (Burtch and Chan 2019 ). Government policy makers can collaborate with schools to support students from low-income households through the provision of home computers aiming to reduce the effect of socio-economic inequalities among students (Wei et al. 2011 ). Policies raising the priority of IT, protecting property rights, and enhancing freedom of the press and openness, can help to stimulate educational advances, labor-force participation and income growth, all of which contribute to advancing technology use (Pick and Azari 2011 ). Policy measures should allow room for local adaptations, as contextual and local elements seem to play a role for technology users and could influence policy success (Racherla and Mandviwalla 2013 ). Effective evaluation mechanisms make it easier to develop new policies addressing digital divides (Chang et al. 2012 ) helping policy-makers to refine initiatives targeting certain segments of society, such as elderly people and socio-economically disadvantaged groups (Hsieh et al. 2011 ).

Contemporary workplaces can help by taking greater responsibility for IT education of their employees even when they are close to retirement. Developing the digital skills of seniors while they are still employed is important for preventing digital exclusion after retirement (Rockmann et al. 2018 ). Overall, employment has a pivotal role in explaining citizen usage of e-government initiatives (Sipior et al. 2011 ). As an employee, an individual may have access to the Internet at the place of employment. Furthermore, employment demands may increase the confidence of an individual in performing new tasks. Thinking beyond workplaces, policies that leverage existing communities, social structures, and local actors can also help in reducing digital inequalities (Racherla and Mandviwalla 2013 ). Such policies can stimulate public/private partnerships with grassroots organizations that already have “hooks” in local communities. Moreover, long-term government policies could set a goal of encouraging growth in social capital within communities (Pick et al. 2018 ).

Proper training and education can help mitigate digital inequalities (Van Dijk 2012 ). For instance, platform operators can provide coaching services for underprivileged populations (Burtch and Chan 2019 ). Furthermore, information campaigns also have a significant role to play, digital divides may be narrowed if vendors engage in trust-building campaigns (Fox and Connolly 2018 ). Integrating digital education into curricula can also contribute to reducing digital inequalities (Reinartz et al. 2018 ), and education campaigns can stimulate the adoption and usage of ICTs bridging rural-urban digital gaps. Rural communities typically lag in digital skills, and digital literacy training programs can improve digital engagement in rural communities. Digital literacy programs targeting senior citizens can help them develop the necessary skills and abilities to use digital mobile devices so that they could be part of the Digital Society (Carvalho et al. 2018 ; Fox and Connolly 2018 ; Klier et al. 2020 ). Educational efforts for the elderly must be practically oriented in order to show directly what is to be gained by becoming more digital (Holgersson and Söderström 2019 ). In addition, social networks, friends and family are important for supporting the training of disadvantaged people in technologies; family emotional and cognitive support can increase the elderly’s digital capabilities, reduce computer anxiety and increase trust and motivation for learning (Xiong and Zuo 2019 ).

The design and development of ICT solutions should take into account individual differences for creating proper stimuli to different user groups. For instance, the use of governmental e-services can be improved by making them more engaging, interactive, and personal to address a country’s or region’s cultural norms (Zhao et al. 2014 ). This makes the role of appropriate design for overcoming the digital divide a center of attention. Lameijer et al. ( 2017 ) propose that design-related issues should be considered and evaluated to better understand technology adoption patterns among elderly. Also, the study by Klier and colleagues showed that there is a potential to shift older individuals towards a more active engagement with digital media by ensuring ease of use in the design of digital services (Klier et al. 2020 ). Furthermore, the needs of groups with disabilities ought to be taken into account when designing information systems for the general public (Pethig and Kroenung 2019 ). It is important to integrate assistive functionalities in general IS to emphasize authentic inclusiveness. Overall, research points to the importance of functionalities that suit the needs of specific user groups to stimulate the use of digital technologies.

4 Crossing Digital Divides: a Research Agenda

The evolution of IS research on the digital divide during the last decade shows the richness of this research area. As digitalization becomes pervasive in our societies, digital inequalities emerge in different contexts and communities renewing the interest on digital divide research. In recent years, researchers have been shifting away from macro-level studies and are re-orienting towards developing nuanced and contextualized insights about digital inequalities. The analysis of published research allows the identification of gaps and opportunities for further research. Furthermore, there are specific research directions proposed in several of the reviewed papers. The synthesis of suggestions from the papers reviewed with the results of our analysis led to the identification of three research avenues that bring exciting opportunities for researchers to engage with topics that are highly relevant with our digitalization era. Specifically, we suggest a research agenda that proposes: [1] extending established digital divide models with new variables and use of theory, [2] examining the effects of interventions, and [3] addressing societal challenges and especially sustainability goals through the lens of digital divide. Social inclusion and digital equality are crucial for a sustainable digitalized society.

4.1 Avenue I: Extending Established Digital Divide Models and Use of Theory

Extant research shows that physical access divides are being reduced in technologically and economically advanced societies but, inequalities in use persist (Hsieh et al. 2011 ; Lameijer et al. 2017 ). These use inequalities are found to be related to socioeconomic characteristics and also, personality traits, motivation and digital skills. A better understanding of the complex phenomenon of digital divide is needed combining multiple aspects to form comprehensive models (Choudrie et al. 2018 ) and further explore the concept itself to get more explanatory power (Lameijer et al. 2017 ). The emphasis, to date, has been on describing the digital divide by identifying gaps between actual technology access and use against an ideal situation. Work should be undertaken to investigate different national, social and cultural settings (Niehaves and Plattfaut 2010 ) across geographical contexts (Niehaves and Plattfaut 2014 ) and the influence of institutional and environmental factors on individuals’ ability and motivation to access and use technology (Racherla and Mandviwalla 2013 ). Furthermore, researchers may explore the values and interests of those abstraining from the use of digital resources and the implications of the overemphasis to digital inclusion (Díaz Andrade and Techatassanasoontorn 2020 ).

Further research is also needed to extend established models with new variables. Future investigations may add variables related to social theories (Abdelfattah et al. 2010 ; Hsieh et al. 2011 ; Niehaves and Plattfaut 2014 ), personal traits models (Ebermann et al. 2016 ), and capital theory (Hsieh et al. 2011 ; Reinartz et al. 2018 ). Additionally, future research should consider testing psychological variables (Niehaves and Plattfaut 2010 ) and additional socio-economical aspects (Hsieh et al. 2011 ; Reisdorf and Rikard 2018 ) including support from friends and family (Xiong and Zuo 2019 ; Holgersson and Söderström 2019 ) to develop a more fine-grained understanding of the association between the digital divide phenomenon and contributing variables (Hsieh et al. 2011 ; Niehaves and Plattfaut 2014 ; Fox and Connolly 2018 ). Qualitative research is important for revealing factors that influence inequalities and can become the basis for model building and testing using quantitative data.

Interestingly, fully developed theoretical frameworks that have been extensively used in other streams of exploratory information systems research related to the introduction and use of ICTs were not present in the papers reviewed. For instance, Activity theory and Institutional theory can be used as lenses for understanding and analyzing the digital divide phenomenon. Activity theory (Allen et al. 2011 ; Engeström 1999 ) can help in developing a nuanced understanding of the relationship between ICT artifacts and purposeful individuals taking into account the environment, culture, motivations, and complexity of real-life settings. Institutional theory (Jepperson 1991 ; Scott 2005 ) can contribute to developing insights related to societal structures, norms and routines shifting attention to units of analysis that cannot be reduced to individuals’ attributes or motives. Overall, we observed that digital divide research could benefit from better leveraging theory to extend established digital divide models.

4.2 Avenue II: Examining the Effects of Interventions to Cross the Digital Divide

Measures for crossing digital divides include policy interventions, training and design. Information Systems research can be especially relevant by developing design knowledge for the development and deployment of digital technology artifacts in different settings. Although several measures are proposed in the literature, further work is required to research the effect of interventions to avoid the exclusion of citizens from the digital realm addressing inequalities (Alam and Imran 2015 ; Reisdorf and Rikard 2018 ; Reinartz et al. 2018 ). In particular, appropriate design approaches for digital technologies should be investigated and tested to avoid involuntary exclusion of marginalized groups, elderly people or any other group of individuals affected by digital inequalities (Rockmann et al. 2018 ; Lameijer et al. 2017 ; Alam and Imran 2015 ; Fox and Connolly 2018 ). Additionally, comparative research can be undertaken investigating the effects and attractiveness of different design solutions in different cultural settings (Pethig and Kroenung 2019 ). Overall, although many studies include insights related to measures for bridging digital divides, there is a clear need for studies with a longitudinal research design to investigate the impact of measures over time. Interestingly, little research has been performed up to now on the potentially negative unexpected effects of measures for bridging digital divides (Díaz Andrade and Techatassanasoontorn 2020 ). This is certainly an area that needs to be further developed. The use of technologies might lead to advantages or disadvantages, which are unevenly distributed in society. Focusing only on benefits, researchers miss the opportunity to connect to emerging literature on the dark side of Internet and unexpected outcomes of digitalization including privacy risks. Scholars of information systems can develop novel avenues of critical thinking on the effects of interventions to cross the digital divide.

4.3 Avenue III: Linking Digital Divide Research With Research on Sustainability

There were no studies in our literature review that focused specifically on sustainability topics, and future research should pay attention to this gap. The United Nations´ sustainability goals focus on reducing inequality within and among countries to avoid biased economic development, social exclusion, and environmentally untenable practices. Important dimensions of sustainable development are human rights and social inclusion, shared responsibilities and opportunities (United Nations 2020 ). An essential part of social inclusion in our societies is e-inclusion (Pentzaropoulos and Tsiougou 2014 ). At the same time, it is important to research the risks and ethical implications of depriving individuals from offline choices (Díaz Andrade and Techatassanasoontorn 2020 ). Furthermore, we need to support sustainability in rural areas reducing the urban - rural digital divide. Sustainability researchers have identified the issue pointing to the vulnerabilities of rural communities that are in particular need of bridging inequalities (Onitsuka 2019 ). Future empirical studies on the digital divide should therefore pay attention to sustainability topics in terms of social exclusion and digital inequality to better understand underlying factors and potential remedies.

The covid-19 pandemic made digital inequalities even more evident. In periods of social distancing to minimize infection risks, individuals sustain their connections with colleagues, friends, and family through online connections. Furthermore, people need digital skills to keep updated on crucial information and to continue working when possible using home offices and digital connections. In addition, recent crisis response experiences have shown that switching to digital education may lead to exclusion of the few that cannot afford physical digital tools (Desrosiers 2020 ), or do not have access to sustainable infrastructures and ICT access. This crisis has shown that digital divides can become a great challenge aggravating inequalities experienced by marginalized communities such as urban poor and under-resourced businesses. Digital inequalities are a major factor of health-related and socio-economical vulnerability (Beaunoyer et al. 2020 ).

The role of Information Systems researchers is critical for the development of digital capital contributing to sustainable development. Digital capital refers to the resources that can be utilized by communities including digital technology ecosystems and related digital literacy and skills. General policy measures related to stimulating regional economic growth, strengthening tertiary education, or discouraging early leaving from education can be developed by scientists in other domains. However, thinking about inclusive configurations of digital infrastructures and ecosystems and developing related design principles entails specialized knowledge from the Information Systems domain. Furthermore, Information Systems researchers can provide insights about the development of capabilities required for leveraging digital resources such as digital infrastructures (Hustad and Olsen 2020 ; Grisot and Vassilakopoulou 2017 ), big data and business analytics (Mikalef et al. 2020 ). Innovative approaches for leveraging digital resources will be pivotal for addressing grand challenges related to poverty, healthcare and climate change. Information Systems researchers can contribute insights for bridging digital divides to promote an agenda towards a sustainable future.

5 Conclusions

The present work takes stock of Information Systems research on the digital divide by synthesizing insights from publications in the 2010–2020 period. The review process was performed with rigor while selecting and critically assessing earlier research. Nevertheless, this work is not without limitations. We have confined the literature search within one specific discipline (Information Systems research). This limits the breadth of the review but facilitates comprehensiveness and depth in the development of insights about the body of literature analyzed. Furthermore, focusing on Information Systems research facilitates the development of a research agenda that is relevant to the target discipline through the identification of gaps and extrapolations from previous work.

The review showed that within digital divide research, the attention of Information Systems research has gradually shifted from access to use and now needs to shift further towards better understanding use outcomes. Digital inequalities are a serious threat to civil society in an era where societies are rapidly going digital. For instance, daily activities such as paying bills, filling in application forms, filing tax returns, are all expected to be carried out electronically. There are high expectations for active citizens´ role based on online services (Axelsson et al. 2013 ; Vassilakopoulou et al. 2016 ); hence, we need to be concerned of digital inequalities ensuring fairness and inclusiveness. Furthermore, digital resources such as big data and business analytics are key enablers of sustainable value creation within societies (Pappas et al. 2018 ; Mikalef et al. 2020 ). Bridging digital divides is critical for sustainable digitalized societies. The findings of this literature review can provide a foundation for further research and a basis for researchers to orient themselves and position their own work.

Abdelfattah, B. M. (2012). Individual-multinational study of internet use: the digital divide explained by displacement hypothesis and knowledge-gap hypothesis. In  AMCIS 2012 Proceedings . 24. https://aisel.aisnet.org/amcis2012/proceedings/AdoptionDiffusionIT/24 .

Abdelfattah, B. M., Bagchi, K., Udo, G., & Kirs, P. (2010). Understanding the internet digital divide: an exploratory multi-nation individual-level analysis. In  AMCIS 2010 Proceedings . 542. https://aisel.aisnet.org/amcis2010/542 .

AIS (2019). Association for information systems. Senior scholars’ basket of journals . https://aisnet.org/page/SeniorScholarBasket . Accessed 10 Jan 2019.

Alam, K., & Imran, S. (2015). The digital divide and social inclusion among refugee migrants: A case in regional Australia. Information Technology & People, 28 (2), 344–365.

Article   Google Scholar  

Allen, D., Karanasios, S., & Slavova, M. (2011). Working with activity theory: Context, technology, and information behavior. Journal of the American Society for Information Science and Technology, 62 (4), 776–788.

Aricat, R. G. (2015). Is (the study of) mobile phones old wine in a new bottle? A polemic on communication-based acculturation research. Information Technology & People, 28 (4), 806–824.

Axelsson, K., Melin, U., & Lindgren, I. (2013). Public e-services for agency efficiency and citizen benefit—Findings from a stakeholder centered analysis. Government Information Quarterly, 30 (1), 10–22.

Barzilai-Nahon, K. (2006). Gaps and bits: Conceptualizing measurements for digital divide/s. The Information Society, 22 (5), 269–278.

Beaunoyer, E., Dupéré, S., & Guitton, M. J. (2020). COVID-19 and digital inequalities: Reciprocal impacts and mitigation strategies. Computers in Human Behavior, 111 , 106424.

Bucea, A. E., Cruz-Jesus, F., Oliveira, T., & Coelho, P. S. (2020). Assessing the role of age, education, gender and income on the digital divide: evidence for the European Union. Information Systems Frontiers . https://doi.org/10.1007/s10796-020-10012-9 .

Burtch, G., & Chan, J. (2019). Investigating the relationship between medical crowdfunding and personal bankruptcy in the United States: evidence of a digital divide. MIS Quarterly, 43 (1), 237–262.

Carvalho, C. V. d., Olivares, P. C., Roa, J. M., Wanka, A., & Kolland, F. (2018). Digital information access for ageing persons. In ICALT 2018 Proceedings  the 8th International Conference on Advanced Learning Technologies, IEEE, 345–347.

Chang, S.-I., Yen, D. C., Chang, I.-C., & Chou, J.-C. (2012). Study of the digital divide evaluation model for government agencies–a Taiwanese local government’s perspective. Information Systems Frontiers, 14 (3), 693–709.

Choudrie, J., Pheeraphuttranghkoon, S., & Davari, S. (2018). The digital divide and older adult population adoption, use and diffusion of mobile phones: a quantitative study. Information Systems Frontiers, 22 , 673–695. https://doi.org/10.1007/s10796-018-9875-2 .

Davis, J. G., Kuan, K. K., & Poon, S. (2020). Digital exclusion and divide in the United States: exploratory empirical analysis of contributing factors. In AMCIS 2020 Proceedings . 1. Fully Online Event. https://aisel.aisnet.org/amcis2020/social_inclusion/social_inclusion/ .

Desrosiers, M.-E. (2020). As universities move classes online, let’s not forget the digital divide, Policy Options Politiques . https://policyoptions.irpp.org/magazines/march-2020/as-universities-move-classes-online-lets-not-forget-the-digital-divide/ . Accessed 25 Mar 2020.

Dewan, S., Ganley, D., & Kraemer, K. L. (2010). Complementarities in the diffusion of personal computers and the Internet: Implications for the global digital divide. Information Systems Research, 21 (4), 925–940.

Díaz Andrade, A., & Doolin, B. (2016). Information and communication technology and the social inclusion of refugees. MIS Quarterly, 40 (2), 405–416.

Díaz Andrade, A., & Techatassanasoontorn, A. A. (2020). Digital enforcement: Rethinking the pursuit of a digitally-enabled society. Information Systems Journal, 12306 , 1–14.

Google Scholar  

DiMaggio, P., Hargittai, E., Celeste, C., & Shafer, S. (2004). Digital inequality: From unequal access to differentiated use. In Social inequality (pp. 355–400). New YorK: Russell Sage Foundation.

Ebermann, C., Piccinini, E., Brauer, B., Busse, S., & Kolbe, L. (2016). The impact of gamification-induced emotions on In-car IS adoption - the difference between digital natives and digital immigrants. In 49th Hawaii International Conference on System Sciences (HICSS 2016) Proceedings, IEEE, 1338–1347.

Engeström, Y. (1999). Activity theory and individual and social transformation. In Y. Engeström, R. Miettinen, & R. L. Punamäki (Eds.), Perspectives on activity theory (Vol. 19, pp. 19–37). Cambridge: Camebridge University Press.

Fox, G., & Connolly, R. (2018). Mobile health technology adoption across generations: Narrowing the digital divide. Information Systems Journal, 28 (6), 995–1019.

Grisot, M., & Vassilakopoulou, P. (2017). Re-infrastructuring for eHealth: Dealing with turns in infrastructure development. Computer Supported Cooperative Work (CSCW), 26 (1), 7–31. https://doi.org/10.1007/s10606-017-9264-2 .

Gunkel, D. J. (2003). Second thoughts: toward a critique of the digital divide. New Media & Society, 5 (4), 499–522.

Holgersson, J., & Söderström, E. (2019). Bridging the gap - Exploring elderly citizens’ perceptions of digital exclusion. In ECIS 2019 Proceedings.  https://aisel.aisnet.org/ecis2019_rp/28 .

Hsieh, J. J., Rai, A., & Keil, M. (2011). Addressing digital inequality for the socioeconomically disadvantaged through government initiatives: Forms of capital that affect ICT utilization. Information Systems Research, 22 (2), 233–253.

Hustad, E., & Olsen, D. H. (2020). Creating a sustainable digital infrastructure: the role of service-oriented architecture. Presented at the Centeris conference 2020, forthcoming in Procedia Computer Science , preprint available at: https://www.researchgate.net/publication/346989191_Creating_a_sustainable_digital_infrastructure_The_role_of_service-oriented_architecture .

Int.Telecom.Union (2019). Facts and figs. 2019: measuring digital development. https://www.itu.int/en/ITU-D/Statistics/Documents/facts/FactsFigures2019.pdf . Accessed 25 Apr 2020.

Jepperson, R. L. (1991). Institutions, institutional effects, and institutionalism. In W. W. Powell, & P. J. DiMaggio (Eds.), The new institutionalism in organizational analysis (pp. 143–163). Chicago: University of Chicago Press.

Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele University Technical Report, UK, TR/SE-0401,  1–26.

Klier, J., Klier, M., Schäfer-Siebert, K., & Sigler, I. (2020). #Jobless #Older #Digital – Digital media user of the older unemployed. In ECIS 2020 Proceedings . Fully Online Event. https://aisel.aisnet.org/ecis2020_rp/206 .

Lameijer, C. S., Mueller, B., & Hage, E. (2017). Towards rethinking the digital divide–recognizing shades of grey in older adults’ digital inclusion. In ICIS 2017 Proceedings . 11. http://aisel.aisnet.org/icis2017/General/Presentations/11 .

Lee, A. S. (2001). Editor’s comments: What are the best MIS programs in US business schools? MIS Quarterly, 25 (3), iii–vii.

Luo, M. M., & Chea, S. (2018). Internet village motoman project in rural Cambodia: bridging the digital divide. Information Technology & People, 21 (1), 2–20.

Ma, J., & Huang, Q. (2015). Does better Internet access lead to more adoption? A new empirical study using household relocation. Information Systems Frontiers, 17 (5), 1097–1110.

Middleton, K. L., & Chambers, V. (2010). Approaching digital equity: is wifi the new leveler? Information Technology & People, 23 (1), 4–22.

Mikalef, P., Pappas, I. O., Krogstie, J., & Pavlou, P. A. (2020). Big data and business analytics: A research agenda for realizing business value. Information & Management, 57 (1), 103237. https://doi.org/10.1016/j.im.2019.103237 .

NTIA. (1999). Falling through the net: Defining the digital divide. A report on the telecommunications and information technology gap in America. National Telecommunications and Information Administration .  https://www.ntia.doc.gov/legacy/ntiahome/fttn99/contents.html . Accessed 20 Oct 2019.

Niehaves, B., & Plattfaut, R. (2014). Internet adoption by the elderly: employing IS technology acceptance theories for understanding the age-related digital divide. European Journal of Information Systems, 23 (6), 708–726.

Niehaves, B., & Plattfaut, R. (2010). The age-divide in private internet usage: a quantitative study of technology acceptance. In  AMCIS 2010 Proceedings . 407. https://aisel.aisnet.org/amcis2010/407 .

OECD. (2001). Understanding the digital divide. OECD Digital Economy Papers ,  49 , OECD Publishing, Paris, France. https://doi.org/10.1787/236405667766 .

Onitsuka, K. (2019). How social media can foster social innovation in disadvantaged rural communities. Sustainability, 11 (2697), 1–24.

Ortiz de Guinea, A., & Paré, G. (2017). What literature review type should I conduct? 1. In The Routledge Companion to Management Information Systems (pp. 73–82). Abingdon: Routledge.

Pappas, I. O., Mikalef, P., Giannakos, M. N., Krogstie, J., & Lekakos, G. (2018). Big data and business analytics ecosystems: paving the way towards digital transformation and sustainable societies. Information Systems and eBusiness Management, 16 (3), 479–491.

Park, S., Freeman, J., Middleton, C., Allen, M., Eckermann, R., & Everson, R. (2015). The multi-layers of digital exclusion in rural Australia. In 48th Hawaii International Conference on System Sciences   (HICSS 2015) Proceedings, IEEE, 3631–3640.

Pentzaropoulos, G. C., & Tsiougou, D. (2014). E-inclusion policies for contemporary knowledge economies and societies: an examination of the main issues. Journal of Social Research & Policy, 5 (1), 77–89.

Pethig, F., & Kroenung, J. (2019). Specialized information systems for the digitally disadvantaged. Journal of the Association for Information Systems, 20 (10), 1412–1446.

Pick, J., & Azari, R. (2011). A global model of technological utilization based on governmental, business-investment, social, and economic factors. Journal of Management Information Systems, 28 (1), 49–84.

Pick, J., & Sarkar, A. (2016). Theories of the digital divide: Critical comparison. In 49th Hawaii International Conference on System Sciences (HICSS 2016) Proceedings, IEEE,  3888–3897.

Pick, J., Sarkar, A., & Parrish, E. (2018). Internet use and online activities in US States: geographic disparities and socio-economic influences. In the 51st Hawaii International Conference on System Sciences (HICSS 2018) Proceedings, IEEE, 3853–3863.

Racherla, P., & Mandviwalla, M. (2013). Moving from access to use of the information infrastructure: A multilevel sociotechnical framework. Information Systems Research, 24 (3), 709–730.

Reinartz, A., Buhtz, K., Graf-Vlachy, L., & König, A. (2018). Mechanisms of engagement with, and disengagement from, Internet applications: A qualitative study of online job search. In International Conference on Information Systems (ICIS) .

Reisdorf, B. C., & Rikard, R. V. (2018). Digital rehabilitation: a model of reentry into the digital age. American Behavioral Scientist, 62 (9), 1273–1290.

Rockmann, R., Gewald, H., & Haug, M. (2018). Equal access for everyone? A digital divide cascade for retired senior citizens. In  ECIS 2018 Proceedings , 30. https://aisel.aisnet.org/ecis2018_rp/30 .

Schryen, G., Wagner, G., & Benlian, A. (2015) Theory of knowledge for literature reviews: an epistemological model, taxonomy and empirical analysis of IS literature. In ICIS 2015 Proceedings .  https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1648&context=icis2015 .

Scott, W. R. (2005). Institutional theory: Contributing to a theoretical research program. In Ken G. Smith and Michael A. Hitt (eds.) Great minds in management: The process of theory development ,  37 (2), 460–484. Oxford UK: Oxford University Press.

Sipior, J. C., Ward, B. T., & Connolly, R. (2011). The digital divide and t-government in the United States: using the technology acceptance model to understand usage. European Journal of Information Systems, 20 (3), 308–328.

Srivastava, S. C., & Shainesh, G. (2015). Bridging the service divide through digitally enabled service innovations: evidence from indian healthcare service providers. MIS Quarterly, 39 (1), 245–267.

Talukdar, D., & Gauri, D. K. (2011). Home Internet access and usage in the USA: Trends in the socio-economic digital divide. Communications of the Association for Information Systems, 28 (1), 85–98.

UnitedNations (2020). Getting to know the sustainable development goals. https://www.un.org/sustainabledevelopment/sustainable-development-goals/ . Accessed 15 Mar 2020.

United Nations (2018). E-government survey 2018, Gearing E‐government to support transformation towards sustainable and resilient societies. https://www.unescap.org/resources/e-government-survey-2018-gearing-e-government-support-transformation-towards-sustainable . Accessed 15 Mar 2020.

Van Deursen, A. J., & Helsper, E. J. (2015). The third-level digital divide: Who benefits most from being online? In Communication and information technologies annual (pp. 29–52). Bingley: Emerald Group Publishing Limited.

Van Dijk, J. A. (2006). Digital divide research, achievements and shortcomings. Poetics, 34 (4–5), 221–235.

Van Dijk, J. A. (2012). The evolution of the digital divide: The digital divide turns to inequality of skills and usage. Digital Enlightenment Yearbook, 2012, 57–75.

Vassilakopoulou, P., Grisot, M., & Aanestad, M. (2016). Enabling electronic interactions between patients and healthcare providers: a service design perspective. Scandinavian Journal of Information Systems, 28 (1), 71–90.

Venkatesh, V., Sykes, T. A., & Venkatraman, S. (2014). Understanding e-Government portal use in rural India: role of demographic and personality characteristics. Information Systems Journal, 24 (3), 249–269.

Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly, 26 (2), xiii–xxiii.

Wei, K.-K., Teo, H.-H., Chan, H. C., & Tan, B. C. (2011). Conceptualizing and testing a social cognitive model of the digital divide. Information Systems Research, 22 (1), 170–187.

Xiong, J., & Zuo, M. (2019). How does family support work when older adults obtain information from mobile internet? Information Technology & People, 32 (6), 1496–1516.

Zhao, F., Collier, A., & Deng, H. (2014). A multidimensional and integrative approach to study global digital divide and e-government development. Information Technology & People, 27 (1), 38–62.

Download references

Acknowledgements

We want to acknowledge June Lithell Hansen and Andreas Skaiaa for their contribution in an early stage of this study during fall 2018. The contribution was part of their master course work performed at the University of Agder.

Author information

Authors and affiliations.

University of Agder, Kristiansand, Norway

Polyxeni Vassilakopoulou & Eli Hustad

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Polyxeni Vassilakopoulou .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic Supplementary Material

(DOCX 35 kb)

Rights and permissions

Reprints and permissions

About this article

Vassilakopoulou, P., Hustad, E. Bridging Digital Divides: a Literature Review and Research Agenda for Information Systems Research. Inf Syst Front 25 , 955–969 (2023). https://doi.org/10.1007/s10796-020-10096-3

Download citation

Accepted : 06 December 2020

Published : 06 January 2021

Issue Date : June 2023

DOI : https://doi.org/10.1007/s10796-020-10096-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Digital divide
• Digitalization
• Digital inequalities
• Information systems research
• Sustainability
• Find a journal
• Publish with us
• Track your research

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved September 16, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health (m-health) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE: Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Sep 4, 2024 9:40 AM
• URL: https://libguides.usc.edu/writingguide

• Help & FAQ

A systematic literature review on Internet of things in education: Benefits and challenges

• Engineering Division (Great Valley)
• Institute for Computational and Data Sciences (ICDS)

Research output : Contribution to journal › Article › peer-review

With close to 20.4 billion devices connected to the Internet to be deployed by 2020, Internet of things (IoT) is already being leveraged in diverse sectors. Now, because of the ubiquitous nature of IoT devices, schools and academic institutions are looking to incorporate IoT in educational activities. With the increased use of IoT in the education domain, it is of utmost importance to study how this technology with its distinguished system functions such as sensing and decision making can support and challenge the pedagogical processes for all interrelated actors (faculty, students, and staff) as well as all involved assets (e.g., libraries, classrooms, and labs). Although there have been several contributions on the inclusion of IoT into the education domain, there is still a lack of consolidated and coherent views on this subject. Hence, we are motivated to close the gap of knowledge and embarked on mapping out the published studies available. This study presents the results of a systematic literature review focusing on the benefits and the challenges faced in education in integrating IoT into the curriculum and educational environments. Different mapping views of the extracted studies are provided as long as a summary of the already implemented tools and a list of gap research questions yet to be investigated.

All Science Journal Classification (ASJC) codes

• Computer Science Applications

Access to Document

• 10.1111/jcal.12383

Other files and links

• Link to publication in Scopus
• Link to the citations in Scopus

Fingerprint

• Education Engineering & Materials Science 100%
• Internet of things Engineering & Materials Science 98%
• Internet Social Sciences 73%
• literature Social Sciences 49%
• education Social Sciences 36%
• Curricula Engineering & Materials Science 20%
• educational activities Social Sciences 17%
• Students Engineering & Materials Science 14%

T1 - A systematic literature review on Internet of things in education

T2 - Benefits and challenges

AU - Kassab, Mohamad

AU - DeFranco, Joanna

AU - Laplante, Phillip

N1 - Publisher Copyright: © 2019 John Wiley & Sons, Ltd

PY - 2020/4/1

Y1 - 2020/4/1

N2 - With close to 20.4 billion devices connected to the Internet to be deployed by 2020, Internet of things (IoT) is already being leveraged in diverse sectors. Now, because of the ubiquitous nature of IoT devices, schools and academic institutions are looking to incorporate IoT in educational activities. With the increased use of IoT in the education domain, it is of utmost importance to study how this technology with its distinguished system functions such as sensing and decision making can support and challenge the pedagogical processes for all interrelated actors (faculty, students, and staff) as well as all involved assets (e.g., libraries, classrooms, and labs). Although there have been several contributions on the inclusion of IoT into the education domain, there is still a lack of consolidated and coherent views on this subject. Hence, we are motivated to close the gap of knowledge and embarked on mapping out the published studies available. This study presents the results of a systematic literature review focusing on the benefits and the challenges faced in education in integrating IoT into the curriculum and educational environments. Different mapping views of the extracted studies are provided as long as a summary of the already implemented tools and a list of gap research questions yet to be investigated.

AB - With close to 20.4 billion devices connected to the Internet to be deployed by 2020, Internet of things (IoT) is already being leveraged in diverse sectors. Now, because of the ubiquitous nature of IoT devices, schools and academic institutions are looking to incorporate IoT in educational activities. With the increased use of IoT in the education domain, it is of utmost importance to study how this technology with its distinguished system functions such as sensing and decision making can support and challenge the pedagogical processes for all interrelated actors (faculty, students, and staff) as well as all involved assets (e.g., libraries, classrooms, and labs). Although there have been several contributions on the inclusion of IoT into the education domain, there is still a lack of consolidated and coherent views on this subject. Hence, we are motivated to close the gap of knowledge and embarked on mapping out the published studies available. This study presents the results of a systematic literature review focusing on the benefits and the challenges faced in education in integrating IoT into the curriculum and educational environments. Different mapping views of the extracted studies are provided as long as a summary of the already implemented tools and a list of gap research questions yet to be investigated.

UR - http://www.scopus.com/inward/record.url?scp=85074992670&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074992670&partnerID=8YFLogxK

U2 - 10.1111/jcal.12383

DO - 10.1111/jcal.12383

M3 - Article

AN - SCOPUS:85074992670

SN - 0266-4909

JO - Journal of Computer Assisted Learning

JF - Journal of Computer Assisted Learning