mental health application thesis

SYSTEMATIC REVIEW article

Effectiveness of mobile app-based psychological interventions for college students: a systematic review of the literature.

• 1 Department of Education and Psychology, University of Aveiro, Aveiro, Portugal
• 2 Department of Psychology and Education, Portucalense Institute for Human Development (INPP), Universidade Portucalense, Porto, Portugal

Serious mental health disorders are increasing among college students and university counseling services are often overburdened. Mobile applications for mental health have been growing exponentially in the last decade and they are emerging in university settings as a promising tool to promote and intervene in college students' mental health. Additionally, considering the recent covid-19 pandemic, mHealth interventions, due to its nature and possibilities, may play an important role in these institutions. Our main objectives are to explore mhealth interventions in universities, regarding its conceptual framework, acceptability and efficacy outcomes and understand its impact and contributions to address treatment delivery and psychological difficulties resulting from covid-19 pandemic. The literature search was conducted in scientific databases, namely, Web of Science, Pubmed, and Scopus. A search in app stores was not conducted, thus regarding commercially available apps, only those found in our database search were included in our review. We selected studies with mobile applications addressing psychological interventions for college students. A total of 2,158 participants were included in the 8 selected studies and most interventions were delivered through mobile apps only and based in cognitive behavioral therapy. Results suggested that college students accept and adhere to these interventions and preliminary evidence of efficacy was demonstrated in different disorders, such as stress, anxiety, depression and risky behaviors such as alcohol and tobacco abuse and sexual knowledge. We conclude that universities, particularly college counseling services, may benefit from mhealth interventions, not only to address college students' mental health but to decrease some of its difficulties related to lack of human resources. Specifically in covid-19 pandemic context, these interventions may contribute significantly by promoting and delivering psychological interventions at a safe distance.

Introduction

Over the last decade numerous mental health mobile applications have been developed and made available for users ( Bakker et al., 2016 ). Smartphones demonstrate numerous advantages such as great computing capacity, mobility, and more rapid and efficient access to information by using mobile applications ( Donker et al., 2013 ). The enthusiasm of smartphones for healthcare initiatives led to the emergence of a novel field called mHealth ( Ben-Zeev et al., 2014 ) defined as the use of mobile technologies to deliver or support psychological or mental health interventions and includes mobile devices such as smartphones, tablets, Personal Digital Assistants, and wearable devices ( Clough and Casey, 2015b ; Alyami et al., 2017 ). In clinical settings, mHealth may enhance face-to-face treatments, increase patient engagement in therapy sessions and adherence to therapy principles; provide better use of clinician time and resources and improve treatment outcome and risk of relapse ( Clough and Casey, 2015b ). Several studies have shown that mental health apps and cognitive behavioral therapy (CBT)-based apps are efficacious ( Rathbone et al., 2017 ; Linardon et al., 2019 ). However, despite clinical potential, interest and early supporting evidence, one factor that seems to limit mental health apps is low engagement or poor adherence to the intervention ( Torous et al., 2018 ).

One of the areas were mental health apps can have a significant impact is in universities. College years are a sensitive period to the onset of several mental health disorders ( Kessler et al., 2007 ) and many studies have reported a significant rise in serious mental health illness among college students ( Hunt and Eisenberg, 2010 ; Storrie et al., 2010 ; Auerbach et al., 2018 ). Major Depressive Disorder (MDD) and Generalized Anxiety Disorder (GAD) were identified as the most common disorders found in college students ( Auerbach et al., 2018 ). University counseling services constitute a valuable resource to support college student mental health and wellness ( Spooner, 2000 ) and a challenge that seems to be common across several counseling services is the growing student demand for these services and the limited resources to face these demands ( Johnson and Kalkbrenner, 2017 ; Shaw et al., 2017 ; Auerbach et al., 2018 ; Lee and Jung, 2018 ). College students are also large consumers of technology and communicate frequently online ( Shaw et al., 2017 ). A study by Wilansky et al. (2016) referred that mobile applications may increase youth adherence to Cognitive Behavioral Therapy (CBT) and improve treatment outcomes. Research suggests that mHealth is already being used to increase students' awareness and to deliver health-related interventions with increasing popularity; preliminary findings indicate that students are open and willing to use these interventions ( Johnson and Kalkbrenner, 2017 ).

Mobile technologies for mental health assume an important role considering our current reality of pandemics resulting from covid-19 infectious disease. Covid-19 is an infectious disease cause by a coronavirus that rapidly expanded worldwide, and some of the protective measures include physical distancing, wearing a mask, avoiding crowds and close contact, and regularly cleaning your hands ( World Health Organization, 2020 ). College students, alongside with children and health workers, are one of the most exposed groups to develop post-traumatic stress disorder, anxiety, depression and other symptoms of distress ( Saladino et al., 2020 ). Studies conducted during covid-19 pandemic in China concluded that almost half of Chinese college students that participated in the study experienced anxiety symptoms ( Fu et al., 2021 ) and are more likely to suffer from stress, anxiety and depression than the general population ( Li et al., 2020 ). Several studies highlight the need to monitor students' mental health during the pandemic and the delivery of timely and appropriate interventions ( Cao et al., 2020 ; Fu et al., 2021 ) such as the importance of technological devices or digital interventions ( Saladino et al., 2020 ). Covid-19 brought several challenges to mental health services delivery, thus many therapists rapidly adhered to telehealth to replace in-person contact ( Taylor et al., 2020 ). The same authors state that this disease presents an imperative for mental health services to make digital mental health interventions available in routine care and not only in response to covid-19 crisis.

Previous systematic reviews with college students and mobile interventions often explore a wide range of mHealth interventions and technology (e.g., Johnson and Kalkbrenner, 2017 ). Our review will focus on (1) mental health mobile applications that include a psychological intervention targeting a mental disorder, (2) college students, and (3) randomized controlled trials and acceptability and feasibility studies. We aim to explore how mobile apps are being developed to address college students' mental health in universities, if they accept and adhere to these interventions and if these interventions demonstrate efficacy. A search will be made for peer-reviewed articles of mental health mobile apps in scientific databases. The present review will not conduct a search in app stores mainly because acceptability and efficacy outcomes are not usually reported in app stores and because it would demand a different type of search strategy. Thus, in the current review we aim to review all published literature, in scientific databases, on psychological interventions using mobile applications, in the last 12 years, for college students. Our main objective is to review efficacy outcomes, through randomized controlled trials, of mobile app-based psychological interventions compared to traditional therapy or a waiting list control group in reducing psychological symptomatology among college students. Additionally, we intend to explore how mobile interventions are being accepted by college students and which conceptual frameworks are being used to develop these interventions. Considering the recent context of covid-19 pandemics, we aim to reflect on the impact and contributions of mHealth interventions for universities and college students.

We used the search method of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) ( Moher et al., 2009 ).

Eligibility Criteria and Information Sources

Inclusion criteria considered (1) target population: college students; (2) types of intervention: psychological interventions delivered through mobile applications (self-guided); mobile applications combined with web-based interventions or mobile applications combined with face-to-face treatments; (3) Primary outcome measures that target specific psychological disorders or symptomatology (i.e., anxiety, depression, social anxiety, stress, PTSD, alcohol abuse); (4) clear report of the psychological intervention, specifying theoretical basis or treatment model and therapeutic techniques; (5) Types of studies: randomized controlled trials (RTC) or quasi-experimental designs that clearly report efficacy outcomes and feasibility and acceptability studies since they contribute with valuable information about conceptual framework and some provide preliminary effectiveness results; (6) written in English; (7) published in the selected scientific databases. Exclusion criteria consisted in (1) studies with young adults (not students); (2) mobile interventions based on text messages; (3a) mobile interventions targeting physical or medical conditions (e.g., diabetes, physical activity, nutrition, weight control etc.); (3b) studies about mobile learning apps (e.g., anatomy); (3c) studies about smartphone addiction; (4) internet and computerized based interventions; (5) study protocols.

Our main objective is to review conceptual framework, acceptability, and efficacy outcomes of mobile app interventions addressing mental health for college students. A search of mobile apps commercially available in the app store was not conducted in this review since, although important, demands a different type of search and selection process, and often don't report acceptability and efficacy results (in the app store). Thus, we considered that it would be more suited to do a review, with this group of apps, separately. A narrative approach was used for extraction and synthesis of the data. Studies were identified through three major electronic databases, namely, Web of Science, Pubmed, and Scopus. An update literature search was performed in January 2021 using the same information sources.

Search and Study Selection Process

The following search keywords were considered “mobile interventions,” “smartphones,” “mobile application,” “mHealth,” “mobile technology,” “college students,” “students,” “university,” “campus.” Two authors independently conducted a thorough search in the three major scientific databases with the mentioned keywords, using primarily the combination “mobile interventions” AND “college students” with year filter between 2008 and 2019. A search update was performed in January 2021 with the same study selection process. In a first instance, studies including keywords in titles and/or abstracts were selected for further thorough review. After identifying eligible studies, duplicates were removed, and full papers were examined regarding eligibility criteria. A list of studies was produced by each author. Afterwards, both authors discussed their list of included studies, and by agreement, a final list of studies was produced.

Data Extraction

Data extraction was performed by two independent researchers and included year of publication, demographic characteristics of participants, study design (RCT, quasi-experimental studies, single-arm pre-test post-test), study participants and interventions (i.e., population, conditions, sample size, outcome measures, mobile app characteristics, theoretical basis, and intervention modality), main results and findings.

Assessment of Methodological Quality

The present review resorted to critical appraisal tools from the Joanna Briggs Institute for randomized controlled trials and quasi experimental studies (non-randomized experimental studies). The Checklist for Randomized Controlled Trials [ The Joanna Briggs Institute (JBI), 2017a ] was utilized to assess the methodological quality of the included RCTs and the Checklist for Quasi-Experimental Studies (non-randomized experimental studies) [ The Joanna Briggs Institute (JBI), 2017b ] to assess methodological quality of quasi-experimental studies and studies with a one group pre-test post-test design. Each study was assessed using JBI checklists for RCT or quasi-experimental studies.

Study Selection

As we can see in Figure 1 our search identified 957 published articles. Afterwards, we removed 23 duplicates and a review of title and abstracts excluded 904 articles. A total of 30 full-text articles were assessed for eligibility, where 11 were excluded due to motives of being a study protocol, thus not presenting feasibility or efficacy outcomes; lack of a psychological intervention or a psychological disorder; being web-based intervention or having no access to article full text. A total of 19 studies were included and examined in accordance with inclusion criteria.

Figure 1 . PRISMA flow diagram. From: Moher et al. (2009) .

Demographic Characteristics

A total of 3,399 college students were included in the selected studies ( n = 19) for this systematic review. Eleven studies included college students with self-reported psychological symptomatology (i.e., elevated stress, generalized anxiety disorder (GAD), PTSD), two studies included first-year college students and the remaining studies included non-treatment seeking college students ( n = 6). Most studies occurred in the USA ( n = 12), others occurred in Germany ( n = 1), Sweden ( n = 1), Canada ( n = 1), United Kingdom (UK) ( n = 2), Australia ( n = 1), and Iran ( n = 1).

Intervention Characteristics

Mobile intervention apps for college students target anxiety ( n = 7), depression ( n = 7), stress ( n = 5), alcohol consumption and risky drinking ( n = 4), smoking ( n = 1), and sexual behaviors ( n = 1), Post-traumatic stress disorder (PTSD) ( n = 1). Table 1 resumes all further interventions characteristics.

Table 1 . Mobile interventions characteristics.

We considered that most studies, with self-guided apps, focus on prevention ( n = 15) and the studies that included human support (therapists and coaches) and a TAU group were more focused on a treatment approach ( n = 4). However, many studies with self-guided apps, included students with elevated psychological symptomatology (i.e., elevated stress, diagnosed PTSD, or GAD), and it isn't always clear the nature of their intervention.

Intervention modality varied between a combination of internet and mobile app intervention ( n = 2) and mobile app intervention only ( n = 17), from these 17 studies, two apps were combined with a wearable band to permit passive data collection. When combining mobile apps with internet interventions, the mobile app functioned mostly as a tool offering support for homework assignment or working as a diary app by enabling monitoring of mood fluctuations or stress levels [e.g., Harrer et al. (2018) ]. Human support was considered in 4 mobile apps (Lantern; TAO; StudiCare Stress; Mind the Moment), two mobile interventions included therapists and two included a coach, StudiCare Stress app included a trained master's student in Psychology (named an eCoach) and Lantern app included a coach with various educational backgrounds. Human support varied from weekly 10–12 min brief videoconferences, to 2 face to face sessions and online sessions only.

Regarding conceptual framework most researchers used CBT intervention or CBT third wave techniques to conceptualize these apps ( n = 17). Most CBT apps include mindfulness exercises ( n = 11), some are solely based on mindfulness ( n = 4) or acceptance and commitment therapy (ACT) ( n = 2). One mobile app is focused on CBT and a biofeedback intervention (BioBase app). Some used CBT intervention as a part of a larger program such as GET.ON Stress, a stress management program, adapted to college students; or BASICS, an alcohol intervention program for college students. In some cases, CBT was combined with other psychological models such as Lazarus Transactional Model of Stress (GET.ON Stress program) or the Unified Theory of Use and Acceptance of Technology (UTAUT). The StudiCare Stress app also included an adherence-focused guidance concept according to the human accountability model. Only two studies did not resort to CBT, the SmarTrek app that used motivational interviewing and the SEX101 that used two psychological models, the Theory of Reasoned Action (TRA; Fishbein and Ajzen, 1975 ) and the Transtheoretical Model (TTM) of behavior Change ( Prochaska and DiClemente, 1984 ). Additionally, SmartTreak and MtM added an Ecological Momentary Intervention (EMI) and Witkiewitz et al. (2014) , BioBase app and ACT daily included an Ecological Momentary Assessment (EMA).

As for specific techniques more than half of the mobile apps include mindfulness exercises; other included psychoeducation or general information about the target disorder; include data collection self-monitoring; exposure; systematic desensitization and relaxation exercises. Other features refer to quizzes and interactive games; virtual coach; passive sensing through sensorband; all apps for risky drinking and excessive smoking included personalized feedback on drinking patterns and motives for drinking, feedback includes information about smoking and “urge-surfing” or strategies to increase student's emotional awareness. All apps were designed to provide education, collect data, monitor/track behavior, some provide personalized feedback or guidance in CBT exercises (in some cases homework assignments).

Few studies gave information regarding privacy and security. For example, Benton et al. (2016) referred that TAO security and privacy included authentication, password protection, and encryption of databases and Lee and Jung (2018) stated that data was collected and stored on secure systems and accessed through computers with password protection and encryption.

Methodological Quality

Tables 2 , 3 resumes the methodological characteristics of the included studies. Eleven studies are randomized controlled trials (RCT) ( Witkiewitz et al., 2014 ; Gajecki et al., 2017 ; Harrer et al., 2018 ; Lee and Jung, 2018 ; Fish and Saul, 2019 ; Huberty et al., 2019 ; Bruehlman-Senecal et al., 2020 ; Flett et al., 2020 ; McCloud et al., 2020 ; Newman et al., 2020 ; Ponzo et al., 2020 ) and two studies are considered quasi-experimental trials ( Benton et al., 2016 ; Borjalilu et al., 2019 ). Four studies considered a single-arm pre-test-post-test study design ( Jackson et al., 2016 ; Leonard et al., 2017 ; Haeger et al., 2020 ; Lattie et al., 2020 ; Reyes et al., 2020 ) and one study included two groups through an iterative process ( Kazemi et al., 2018 ).

Table 2 . Methodological characteristics.

Table 3 . JBI Checklist for randomized controlled trials.

Eleven of the included studies are RCTs and the total sample size ranges from 72 to 330 college student participants; the overall duration of the intervention range from 14 days to 3 months and when we consider follow-ups, the longest trial lasted for 9 months. Most RCTs included as a control group a waiting list control trial ( n = 8). Following JBI critical appraisal tool, we consider that all RCTs reported that participants were randomly assigned to treatment groups, 9 out of 11 studies provided detailed description of the randomization procedure and two studies merely stated that the participants were randomly assigned. As for allocation concealment, three studies provided information about allocation concealment. For example, Harrer et al. (2018) stated that the randomization process was performed by a researcher not involved in the study, and although they weren't able to blind participants to study conditions, during the randomization process, they were able to conceal the allocation from participants, researchers, and e-coches. Ten studies provided information and reported similar groups at baseline. As for blinding participants, or those delivering treatment and even outcome assessors to treatment conditions may be difficult and even unachievable in this type of studies; several studies reported this issue, pointing to the inability to blind their participants to treatment conditions. There were incomplete follow-ups; however they were generally adequately described and analyzed. Six RCTs provided detailed information about intention-to-treat analyses (ITT); the remaining studies excluded participants, lost to follow-up, from analysis. All studies used primary outcome measures with good validity and reliability. The large majority of RCTs also included quantitative and/or qualitative self-report measures to evaluate usability, acceptability, user satisfaction, or app adherence.

The studies by Benton et al. (2016) and Borjalilu et al. (2019) were considered as quasi-experimental studies. The first study included a large sample size ( n = 1,241) with overall duration of the intervention of 7 weeks. They included a wait-list treatment as usual control group and the intervention group received the intervention of study. The primary outcome measure was adequately validated and provided multiple measurements along the intervention as well as pre and post assessment. Differences between groups in terms of follow-up were adequately described and analyzed. This study presented many missing data and the linear mixed-effects models was utilized to estimate parameters for missing values. As for Borjalilu et al. (2019) , they conducted a study with three conditions and 68 college students, who were randomly assigned into the three groups, but no further detailed information was given about the randomization process. There were pre- a post-assessments and follow up was complete. Outcomes were measured in a reliable way and participants, from both groups, were assessed in the same way.

In this review there is a significant number of a single group pre-test-post-test design studies that aimed to evaluate acceptability and feasibility; only one study ( Jackson et al., 2016 ) aimed to evaluate efficacy with this design. Sample sizes were similar between studies, ranging from n = 10 to n = 23, with overall duration (intervention) of 3–4 weeks. Adequate and validated main outcome measures were used. The SEX101 ( Jackson et al., 2016 ) had a larger sample size compared to the previous studies and a follow-up assessment of 3 months after intervention completion. However, the overall duration of the intervention was very small (pre-test and intervention had to be complete in 1 week and it takes 40 min to complete) and some outcome measures were developed by the researchers with few information regarding reliability.

Intervention Outcomes and Effect Sizes

A study conducted by Newman et al. (2020) assessed the efficacy of Lantern, a self-help mobile app to treat generalized anxiety disorder. Study results demonstrated a significant reduction on the DASS stress scores ( d = 0.408) and greater probability of remission from GAD ( d = 0.114). Lantern revealed moderate effects in reducing anxiety, stress, and depression. BioBase is a biofeedback self-guided mobile app combined with wearable device (BioBeam), to treat anxiety in college students. Ponzo et al. (2020) conducted a RCT to assess BioBase efficacy and results indicated that a 4-week intervention significantly reduced anxiety ( d = 0.67), depression (d = 0.99), and increased perceived well-being ( d = 0.65) demonstrating moderate to large effects. Sustained large effects at 2-week follow-up was found for anxiety ( d = 0.81) and perceived well-being ( d = 1.16).

McCloud et al. (2020) conducted a RCT to assess efficacy of Feel Stress Free app for the treatment of depression and anxiety symptoms. Results showed that there was a significant reduction of depression symptoms at week 4 ( d = 0.27) and week 6 ( d = 0.39), and significant reduction of anxiety symptoms at week 4 ( d = 0.58). Overall effect sizes ranged from small to moderate.

Bruehlman-Senecal et al. (2020) studied Nod, a mobile app designed to reduce loneliness during the transition to college. Their RCT results indicated significant condition-by-baseline loneliness interaction to predict week-4 depression (Np 2 = 0.02) and sleep quality (Np 2 = 0.04), suggesting that Nod buffered participants with higher baseline loneliness against heightened midquarter depression and poor sleep quality. Calm, is a mindfulness-based app, and its efficacy was tested among students with elevated stress. The study results of Huberty et al. (2019) found significant differences among conditions in all outcomes, namely, significant reduction in perceived stress ( d = 1.24), significant improvements in mindfulness ( d = 1.11), and self-compassion ( d = 0.84).

Harrer et al. (2018) conducted a randomized controlled trial to evaluate the efficacy of Studicare Stress, a stress management intervention app for college students. Their results indicated significant effects of the intervention compared with the waitlist control group for stress at post-test ( d = 0.69) and at 3-month follow-up, other secondary outcome measures also yielded significant effects such as anxiety ( d = 0.76), depression ( d = 0.63), college related productivity ( d = 0.33), and academic work impairment ( d = 0.34). Thus, Studicare Stress revealed moderate to large intergroup effects for the reduction of perceived stress and other health and college related outcomes.

Lee and Jung (2018) conducted a pilot study to evaluate efficacy of DeStressify, a mindfulness-based app on stress, anxiety, depressive symptomatology, sleep behavior, and other variables. Results indicated that when using the app during 4 weeks, students in the experimental group at post-test reported less trait anxiety ( N p 2 = 0.040); an improve in several quality of life subscales, such as general health, that significantly differed between treatment condition in post-intervention scores ( N p 2 = 0.07). A significant difference was also found in energy or fatigue subscale between treatment conditions ( N p 2 = 0.05). An interaction effect was found in the emotional well-being subscale ( N p 2 = 0.05). The author interpreted the partial eta squared values of 0.0099, 0.0588, and 0.1379 as small, medium, and large effect, respectively, following suggestions by Cohen ( Field, 2009 ). This indicates that we can verify small (trait anxiety) to medium effects for general health, energy or fatigue and emotional well-being.

Telecoach app ( Gajecki et al., 2017 ) was evaluated using a 3-arm randomized controlled trial and results demonstrated that the proportion of students with excessive alcohol consumption declined in both intervention and wait list control group compared to controls at first and second follow-ups. Secondary analysis showed reductions for the intervention group in quantity of drinking at first follow up and in frequency of drinking at both follow-ups. Across both follow-ups the odds ratios for not having excessive weekly alcohol consumption in the intervention group (1.95) was almost twice as high as for controls (1.00). Secondary analysis by gender showed that the odds ratio for not having excessive alcohol consumption among men in the intervention group compared to male controls was higher (2.68) than women in the intervention group (1.71) compared to women controls.

Witkiewitz et al. (2014) conducted a 3-arm randomized controlled trial to evaluate a mobile feedback intervention for heavy-episodic drinking (HED) and smoking among college students, and they concluded that at 1-month follow-up there were significant reduction in number of cigarettes per smoking day in both the mobile intervention ( d = 0.55) and mobile assessment conditions ( d = 0.45) with moderate effects. No significant results were observed on HED or concurrent smoking and drinking. As for Benton et al. (2016) quasi-experimental study, the intervention group showed improvements across time significantly greater than treatment as usual participants, for all primary outcomes except Life Functioning (LF) subscale. The size of these effects ranged from small ( d = 0.16) for LF, Global Mental Health and Well-Being (d = 0.20) to medium for Anxiety ( d = 0.31).

Usability, Acceptability, and Feasibility Outcomes

The large majority of the included studies evaluated acceptability and students' satisfaction with the intervention. From the 19 studies, eight studies explored adherence/satisfaction and six used adequately valid scales or methods to assess usability or satisfaction with app use. Some studies also used metrics obtained through the mobile app ( n = 2). Most studies, created their own items to assess satisfaction with the intervention. Overall, we could observe good retention rates across studies, however as Gajecki et al. (2017) specifically noted in there study, there is a possibility that their fairly high retention levels could result from the desire of some participants to win an iPad (reward to participate in the study) with no actual intention to use the app. Out of the 19 studies, 10 gave rewards to their participants.

All studies that evaluated satisfaction reported moderate to high client satisfaction with the intervention. The MtM app ( Leonard et al., 2017 ) demonstrated that 60% of the participants reported “mostly” or “very” satisfied with the sensorband and 50% with the mobile app. Also, 93.9% of the participants were very satisfied or satisfied with the intervention program of SEX101 app ( Jackson et al., 2016 ). However, this particular study produced large attrition rates (50%) and as the authors of this study noted information regarding app components that need to be improved, added or removed should be collected. In the Witkiewitz et al. (2014) EMA app, over 65% of the participants reported an increase in awareness of their drinking and/or smoking and 60% stated that they would recommend this study to a friend because it provided greater awareness and they could help a friend reduce their drinking and/or smoking. Kazemi et al. (2018) demonstrated good usability of SmartTrek and the best feature reported by students was “Games” and the most useful features was “know your BAC” and “My strategies” that monitored alcohol intake, created behavioral change plans and reminded them of their goals. None of the studies, that provided human support (therapists), explored acceptability and satisfaction of the therapist with the intervention.

Implications and Contributions of mHealth Interventions for College Students in Covid-19 Context

Covid-19 infectious disease emerged in China and rapidly expanded around the globe, leading to an unexpected pandemic, which completely changed our daily lives and significantly limited physical and social contact with significant repercussions to our physical and mental health. Specifically in college students that live in a constant and thriving social interaction, covid-19 pandemic had a strong negative impact on mental health and may have contributed to the increase of several preexisting barriers and limitations to college counseling services. Considering these restraints, mHealth interventions may play an important role in a pandemic context due to its ubiquitous, remote and innovative functionalities that may facilitate access to evidence base treatments for mental health and also, its provider and facilitator (therapist).

Taking into consideration the included studies and their characteristics, acceptability, satisfaction and efficacy outcomes, we may determine that these interventions can significantly contribute in several important aspects related to college students' mental health. To our understanding, mobile app technologies may significantly contribute to promote mental health in college students targeting several specific disorders, such as anxiety, stress, depression, smoking, and alcohol abuse. It is also attainable to support students with coping strategies for elevated stress, anxiety, smoking, and alcohol abuse. Through mobile technologies, therapists may monitor and keep track of their patients' symptomatology and well-being, check homework assignments, and contact their patients' regularly through chat or messages, remotely. Overall, mobile technologies provide spontaneous and remote access to app content whenever we want, particularly in the comfort of our home. It helps us maintain physical distance from mental health professionals and counseling services without interrupting treatment.

Summary of Evidence

Our search for studies addressing mobile health apps for college students in university settings gathered 19 studies with different conceptual frameworks and study designs. In this review we could verify an increase in studies using mobile interventions for college students over the years, particularly in the last year, which may indicate an increasing trend in mobile use for the delivery of health interventions for college students. The large majority of studies are being developed in North America and Europe.

Regarding target disorders we can verify that most apps target anxiety, depression and stress, others target risky or excessive drinking, PTSD and sexual behaviors. Overall, mobile interventions showed promising results to reduce psychological symptomatology associated with stress, depression, anxiety and general student's mental health. As for drinking, smoking, and sexual behaviors, the included apps seemed to reduce excessive drinking and smoking and increase contraceptive use and knowledge but not the intention to reduce sexual risk behaviors or actual risk reduction. Most of the mobile interventions showed medium to large effect sizes for the main variables the app was designed to intervene, which may indicate that these interventions are well conceptualized and grounded according to the best available empirical evidence. Some of the included studies aimed to evaluate acceptability and feasibility and overall, these apps demonstrate good acceptability and feasibility among college students, supporting the hypothesis that students may accept and adhere to these interventions.

When we explore conceptual frameworks of these mobile apps we verify that many studies adopted CBT as the main intervention, particularly Mindfulness exercises. Effectively, CBT is well-established and particularly known as an effective treatment for several mental health disorders, and have demonstrated its efficacy when delivered through apps ( Rathbone et al., 2017 ). In some studies the intervention was complemented with psychological models, which have been shown to increase intervention efficacy ( Webb et al., 2010 ). Aside from psychological models/theories for behavioral change, one study incorporated a technological model, namely the Unified Theory of Use and Acceptance of Technology (UTUAT). There seems to be a strong application of psychological models and intervention techniques, indicating that there is a concern in adequately conceptualizing these interventions following evidence base principles. However, considering that we are studying mobile health interventions with significant emphasis in technology, very few studies incorporated technological models. Also, security and privacy features are also rarely mentioned and increasingly relevant in this type of interventions, best practices should be known and shared, reflecting in a mobile app quality indicator.

Regarding therapist role in mobile interventions, only 4 studies incorporated human support, two studies included therapists and two studies included a trained psychology student. From the mentioned studies, one used the human accountability model to inform this support. We consider that even though most of these apps intend to reduce therapist time and subsequently reduce therapist caseload and overburdened, this process may be optimized and better conceptualized using human support models. Moreover, evidence shows that app based interventions with therapist support has shown to produce larger effects ( Linardon et al., 2019 ).

As for methodological quality of the included studies, most studies aimed to evaluate efficacy and resorted to a randomized controlled trial, which is natural since RCTs are known as the golden standard to evaluate efficacy. All trials randomly assigned their participants to treatment conditions; however the number of studies that performed randomization concealment and blinding was almost non-existent. This reflects the difficulty of concealment and blinding in these type of studies and the limitation of the RCT study design when assessing efficacy in this type of interventions. Most studies also use a waiting list control group; given that many studies included students with elevated psychological symptomatology (that have to wait weeks/months to get access to the intervention) and the difficulty of blinding participants with this type of comparator we wonder if this is the best control group to use in this studies. Other research designs are also being explored in these studies and should be considered, so we can obtain efficacy results timelier and reliably ( Clough and Casey, 2015a ). Many studies adopted a pre-test post-test study design in order to evaluate acceptability and feasibility, even though this research design is considered a weak experimental study design, we consider that for the purpose and objectives of the studies this design was well-applied. Good overall retention rates may indicate treatment feasibility and acceptability. However, most studies were of short duration, with small samples and in controlled settings, with the addition of significant rewards. Additionally, many outcome measures were self-reported and not always congruent with app adherence rates. User metrics (e.g., how many times a participant accessed the app) provided by mobile apps may contribute to more accurate indicators of use and adherence to the intervention. Also, qualitative studies exploring perceived usefulness and user experience with the app intervention may also contribute to understand and overcome some barriers of adherence and engagement. Rewards are sometimes our best option to find participants, however when we are studying acceptability and adherence to these interventions, rewards may produce biased results. Recent studies opted to reward outcome measures completion, rather than app use.

A final question that emerged while exploring the studies is associated with the limited visual content of the apps included in the studies. Few studies included images/visual content of the mobile apps; some studies reported how they developed the app but provided little information about app design. A study by Torous et al. (2018) concluded that most mhealth apps suffer from low engagement and adherence and this may be, along with other issues, due to poor usability and because most apps are not user-friendly. It is important that researchers provide more frequently studies regarding user's needs and report multidisciplinary teams when building (native) apps, since this area often needs involvement of psychologists, software engineers, and designers/interaction designers. Also these tools, in clinical settings (e.g., counseling services), should be designed and optimized regarding all end users: students and therapists. Therapists' point of view and evaluation was often forgotten in the included studies that involved therapists.

Mobile apps may be customized and designed under practically unlimited possibilities. They can be developed to promote, prevent or intervene in a specific mental health disorder; to promote well-being and to deliver treatment under different levels of therapist support in different mental health services. Thus, they can be implemented and tailored according to specific needs. It is important to continue studying these interventions using user-centered designs and rigorous efficacy and effectiveness studies. We consider that universities, including college counseling services, may benefit from mhealth interventions, not only to address college student mental health but to decrease some of its difficulties related to few human resources. In a context of quarantine and confinement at home, where physical and social distance is imperative, these interventions assume special importance. They facilitate mental health promotion and support therapist and patient contact at a safe distance, avoiding treatment interruption.

Limitations

The current review presents a major limitation since we limited our search scope to the mentioned databases. Registered clinical trials and commercially available apps in app stores were not included, thus we may have missed already developed or apps that are being currently studied for college students. We may have failed to identify studies with relevant information regarding the application of mHealth intervention in college settings when we didn't consider “young adults,” since it may not include college students or occur in college settings.

The current systematic review shows that mobile apps for mental health intervention in college students exists and demonstrates good acceptability and feasibility. They also demonstrate efficacy among students. Overall we may conclude that mHealth interventions may turn out to be a great resource and tool to implement in counseling services, offering therapists and students many advantages. Particularly in the current pandemic context, these interventions demonstrate innumerous possibilities and promising solution to address college students' mental health and overcome many barriers associated with treatment access.

Future studies addressing mobile apps in college students, should invest in user-centered design studies so we can better understand what students and therapists (also attending university counseling services workflow) value more in a mobile based psychological intervention, to better adapt and tailor the intervention to user's needs. Effectively, acceptability and feasibility results among therapists are lacking in studies that use mobile intervention with therapist support. Future investigations should also explore diversity when developing and studying future apps, examining the applicability and efficacy of other theories/models. Also, we consider that studies should describe the development process of the mobile application (e.g., by including visual content) so we can better understand what is actually being evaluated and how it may impact efficacy results, in terms of usability and design. Lastly, students are large consumers of technology and so it may be important to invest more in these interventions, doing larger studies with more students, with superior methodological quality and avoiding large monetary rewards.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author Contributions

CO searched for studies to include in the systematic review and wrote sections of the manuscript. AP and PV revised the manuscript and contributed to the conception of the study. CN, JG, and BA contributed to organize data extraction and the search of studies in the scientific databases. All authors contributed to the manuscript revision, read, and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Alyami, M., Giri, B., Alyami, H., and Sundram, F. (2017). Social anxiety apps: A systematic review and assessment of app descriptors across mobile store platforms. Evid. Based Ment. Health 20, 65–70. doi: 10.1136/eb-2017-102664

PubMed Abstract | CrossRef Full Text | Google Scholar

Auerbach, R. P., Mortier, P., Bruffaerts, R., Alonso, J., Benjet, C., Cuijpers, P., et al. (2018). WHO world mental health surveys international college student project: prevalence and distribution of mental disorders. J. Abnorm. Psychol . 127, 623–638. doi: 10.1037/abn0000362

Bakker, D., Kazantzis, N., Rickwood, D., and Rickard, N. (2016). Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Mental Health 3:e7. doi: 10.2196/mental.4984

PubMed Abstract | CrossRef Full Text

Benton, S. A., Heesacker, M., Snowden, S. J., and Lee, G. (2016). Therapist-assisted, online (TAO) intervention for anxiety in college students: TAO outperformed treatment as usual. Prof. Psychol. 47, 363–371. doi: 10.1037/pro0000097

CrossRef Full Text | Google Scholar

Ben-Zeev, D., Schueller, S. M., Begale, M., Duffecy, J., Kane, J. M., and Mohr, D. C. (2014). Strategies for mHealth research: lessons from 3 mobile intervention studies. Administr. Policy Mental Health Mental Health Serv. Res . 42, 157–167. doi: 10.1007/s10488-014-0556-2

Borjalilu, S., Ali Mazaheri, M., and Talebpour, A. (2019). Effectiveness of mindfulness-based stress management in the mental health of Iranian university students: a comparison of blended therapy, face-to-face sessions, and mHealth app (Aramgar). Iran. J. Psychiatry Behav. Sci . 13:84726. doi: 10.5812/ijpbs.84726

Bruehlman-Senecal, E., Hook, C. J., Pfeifer, J. H., FitzGerald, C., Davis, B., Delucchi, K. L., et al. (2020). Smartphone app to address loneliness among college students: pilot randomized controlled trial. JMIR Mental Health 7:21496. doi: 10.2196/21496

Cao, W., Fang, Z., Hou, G., Han, M., Xu, X., Dong, J., et al. (2020). The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Res . 287, 1–5. doi: 10.1016/j.psychres.2020.112934

Clough, B. A., and Casey, L. M. (2015a). Smart designs for smart technologies: research challenges and emerging solutions for scientist-practitioners within e-mental health. Prof. Psychol. Res. Pract . 46, 429–436. doi: 10.1037/pro0000053

Clough, B. A., and Casey, L. M. (2015b). The smart therapist: a look to the future of smartphones and mHealth technologies in psychotherapy. Prof. Psychol. Res. Pract . 46, 147–153. doi: 10.1037/pro0000011

Donker, T., Petrie, K., Proudfoot, J., Clarke, J., Birch, M. R., and Christensen, H. (2013). Smartphones for smarter delivery of mental health programs: a systematic review. J. Med. Internet Res . 15:2791. doi: 10.2196/jmir.2791

Field, A. (2009). Discovering Statistics Using SPSS. 3rd Edn . London: SAGE Publications Inc.

Google Scholar

Fish, M. T., and Saul, A. D. (2019). The gamification of meditation: a randomized-controlled study of a prescribed mobile mindfulness meditation application in reducing college students' depression. Simulat. Gaming 50, 419–435. doi: 10.1177/1046878119851821

Fishbein, M., and Ajzen, I. (1975). Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research . Reading, MA: Addison-Wesley.

Flett, J. A. M., Conner, T. S., Riordan, B. C., Patterson, T., and Hayne, H. (2020). App-based mindfulness meditation for psychological distress and adjustment to college in incoming university students: a pragmatic, randomised, waitlist-controlled trial. Psychology Health 35, 1049–1074. doi: 10.1080/08870446.2019.1711089

Fu, W., Yan, S., Zong, Q., Anderson-Luxford, D., Song, X., Lv, Z., et al. (2021). Mental health of college students during the COVID-19 epidemic in China. J. Affect. Disord . 280, 7–10. doi: 10.1016/j.jad.2020.11.032

Gajecki, M., Andersson, C., Rosendahl, I., Sinadinovic, K., Fredriksson, M., and Berman, A. H. (2017). Skills training via smartphone app for university students with excessive alcohol consumption: a randomized controlled trial. Int. J. Behav. Med . 24, 778–788. doi: 10.1007/s12529-016-9629-9

Haeger, J. A., Davis, C. H., and Levin, M. E. (2020). Utilizing ACT daily as a self-guided app for clients waiting for services at a college counseling center: a pilot study. J. Am. Coll. Health . 12, 1–8. doi: 10.1080/07448481.2020.1763366

Harrer, M., Adam, S. H., Fleischmann, R. J., Baumeister, H., Auerbach, R., Bruffaerts, R., et al. (2018). Effectiveness of an internet- and app-based intervention for college students with elevated stress: randomized controlled trial. J. Med. Internet Res . 20:9293. doi: 10.2196/jmir.9293

Huberty, J., Green, J., Glissmann, C., Larkey, L., Puzia, M., and Lee, C. (2019). Efficacy of the mindfulness meditation mobile app “calm” to reduce stress among college students: randomized controlled trial. JMIR MHealth UHealth 7:14273. doi: 10.2196/14273

Hunt, J., and Eisenberg, D. (2010). Mental health problems and help-seeking behavior among college students. J. Adolesc. Health 46, 3–10. doi: 10.1016/j.jadohealth.2009.08.008

Jackson, D. D., Ingram, L. A., Boyer, C. B., Robillard, A., and Huhns, M. N. (2016). Can technology decrease sexual risk behaviors among young people? Results of a pilot study examining the effectiveness of a mobile application intervention. Am. J. Sexual. Educ . 11, 41–60. doi: 10.1080/15546128.2015.1123129

Johnson, K. F., and Kalkbrenner, M. T. (2017). The utilization of technological innovations to support college student mental health: mobile health communication. J. Technol. Hum. Serv . 35, 314–339. doi: 10.1080/15228835.2017.1368428

Kazemi, D. M., Borsari, B., Levine, M. J., Shehab, M., Nelson, M., Dooley, B., et al. (2018). Real-time demonstration of a mHealth app designed to reduce college students hazardous drinking. Psychol. Serv . 16, 255–259. doi: 10.1037/ser0000310

Kessler, R. C., Amminger, G. P., Aguilar-Gaxiola, S., Alonso, J., Lee, S., and Ustun, T. B. (2007). Age of onset of mental disorders: a review of recent literature. Curr. Opin. Psychiatry 20, 359–364. doi: 10.1097/YCO.0b013e32816ebc8c

Lattie, E., Cohen, K. A., Winquist, N., and Mohr, D. C. (2020). Examining an app-based mental health self-care program, intellicare for college students: single-arm pilot study. JMIR Mental Health 7, 1–15. doi: 10.2196/21075

Lee, R. A., and Jung, M. E. (2018). Evaluation of an mhealth app (destressify) on university students' mental health: pilot trial. J. Med. Internet Res . 5:e2. doi: 10.2196/mental.8324

Leonard, N. R., Silverman, M., Sherpa, D. P., Naegle, M. A., Kim, H., Coffman, D. L., et al. (2017). Mobile health technology using a wearable sensorband for female college students with problem drinking: an acceptability and feasibility study. JMIR MHealth UHealth 5, 1–16. doi: 10.2196/mhealth.7399

Li, S., Wang, Y., Yang, Y., Lei, X., and Yang, Y. (2020). Analysis of influencing factors of anxiety and emotional disorders in children and adolescents during home isolation during the epidemic of novel coronavirus pneumonia. Chin. J. Child Health Care 28, 407–410. doi: 10.11852/zgetbjzz2020-0169

CrossRef Full Text

Linardon, J., Cuijpers, P., Carlbring, P., Messer, M., and Fuller-Tyszkiewicz, M. (2019). The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry 18, 325–336. doi: 10.1002/wps.20673

McCloud, T., Jones, R., Lewis, G., Bell, V., and Tsakanikos, E. (2020). Effectiveness of a mobile app intervention for anxiety and depression symptoms in university students: randomized controlled trial. JMIR MHealth UHealth 8, 1–22. doi: 10.2196/15418

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and The PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med . 6:e1000097. doi: 10.1371/journal.pmed.1000097

Newman, M. G., Jacobson, N. C., Rackoff, G. N., Bell, M. J., and Taylor, C. B. (2020). A randomized controlled trial of a smartphone-based application for the treatment of anxiety. Psychotherapy Res . 31, 443–454. doi: 10.1080/10503307.2020.1790688

Ponzo, S., Morelli, D., Kawadler, J. M., Hemmings, N. R., Bird, G., and Plans, D. (2020). Efficacy of the digital therapeutic mobile app biobase to reduce stress and improve mental well-being among university students: randomized controlled trial. JMIR MHealth UHealth 8:17767. doi: 10.2196/17767

Prochaska, J. O., and DiClemente, C. C. (1984). The Transtheoretical Approach: Crossing the Traditional Boundaries of Change . Homewood, IL: J. Irwin.

Rathbone, A. L., Clarry, L., and Prescott, J. (2017). Assessing the efficacy of mobile health apps using the basic principles of cognitive behavioral therapy: systematic review. J. Med. Internet Res. 19:e399. doi: 10.2196/jmir.8598

Reyes, A. T., Bhatta, T. R., Muthukumar, V., and Gangozo, W. J. (2020). Testing the acceptability and initial efficacy of a smartphone-app mindfulness intervention for college student veterans with PTSD. Arch. Psychiatr. Nurs . 34, 58–66. doi: 10.1016/j.apnu.2020.02.004

Saladino, V., Algeri, D., and Auriemma, V. (2020). The psychological and social impact of covid-19: new perspectives of well-being. Front. Psychol. 11:577684. doi: 10.3389/fpsyg.2020.577684

Shaw, B. M., Lee, G., and Benton, S. (2017). “Work smarter, not harder: expanding the treatment capacity of a university counseling center using therapist-assisted online treatment for anxiety,” in Career Paths in Telemental Health , eds M. Maheu, K. Drude, and S. Wright (Cham: Springer), 197–204. doi: 10.1007/978-3-319-23736-7_19

Spooner, S. E. (2000). “The college counseling environment,” in College Counseling: Issues and Strategies for a New Millennium , eds D. C. Davis and K. M. Humphrey (Alexandria, VA: American Counseling Association), 3–14.

Storrie, K., Ahern, K., and Tuckett, A. (2010). A systematic review: students with mental health problems-A growing problem. Int. J. Nurs. Pract . 16, 1–6. doi: 10.1111/j.1440-172X.2009.01813.x

Taylor, C. B., Fitzsimmons-Craft, E. E., and Graham, A. K. (2020). Digital technology can revolutionize mental health services delivery: the COVID-19 crisis as a catalyst for change. Int. J. Eating Disord . 53, 1155–1157. doi: 10.1002/eat.23300

The Joanna Briggs Institute (JBI) (2017a). Checklist for Randomized Controlled Trials . Adelaide, SA: Joanna Briggs Institute.

The Joanna Briggs Institute (JBI) (2017b). Checklist for Quasi-Experimental Studies . Adelaide, SA: Joanna Briggs Institute.

Torous, J., Nicholas, J., Larsen, M. E., Firth, J., and Christensen, H. (2018). Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid. Based Ment. Health 21, 116–119. doi: 10.1136/eb-2018-102891

Webb, T. L., Joseph, J., Yardley, L., and Michie, S. (2010). Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J. Med. Internet Res . 12, 1–18. doi: 10.2196/jmir.1376

Wilansky, P., Eklund, M., Milner, T., Kreindler, D., Kovacs, T., Shooshtari, S., et al. (2016). Cognitive behavior therapy for anxious and depressed youth: improving homework adherence through mobile technology. JMIR Res. Protoc . 5:e209. doi: 10.2196/resprot.5841

Witkiewitz, K., Desai, S. A., Bowen, S., Leigh, B. C., Kirouac, M., and Larimer, M. E. (2014). Development and evaluation of a mobile intervention for heavy drinking and smoking among college students. Psychol. Addict. Behav . 28, 639–650. doi: 10.1037/a0034747

World Health Organization (2020). Coronavirus Disease (COVID-19) . Available online at: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/question-and-answers-hub/q-a-detail/coronavirus-disease-covid-19 (accessed December, 21, 2020).

Keywords: college students, mental health, mHealth, cognitive-behavioral therapy, counseling services

Citation: Oliveira C, Pereira A, Vagos P, Nóbrega C, Gonçalves J and Afonso B (2021) Effectiveness of Mobile App-Based Psychological Interventions for College Students: A Systematic Review of the Literature. Front. Psychol. 12:647606. doi: 10.3389/fpsyg.2021.647606

Received: 30 December 2020; Accepted: 06 April 2021; Published: 11 May 2021.

Reviewed by:

Copyright © 2021 Oliveira, Pereira, Vagos, Nóbrega, Gonçalves and Afonso. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Carla Oliveira, carlaandreia@ua.pt

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

• Bibliography
• More Referencing guides Blog Automated transliteration Relevant bibliographies by topics
• Automated transliteration
• Relevant bibliographies by topics
• Referencing guides

Counseling & Human Services Theses & Dissertations

Theses and dissertations published by graduate students in the Department of Counseling and Human Services, College of Education, Old Dominion University, since Fall 2016 are available in this collection. Backfiles of all dissertations (and some theses) have also been added.

In late Fall 2025, all theses will be digitized and available here. In the meantime, consult the Library Catalog to find older items in print.

Theses/Dissertations from 2024 2024

Dissertation: The Lived Experiences of Counselors-in-Training Completing Practicum and Internship During COVID-19 , Shelby Lauren Dillingham

Dissertation: Climbing the Academic Ladder While Black: Exploring the Experiences of Institutional Belongingness for Black Counselor Education and Supervision Doctoral Students at Predominantly White Institutions , Jasmine Alicia Hawa Griffith

Dissertation: Understanding the Role of Adverse Childhood Experiences on Resilience in Police Officers , Wayne F. Handley

Thesis: The Experiences of Multilingual Learning-Focused School Counselors: A Phenomenological Investigation , Chelsea Jo Hilliard

Dissertation: Partners in Crisis: A Phenomenological Exploration of Collaboration Between Crisis Intervention Team Officers and Mental Health Professionals , Jessica L. Huffman

Dissertation: Exploring the Experiences of School Counselors Advocating for Social Justice Within Urban PK-12 Charter Schools , Betsy Marina Perez

Dissertation: Supervision Needs of Novice Behavioral Health Providers in Integrated Primary Care Settings: A Delphi Study , Nicholas D. Schmoyer

Dissertation: Wellness as a Predictor of Turnover Intention in the Active Duty Military , Thomas Clifford Seguin

Dissertation: Consensus on Best Practices in Neuroscience-Informed Treatment of Combat-Related Posttraumatic Stress in Military Veterans: A Delphi Study of Experts in the Field of Mental Health , Danielle Lauren Winters

Theses/Dissertations from 2023 2023

Dissertation: Providing Family Centered Care Within Pediatric Integrated Care Settings , Emily D. Bebber

Dissertation: The Lived Experiences of 911 Dispatchers With Compassion Fatigue: An Interpretive Phenomenology , Angela Johnson

Dissertation: Exploring the Lived Experiences of Self-Identified Politically Conservative Students in Graduate Counseling Programs in Public Universities , Elizabeth A. Orrison

Theses/Dissertations from 2022 2022

Dissertation: Mental Health Counselors’ Perceptions of Professional Identity as Correctional Counselors in an Integrated Behavioral Health Care Setting , Jeanel L. Franklin

Dissertation: Complex Thought for Complex Work: Preparing Cognitively Complex Counselors for Work in Diverse Settings , Alexandra C. Gantt

Dissertation: Trauma and Crisis Counselor Preparation: The Relationship of an Online Trauma and Crisis Course and Counseling Self-Efficacy , Julia Leigh Lancaster

Dissertation: An Investigation of Healthcare Professionals’ Perspectives on the Tasks of Mental Health Counselors in Hospital Settings , Suelle Micallef Marmara

Dissertation: Broaching Race and Race-Related Issues: Phenomenological Inquiry of Doctoral Student Supervisors of Counselor Trainees , Judith Wambui Preston

Dissertation: A Phenomenological Investigation of School Counselor Antiracist Social Justice Practices , Stephanie Deonca Smith-Durkin

Dissertation: Counselor Perceived Competence Diagnosing Disorders from DSM-5 Diagnostic Categories Survey Results and the Development and Validation of Scores on the Diagnostic Self-Efficacy Scale , Erin Elizabeth Woods

Dissertation: The Experiences of School Counselors Providing Virtual Services During Covid-19: A Phenomenological Investigation , Allison Kathryn Worth

Dissertation: A Comparison of Sorority Women and Non-Sorority Women’s Alcohol Use: Perception, Rate of Use, and Consequences , Betsy Zimmerman

Theses/Dissertations from 2021 2021

Dissertation: Effectiveness of College Counseling Interventions in International Student Adjustment to United States Higher Education Systems: A Meta-Analysis , Joshua Ebby Abraham

Dissertation: What Are We Missing?: A Comparison of Experiences of Race-Based Trauma by Black Americans and Black Jamaicans , Bianca R. Augustine

Dissertation: Site Supervisors' Perspectives on Supervision of Counselor Trainees in Integrated Behavioral Health (IBH) Settings: A Q Methodology Approach , Yeşim Giresunlu

Dissertation: The Impact of a Crisis Intervention Team Program on Psychiatric Boarding , Kurtis Hooks

Dissertation: A Case Study on the Application and Implementation of Positive Behavioral Interventions and Supports for Students with Emotional Disabilities in Alternative Education , Kira Candelieri Marcari

Dissertation: Initial Development of the Escala de Fortaleza en Jóvenes para Padres , David Moran

Dissertation: Incivility of Coworker Behaviors and Minority Firefighters’ Belongingness in the Workplace , Alyssa Reiter

Dissertation: A Meta-Analysis of Three Years of Data on Outcomes of Therapy Groups for Inmates in the Virginia Department of Corrections , Abie Carroll Tremblay

Theses/Dissertations from 2020 2020

Dissertation: Improving Veterans’ Psychological Well-Being with a Positive Psychology Gratitude Exercise , Clara Im Adkins

Dissertation: An Examination of the Relationship Among Social Services Support, Race, Ethnicity and Recidivism in Justice Involved Mothers , Ne’Shaun Janay Borden

Dissertation: Development and Validation of the Students With Learning Disabilities School Counselor Self- Efficacy Scale: A Psychometric Study , Rawn Alfredo Boulden, Jr.

Dissertation: Minority Counselor Multicultural Competence in the Current Sociopolitical Climate , Kathleen Brown

Dissertation: A Meta-Analysis of Group Treatment Outcomes for Veterans with Substance Use Disorders , Robert “Tony” Dice

Dissertation: Investigating the Impact of the FAVA Well-Being Protocol on Perceived Stress and Psychological Well-Being With At-Promise High School Students , Renee L. Fensom

Dissertation: Mental Health Counselors' Perceptions on Preparedness in Integrated Behavioral Healthcare in Underserved Areas , Kyulee Park

Dissertation: Group Treatment Effectiveness for Substance Use Disorders: Abstinence vs. Harm Reduction , Jill D. Parramore

Dissertation: Best Practices in Clinical Supervision: What Must Supervisees Do? , Johana Rocha

Dissertation: A Phenomenological Investigation of Counselors’ Experiences With Clients Affected by Problematic Internet Pornography Use , Kendall R. Sparks

Dissertation: Counselor Education Doctoral Students’ Research Self-Efficacy: A Concept Mapping Approach , Zahide Sunal

Dissertation: A Systemic Review and Meta-Analysis of Psychoeducational Groups for the Treatment of Psychopathology Resulting from Child Sexual Abuse , Alexis Lynnette Wilkerson

Dissertation: School Counseling Professionals’ Experiences Using ASCA’s Mindsets & Behaviors for Student Success to Achieve College and Career Readiness , George Wilson

Theses/Dissertations from 2019 2019

Dissertation: Exploring the Lived Experiences of Career Oriented Military Spouses Pursuing Education for Career Advancement , Melody D. Agbisit

Dissertation: Reshaping Counselor Education: The Identification of Influential Factors on Multisystemic Therapy , T'Airra C. Belcher

Dissertation: An Investigation of Posttraumatic Growth Experienced By Parents After a Miscarriage , Barbara Elizabeth Powell Boyd

Dissertation: The Psychometric Properties of the School Counseling Internship Competency Scale , Melanie Ann Burgess

Dissertation: Intersectional Identities and Microaggressions: The Experience of Transgender Females , Cory Daniel Gerwe

Dissertation: Comparing Higher Order Value Differences By Religious and Spiritual Association and Implications for Counseling: An Exploratory Study , Gregory C. Lemich

Dissertation: The Effects of Supervisory Style and Supervisory Working Alliance on Supervisee Disclosure in Supervision: A Moderated Mediation Analysis , Chi Li

Dissertation: A Comparison of College Student-Athletes With Attention-Deficit Hyperactivity Disorder (ADHD) and Nonathletes With ADHD: Academic Adjustment, Severity of Mental Health Concerns, and Complexity of Life Concerns , Sonja Lund

Dissertation: An Experimental Study of Research Self-Efficacy In Master’s Students , Nicola Aelish Meade

Theses/Dissertations from 2018 2018

Dissertation: Multilevel Confirmatory Factor Analysis of the Family Adjustment Measure with Lower-Income, Ethnic Minority Parents of Children with Disabilities , Vanessa Nicole Dominguez-O'Hare

Dissertation: Risk and Resiliency Factors Affecting the College Adjustment of Students with Intersectional Ethnocultural Minority and LGBTQ Identities , Stacey Christina Fernandes

Dissertation: The Relationship Between Childhood Adversity and Adult Relationship Health for Economically Marginalized, Racially and Ethnically Diverse Individuals , Sandy-Ann M. Griffith

Dissertation: An Exploration of Practicum Students' Experiences of Meaning-Making Through Altruism , Debra Paige Lewis

Dissertation: Addiction Counselors' Perceptions of Clinical Supervision Practices , Marla Harrison Newby

Dissertation: Exploring the Variant Experiences Through Which Racial/Ethnic Minorities Select Art Therapy as a Career , Mary Ritchie Roberts

Dissertation: Psychosocial Determinants of Medication Adherence among HIV-Positive Individuals in Mexico City , Anthony Vajda

Theses/Dissertations from 2017 2017

Dissertation: Cross-Racial Trust Factors: Exploring the Experiences of Blacks Who Have Had White Mentors in the Counseling Profession , Eric Montrece Brown

Dissertation: Personality, Motivation, and Internet Gaming Disorder: Understanding the Addiction , Kristy L. Carlisle

Dissertation: The Relationship Between Trauma Exposure and College Student Adjustment: Factors of Resilience as a Mediator , Amber Leih Jolley

Dissertation: Establishing the Psychometric Properties of the Understanding Mental Health Scale: A Dissertation Study , Michael Thomas Kalkbrenner

Dissertation: Attitudes and Actions that Adoptive Parents Perceive as Helpful in the Process of Raising Their Internationally Adopted Adolescent , Marina V. Kuzmina

Dissertation: Towards a New Profession: Counselor Professional Identity in Italy. A Delphi Study , Davide Mariotti

Dissertation: Exploring the Relationship Between Depression and Resilience in Survivors of Childhood Trauma , Marquis A. Norton

Dissertation: Understanding the Experiences of Women with Anorexia Nervosa Who Complete an Exposure Therapy Protocol in a Naturalistic Setting , Gina B. Polychronopoulos

Dissertation: An Exploratory Factor Analysis Examining Experiences and Perceptions of Campus Safety for International Students , Sonia H. Ramrakhiani

Dissertation: The Role of Self-Care and Hardiness in Moderating Burnout in Mental Health Counselors , Traci Danielle Richards

Theses/Dissertations from 2016 2016

Dissertation: Examining Changes in College Counseling Clients’ Symptomology and Severity over an Eight Year Span , Caroline Lee Bertolet

Dissertation: Initial Development and Validation of the Transgender Ally Identity Scale for Counselors , Jamie D. Bower

Dissertation: A Counselor’s First Encounter with Non-Death Loss: A Phenomenological Case Study on New Counselor Preparation and Experience in Working with Non-Death Loss , Charles P. Carrington

Dissertation: The Relationship Between Counselors' Multicultural Counseling Competence and Poverty Beliefs , Madeline Elizabeth Clark

Dissertation: Counselors’ Perceived Preparedness for Technology-Mediated Distance Counseling: A Phenomenological Examination , Daniel C. Holland

Theses/Dissertations from 2015 2015

Dissertation: Factors Associated with Family Counseling Practices: The Effects of Training, Experience, and Multicultural Counseling Competence , Amanda A. Brookshear

Dissertation: An Examination of Supervisory Working Alliance, Supervisee Demographics, and Delivery Methods in Synchronous Distance Supervision , Robert Milton Carlisle III

Dissertation: A Phenomenological Investigation of Counselors' Perceived Degree of Preparedness When Working with Suicidal Clients , Heather Danielle Dahl

Dissertation: African American Pastors and Their Perceptions of Professional School Counseling , Krystal L. Freeman

Dissertation: A Phenomenological Investigation of Wellness and Wellness Promotion in Counselor Education Programs , Brett Kyle Gleason

Dissertation: Examining Disordered Eating Amongst Sorority Women , Andrea Joy Kirk

Dissertation: Bhutanese Counselors' Experiences with Western Counseling: A Qualitative Study , Susan V. Lester

Dissertation: An Exploration of Health Providers' Responses to Intimate Partner Violence (IPV) in Malaysia , Kee Pau

Dissertation: A Mixed Methods Study of the Intersection of Sexual Orientation and Spiritual Development in the College Experience , Kevin C. Snow

Theses/Dissertations from 2014 2014

Dissertation: Ethical and Legal Knowledge, Cognitive Complexity, and Moral Reasoning in Counseling Students , Matthew W. Bonner

Dissertation: A Grounded Theory of the College Experiences of African American Males in Black Greek-Letter Organizations , David Julius Ford Jr.

Dissertation: The Experiences of School Counselors with Court Involvement Related to Child Custody , Crystal E. Hatton

Dissertation: A Grounded Theory of Suicidality in Children Ten and Younger , Katherine Angela Heimsch

Dissertation: School Counseling Program Models Utilized By School Districts , Tracy L. Jackson

Dissertation: The Relationship Among Counseling Supervision Satisfaction, Counselor Self-Efficacy, Working Alliance and Multicultural factors , Jennifer Dawn Logan

Dissertation: Development of the Profession of Counseling in Kenya, Uganda, and Tanzania: A Grounded Theory Study , Mueni Joy Maweu Mwendwa

Dissertation: Resident Assistants' Self- Efficacy for Participation in Counseling Activities , Miranda Johnson Parries

Dissertation: Role Ambiguity of Counseling Supervisors , Aaron Gabriel Shames

Dissertation: Degree of Implementation of the American School Counselor Association National Model and School Counselor Burnout , Katrina Marie Steele

Dissertation: College Health and Mental Health Outcomes on Student Success , Daniel Joseph St. John

Dissertation: Supervisor Perceptions of Their Multicultural Training Needs for Working with English Language Learning Supervisees , Hsin-Ya Tang

Theses/Dissertations from 2013 2013

Dissertation: Experiences of Resident Assistants with Potentially Suicidal Students: Identification, Referral, and Expectations , Katherine M. Bender

Dissertation: Counselor Demographics, Client Aggression, Counselor Job Satisfaction, and Confidence in Coping in Residential Treatment Programs , Erik Braun

Dissertation: School Personnel Perceptions of Professional School Counselor Role and Function , Caron N. Coles

Dissertation: Factors That Influence Minority Student Enrollment at Various Levels of Postsecondary Education , LaShauna Mychal's Dean

Dissertation: Experiences and Perceptions of Mental Health Professionals Considered Effective in the Diagnosis and Treatment of Adults with Attention Deficit Hyperactivity Disorder , Bonita H. Erb

Dissertation: A Qualitative Study of the Experiences of Gatekeeping Among PhD Counselor Educators , Carol A. Erbes

Advanced Search

• Notify me via email or RSS
• Collections
• Disciplines
• Author Guidelines
• Department of Counseling & Human Services
• Other Digital Collections
• ODU Libraries
• Old Dominion University

Home | About | FAQ | My Account | Accessibility Statement

Privacy Copyright

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Psychiatry

Evaluating the Implementation of a Mental Health App for Overseas Filipino Workers in Macao China: A Mixed-Methods Study of Stakeholders’ Perspectives

Andrian liem.

1 Jeffrey Cheah School of Medicine and Health Sciences, Monash University Malaysia, Bandar Sunway, Malaysia

Karmia A. Pakingan

2 Department of Psychology, De La Salle University, Manila, Philippines

Melissa R. Garabiles

3 Scalabrini Migration Center, Quezon City, Philippines

Hao Fong Sit

4 Department of Psychology, The University of Hong Kong, Hong Kong, Hong Kong SAR, China

Sebastian Burchert

5 Division of Clinical Psychological Intervention, Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany

Agnes I. F. Lam

6 Department of Communication, University of Macau, Macau, Macau SAR, China

7 Centre for Macau Studies, University of Macau, Macau, Macau SAR, China

Brian J. Hall

8 Center for Global Health Equity, NYU Shanghai, Shanghai, China

9 Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, United States

Associated Data

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Overseas Filipino workers (OFWs) is one of the largest communities of international migrant workers. They face systemic barriers to fulfilling their health needs. The COVID-19 pandemic worsened this condition and provided a context to evaluate the utility of a digital mental health intervention delivered within a stepped-care model to address OFW mental health. Using an implementation science framework, this study aimed to evaluate stakeholders’ perspectives on the implementation of Kumusta Kabayan, a mobile phone-based mental health app, for OFWs in Macao.

A mixed-methods convergent design was used by conducting two parallel steps, including quantitative and qualitative measures. The quantitative data was collected from Filipino team members and local non-governmental organization (NGO) staff members ( N = 12). The qualitative data were gathered from interviews with OFWs in Macao who used the app ( N = 25; 80% females, 76% domestic workers).

From the online survey, the staff members of the local partner NGO and the Filipino team members strongly perceived that their organization could adapt Kumusta Kabayan to their program and generally evaluated that Kumusta Kabayan achieved its goal and was received well by OFWs. In the interviews, the OFW app users shared their experiences in using Kumusta Kabayan, which was thematically organized into six aspects of the participants’ experience: (1) promotional channel and expectation; (2) when to use the app and in what language; (3) lessons learnt; (4) memorable aspects; (5) key facilitators and barriers; and (6) suggestions.

Kumusta Kabayan was well accepted and shows potential to be integrated into the existing support services for OFWs in Macao. This app has the promise of being scaled-up for OFWs in other countries by collaborating with local and overseas stakeholders. Lessons learnt from this evaluation could also be implemented in wider digital mental health services in different settings.

Introduction

Overseas Filipino workers (OFWs) is one of the largest international migrant worker communities, with an estimated number of over 10 million worldwide ( 1 ). They work in both high-skill and low-skill sectors across countries to improve their lives and the livelihoods of their family members who remain in the Philippines ( 2 ). However, OFWs and other migrant workers struggle to fulfill their health needs because of systemic barriers they experience including language difficulties, adverse working conditions, lack of available healthcare providers, and inadequate social protection for migrant workers in the host countries ( 3 , 4 ). Consequently, migrant workers, including OFWs, experience higher risks for developing mental disorders like anxiety and depression than local people in the host countries ( 5 – 7 ).

The COVID-19 pandemic worsened this condition as migrant workers were forced to take temporary unpaid leaves from employment or suddenly lost their jobs and social protections ( 8 , 9 ). Additionally, OFWs and other migrants who worked as domestic workers encountered other challenges like excessive work load from their employers who began to work from home and some were not permitted to take their normal day-off due to employer’s fears of COVID-19 contamination and lack of trust in the domestic workers’ hygiene ( 10 , 11 ). Due to cultural and language barriers, migrant workers were also at risk of having inadequate knowledge and awareness of the latest COVID-19 situation and policies in their host countries ( 12 ). Consequently, OFWs and other migrant workers’ mental health deteriorated and they have to survive with the limited mental health support in the host countries ( 13 ).

The inequity in healthcare access during the pandemic may be reduced by implementing digital mental health interventions within a stepped-care approach, which includes providing the least intensive intervention based on clients’ needs and monitoring them regularly ( 14 ). Digital mental health interventions, including via the internet and mobile phones, were found to be consistently effective in reducing mental disorders like depression and anxiety ( 15 , 16 ). Furthermore, mobile phone-based health apps can also accurately screen mood disorder symptoms ( 17 ). Furthermore, a previous study on OFWs’ attitude toward digital health interventions found that the community showed their acceptance and willingness in accessing mobile phone-based mental health interventions ( 18 ). However, mobile phone-based mental health applications for improving mental health of OFWs and other migrant workers were still limited as previous interventions were mostly developed for inpatient adults ( 15 ).

With the limited internet and mobile phone-based mental health interventions for non-clinical groups, the World Health Organization (WHO) developed Step-by-Step, an online guided self-help intervention for people with depressive symptoms ( 19 ). Step-by-Step is an online version of the WHO’s offline transdiagnostic psychological intervention for common mental health problems named Problem Management Plus (PM+) ( 19 , 20 ). The Step-by-Step website version has been piloted among Lebanese, Palestinian, and Syrian adults in Lebanon ( 21 ) and the mobile application version was tested among adult Syrian refugees residing in Germany, Sweden and Egypt ( 22 ). These studies showed that Step-by-Step was a promising stepped-care digital mental health intervention for OFWs. The Step-by-Step program was also culturally adapted for use among Filipino overseas workers.

The current study evaluated stakeholders’ perspectives on the implementation of Step-by-Step for OFWs in Macao using a mixed-methods design. The study utilized an implementation science approach, which emphasized the practical aspects of how to scale-up an evidence-based intervention ( 24 ). The current study presents the results of the implementation evaluation of the program utilizing questionnaire data collected from our local non-governmental organization (NGO) partner and e-helpers (digital lay health workers) in the Philippines as well as interviews with OFWs in Macao as end users of the Filipino version of Step-by-Step named Kumusta Kabayan, which means “Hello, fellow Filipino.” The roles of each stakeholder member are presented in Figure 1 below. Using an implementation science framework, this study was designed to accelerate the translation of evidence-based intervention into real-world system of care and address inequities in healthcare delivery for migrant workers ( 25 , 26 ).

Stakeholder members’ roles.

Materials and Methods

This implementation science evaluation study used a mixed-methods parallel convergent design ( 27 ). The evaluation was conducted in two parallel steps, using quantitative and qualitative measures, which complemented each other and provided more comprehensive evaluation results ( 28 ). Details of the study is described in sub-sections below. Ethical approval for the study was granted from the Research Ethics Committee at the University of Macau (SSHRE19-APP074-FSS).

Sampling and Procedure

For the quantitative measure, total population sampling ( 29 ) was used where the lead author sent message invitations to the anonymous online evaluation survey to all e-helpers, clinical supervisors, staff and volunteers of the local NGO partner. The online survey for staff and volunteers of the local NGO partner were provided in English and Chinese to increase the participants’ understanding. Before completing the survey, electronic informed consent forms were provided to the participants.

For the qualitative measure, participants were recruited with a convenience census-approach sampling. All participants who participated in the Kumusta Kabayan ( N = 206) and were screened with the Patient Health Questionnaire (PHQ)-9 for depressive symptoms ( 6 ) were considered as eligible respondents and were asked if they were willing to participate in the interview. Participants who either completed or did not complete the program were purposively invited to represent diverse experiences. Not all potential interviewees joined the interviews as illustrated in Figure 2 .

Participants recruitment for interviews from both completer and non-completer groups.

Before the interviews were audio-recorded, participants were informed about the aim of interview and verbally asked for their consent. One-to-one audio interviews from October 2020 to February 2021 were conducted by e-helpers using voice over IP services. Interviews lasted an average of 25 min (SD = 8) and were conducted in English or Tagalog. Participants were compensated with a food voucher worth MOP 100 (≈USD $12) for their time. In order to improve participants’ openness, they were assigned to a different e-helper for the interview if they were previously assisted by an e-helper during the intervention period. Moreover, e-helpers were also trained for doing the interviews using a role play method prior to the data collection.

Participants

The demographic information (e.g., age and sex) of participants in the survey ( N = 12) was not asked to protect their anonymity, given the small sample sizes of e-helpers and NGO partner members. Participants in this quantitative measure consisted of a team in the Philippines ( n = 8), including two clinical psychologists, two Master of Psychology students, and four Bachelor of Psychology students; and a team in Macao ( n = 4), including three social workers and one logistical staff.

The interviewed OFW participants’ demographic information ( N = 25) was retrieved from their biodata in the app. The interviewed participants were mostly female ( n = 20), with an average age of 39 years old (SD = 8), and dominated by a domestic work job ( n = 19). On average, they have been working as OFWs for 59 months (SD = 78), having 66 working hours per week (SD = 16), and with a monthly salary of MOP 4,664 (SD = 1,182) (≈USD $580, exchange rate in July 2021). The interviewed participants consisted of both completers and non-completers. The demographic information of interviewees’ and non-interviewees were comparable ( Supplementary Material ).

Instruments

The online survey was adapted from previous studies on evaluation of innovative health programs promotion and implementation ( 30 – 33 ). The measures included organizational climate scale, awareness and concern scale, perceived advantage scale, perceived complexity scale, and level of success scale. Also, one open-ended item to ask for general feedback and suggestion was located at the end of the online survey (see Supplementary Material ).

The 11-item organizational climate scale consisted of three parts: mission, teamwork, and infrastructure support. An item from this measure is “Our team is willing to take a chance on a good idea” that was measured with Likert’s scale from 1 (strongly disagree) to 5 (strongly agree), where higher summed scores represent more positive organizational climate in adapting a new program. This scale was given only to the team in Macao due to the nature of the items. The internal reliability (Cronbach’s alpha) of this measure was 0.94. The 9-item awareness and concern scale consisted of three parts: awareness, concern, and interest. An item example of this measure is “I am aware that Kumusta Kabayan addresses depressive symptoms.” Item responses were on a Likert-type scale from 1 (not at all true) to 4 (very true), where higher summed scores represent higher awareness and concern of participants to the program. The Cronbach’s alpha of this measure was 0.77.

The perceived advantage scale was measured with a 3-item Likert scale from 1 (strongly disagree) to 5 (strongly agree), where higher summed scores represent greater perceived level of program’s advantage. An item from this measure is “The Kumusta Kabayan program can improve the mental health of Filipino workers in Macao.” The Cronbach’s alpha of this measure was 0.71. The perceived complexity scale was measured with a single item of organizational level (Part A, “The Kumusta Kabayan would be difficult for our organization/institution to explain to potential Filipino migrant worker users”) and 2-item Likert scale (Part B) from 1 (strongly disagree) to 5 (strongly agree), where higher total score represents higher perceived complexity of the program. A sample item from Part B is “The Kumusta Kabayan program would be hard for Filipino migrant workers to understand.” The Cronbach’s alpha of this measure (Part B) was 0.74. These reliability coefficients around 0.70 are considered moderately reliable ( 34 ). The last question, level of success, asked the participants to choose one of eight continuum options based on their experience and observation. It started from Option 1 (The Kumusta Kabayan program not only failed to meet its goals, but it caused a loss of resources and create other problems) to complete successful perception of Option 8 (The Kumusta Kabayan program fulfilled all of its goals and also provided other benefits for Filipino migrant workers and/or for other stakeholders).

For the qualitative measure, all e-helpers followed the interview guide ( Supplementary Material ) that was developed for this study. The guideline was pilot tested among e-helpers and with three participants. During the interviews, e-helpers introduced themselves and asked for participants’ verbal consent. Consented participants were asked about the following themes: referral channel and initial thoughts about the program, period of use, lessons learnt, key facilitators and barriers, and improvement suggestions for the program.

Data Analysis

Results from the online survey were descriptively analyzed. The inferential analysis was not conducted due to the limited number of responses and demographic variables of online survey participants were not collected to protect participants’ anonymity. Data from the research team in Macao and the Philippines were visually compared to show any contrast or similarity between the stakeholders in evaluating the program.

Interview audio-recordings were transcribed by the e-helper who did the interview within 2 weeks from the interview then sent to the clinical supervisor and research coordinators. Following a Rapid Evaluation and Assessment Method ( 35 ) that was used in a similar evaluation study ( 36 ), the lead author with a doctoral degree in psychology and mixed-method research experience read each transcript and summarized the information into a table for each participant using a deductive content analysis approach (i.e., promotional channel and initial expectation). This summary table was sent to other research team members and clinical supervisors for iterative analysis and discussion using an inductive content analysis method, in which the interpretation focused on the explicit data given by the participants ( 37 , 38 ). Steps in this method included open coding, creating categories, and abstraction for each aspect. Data saturation was reached in the interview with the 14th participant where the responses shared similarity with previous responses. The Consolidated Criteria for Reporting Qualitative Studies (COREQ) ( 39 ) was used as a guideline for the qualitative portion of the study.

Reflexivity

This study was part of an ongoing implementation study that involved research team members from multiple sectors and countries that might have different perspectives and cultural backgrounds. Therefore, at the beginning of the study, regular intensive meetings were conducted to build mutual understanding and improve transparency, including the challenges that might be faced and alternative solutions. Along the way, regular meetings were also conducted among e-helpers, clinical supervisors, and research coordinators to discuss any experiences and reflective thoughts during the interviews. The research coordinators emphasized that all opinions shared within the meeting were valid and important to encourage members to share both positive and negative experiences. The e-helpers also had their regular meetings with their clinical supervisor, in which the information in the minutes could be anonymized when necessary so they could openly share their thoughts without worry of being judged. The same step was also conducted when collecting quantitative feedback by using an anonymous online survey.

Findings were presented in the two sub-sections below. The quantitative results from local NGO staff members and research team members in the Philippines ( N = 12) cover organizational climate, awareness and concern, the app’s advantage and complexity, and level of success. Findings from the interviews with OFWs in Macao ( N = 25) cover six aspects from promotional channel and expectation to suggestions.

Online Evaluation of Local and Overseas Partner Member Perspectives

Table 1 displays the quantitative findings from the local NGO partner and overseas research team members. The median score of organizational climates from the teams in Macao and the Philippines are 4 and 5, respectively. The local partner NGO staff members and overseas members strongly perceived that their organization could positively incorporate Kumusta Kabayan into its program. However, staff members in Macao indicated their worry on infrastructure support, with a relatively lower score on this sub-scale than on other sub-scales and when compared with e-helpers in the Philippines. On the awareness and concern, both groups were highly aware of the app’s purpose, with score median of 4 in Macao and the Philippines. Both groups agreed that Kumusta Kabayan would be helpful for the OFWs. However, their perception toward the complexities of the app was ambiguous. For level of success, the teams in Macao and the Philippines generally evaluated that Kumusta Kabayan had achieved its goal.

Characteristics of participants in online survey.

A = From 1 (strongly disagree) to 5 (strongly agree); B = From 1 (not at all true) to 4 (very true).

Interviews With Overseas Filipino Workers in Macao

Findings from the interviews with the completers and non-completers are organized into six aspects of the participants’ experience, including (1) promotional channel and expectation; (2) when to use and in what language; (3) lessons learnt; (4) memorable aspects; (5) key facilitators and barriers; and (6) suggestions. Relatable quote(s), regardless of participants’ completion status, were provided for each aspect with information of participant’s number at the end of the quote.

Promotional Channel and Expectation

Participants learned about the program from several promotional channels including social media like Facebook and flyers that were distributed in churches and restaurants where OFWs gather on weekends ( Supplementary Material ). Some of them read the news about about Kumusta Kabayan’s launching and talked about it with friends. Participants were also informed about the program from The Philippines Consulate in Macau and the local NGO partner, Caritas Macau. Participants commonly expected that the program could assist them in managing emotions, sharing experiences, or connecting with other OFWs, particularly because they recognized the name of the program in Tagalog. However, some participants who learned about the program from the Consulate thought that Kumusta Kabayan was part of the employment and visa process or it would provide monetary aid.

“I just knew about this application from my friends here in Macao so I installed it like them. My first impression about the Kumusta Kabayan is that it would be helpful for us, our fellow Filipinos to check up on each other, to check up on how they are doing. At first, I really do not know. But, while reading the questionnaires, it is really related to the lives of the OFW and also to our personal life, so in a way it touches us .” (Participant #94).

When to Use and in Which Language

Participants who completed the program typically used the app in the morning before work, sometimes in the afternoon, or at night before sleeping, depending on their free time. They used the mobile app almost every day to check for new content or use the mood tracker. Participants who partially completed or discontinued the program used the app only on their day-off on weekends. However, during weekends, they still had difficulties finding time to use the app as they were also busy with extra works or were too tired. Both English and Filipino language versions were used by participants. Some words related to psychological questions were reported hard to understand, especially in English.

“ Sometimes I was able to open it but I did not continue answering … Recently I wasn’t able to use it because I got busy with my work. … When I answer it, when I want to finish the questionnaire, there are times that I wasn’t able to finish when my boss calls me, and also when their child [who I take care of] needs me. I answer it during my rest hour. When the child is sleeping, that’s when I open it. ” (Participant #28).

“ English, for me it’s easy. But, I can’t speak for other. Ay Ineng [young girl], I don’t want to sound overconfident. But, I somehow understand the questions since I’m also a college graduate. That means I understand it. But, for those that didn’t aim a higher degree, they may not be able to answer it well. I’m not generalizing. But, in my opinion, the question and the English are okay. It’s easier for me to understand if it’s in English. ” (Participant #40).

Lessons Learnt

Participants explained some lessons that they learnt from the program. The basic lesson is doing reflective thinking or self-checking and managing their emotions and feelings. Participants learnt how emotions and feelings are connected to their physical state so they also understand why doing exercise is important. When feeling stressed, participants learnt from the app to divide their focus and to do self-care by doing activities they enjoy like cooking or applying breathing exercise. Moreover, they learnt the importance of making, opening themselves to others, and finding support from the local society.

“ Before when I hadn’t installed the Kumusta Kabayan, I was always stressed. I don’t go out of the boarding house. I just go from work to home all the time. But after opening Kumusta Kabayan, it was a huge help for me especially the things I’m reading from the app, including the questions from Kumusta Kabayan. There were times, when I was reading the questions or stories, then suddenly I started crying. I said to myself “This Kumusta Kabayan is such a big help for me because before I’ve forgotten to interact with other people. But this time, I’m doing well. Now I’m able to socialize with other people all because of Kumusta Kabayan. ” (Participant #207).

Memorable Aspects

Participants felt grateful for the program that they never knew existed before joining the study. The name of the program, Kumusta Kabayan, also made it easy for participants to understand the program’s aim. They praised the story and the characters that highly resembled participants’ lives as OFWs. Having someone like them overcome life’s challenges as an OFW made them hopeful and boosted their confidence. The psychological questions encouraged participants to reflect and disclose their feelings to their friends, including things they never told others before. From the content, participants remember breathing exercises that they could apply in their daily lives, especially when they get stressed.

“ What I like about the app are the stories that are really, sometimes it gives me a boost of self-confidence most especially that we live alone and slowly losing hope. But after reading the stories there is still hope. Something like that, we just have to keep fighting. You can do it. It’s just a problem and it will eventually end. Just like the saying that goes in every problem there is a solution. ” (Participant #50).

“ Contents related to my experiences, experiences of an OFW … about missing your family, and also stress. … and there were times when you encountered a misunderstanding with your employer. Something like that, those kinds of situations. I liked the story, I feel excited also. The character [Ate Sonia] is very inspiring too. The activity is very relaxing, especially the exercising part, Inhale-Exhale [breathing exercise]. It helped me relax my mind. ” (Participant #194).

Key Facilitators and Barriers

The main facilitators for successful implementation of Kumusta Kabayan were the accessibility of this app on Play Store and App Store, and the app’s small size. On another side, four key barriers for participants using the app were recognized. First is the time barrier that the majority of participants were migrant domestic workers with only one day-off. Some of them had extra work to do during their day-off and some used it for resting, and they had minimal time to prioritize the app. Second are technological barriers including lack of phone memory to accommodate the app, incompatible operating system, poor signal reception, limited or no mobile data, limited or unstable Wi-Fi, and technical glitch where the app always return to the start and did not record their responses to questionnaires. Third is the lack of the digital literacy where participants changed their phone or formatted it so they had to re-install the app. However, they could not re-install by themselves and needed assistance by research assistants. Fourth are human factors including forgetting their username or password that was case sensitive, changing phone numbers so they could not receive access code when they forgot username or password, and psychological barriers because the questions were triggering of homesickness, as explained by Participant 94 below:

“… while answering the questionnaire, we are also in another country, it feels difficult, you really take in the emotions of each question that you will be answering. It is like you are going back to the time before you leave the Philippines, you are thinking about the past and how can you be better compared before. ” (Participants #94).

Suggestions

Participants suggested several points for the improvement of Kumusta Kabayan, namely (1) connectivity, (2) connectedness, (3) content, (4) feedback or brief explanation on participants’ responses to the questionnaires, (5) add-ons, and (6) promotional methods. First was about connectivity, which they recommended to connect the app with their social media account so they do not need to remember another username and password. Participants also shared their expectation for Kumusta Kabayan that does not require an internet connection because they did not always have mobile data or Wi-Fi. However, on another side, some participants disclosed that their mobile phone memory was full so they had to use the website version. Second was about connectedness, where participants would like to see a video call feature with the e-helper and group chat feature with other participants. This suggestion was motivated by participants’ need to be connected and to share stories with other OFWs and people in their home country, which would ease their homesickness. Third was about the content, which participants suggested to also integrate their positive and uplifting and positive experiences of being OFWs besides the struggles to balance the negative feelings as illustrated below. In addition, participants who came from various occupational backgrounds recommended making several storylines and characters that users could choose base on their type of job. Fourth was about feedback or brief explanation on their responses to the questionnaires so they could understand their condition. Fifth was about add-ons, which participants suggested to add entertainment features and news on the app so they still could use the app while waiting for the next session to be opened. Sixth was about the promotional methods, which participants suggested to involve the OFWs who are also YouTubers or TikTokers to promote Kumusta Kabayan because many OFWs watch these social media celebrities.

“… maybe give many examples of the OFWs’ lives, not only the sad stories. … they should put stories about the bright side of being an OFW so that people will be encouraged to use the application. … because sometimes when the story is sad, the delivery of it is also negative, it should be something nice too. ” (Participant #28).

This study evaluated the stakeholders’ perspectives on the Kumusta Kabayan implementation for OFWs in Macao using a mixed-methods design. Three stakeholders were involved in this study: staff members of local NGO partner, research team members in the Philippines, and OFWs in Macao as the app users. The quantitative measure among NGO staff members and Filipino team members found that Kumusta Kabayan was perceived fit to be integrated into their organizations. In particular, the NGO staff members strongly agreed that Kumusta Kabayan was in line with the NGO’s aim of assisting migrant workers.

On the other hand, infrastructure support was a concern to both NGO staff members and Filipino members. Infrastructure support covered the available organization’s equipment and technology, human resources, and financial situation. Their concerns might be associated with the sustainability of the program as staff numbers in the NGO were limited and Filipino e-helpers were volunteer students. Moreover, the app maintenance fee was relatively high (≈USD $2,367/year) for the NGO.

Generally, the NGO staff members and overseas members perceived that Kumusta Kabayan achieved its goals. From the survey, some Filipino members chose an evaluation option that despite this success, the app caused other problems. This finding might be associated with the difficulty experienced by users when they forgot their passwords. For safety reasons, a one-time-password (OTP) would be sent to user’s phone number registered in the system if user forgot their password. However, some app users changed mobile phone numbers during the program so they could not retrieve the OTP and could not access their account privacy settings. This unfortunately led to users being unable to use the app in some instances. Future iterations of the application should provide easier and more stable access to the application.

Moreover, the OFWs also used a variety of smartphone brands and not all of the operating systems were compatible with the app. Users with inadequate digital literacy could not follow the instructions to check their smartphone configuration nor to download and install the app directly which frustrated them. This finding reflected similar results in Step-by-Step testing among refugees in European countries where technical literacy was the main barrier for them for accessing the app ( 22 ). Therefore, OFWs in the interviews suggested an easier log-in method such as linking it with their social media accounts. However, this recommendation should be thought through carefully so it will not compromise user’s data privacy. An alternative is posting FAQs, including the steps in accessing the app, in the Kumusta Kabayan program’s social media account.

Users also suggested that video calls and group chat features should be incorporated into the app. This suggestion might be driven by social isolation and loneliness among people from communal cultures ( 40 ). Moreover, the mobility restriction during the pandemic amplified the social isolation because migrant workers, particularly the migrant domestic workers, were not allowed by their employers to take the day-off or asked to stay at home during their day-off ( 9 ). This also explains why users were very touched and appreciative of the storyline because it was relatable and educational. This finding highlighted the importance of cultural adaptation as the storyline was developed carefully by involving the OFWs themselves in the adaptation process ( 23 ). However, from the interviews it was found that some users would like to see more characters from different types of jobs because of different challenges faced by these OFWs.

In the interviews we also explored the main barriers for OFWs when using the app, including from the users who discontinued using it. The primary reason was they did not have time to use the app because they were busy with their jobs, which some also did extra work during their day-off. The demographic information showed that our participants could work up to 14 h per day on average. OFWs who worked as domestic workers had difficulty opening the app because they were also taking care of the children or older adults in the family. This finding supported previous study on migrant workers’ health that migrant domestic workers were at high risk for injuries, mental problems, and low quality of life because of long working hours, low wages, and inadequate health insurance ( 41 ). Furthermore, the working condition of OFWs shared by the users also explained their suggestion about entertainment add-ons so they could relax when using the app. Moreover, many users opened the app at night after long working hours and could not concentrate on reading on their screen. Therefore, a format of video or animation could be an alternative in delivering the content for OFWs.

Lastly, the interviews enlightened us about the promotional channels and methods. It was clear that using social media as an online method and face-to-face promotion at church and restaurants worked well in recruiting potential users. Moreover, collaboration with the local NGO that was well-known for its programs in assisting migrant workers enhanced the trust of OFWs for the app. However, when the app was launched and covered by the mass media, the local people showed divided attitude toward this program as some viewers commented that the citizens should be taken care more than the foreigners [i.e., ( 42 )]. This resistance and sentiment toward migrant workers confirmed a previous study on fear and disgust toward migrant workers, particularly domestic workers, as their professions were perceived as low-end 3D jobs (dangerous, difficult, and dirty) ( 43 ). The insight emerged from the interviews was to collaborate with OFWs who were popular YouTubers and TikTokers to promote the app on their channels. Also, the interviews revealed that collaborations with the Government’s representatives either from the host country or sender country, was like a double-edged sword because it could reach many OFWs but not all of them trust these entities and were thinking that the app was a surveillance program.

Implications for Practice and Future Studies

The findings from this study could be extended to the improvement of mobile mental health apps implementation, particularly for migrant workers. Moreover, lessons learnt from this evaluation could also be useful to be implemented in wider digital mental health services for other group of users. For example, the very appreciated storyline and characters were designed from a cultural adaptation process, which also being a critical step for in designing digital health intervention for Chinese young adults ( 44 ). The local partner and Filipino e-helpers contributed substantially to the successful implementation of Kumusta Kabayan. Therefore, it is also important to involve the stakeholders at the earliest possible time during the planning phase, to minimize the potential barriers in implementing digital mental health. Also, it is crucial to assess the digital literacy level of potential users when designing the app as having a smartphone does not guarantee that the owner could use it easily.

Another practical implication, the users’ recommendations of adding additional features including feedback on questionnaires, video chat, group chat, entertainment, and news could be integrated with gamification principles to improve user engagement and increase retention ( 45 ). For example, an app user might creatively design an avatar for their profile picture that would increase the connectedness to the app. Leaderboard and progress bar ( 46 ) could also be introduced to encourage users to complete the challenges during the sessions such as planning and executing the social activities as a part of behavioral activation intervention. Users might collect virtual coins from completing activities or just from opening the app that these coins can be used to access the suggested add-ons. Future studies could investigate the effectivity of these gamification principles in mobile phone app for migrant workers. Additionally, a future study could explore the virtual connections among app users within the app online community using a digital ethnography method ( 47 ).

Limitations

This study has two limitations. First, despite its anonymity, evaluation from the local and overseas partners relied on self-report questionnaires that might lead to social desirability. Second, despite data saturation and engagement with discontinued users, evaluation from the OFWs might be affected by self-selection bias as disappointed users might ignore the interview invitation so their perspective was less represented. Regardless of these limitations, the quantitative and qualitative findings in our study were complimentary of one another and provided key directions to improve subsequent implementation of Kumusta Kabayan among OFWs in Macao and elsewhere.

This study reported the implementation evaluation of Kumusta Kabayan, a mobile mental health app for OFWs, from the perspective of stakeholders including staff members of local NGO as a collaborator, team members in the Philippines, and OFWs in Macao as the app users. The Kumusta Kabayan program was accepted by the users because the character’s story highly resonated with their lives as OFWs and the app was relatively easy to use. The Kumusta Kabayan has the potential of being integrated into the existing support services for migrant workers in Macao by collaborating with the local NGO stakeholder. This app could be scaled-up for OFWs in other countries by paying attention to the key facilitators, barriers, and the suggestions shared by the users that reported in this study.

Data Availability Statement

Ethics statement.

The studies involving human participants were reviewed and approved by the ethical approval for the study was granted from the Research Ethics Committee at the University of Macau (SSHRE19-APP074-FSS). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

AL, HS, and BH: study conceptualization and design. AIFL and BH: secured funding. AL, AIFL, BH, HS, KP, MG, and SB: data collection and final draft writing, review, and editing. AL, KP, and MG: data analysis. AL: draft manuscript writing. All authors contributed to the article and approved the submitted version.

Conflict of Interest

BH is co-editing the Frontiers special topic “Global mental health among marginalized communities in pandemic emergencies” but was not directly involved in the review of this article. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank Caritas Macau, the Macau Foundation, Centre for Macau Studies – University of Macau, and the Philippine Consulate General in Macau (SAR) for their support and partnership in this research; also Hybrid Heroes, the application developer team for the technical assistance and research assistants at the Global and Community Mental Health Research Group for their support in the research and team members from De La Salle University, particularly Professor Allan B. Bernardo, Alma Grace De Vera, Andrae Joseph Yap, Angelica Julienne Tan, Genevive Koa Chua, Jamilla Marie B. Luces, Jesus Antonio D. Marcelo, and Michaella Lobaton. We also thank the overseas Filipino migrant workers in Macao for their time, effort, and guidance in the adaptation of Step-by-Step.

BH and AIFL were supported by the Research Services and Knowledge Transfer Office (RSKTO), University of Macau and the Macau Foundation. Funding for this work was provided by the Li Ka Shing Foundation.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2022.836156/full#supplementary-material

Climate and Mental Health Award: Uncovering mechanisms between heat and mental health

This award will fund projects to advance our understanding of how heat impacts anxiety, depression and psychosis in the most impacted groups globally through biological, psychological and/or social mechanisms. 

Successful applications will propose translational opportunities for climate-resilient solutions and/or mental health interventions.

• Share with Facebook
• Share with X
• Share with LinkedIn
• Share with Email

£1 - 3 million per project

3 - 5 years

Upcoming application stage

Who can apply  .

To apply for this award, teams must bring together mental health and climate expertise as well as additional expertise as relevant. Consistent with our approach to mental health research, we expect research teams to include relevant lived experience of mental health problems, unless there is a strong justification for not doing so. We are open to any methods of involvement that teams choose. However, it is expected that lived experience expertise is involved in the most appropriate and ethical ways to inform multiple aspects and stages of the proposed research project.

Review our guidance on how to involve people with lived experience of mental health challenges in your research .

Is your research right for this call?  

We expect proposed research projects to:

• focus on heat as a stressor of relevance to climate change
• focus on anxiety, depression and/or psychosis
• focus on mechanisms underpinning the relationship between heat and mental health
• consider and clearly describe the potential impact of the proposed project and how, if successful, it would contribute to translational work (either directly or over time) supporting real-world application
• include mental health expertise as well as climate expertise

We expect applications to:

• include relevant lived experience expertise of anxiety, depression and/or psychosis on the project team, unless there is a strong justification for not doing so
• appropriately reference evidence for the association between heat and the mental health outcome of choice
• focus on communities most affected by heat and/or mental health challenges
• propose methods that will uncover causal insights concerning the mechanisms underpinning the relationship between heat and mental health

Projects with human participants must use, as a minimum, one or more of our recommended common measures in mental health research in the collection of new data. Teams may also collect data using any other measure(s).

Is your organisation right for this call?  

Your administering organisation can be a:

• higher education institution
• research institute
• non-academic healthcare organisation
• not-for-profit or non-governmental research organisation

Coapplicants and collaborators can be based at commercial organisations.  

Read about the different applicant roles at Wellcome .

Key dates  

Application process timeline.

You must submit your application by 17:00 GMT on the deadline day. We don’t accept late applications.

Full details of the award are published and the call opens to applications

Application deadline, sign up to our research newsletter.

Stay up to date with funding news, policy changes and events with our monthly email.

Learn about  all our newsletters and subscribe .

By clicking subscribe, you agree to receive this newsletter. You can unsubscribe any time. For information about how we handle your data, please read our privacy notice .

• DOI: 10.55041/ijsrem29138
• Corpus ID: 268429998

An Investigation on Students Learnings & Mental Health Influenced by Artificial Intelligence

• Ijsrem Journal
• Published in INTERANTIONAL JOURNAL OF… 7 March 2024
• Computer Science, Education, Psychology

Related Papers

Showing 1 through 3 of 0 Related Papers

Breezy Mental Health 17+

Breezy mental health, llc, iphone screenshots, description.

Introducing the Breezy Mental Health, LLC Application - Your hub for scheduling appointments, messaging your provider, reviewing medications and lab results, receiving updates, accessing visit summaries, and more!

App Privacy

The developer, Breezy Mental Health, LLC , indicated that the app’s privacy practices may include handling of data as described below. For more information, see the developer’s privacy policy .

Data Not Collected

The developer does not collect any data from this app.

Privacy practices may vary, for example, based on the features you use or your age. Learn More

Information

• App Support
• Privacy Policy

You Might Also Like

HIV In Children

Psyter Provider

Pathways To Care

Pediatric Facts Made Quick

Copyright © 2024 Apple Inc. All rights reserved.