Comparison of mental health symptoms before and during the covid-19 pandemic: evidence from a systematic review and meta-analysis of 134 cohorts

Objective To synthesise results of mental health outcomes in cohorts before and during the covid-19 pandemic.

Design Systematic review.

Data sources Medline, PsycINFO, CINAHL, Embase, Web of Science, China National Knowledge Infrastructure, Wanfang, medRxiv, and Open Science Framework Preprints.

Eligibility criteria for selecting studies Studies comparing general mental health, anxiety symptoms, or depression symptoms assessed from 1 January 2020 or later with outcomes collected from 1 January 2018 to 31 December 2019 in any population, and comprising ≥90{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} of the same participants before and during the covid-19 pandemic or using statistical methods to account for missing data. Restricted maximum likelihood random effects meta-analyses (worse covid-19 outcomes representing positive change) were performed. Risk of bias was assessed using an adapted Joanna Briggs Institute Checklist for Prevalence Studies.

Results As of 11 April 2022, 94 411 unique titles and abstracts including 137 unique studies from 134 cohorts were reviewed. Most of the studies were from high income (n=105, 77{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) or upper middle income (n=28, 20{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) countries. Among general population studies, no changes were found for general mental health (standardised mean difference (SMD)_change 0.11, 95{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} confidence interval −0.00 to 0.22) or anxiety symptoms (0.05, −0.04 to 0.13), but depression symptoms worsened minimally (0.12, 0.01 to 0.24). Among women or female participants, general mental health (0.22, 0.08 to 0.35), anxiety symptoms (0.20, 0.12 to 0.29), and depression symptoms (0.22, 0.05 to 0.40) worsened by minimal to small amounts. In 27 other analyses across outcome domains among subgroups other than women or female participants, five analyses suggested that symptoms worsened by minimal or small amounts, and two suggested minimal or small improvements. No other subgroup experienced changes across all outcome domains. In three studies with data from March to April 2020 and late 2020, symptoms were unchanged from pre-covid-19 levels at both assessments or increased initially then returned to pre-covid-19 levels. Substantial heterogeneity and risk of bias were present across analyses.

Conclusions High risk of bias in many studies and substantial heterogeneity suggest caution in interpreting results. Nonetheless, most symptom change estimates for general mental health, anxiety symptoms, and depression symptoms were close to zero and not statistically significant, and significant changes were of minimal to small magnitudes. Small negative changes occurred for women or female participants in all domains. The authors will update the results of this systematic review as more evidence accrues, with study results posted online (https://www.depressd.ca/covid-19-mental-health).

Review registration PROSPERO CRD42020179703.

Eligible studies

We included studies on any population that compared eligible outcomes assessed from 1 January 2018 to 31 December 2019 when China first reported covid-19 to the World Health Organization,18 with the same outcomes collected from 1 January 2020 or later. Studies had to report data from cohorts comprising at least 90{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} of the same participants between pre-covid-19 and pandemic periods or to use statistical methods to account for missing data. We did not include repeated cross sectional surveys or studies with fewer than 100 participants.

Eligible outcomes included continuous scores on validated mental health symptom questionnaires, proportion of participants above a threshold on a validated symptom questionnaire, or proportion of participants meeting criteria for a mental disorder using a validated diagnostic interview. In our ongoing living systematic review, we defined outcomes broadly to include, for example, anxiety symptoms, depression symptoms, general mental health, stress, loneliness, anger, grief, and burnout. In the present report, we included only general mental health (eg, general symptoms, mental health related quality of life), anxiety symptoms, and depression symptoms because few studies reported other outcomes. Results for other outcome domains are available online (https://www.depressd.ca/covid-19-mental-health).

Identification and selection of eligible studies

Using a strategy designed by an experienced health sciences librarian, we searched Medline (Ovid), PsycINFO (Ovid), CINAHL (EBSCO), Embase (Ovid), Web of Science Core Collection: Citation Indexes, China National Knowledge Infrastructure, Wanfang, medRxiv, and Open Science Framework Preprints. Because of the need for rapid evidence early in the pandemic, we did not formally peer review the search strategy; however, covid-19 terms were developed in collaboration with other librarians working on the topic and updated as covid-19 specific subject headings became available (see supplementary material 2). We initially searched from 31 December 2019 to 13 April 2020, then daily until 28 December 2020, and thereafter weekly.

Search results were uploaded into DistillerSR (Evidence Partners, Ottawa, Canada), where we identified and removed duplicate references. Two independent reviewers evaluated titles and abstracts in random order. If either reviewer deemed a study potentially eligible, a full text review was completed—also by two independent reviewers. Any discrepancies were resolved through consensus, with a third investigator consulted as necessary. An inclusion and exclusion coding guide was developed and pre-tested, and team members were trained over several sessions (see supplementary material 3).

Data extraction and synthesis

One reviewer extracted data from each included study using a standardised form in DistillerSR, and a second reviewer validated the data using the DistillerSR Quality Control function. Reviewers extracted publication characteristics (eg, first author, publication year, journal); population characteristics, including eligibility criteria, recruitment method, number of participants, and population group (general population, older adults, young adults, children and adolescents, parents, university students, people with pre-existing medical conditions, people with pre-existing mental health conditions, medical staff, and groups defined by sex or gender in the present report, although we extracted any group for which we found data); mental health outcomes and assessment timing; and adequacy of study methods and reporting. We used World Bank classifications for country income and region.19 To assess risk of bias and adequacy of study methods and reporting (sampling frame, recruitment methods, sample size, setting and participant descriptions, participation or response rate, outcome assessment methods, standardisation of assessments, statistical analyses, follow-up rate), we adapted the Joanna Briggs Institute Checklist for Prevalence Studies20 (see supplementary material 4).

For continuous outcomes, we extracted estimates as the SMD effect size with 95{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} confidence intervals for change from pre-covid-19 to during the pandemic. If such data were not provided, we calculated Hedges g,21 as described elsewhere.22 Positive SMD estimates represent a worsening of mental health and negative estimates represent an improvement. For proportions, we calculated missing 95{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} confidence intervals using Agresti and Coull’s approximate method for binomial proportions.23 For proportion changes, we generated missing 95{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} confidence intervals using Newcombe’s method for differences between binomial proportions based on paired data,24 which requires the number of participants above a threshold to be known at both assessment points. If these data were not available, we assumed that 50{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} of cases above a threshold during pre-covid-19 assessments continued to be above the threshold during the pandemic. We confirmed that results did not differ substantively if we assumed values within a plausible range (30-70{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}).

Owing to pitfalls in interpreting proportions of participants who crossed a dichotomous threshold, we prioritised continuous data (see box 1). For each population group with continuous outcomes for at least two studies in a domain, we pooled SMDs through restricted maximum likelihood random effects meta-analysis. Heterogeneity was assessed using the I² statistic. For studies with more than one continuous outcome in a domain (eg, two depression symptom measures), we pooled SMDs within the study before fitting the meta-analysis.

Box 1

Interpreting standardised mean differences effect sizes and changes in proportion above a threshold on mental health measures

Changes in symptoms assessed with mental health patient reported outcome measures in covid-19 have been reported as changes in continuous scores and as the proportion of study participants above a threshold. Continuously measured symptom changes are presented in terms of standardised mean difference (SMD), which describe change in terms of within group standard deviations rather than raw change scores, which are measure specific and not easily compared across measures. To illustrate, the figure below shows the amount of change, assuming a normal distribution, for a SMD of 0.25. The hypothetical purple distribution represents pre-covid-19 scores and the orange distribution represents post-covid-19 scores, with a mean symptom increase of SMD 0.25. With a threshold located at 1 standard deviation above the pre-covid-19 mean, the proportion of participants above the threshold would change from 16{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} to 23{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}. With a threshold 2 standard deviations above the pre-covid-19 mean, the proportion would change from 2{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} to 4{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}.

When studies report an increase or decrease in the proportion of participants above a measure threshold, dichotomous thresholds used for this purpose are sometimes labelled as thresholds for clinically significant symptoms or as reflecting the presence of a condition, such as depression.7 These designations are not, however, based on evidence that a threshold represents a meaningful divide between impairment and non-impairment and do not reflect the presence of a mental disorder. Most commonly, these designations reflect a point on a measure that balances sensitivity and specificity when used for screening, which does not inform when score levels might become clinically meaningful.7891011

Thresholds on different symptom measures are often located at different places in the symptom distribution. This can lead to divergent estimates of proportions crossing a threshold, depending on the measure used, rather than because of actual differences in symptom changes. As the figure shows, the same change in symptoms in a hypothetical study sample would result in a 7{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} increase in participants at or above the threshold on one measure (first vertical line, 1 standard deviation above pre-covid-19 distribution mean) but an increase of only 2{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} on another (second vertical line, 2 standard deviations above pre-covid-19 distribution mean).

RETURN TO TEXT

Although we have prioritised interpretation of changes in continuous score, in the supplementary material we also report proportions above thresholds, as they can be informative, such as when they are reported for two time points in the same study or as an indicator if some level of change may have occurred. We have, however, avoided interpretation of the magnitudes of proportions above thresholds.

Meta-analyses were performed in R (R version 3.6.3, Rstudio Version 1.2.5042) using the rma.uni function in the metafor package.25 Forest plots were generated using the forest.rma function in metafor. We characterised changes as minimal (SMD <0.20), small (0.20), medium (0.50), or large (0.80).26

Patient and public involvement

Sarah Markham, an experienced patient advisor and member of The BMJ’s international patient panel, was a research team member from project inception. She provided input on the design of the study, underwent training on study procedures, and was involved in selection of eligible studies. She provided comments on the content of this article.

Search results and study selection

As of 11 April 2022 our searches identified 94 411 unique citations. We excluded 92 457 after review of titles and abstracts and 1523 after full text review, leaving 431 studies with longitudinal data. Of those, 276 studies did not include pre-covid-19 data, 11 only assessed outcomes not included in the present report (eg, loneliness), one used the same outcome measure but for different time periods pre-covid-19 (worst month in the past year) and during the pandemic (past month), and six duplicated data from another study, leaving 137 unique studies from 134 cohorts (fig 1).

Fig 1

Flow of studies through review

Characteristics of included studies

Supplementary material 5 provides a more detailed version of study characteristics and outcomes. Supplementary table 1 shows the characteristics of the included studies. All cohort studies reported covid-19 outcome data collected in 2020 (nine studies from Asia in January or February, 125 studies from Asia and elsewhere in March or later). Only three cohort studies reported results from multiple time points in 2020, and one also reported results from 2021. Overall, 105 (77{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) studies were from high income countries, 28 (20{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) studies from upper middle income countries, one (1{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) study from a mix of high income and upper middle income country samples, three (2{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) studies from lower middle income countries, and none from low income countries. Fifty two (38{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) studies were from Europe and Central Asia, 46 (34{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) from East Asia and the Pacific, 28 (20{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) from North America, and 11 (8{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) from other regions. Study populations comprised adults in 104 (76{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) studies; children or adolescents in 30 (24{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) studies, including 27 that focused mostly or entirely on adolescents (10-19 years); and a mixture of children (<9 years) and adolescents in three (2{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) studies. No study focused only on children.

Risk of rias and adequacy of study methods and reporting

Supplementary table 2 shows ratings for risk of bias and adequacy of methods and reporting. Of the 137 included studies, 37 (27{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) used sampling frames that closely represented the target population, 32 (23{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) used census or random sampling methods, 13 (9{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) had response rates of ≥75{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}, and 43 (31{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) successfully followed up with ≥75{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} of participants or included methods to deal with loss to follow-up. “Yes” ratings were between 73{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} and 100{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} for sample size, participant and setting description, valid assessment methods, standard outcome collection methods, and appropriately analysed results.

Change in mental health symptoms

Supplementary tables 3-5 show changes in mental health symptoms for individual studies by population category: supplementary table 3 for general mental health symptoms, supplementary table 4 for anxiety symptoms, and supplementary table 5 for depression symptoms. Table 1 shows the results of meta-analyses. Among the three cohorts in which changes were assessed in March to June 2020 and again in September to November 2020 or in 2021, symptoms from later assessments were stable or improved from early 2020. No meta-analysis results changed in sensitivity analyes (see supplementary material 5).

General mental health

The forest plots in supplementary figures 1a-k show the SMD change for general mental health in studies of general populations and subgroups. The estimated change in the general population cohorts was minimal and not significant (supplementary figure 1a; 11 cohorts, n=30 185; SMD_change 0.11, 95{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} confidence interval −0.00 to 0.22; I²=97{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}). Among subgroups, a small, significant worsening of general mental health was observed for women or female participants (supplementary figure 1b; six cohorts, n=10 329; SMD_change 0.22, 0.08 to 0.35; I²=91{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) and a small to medium significant worsening for parents (supplementary figure 1h; three cohorts, n=932; SMD_change 0.39, 0.21 to 0.56; I²=57{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}). Symptoms improved by a small amount among people with pre-existing mental health conditions (supplementary figure 1j; two cohorts, n=457; SMD_change −0.22, −0.35 to −0.09; I²=0{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}). No other subgroup changes were statistically significantly different from zero. I² across analyses was high (57{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} to 99{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}), except among people with pre-existing mental health conditions (0{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}).

Anxiety symptoms

The forest plots in supplementary figures 2a-l show the SMD change for anxiety symptoms in studies of general populations and subgroups. The estimated change in the general population cohorts was not significant and was close to zero (supplementary figure 2a; four cohorts, n=2632; SMD_change 0.05, −0.04 to 0.13; I²=37{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}). Anxiety symptoms worsened significantly by small amounts among women or female participants (supplementary figure 2b; five cohorts, n=3500; SMD_change 0.20, 0.12 to 0.29; I²=41{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}) and parents (one cohort, n=147; SMD_change 0.25, 0.02 to 0.49). Estimates were non-statistically significant and close to zero for all other subgroups. I² ranged from 0{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} to 41{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} for the general population, women or female participants, and men or male participants but was higher for the other subgroups (80{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} to 98{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}).

Depression symptoms

The forest plots in supplementary figures 3a-m show the change in SMDs for depression symptoms in studies of general populations and subgroups. The estimated change in general population cohorts increased statistically significantly by a minimal amount (supplementary figure 3a; four cohorts, n=3470; SMD_change 0.12, 0.01 to 0.24; I²=81{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}). The estimated change also increased significantly by minimal to small amounts among women or female participants (supplementary figure 3b; seven cohorts, n=3851; SMD_change 0.22, 0.05 to 0.40, I²=89{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}), older adults (supplementary figure 3d; seven cohorts, n=7419; SMD_change 0.22, 0.06 to 0.38, I²=95{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}), university students (supplementary figure 3f; 19 cohorts, n=26 164; SMD_change 0.14, 0.01 to 0.26, I²=98{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}), and people who self-identified as belonging to a sexual or gender minority group (supplementary figure 3k, three cohorts, n=3741; SMD_change 0.19, 0.10 to 0.28; I²=67{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}). The estimated change improved minimally for people with pre-existing mental health conditions (supplementary figure 3j, three cohorts, n=12 352; SMD_change −0.05, −0.08 to −0.03; I²=0{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}). The I² statistic was 0{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} for people with pre-existing mental health conditions and 67{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} to 98{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} in other analyses.

Comparison with other studies

Our finding that mental health was either unchanged or worsened by minimal to small amounts in the general population and subgroups is consistent with results from a more limited systematic review of 65 longitudinal studies from early in the pandemic13 and somewhat smaller than a second systematic review of 43 studies.14 We know of only one study that has evaluated mental disorders using validated diagnostic methods. That study, from Norway,27 which was not eligible for our review, evaluated prevalence of current mental disorders in a series of cross sectional random samples collected from 28 January to 11 March 2020 (n=563, 15.4{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}, 95{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} confidence interval 12.5{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} to 18.8{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}), 12 March to 31 May 2020 (n=691, 9.0{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}, 7.1{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} to 11.4{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}), 1 June to 31 July 2020 (n=530, 14.3{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}, 11.5{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} to 17.5{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}), and 1 August to 18 September 2020 (n=370, 11.9{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}, 9.0{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} to 15.6{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e}). The authors concluded that the prevalence of mental health disorders was stable or slightly decreased across the pandemic.

The largest study on suicide during the pandemic2829 included monthly data from official government sources on suicide occurences from 21 countries from 1 January 2019 or earlier to 31 July 2020 and found no evidence of a statistically significant increase in any country or region; statistically significant decreases did, however, occur in 12 countries or regions.28 Results were similar in a subsequent update with data from 33 countries across the first nine to 15 months of the pandemic.29

We found that women or female participants experienced small negative changes, in aggregate, for general mental health, anxiety symptoms, and depression symptoms during the early part of the pandemic. This finding is consistent with a previous analysis of a subset of studies from the present review with direct within study comparisons between mental health of women or female participants and men or male participants (n=10 unique cohorts).30 That study found statistically significantly, albeit minimally greater, worsening of general mental health and anxiety symptoms among women than among men (SMD_{difference-change} 0.15 for both); changes in depressive symptoms were worse but not statistically significant for women (SMD_{difference-change} 0.12).30

Significant worsening of symptoms among women or female members of the population is of concern. This is an aggregate result that, even though small, suggests that the disproportionate effect of the pandemic on women or female members of the population has influenced mental health. In terms of vulnerabilities, most single parent families tend to be headed by women, and women earn less and are more likely to live in poverty than men.3132333435 Women are also overrepresented in healthcare jobs and provide most family and elder care.31323334 Intimate partner violence towards women increased during the pandemic.36 The small overall change in mental health symptoms suggests that many women have been resilient but that among some an important worsening of symptoms occurred. Indeed, although most of our analyses found no changes or minimal to small negative changes in mental health, they do suggest that the pandemic negatively influenced the mental health of some people, which is consistent with, for example, reports of increased visits for mental health and sustance misuse.3738

Nonetheless, the patterns of findings from our review, along with evidence on mental health disorders and suicide, converge to suggest that the effects of covid-19 on mental health are more nuanced than the “tsunami” descriptor or other similar terms used by some investigators and in many media articles.12 Short news cycles that emphasise bad news,3940 anecdotes, and an uncritical reliance on cross sectional studies and unvalidated, difficult-to-interpret survey tools that inquire about mental health and wellbeing during the pandemic among convenience samples might at least partially explain this discrepancy. Illustrating the pitfalls of interpreting studies that ask questions about emotional reactions to covid-19, separate from mental health, a longitudinal study of 2345 young men from Switzerland found that depression symptoms and stress significantly decreased during covid-19 compared with pre-pandemic levels.41 The study also reported results from a series of unvalidated single items, administered only during the pandemic, that queried about emotional reactions to the pandemic and specifically assigned covid-19 as the cause (eg, “due to covid-19, I experienced . . .”); these items suggested high levels of distress, which became the focus of the study’s conclusions without mention of unchanged mental health indices. Together with the findings from our systematic review, this suggests that many or most people have experienced different aspects of covid-19 as highly unpleasant or distressing, that most people have been resilient, and that population level mental health has not changed by large amounts, although for some it has changed negatively.

Policy implications

The lack of evidence of a large scale decline in mental health so far in the context of covid-19 could be because people are resilient and have made the best of a difficult situation. Indeed, although evidence is limited, data suggest, for example, that suicide generally declines during periods of societal conflict.4243444546 War and pandemics have different characteristics, but in both there is a shared threat and common focus on collective action to tackle that threat.

The minimal changes in mental health that we found could also reflect steps that governments have taken to support mental health. The World Health Organization, other pan-national organisations, and governments across the globe have produced strategies for dealing with mental health and have invested in resources to support public mental health,4748 even in countries where mental health had not been a priority previously.4950 It is not known to what degree these efforts have been effective, but it is possible that government action has played an important role. However, the negative changes that we detected in some groups, particularly for women and female cohorts, early in the pandemic, underline the need for continued surveillance to determine the degree to which negative mental health changes may be ongoing and require additional resources.

In terms of research, the results of our study underline gaps in mental health surveillance across countries. Since early in the pandemic, the need for high quality surveys with appropriately representative probabilistic sampling methods and pre-pandemic data has been emphasised.51 However, we found few examples of mental health surveillance frameworks that generated high quality data based on this type of sampling. Investment in more robust mental health surveillance mechanisms that can be used to identify and deal with mental health needs at all times, including in times of crisis, is needed.

Strengths and limitations of this review

Strengths of our systematic review include using rigorous best practice methods; searching nine databases, including two Chinese databases; no restrictions on language; and the ability to update findings as evidence emerged via our living systematic review approach (see https://www.depressd.ca/covid-19-mental-health). Our systematic review also has limitations that suggest some caution in interpreting results. Firstly, we did not peer review our search strategy given the urgency of generating mental health data early in the pandemic. An experienced librarian did, however, develop the search strategy and develop covid-19 search terms in collaboration with other experienced librarians. We reviewed studies included in other reviews and did not identify any that we missed; to the contrary, in our review we identified many more studies than other similar reviews with overlapping search periods.1314 Secondly, aside from several population level randomly sampled surveys, most studies included in our systematic review had limitations related to study sampling frames and recruitment methods, response and follow-up rates, and missing follow-up data. Thirdly, we did not include repeated cross sectional studies, which may have excluded some studies with useful evidence. Fourthly, heterogeneity was high in most of the meta-analyses that we conducted. Fifthly, only a handful of studies reported results from late 2020, although the few studies that did suggested that symptoms were stable or reduced from earlier in the pandemic. Sixthly, although we were able to synthesise results from several vulnerable groups, including older adults and people with pre-existing medical conditions, there were few studies for other groups, such as people with low socioeconomic status, and there were no studies on children (ages 0-9 years). Similarly, there was little evidence from low income or lower middle income countries or from some areas of the world, such as sub-Saharan Africa. Seventhly, we did not include studies with fewer than 100 participants; however, in a previous covid-19 mental health systematic review with 65 total studies, studies with fewer than 100 participants comprised only 1{fc1509ea675b3874d16a3203a98b9a1bd8da61315181db431b4a7ea1394b614e} of total participants.13 Eighthly, the evidence base is rapidly evolving, and main results could change, although our own living systematic review format will allow updating as this occurs. Finally, we did not assess possible publication bias, although the largely null findings suggest that this was not likely to be an important factor.

Conclusions

We reviewed 137 studies with data from 134 unique cohorts. Across population groups, results suggest that, rather than a mental health crisis, at a population level there has been a high level of resilience during covid-19, and changes in general mental health, anxiety symptoms, and depression symptoms have been minimal to small with no changes detected in most analyses. There were few robust studies with vulnerable groups, however, and it is possible that some population groups experience mental health issues that differ from those of the general population or from other groups. The pandemic and the long term ramifications continue to affect societies across the world, and it will be important to continue to assess mental health. The pandemic has affected the lives of many people, and some are now experiencing mental health difficulties for the first time. Governments should continue to ensure that mental health supports are available and respond to population needs. We will update the results of this systematic review as more evidence accrues, with study results posted online (https://www.depressd.ca/covid-19-mental-health).