Introduction

The COVID-19 pandemic and lockdown have a major impact on the psychological and social wellbeing of individuals (Rajkumar, 2020). Especially women in the perinatal period may have been affected, since these women are more vulnerable for environmental stressors (Bales et al., 2015) and more often experience elevated levels of distress (Dennis et al., 2017; Woody et al., 2017), which is a risk factor for maternal mental health problems (Dayan et al., 2010). During the COVID-19 pandemic, pregnant and postpartum women showed elevated levels of depressive and anxiety symptoms, compared to similar pre-pandemic groups (Ceulemans et al., 2020; Davenport et al., 2020; Lebel et al., 2020; Wu et al., 2020). Moreover, pregnant women showed an increase in pregnancy-specific distress symptoms (Boekhorst et al., 2020a, b). Pregnant women expressed more concerns about the health of their unborn baby as well as more fear, sadness and uncertainty regarding their childbirth during the COVID-19 pandemic (Ravaldi et al., 2020). Another study showed that almost half of the participating pregnant women reported high anxiety for vertical transmission of COVID-19 (Saccone et al., 2020). Moreover, the COVID-19 pandemic was associated with a decrease in mother-infant bonding at one month postpartum (Suzuki, 2020).

It seems likely that the lockdown restrictions regarding pregnancy care, work and social activities have affected women in the perinatal period in various other ways as well. It is important to gain insight into the specific aspects of stressors of the COVID-19 pandemic affecting the mental health status in the perinatal period, and to measure these stressors effectively. Therefore, the aim of the current study was to develop a COVID-19 Perinatal Perception Questionnaire (COVID19-PPQ), using input by in-depth interviews of pregnant women, women who gave birth recently and perinatal healthcare professionals.

Method

Participants and Procedure

The current study is part of the Brabant Study, a large longitudinal cohort study following women from 12 weeks of pregnancy until eight to ten weeks postpartum (see Meems et al. (2020) for the design of the study). The study was approved by the Medical Ethics Committee at the Máxima Medical Centre Veldhoven (L64091.015.17). Written informed consent was obtained from all participants.

In the current study, specific aspects of stressors of the COVID-19 pandemic in the perinatal period, were assessed by telephone interviews with a selection of participants from the Brabant Study. Interviews were conducted during the first national lockdown period with nine primiparous and ten multiparous pregnant women between 12 and 28 weeks of pregnancy, and seven primiparous and eight multiparous women who recently gave birth. These interviews took place from 21 April until 20 May 2020. Moreover, telephone interviews were conducted with healthcare professionals (five community midwifes, three clinical midwives, five obstetricians and three maternity nurses) from 14 May until 27 May 2020. All interviews were conducted by researchers from the university. The interviews were recorded and transcribed by research assistants. Subsequently, two researchers independently identified important themes from the interviews with the Brabant Study participants, and two other researchers identified themes from the interviews with the healthcare professionals. Thereafter, possible candidate items for the questionnaire were formulated per theme and discussed by these four researchers and supervised and confirmed by a fifth researcher. Double and less relevant items were eliminated until a consensus was reached. This resulted in 27 candidate items for the pregnancy scale and 21 candidate items for the postpartum scale, assessing the perinatal COVID-19-related stress perception. Items were formatted on a four-point Likert scale, ranging from 1 (‘completely disagree’) to 4 (‘fully agree’).

In July and August 2020, 154 pregnant women (sample Ia) and 90 postpartum women (sample Ib) completed the 27-item pregnancy scale and the 21-item postpartum scale, respectively. Data of sample I were used to conduct an exploratory factor analysis (EFA) and reliability analysis. This resulted in a more refined version of the COVID19-PPQ. This refined version was filled out by 113 pregnant women (sample IIa) and 81 postpartum women (sample IIb) in September and October 2020. Data of this sample II were used to perform a confirmatory factor analysis (CFA). Except for gestational age in sample Ia and IIa, the women in sample I and II had similar characteristics and therefore the data of both samples was subsequently taken together to examine the concurrent validity.

Measures

Depressive Symptoms

Depressive symptoms during pregnancy (12, 20, 28 weeks) and postpartum (8–10 weeks) were assessed using the Dutch version of the ten-item Edinburgh (Postpartum) Depression Scale (E(P)DS) (Bergink et al., 2011; Cox et al., 1987; Pop et al., 1992). The total score ranges from 0 to 30 and a higher score indicates more depressive symptoms. The E(P)DS has been shown to be a valid and reliable instrument to measure depressive symptoms in each trimester of pregnancy (Bergink et al., 2011) and postpartum (Pop et al., 1992). In the current study, the Cronbach’s alpha’s were .85, .84 and .86 per trimester, respectively and .83 postpartum.

Pregnancy-Specific Distress

Measurement of pregnancy-specific distress symptoms was carried out using the negative affect subscale (TPDS-NA, ten-item adapted version) of the Tilburg Pregnancy Distress Scale (TPDS) (Pop et al., 2011). Total scores range from 0 to 30 and higher scores indicate greater levels of pregnancy-specific distress. The TPDS-NA shows good psychometric properties in each trimester of pregnancy (Boekhorst et al., 2020a, b), and the internal consistency and structural validity have been evaluated as excellent (Evans et al., 2015). In the current study, the Cronbach’s alpha’s were .76, .81 and .76, per trimester, respectively.

Postpartum Anxiety Symptoms

The ten-item anxiety subscale of the Dutch version of the Symptom Checklist (SCL-90) was used to assess anxiety symptoms postpartum (8–10 weeks) (Arrindell & Ettema, 1981). Total scores range from 0 to 40 and higher scores reflect more anxiety symptoms. The SCL-90 has shown to have good reliability and validity (Arrindell & Ettema, 2003). The Cronbach’s alpha in the current study was .86.

Descriptive Characteristics

Demographic, psychological and obstetric parameters were obtained at baseline in the first trimester of pregnancy, such as age, high level of education (Bachelor’s or Master’s degree), paid job, living with a partner, previous diagnosis of depression and/or anxiety disorder, parity, previous miscarriage, problems with previous pregnancy and/or delivery.

Statistical Methods

Statistical analyses were performed using the Statistical Package for Social Sciences (SPSS version 24, IBM, Chicago IL, USA). CFA was completed using AMOS (version 24, IBM, Chicago, IL, USA). EFA was performed on the 27-item pregnancy scale and the 21-item postpartum scale in sample I. We used a principal component analysis and scree plot to select factors for retention. Item factor loadings > .40 were regarded to be important. When items loaded on more than one dimension, we retained the item when the difference was at least .20, unless face validity and reliability suggested otherwise. Internal consistency analyses were conducted using Cronbach’s alpha for the total pregnancy and postpartum scale and possible subscales derived from factor analysis.

Thereafter, CFA was performed in sample II on the remaining items from the refined version of the pregnancy and postpartum scale of the COVID19-PPQ. CFA was used to test the model fit of the factor structures found with EFA, assessing the comparative fit index (CFI), normed fit index (NFI), Tucker-Lewis Index (TLI), and the root mean square error of approximation (RMSEA) with lower bound. Adequate model fit can be assumed with a CFI ≥ .80, NFI ≥ .80, TLI ≥ .80, and RMSEA ≤ .08 for adequate and RMSEA ≤ .05 for excellent fit with an appropriate lower bound set at .04 (Browne & Cudeck, 1992; Hu & Bentler, 1998).

To assess concurrent validity of the pregnancy and postpartum scale of the COVID19-PPQ, we correlated the pregnancy scale with the EDS and TPDS-NA in each trimester of pregnancy and the postpartum scale with the EPDS and SCL-90 (Pearson’s r correlations, two-tailed).

Results

Factor Analyses

COVID19-PPQ: Pregnancy Scale

The women participating in samples Ia and IIa had similar characteristics except gestational age (Table 1a). All item scores were normally distributed in sample Ia. See Table 2 for an overview of items. Based on face validity, items 16, 19 and 21 were eliminated. The Kaiser–Meyer–Olkin index was > .60 (.73) with a significant Bartlett’s test of sphericity (p < .001). A three-factor solution was suggested in the scree plot, with a ‘risk of infection’ factor, ‘contact’ factor and ‘future’ factor, explaining 38.6% of the variance (Table 2). A varimax rotation was used since the component correlations between the three factors were < .30 (.28) with direct oblimin rotation. Four items (1, 6, 25 and 26) did not have a loading above .40 and five items (4, 7, 9, 18 and 20) loaded on both factors with a difference smaller than .20 and were therefore omitted. Items 11 and 15 were retained because of face validity and reliability, while item 8 was omitted because of face validity. Reliability analyses resulted in a Cronbach’s alpha of .74 for the four-item risk of infection subscale and a Cronbach’s alpha of .72 for the six-item contact subscale. The four-item future subscale had a Cronbach’s alpha of .70, which increased to .71 after deletion of item 24. The total 13-item pregnancy scale of the COVID19-PPQ had a Cronbach’s alpha of .76.

Table 1 Characteristics of two samples of pregnant women (N = 267) and two samples of postpartum women (N = 171)
Full size table
Table 2 Three-factor solution from explorative factor analysis (EFA) with varimax rotation in 154 (sample Ia) pregnant women who completed the 27-item pregnancy scale of the COVID19-PPQ
Full size table

When we performed a CFA on the pregnancy scale in the second pregnancy sample (IIa) with the items and dimensions of the EFA of the first sample (Ia), we found a poor model fit (CFI = .75, NFI = .68, TLI = .69, RMSEA = .13, lower bound = .11). The standardized residual covariance matrix clearly showed that items 2, 11, 14 and 15 showed relationships that were not well explained by the model and were therefore removed. When we repeated the CFA without these items we found an adequate model fit (CFI = .95, NFI = .88, TLI = .92, RMSEA = .07, lower bound = .03). Then, EFA with varimax rotation was performed on the nine-item pregnancy scale in sample IIa for confirmation, which resulted in a three factor structure with a total explained variance of 67.5%. The three-item risk of infection subscale had a Cronbach’s alpha of .74, while the three-item contact and three-item future subscales both had a Cronbach’s alpha of .71. The total nine-item pregnancy scale showed a Cronbach’s alpha of .71. The items were recoded from 1–4 to 0–3, with total scores ranging from 0 to 27. Higher scores indicated a more negative COVID-19-related stress perception during pregnancy.

COVID19-PPQ: Postpartum Scale

Women in samples Ib and IIb had similar characteristics (Table 1b). All item scores were normally distributed in sample Ib. A Kaiser–Meyer–Olkin index greater than .60 (.76) was found and the Bartlett’s test of sphericity value was significant (p < .001). The scree plot suggested a three-factor solution with 47.3% total explained variance, with a ‘first postpartum week’ factor, ‘COVID-19 measures’ factor and ‘fear for infection’ factor (Table 3). The component correlations between the three factors were smaller than .30 (.18) with direct oblimin rotation and therefore varimax rotation was used. Three items (2, 11 and 15) did not have a loading > .40 and four items (7, 16, 19 and 21) loaded on both factors with a difference < .20 and were therefore eliminated. Reliability analyses showed a Cronbach’s alpha of .77 for the six-item first postpartum week subscale, which increased to .81 after omission of item 1. A Cronbach’s alpha of .77 was found for the five-item COVID-19 measures subscale and a Cronbach’s alpha of .66 for the three-item fear for infection subscale, which increased to .84 after elimination of item 20. The total 12-item postpartum scale of the COVID19-PPQ had a Cronbach’s alpha of .78.

Table 3 Three-factor solution from explorative factor analysis (EFA) with varimax rotation in 90 (sample Ib) postpartum women who completed the 21-item postpartum scale of the COVID19-PPQ
Full size table

Several women that completed the first version of the postpartum scale mentioned that they missed a question on fear for infection of their partner. Therefore, we included an extra question in version two of the scale: “I was afraid that my partner would be infected with the Coronavirus during the delivery/first postpartum week”. CFA was then conducted on the 13-item second version of the postpartum scale in sample IIb and resulted in an inadequate model fit (CFI = .77, NFI = .72, TLI = .75, RMSEA = .09, lower bound = .06). The standardized residual covariance matrix clearly showed that items 5, 10 and 12 showed relationships that were not well explained by the model. These items were subsequently removed and when we repeated the CFA on the remaining ten items we found an excellent model fit (CFI = .98, NFI = .92, TLI = .97, RMSEA = .03, lower bound = .01). We repeated EFA with varimax rotation on the ten-item postpartum scale in sample IIb, for confirmation. This resulted in a three factor structure, explaining 72.1% of the variance. Reliability analyses showed a Cronbach’s alpha of .83 for the postpartum subscale, .77 for the COVID-19 measures subscale, .87 for the fear for infection subscale, and .64 for the total ten-item postpartum scale. The items were recoded from 1–4 to 0–3, and the items of the first postpartum week and COVID-19 measures subscale were reverse coded. Total scores ranged from 0 to 30 with higher scores indicating a more negative COVID-19-related stress perception postpartum. Table 4 shows the pregnancy and postpartum scale of the COVID19-PPQ.

Table 4 The COVID-19 Perinatal Perception Questionnaire (COVID19-PPQ)
Full size table

Concurrent Validity Analyses

Sample I and II were combined for concurrent validity with the E(P)DS, TPDS-NA and SCL-90. We used multiple imputation with ten iterations on the missing values in sample IIa, corresponding to the item that was included in the postpartum scale after EFA. The final ten items of the postpartum scale were included in the imputation process. A normal distribution was shown for all items in the pregnancy and postpartum scale. The Pearson correlations between the pregnancy scale and the EDS and TPDS-NA in each trimester of pregnancy are shown in Table 5a. In sum, the EDS correlated significantly with most pregnancy (sub)scale(s) with small effect size, while the coefficients with the TPDS-NA showed small to medium effect sizes. Table 5b shows that the postpartum scale and the COVID-19 measures and fear for infection subscale were significantly associated with the EPDS, as well as with the SCL-90, all with small effect sizes.

Table 5 Correlation coefficients between the pregnancy scale of the COVID19-PPQ and symptoms of depression and pregnancy-specific distress at each trimester of pregnancy (N = 267) and between the postpartum scale of the COVID19-PPQ and symptoms of postpartum depression and anxiety (N = 171)
Full size table

Discussion

In the current study, the COVID19-PPQ was developed during the first national lockdown period based on input from pregnant women, new mothers and perinatal healthcare professionals. The questionnaire consists of an (a) pregnancy and (b) postpartum scale. We found adequate psychometric properties for both scales: a three-factor structure for the pregnancy scale with appropriate reliability and adequate model fit, and a three-factor structure for the postpartum scale with appropriate reliability and excellent model fit. It must be noted that the Cronbach’s alpha of the total postpartum scale was .64, while a Cronbach’s alpha of > .70 has been recommended (Kline, 1993). This could be due to the reverse coded items of the first postpartum week and COVID-19 measures subscale, especially since the reliability scores of the three subscales were all > .77. This could imply that using the subscale scores instead of the total postpartum scale score may be preferable.

The pregnancy scale consists of three subscales. Items of the risk of infection subscale refer to worries about infection and risk of COVID-19 during pregnancy. The contact subscale contains items involving concerns about cancellations of ultrasounds, ultrasounds without partner and lack of visits from family and friends during the first postpartum week. Items of the future subscale entail worries about finances and work. All subscales appear to have a concern or worry component. Indeed, the total pregnancy scale and its subscales were significantly correlated with pregnancy specific distress symptoms with medium effects size. In addition, there was a significant correlation between the pregnancy (sub)scale(s) and symptoms of depression, but with small effect size.

The postpartum scale contains three subscales, with items of the first postpartum week subscale referring to perception of COVID-19-related changes in the first postpartum week, especially due to fewer visits from family and friends. The COVID-19 measures subscale contains items regarding the perception of the measures and guidelines during delivery. Items in the fear for infection subscale include worries to get infected (self, baby or partner) with COVID-19 during delivery or in the first postpartum week. The COVID-19 measures and fear for infection subscale showed a significant correlation with symptoms of postpartum depression and anxiety with small effect size, and the first postpartum week subscale showed no significant associations. This could imply that, in contrast to pregnant women, there were certain stressors of the COVID-19 pandemic in postpartum women that did not have an association with elevated levels of depression and anxiety. Another explanation could be that there were lower levels of stressors postpartum. Moreover, when looking at effect sizes, perinatal stressors of the COVID-19 pandemic seemed to have a greater association with distress symptoms in pregnant women than in postpartum women.

The concept of the COVID19-PPQ could be clinically relevant in measuring specific COVID-19 stressors that could be related to increased depression and anxiety symptoms in perinatal women in times of COVID-19 (Ceulemans et al., 2020; Davenport et al., 2020; Lebel et al., 2020; Wu et al., 2020). Knowing on which subscale(s) of the pregnancy and postpartum scale perinatal women show higher levels of stressors, could help healthcare professionals in offering proper support to perinatal women during the COVID-19 pandemic. Moreover, the stressors that were identified in the current study could be valuable in developing interventions for perinatal women in the current and future pandemic episodes. A meta-analysis stressed the need for such interventions based on evidence from past epidemics and pandemics (Shorey & Chan, 2020).

The development of the COVID19-PPQ was based on in-depth interviews with perinatal women and healthcare professionals during the first lockdown period, which is a major strength of the current study. A limitation is that compared to the national population, all participants were Dutch, more often highly educated and living with a partner, which could limit generalization of our findings. Therefore, the psychometric properties of the COVID19-PPQ should be re-evaluated in samples with more diversity in ethnicity, education level and type of household.

In conclusion, the COVID19-PPQ seems to be a user-friendly and valid instrument for assessment of perinatal COVID-19-related stress perception, with adequate psychometric properties for both the pregnancy and postpartum scale. Future research should examine the use of this instrument in clinical practice during new episodes of the COVID-19 pandemic.