Skip to content
BY 4.0 license Open Access Published by De Gruyter April 21, 2022

A systematic review on quality of life (QoL) of patients with peritoneal metastasis (PM) who underwent pressurized intraperitoneal aerosol chemotherapy (PIPAC)

  • Zhenyue Li , Louis Choon Kit Wong , Rehena Sultana , Hui Jun Lim ORCID logo , Joey Wee-Shan Tan ORCID logo , Qiu Xuan Tan ORCID logo , Jolene Si Min Wong ORCID logo , Claramae Shulyn Chia ORCID logo and Chin-Ann Johnny Ong ORCID logo EMAIL logo
From the journal Pleura and Peritoneum

Abstract

Background

Pressurized intraperitoneal aerosol chemotherapy (PIPAC) has recently emerged as a palliative alternative for patients with unresectable peritoneal metastasis (PM). Quality of life (QoL) has increasingly been used as an endpoint to evaluate treatment outcomes. This review aims to identify evidence on how PIPAC would impact the QoL of PM patients.

Content

A systematic review was performed on articles identified from Medline, EMBASE, PsycInfo, and Web of Sciences. A meta-analysis was conducted on further selected studies. ACROBAT-NRSI was attempted to assess the risk of bias (RoB).

Summary

Nine studies using the EORTC QLQ-C30 questionnaire to assess QoL after repeated PIPAC cycles were identified. Majority was found to be moderately biased and a great extent of heterogeneity was observed. Four studies on PM from either gastric cancer (GC) or epithelial ovarian cancer (EOC) were included for meta-analysis. In 31 GC patients and 104 EOC patients, QoL remained stable in 13/14 and 11/14 EORTC QLQ-C30 scales. PIPAC was inferior to cytoreductive surgery with hyperthermic intraperitoneal chemotherapy (CRS/HIPEC) in global QoL and functioning but superior in symptom reduction.

Outlook

PIPAC is a well-tolerated option for most GC and EOC patients with irresectable PM. Future trials are warranted to confirm the findings.

Introduction

Peritoneal metastasis (PM) refers to the formation of peritoneal tumors within the abdominal cavity. It is commonly observed in advanced stages of colorectal carcinoma (CRC), gastric carcinoma (GC), as well as epithelial ovarian carcinoma (EOC). Rarer causes of PM include primary peritoneal cancer, primary mesothelioma, and tumors arising from the small bowel, pancreas, and appendix [1]. Historically, PM was associated with a uniformly dismal prognosis due to delayed diagnosis, aggressive progression, and treatment resistance [1], [2], [3], [4], [5], [6]. Ascites and intestinal obstruction as late-stage symptoms are found in approximately 50% of patients with PM [5]. The various nonspecific complaints of PM patients during their initial presentations make early detection challenging. Surgery is potentially curative but patients are highly selected. Cytoreductive surgery with hyperthermic intraperitoneal chemotherapy (CRS/HIPEC) is the standard of care for PM patients with pseudomyxoma peritonei, peritoneal mesothelioma, and colorectal carcinoma with limited peritoneal spread [7]. More clinical trials are being conducted to investigate its efficacy in gastric and ovarian cancers. However, most patients are not eligible for CRS/HIPEC due to extensive disease not amenable for surgery, poor tumor biology, or lack of surgical fitness [7]. Systemic chemotherapy remains the standard of care for palliative patients but often show limited benefits due to drug resistance and poor distribution across the peritoneal–plasma barrier [2, 3, 8].

Recent advancements and the introduction of pressurized intraperitoneal aerosol chemotherapy (PIPAC) could potentially overcome the mechanisms of drug resistance and impaired bioavailability. PIPAC, performed laparoscopically, delivers a pressurized suspension of chemotherapeutic agents into the abdominal cavity via the use of a micropump and high-pressure injector. The rapid action and smaller dosage of therapeutic aerosols have been well-appreciated in pulmonary medicine [9]. Additionally, favorable aerosol deposition and lack of required maneuver coordination from patients make intraperitoneal administration easier [9]. Increased intra-abdominal pressure also facilitates intramural drug distribution [3]. PIPAC has therefore emerged as a promising palliative alternative for patients with PM [4]. Although PIPAC has been shown to improve outcome in selected patients in early phase studies, it remains unclear how the administration of PIPAC can improve the quality of life (QoL) of patients.

Biomedical data are traditionally evaluated as endpoints in clinical studies to identify treatment or intervention benefits. In the past decades, QoL research has increased worldwide to add onto biomedical endpoints for a more holistic assessment [10]. Self-reported QoL is important in understanding functioning, symptom relief, and rehabilitation of patients. It could aid medical professionals in treatment modification and improvement while prepare patients on potential consequences [10], [11], [12]. Here, we set out to evaluate the changes in QoL in PM patients who undergo PIPAC for palliation by undertaking a systematic review and meta-analysis.

Methods

Search strategy

A systematic literature search in Medline, EMBASE, PsycInfo, Web of Sciences, and Cochrane Central Register of Controlled Trials (CENTRAL) was conducted and analyzed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [13]. Searches were performed independently by two reviewers (ZL and LCKW) for internal reliability. Our search strategy was generated based on the intervention and outcome components of the PICO formula [14] (Table 1). Studies in English, reporting on PIPAC together with its variations, and QoL together with its variations, were included. Our search was limited to the time period from 2011 to August 2020 before data extraction as the first in-human PIPAC was performed in 2011. Articles with full-text access were subjected to abstract screening and review articles were excluded. Full-text screening was subsequently performed to identify independent studies with available numerical or graphical questionnaire-based QoL scores. The reference lists of the excluded review articles were further scrutinized for potential unidentified research studies. Figure 1 illustrates the search flow.

Table 1:

PICO [14] and search strategy.

PICO

Patients Patients with peritoneal metastasis (PM)
Intervention Pressurized intraperitoneal aerosol chemotherapy (PIPAC)
Comparative groups Trials having one treatment group consisting of patients with PM who underwent PIPAC plus or minus systemic chemotherapy, and one control group consisting of patients with PM who underwent only systemic chemotherapy
Outcome Quality of life (QoL)

Search strategy

Intervention (PIPAC[All Fields] OR “Pressurized IntraPeritoneal Aerosol Chemotherapy”[All Fields] OR (“pressurized”[All Fields] AND “intraperitoneal”[All Fields] AND “aerosol”[All Fields] AND “chemotherapy”[All Fields]))

AND

Outcome ((“quality of life”[MeSH Terms] OR (“quality”[All Fields] AND “life”[All Fields]) OR “quality of life”[All Fields]) OR QoL[All Fields] OR QLQ[All Fields] OR questionnaire[All Fields])
  1. PM, peritoneal metastasis; PIPAC, pressurized intraperitoneal aerosol chemotherapy; QoL, quality of life.

Figure 1: 
PRISMA flow diagram of the literature search algorithm.
Figure 1:

PRISMA flow diagram of the literature search algorithm.

Meta-analysis

The meta-analysis was carried out with our best efforts to consolidate reported and unreported data. Data extractions mainly included: (1) PM histology, (2) number of PIPAC cycles, (3) intervals between PIPAC cycles, (4) number of patients who responded to the QLQ-C30 questionnaire for each PIPAC cycle, and finally (5) reported means and standard deviations of QoL. Most data can be found from the publications. The means of the QoL scores, with the exception of the GHS scale, of PM patients with EOC were approximated from the respective graphs. Corresponding authors of the included studies were contacted for PIPAC intervals and the number of patients in each PIPAC cycle.

PM from different primary sites is associated with distinctive symptoms, disease progression, and therapeutic chemotherapy drugs delivered through PIPAC. Thus, studies of single histology are clinically meaningful for analysis. Studies [15], [16], [17], [18] of single histology (GC or EOC) were identified for meta-analysis. EORTC QLQ-C30 assesses QoL in 30 questions from 15 scales – one global health status scale, five functioning scales, three multiple-item symptom scales, and six single-item symptom scales. Patient responses in each scale are then converted to a 0–100 range-based score. Due to a lack of data in the financial scale, our meta-analysis only incorporated 14/15 QLQ-C30 indices. In reference to how PIPAC affects QoL, we compared the QoL indices on the EORTC QLQ C30 questionnaire with data for 27 patients who underwent CRS/HIPEC [19].

Statistically, GC and EOC were repeatedly measured over time for each patient. With regard to time, patients with GC underwent PIPAC at week 8, 16, and 24, and patients with EOC underwent PIPAC at week 5, 10, 15, 20, 25, and 30. Differences between subgroups of GC or EOC patients at each timepoint were estimated using meta-regression based on repeated measures mixed-models (repeated ANOVA). This model accounts for the dependence among repeated measurements on the same patient. Mixed-models for continuous data was employed, in SAS v9.4 (SAS Institute, Cary, North Carolina; PROC MIXED), with time (in weeks), histology (GC or EOC), and time by histology interaction as fixed-effects (for example. GC at week 8), and time as a random-effect with a variance component variance–covariance matrix. The estimation method was based on a maximum likelihood technique and the variance–covariance matrix of the parameter estimates computed.

Risk of bias

A risk of bias assessment based on QoL as the sole outcome was performed using the Cochrane Risk of Bias Assessment Tool: for Non-Randomized Studies of Interventions (ACROBAT-NRSI) tool [20]. Funnel plots were used to examine publication bias.

Results

Systematic search

A total of 29, 60, 0, 7, and 5 results were generated respectively from Medline, EMBASE, PsycInfo, Web of Sciences, and CENTRAL. After full-text review, eight studies were shortlisted [15], [16], [17], [18, 21], [22], [23], [24]. One relevant study was included from reference list search of previously excluded 21 review articles [25]. All shortlisted studies utilized the EORTC QLQ-C30 questionnaire. Homogeneity was achieved in all nine studies regarding the cohort study design, the PIPAC procedure, and QoL measurement. Heterogeneity was found regarding patient selection criteria, PM histology, number of PIPAC cycles, PIPAC administration interval, and EORTC QLQ-C30 administration intervals (Table 2).

Table 2:

Details and characteristics of the relevant nine studies identified.

Publications Year n Histology Time of questionnaire administration PIPAC interval No. of responses per PIPAC cycle (c) Representation of QoL scores
GC EOC CRC DMPM Others
Gockel et al. [15]c 2018 24 24 Before 3 PIPACs 8 weeks Not reported Mean ± SEM of 15 scales
Struller et al. [16]c 2019 14 14 Day 1 and after PIPAC 1 8 weeks c1=c2=14 Mean ± SD of 15 scales
Tempfer et al. [17]c 2015 58a 84a 6 1 1 day before 8 PIPACs 4–6 weeks c1=58, c2=31, c3=22, c4=9, c5=6, c6=4, c7=2, c8=1 Graph of mean ± 95% CI of 15 scales
Tempfer et al. [18]c 2015 46 46 1 day before 3 PIPACs 4–6 weeks c1=46, c2=39, c3=28 Graph of mean ± 95% CI of 14 scales
Farinha et al. [22]b 2017 42 3 21 14 1 3 Before and after and follow up of 3 PIPACs 6 weeks Not reported Graph of 15 scales
Odendahl et al. [21]b 2015 48 18 8 8 1 13 Day 1 and after 2 PIPACs 6 weeks c1=c2=c3=48 Graph of 9 scales
Graversen et al. [23] 2018 35 5 5 12 1 12 Day 1 and 60 60 days Not reported Graph of 1 scale
Giger–Pabst et al. [24] 2014 29 29 Not reported 6 weeks c1=21, c2=18, c3=11, c4/5=11 Graph of 15 scales
Robella et al. [25] 2017 14 6 3 2 2 1 Not reported 6 weeks Not reported Not reported

Total 310 70 167 36 40 30
  1. aThe study was approximated as having single histology. 91 patients were recruited with specified histology but only 58 were interviewed with QLQ C-30. bStudies with primary focus on quality of life. cStudies selected for meta-analysis. GC, gastric cancer; EOC, epithelial ovarian cancer; CRC, colorectal cancer; DMPM, diffuse malignant peritoneal mesothelioma; PIPAC, pressurized intraperitoneal aerosol chemotherapy; QoL, quality of life.

Meta-analysis

Global health status (GHS) scale

Calculated means ± 95% confidence intervals (CI) of the global health status (GHS) scale from GC patients, at weeks 8, 16, and 24, were 53.2 ± 6.9, 49.9 ± 6.3, and 48 ± 12, respectively. Calculated means ± 95% CIs of GHS from EOC patients at weeks 5, 10, 15, 20, 25, and 30 were 51.7 ± 1.2, 58.4 ± 1.4, 59 ± 2.5, 52.8 ± 4.5, 66.7 ± 5.6, and 60.4 ± 5.0, respectively (Figure 2). There was no significant difference in the QoL of GC patients who underwent PIPAC at the three reported timepoints. Overall, no deterioration of GHS was noted. Patients who underwent PIPAC had poorer GHS than patients who underwent CRS/HIPEC (Figure 2).

Figure 2: 
Line graph depicting the global health status (GHS) scale of QoL as measured by EORTC QLQ-C30 across time.
The three-point trendline depicts GHS scores before the first, second, and third cycles of PIPAC (c) in patients with peritoneal metastasis (PM) from gastric cancer (GC); c1=31, c2=22, c3=5. The six-point trendline depicts GHS scores before the first, second, third, fourth, fifth, and sixth cycles of PIPAC in patients with PM from epithelial ovarian cancer (EOC); c1=104, c2=70, c3=50, c4=9, c5=6, c6=4. Error bars represent 95% CI. No significant difference was found comparing means within GC and EOC. As a reference, the dashed line represents a snapshot of GHS in 6–18 months after CRS/HIPEC in a cohort of patients (n=27) with PM from a mixed histological origin.
Figure 2:

Line graph depicting the global health status (GHS) scale of QoL as measured by EORTC QLQ-C30 across time.

The three-point trendline depicts GHS scores before the first, second, and third cycles of PIPAC (c) in patients with peritoneal metastasis (PM) from gastric cancer (GC); c1=31, c2=22, c3=5. The six-point trendline depicts GHS scores before the first, second, third, fourth, fifth, and sixth cycles of PIPAC in patients with PM from epithelial ovarian cancer (EOC); c1=104, c2=70, c3=50, c4=9, c5=6, c6=4. Error bars represent 95% CI. No significant difference was found comparing means within GC and EOC. As a reference, the dashed line represents a snapshot of GHS in 6–18 months after CRS/HIPEC in a cohort of patients (n=27) with PM from a mixed histological origin.

Functioning scales

Transient changes were detected in all five functioning scales, but overall, no deterioration in function in PM patients with either GC or EOC was noted. Patients who underwent PIPAC had poorer functioning than patients who underwent CRS/HIPEC (Figure 3).

Figure 3: 
Line graphs depicting the five functioning scales of QoL as measured by EORTC QLQ-C30 across time.
Line graphs of (A) cognitive functioning, (B) emotional functioning, (C) physical functioning, (D) role functioning, and (E) social functioning. Each three-point trendline depicts its respective functioning scores before the first, second, and third cycles of PIPAC (c) in patients with PM from gastric cancer (GC); c1=31, c2=22, c3=5. Each six-point trendline depicts its respective functioning scores before the first, second, third, fourth, fifth, and sixth cycles of PIPAC in patients with PM from epithelial ovarian cancer (EOC); c1=104, c2=70, c3=50, c4=9, c5=6, c6=4. Error bars represent 95% CIs. In each respective functioning scale of the five, no significant difference was found comparing means within GC and EOC. As a reference, each dashed line represents a snapshot of its respective functioning score in 6–18 months after CRS/HIPEC in a cohort of patients (n=27) with PM from a mixed histological origin.
Figure 3:

Line graphs depicting the five functioning scales of QoL as measured by EORTC QLQ-C30 across time.

Line graphs of (A) cognitive functioning, (B) emotional functioning, (C) physical functioning, (D) role functioning, and (E) social functioning. Each three-point trendline depicts its respective functioning scores before the first, second, and third cycles of PIPAC (c) in patients with PM from gastric cancer (GC); c1=31, c2=22, c3=5. Each six-point trendline depicts its respective functioning scores before the first, second, third, fourth, fifth, and sixth cycles of PIPAC in patients with PM from epithelial ovarian cancer (EOC); c1=104, c2=70, c3=50, c4=9, c5=6, c6=4. Error bars represent 95% CIs. In each respective functioning scale of the five, no significant difference was found comparing means within GC and EOC. As a reference, each dashed line represents a snapshot of its respective functioning score in 6–18 months after CRS/HIPEC in a cohort of patients (n=27) with PM from a mixed histological origin.

Multiple-itemed symptom scales

In patients with GC, significant decrease in fatigue was noticed from baseline compared to before the third cycle of PIPAC, and similarly from before the second cycle of PIPAC to before the third cycle of PIPAC. This suggests progressive fatigue in patients with GC patients who undergo repeated PIPAC procedures. With regard to nausea or vomiting, significant decreases were found in patients with EOC from baseline to before the second, third, fourth, and fifth cycles of PIPAC, respectively, suggesting worsening nausea or vomiting in EOC patients with repeated PIPAC procedures. No deterioration of pain was noted in both GC and EOC patients. Moreover, patients who underwent PIPAC were found to have better multiple-itemed symptom reduction than patients who underwent CRS/HIPEC (Figure 4).

Figure 4: 
Line graphs depicting the three multiple-itemed symptom scales of QoL as measured by EORTC QLQ-C30 across time.
Line graphs of (A) fatigue, (B) nausea and vomiting, and (C) pain. Each three-point trendline depicts its respective multiple-itemed symptom scores before the first, second, and third cycles of PIPAC (c) in patients with PM from gastric cancer (GC); c1=31, c2=22, c3=5. Each six-point trendline depicts its respective multiple-itemed symptom scores before the first, second, third, fourth, fifth, and sixth cycles of PIPAC in patients with PM from epithelial ovarian cancer (EOC); c1=104, c2=70, c3=50, c4=9, c5=6, c6=4. Error bars represent 95% CIs. Significant deterioration was found in the fatigue scale (FA) within GC and in the nausea and vomiting scale (NV) within EOC. As a reference, each dashed line represents a snapshot of its respective multiple-itemed symptom score in 6–18 months after CRS/HIPEC in a cohort of patients (n=27) with PM from a mixed histological origin.
Figure 4:

Line graphs depicting the three multiple-itemed symptom scales of QoL as measured by EORTC QLQ-C30 across time.

Line graphs of (A) fatigue, (B) nausea and vomiting, and (C) pain. Each three-point trendline depicts its respective multiple-itemed symptom scores before the first, second, and third cycles of PIPAC (c) in patients with PM from gastric cancer (GC); c1=31, c2=22, c3=5. Each six-point trendline depicts its respective multiple-itemed symptom scores before the first, second, third, fourth, fifth, and sixth cycles of PIPAC in patients with PM from epithelial ovarian cancer (EOC); c1=104, c2=70, c3=50, c4=9, c5=6, c6=4. Error bars represent 95% CIs. Significant deterioration was found in the fatigue scale (FA) within GC and in the nausea and vomiting scale (NV) within EOC. As a reference, each dashed line represents a snapshot of its respective multiple-itemed symptom score in 6–18 months after CRS/HIPEC in a cohort of patients (n=27) with PM from a mixed histological origin.

Single-itemed symptom scales

Scattered significant increases of dyspnea were identified. Despite data of fewer PIPAC cycles collected for analysis, patients with EOC were found to have worsening diarrhoea from baseline to the second cycle of PIPAC, and from baseline to the third cycle of PIPAC. Transient changes were picked up in appetite loss and insomnia, but overall our analysis showed stabilization in patients with GC and EOC. Statistically significant aggravating constipation was detected in EOC patients from baseline to all subsequent timepoints respectively. Patients who underwent PIPAC were found to have better single-itemed symptom reduction than patients who underwent CRS/HIPEC (Figure 5).

Figure 5: 
Line graphs depicting the single-itemed symptom scales of QoL as measured by EORTC QLQ-C30 across time.
Line graphs of (A) appetite loss, (B) constipation, (C) diarrhea, (D) dyspnea, and (E) insomnia. Each three-point trendline depicts its respective single-itemed symptom scores before the first, second, and third cycles of PIPAC (c) in patients with PM from gastric cancer (GC); c1=31, c2=22, c3=5. Each six-point trendline depicts its respective single-itemed symptom scores before the first, second, third, fourth, fifth, and sixth cycles of PIPAC in patients with PM from epithelial ovarian cancer (EOC); c1=104, c2=70, c3=50, c4=9, c5=6, c6=4. The diarrhea scale (DI) for EOC patients only has three data points before the first, second, and third cycles of PIPAC whereas the constipation scale (CO) for EOC patients only has five data points from the first, second, third, fourth, and fifth cycles of PIPAC. Error bars represent 95% CIs. Significant deterioration was found in DI and CO within EOC. As a reference, each dashed line represents a snapshot of its respective single-itemed symptom score in 6–18 months after CRS/HIPEC in a cohort of patients (n=27) with PM from a mixed histological origin.
Figure 5:

Line graphs depicting the single-itemed symptom scales of QoL as measured by EORTC QLQ-C30 across time.

Line graphs of (A) appetite loss, (B) constipation, (C) diarrhea, (D) dyspnea, and (E) insomnia. Each three-point trendline depicts its respective single-itemed symptom scores before the first, second, and third cycles of PIPAC (c) in patients with PM from gastric cancer (GC); c1=31, c2=22, c3=5. Each six-point trendline depicts its respective single-itemed symptom scores before the first, second, third, fourth, fifth, and sixth cycles of PIPAC in patients with PM from epithelial ovarian cancer (EOC); c1=104, c2=70, c3=50, c4=9, c5=6, c6=4. The diarrhea scale (DI) for EOC patients only has three data points before the first, second, and third cycles of PIPAC whereas the constipation scale (CO) for EOC patients only has five data points from the first, second, third, fourth, and fifth cycles of PIPAC. Error bars represent 95% CIs. Significant deterioration was found in DI and CO within EOC. As a reference, each dashed line represents a snapshot of its respective single-itemed symptom score in 6–18 months after CRS/HIPEC in a cohort of patients (n=27) with PM from a mixed histological origin.

Risk of bias

Table 3 summarizes the risk of bias in selected studies as determined by the ACROBAT-NRSI tool [20]. Seven studies [15], [16], [17], [18, 21, 22, 24] were determined to have moderate risks. Graversen et al. [23] was subject to serious risk of bias and Robella et al. [25] lacks essential information. Specifically, time-varying bias introduced from having lesser patients eligible for subsequent cycles of PIPAC was identified in seven studies [15, 17, 18, 22], [23], [24], [25]. Selection bias was more profound in Gockel et al. [15] that expands treatment to ineligible patients with strong preferences and Odendahl et al. [21] that recruits patients resistant to systemic chemotherapy. PIPAC intervention was considered similar across all institutions and the EORTC QLQ-30 was validated and universally applied. As for departures from PIPAC, Gockel et al. [15], Graversen et al. [23], and Giger–Pabst et al. [24] were subject to bias induced by systemic chemotherapy as co-intervention. Missing data are likely to introduce bias because patients lasted longer for repeated cycles of PIPAC and patients who responded to QLQ-C30 tend to have better QoL. In almost all studies, fewer patients were interviewed with QLQ-C30 than those who were recruited. Patients who responded to QLQ-C30 were even fewer than the ones interviewed, which subject those studies to biases caused by missing data. Finally, studies that reported some scales in QLQ-C30 but not others were suspected of selective outcome and analysis reporting bias. Giger–Pabst et al. [24] reported only GHS, which may contribute to analysis reporting bias. Overall, the pooled analysis from four studies [15], [16], [17], [18] for meta-regression was considered to have moderate risk of bias. No significant publication bias could be observed as determined by the funnel plots (Supplementary Figures S1 and S2).

Table 3:

Risk of bias (RoB) using ACROBAT-NRSI tool [20]. Overall RoB for each study is considered (1) low if the study is judged to be at low RoB for all domains, (2) moderate if the highest risk among all domains is moderate (3) serious if the highest risk among all domains is serious, or (4) critical if the highest risk among all domains is critical [46].

Publications Year Bias due to confounding Bias in selection of participants into the study Bias in measurement of interventions Bias due to departures from intended interventions Bias due to missing data Bias in measurement of outcomes Bias in selection of the reported results Overall bias
Gockel et al. [15]b 2018 Moderate Moderate Low Moderate Moderate Low Low Moderate
Struller et al. [16]b 2019 Low Low Low Low Moderate Low Low Moderate
Tempfer et al. [17]b 2015 Moderate Low Low Low Moderate Low Moderate Moderate
Tempfer et al. [18]b 2015 Moderate Low Low Low Moderate Low Low Moderate
Farinha et al. [22]a 2017 Low Low Low Low Low Low Low Moderate
Odendahl et al. [21]a 2015 Low Moderate Low Low Moderate Low Moderate Moderate
Graversen et al. [23] 2018 Moderate Low Low Moderate Unclear Low Serious Serious
Giger–Pabst et al. [24] 2014 Moderate Low Low Moderate Moderate Low Low Moderate
Robella et al. [25] 2017 Moderate Low Low Low Unclear Low Unclear Unclear
  1. aStudies with primary focus on quality of life (QoL). bStudies selected for meta-analysis.

Discussion

All nine studies have the same cohort study design, PIPAC intervention, and EORTC QLQ-C30 questionnaire used for QoL assessment (Table 2). They nonetheless vary in patient selection criteria, primary tumor origin, number of PIPAC cycles, PIPAC administration interval, and EORTC QLQ-C30 administration time (Table 2). Specifically, Struller et al. [16] and Tempfer et al. [17] are the only studies that provided calculated minimum sample size to guide recruitment. Odendahl et al. [21] and Farinha et al. [22] are the only studies with primary aims in QoL. Other studies that focused on tumor regression, postoperative complications, or median or overall survival, have underreported QoL data. Limitations commonly discussed in the nine studies were (1) small sample size, (2) nonrandomized design, and (3) patients pretreatment heterogeneity. Limitations commonly discussed in the four studies we extracted for the meta-analysis include: (1) small numbers of PIPAC cycles, (2) fewer and fewer patients suitable for repeated PIPAC cycles, and (3) not full return of the QLQ-C30 questionnaire. Moreover, the self-selection of patients undergoing PIPAC may play an important role with respect to their QoL. Patients who report higher QoL are more likely to continue with repeated PIPAC cycles. Contrastingly, patients with lower QoL or suffer from side effects following treatment with PIPAC may elect to stop their treatment. This translates to a patient selection bias where only highly selected patients are enrolled into subsequent PIPAC cycles, possibly inflating the positive QoL results of PIPAC. Accordingly, future studies should adequately address the influence of this confounding factor.

The pooled analysis performed in this study provided us with greater power to detect for differences between groups. No QoL deterioration was found in 13 of 14 scales of patients with GC and no QoL deterioration was found in 11 of 14 scales of patients with EOC. This is consistent with the conclusion drawn from the nine articles that PIPAC is well-tolerated in most patients with PM. Furthermore, the observed differences between the GC and EOC subgroups provide evidence that the histological origin has an influence on the QoL of patients as well as patient outcome, particularly in the social functioning and fatigue scales of QoL. Down-trending of QoL curves across time, on the other hand, might not be contributed entirely to PIPAC, but to the natural aggregating progression of PM. One possible explanation is that without PIPAC, patients might experience even worse QoL. To put our results into perspective, it would have been ideal to compare the QoL of PIPAC patients with PM patients undergoing other palliative cancer therapies, such as systemic chemotherapy, of corresponding histological origins. Unfortunately, there is a paucity of evidence investigating the QoL of patients with advanced PM who have undergone palliative treatment to control their cancer symptoms. Notwithstanding, PIPAC has shown efficacy in clinical survival and tumor regression in patients with PM of CRC, GC, and EOC origins with reported resistance to systemic chemotherapy [18, 26, 27]. This suggests that PIPAC treatment could potentially lead to better QoL in patients as compared to other palliative treatment options such as systemic chemotherapy.

To this end, we compared the QoL of PIPAC patients with 27 patients who underwent CRS/HIPEC. Acknowledging the multiple limitations such as differences in patients’ disease stage, it is our opinion that the comparison remains helpful in contextualizing the relative degree of QoL improvement in PIPAC patients. We found that patients who underwent PIPAC showed lower QoL compared to CRS/HIPEC in the GHS scale and five functioning scales. This is consistent with our understanding that patients selected for a curative modality would have better general and functioning status than palliative patients. Interestingly, patients who underwent PIPAC showed better QoL in the nine symptom scales than CRS/HIPEC, suggesting that PIPAC is a promising palliative treatment for patients with PM who are not eligible for CRS/HIPEC. Another likely explanation could be that data were never collected or extracted long enough postsurgically for CRS/HIPEC to show its superiority over PIPAC. Conclusively, few existing studies have examined the impact of PIPAC on QoL. As the power of this meta-analysis is limited by study numbers and sample size, we would like to call in the future for more carefully conducted two-armed randomized controlled trials in patients with PM from a single origin to confirm our findings. Ideally, a larger number of PM patients with the same histological origin should be recruited under standardized institutional guidelines regarding pretreatment, withdrawal, PIPAC interval, and QoL questionnaire administration and collection. Half of the patients should be randomly selected to receive PIPAC alone or in combination with systemic chemotherapy while the other half should receive systemic chemotherapy as the control group.

The widespread heterogeneity in these nine included studies, namely patient pretreatment, PM histology, PIPAC interval, and QLQ-C30 interval, as well as the inherent moderate risk of bias, has undermined our meta-analysis to some extent. To overcome this limitation, we performed a pooled analysis to increase our sample size and subcategorize PM by GC or EOC histology. The pooled analysis granted us higher power to detect for differences. Furthermore, despite our best efforts to contact corresponding authors, data for patients with EOC were only close estimation because the published work did not report the exact QoL numerical score values. As such, we approximated the data from the published figures. Furthermore, as far as we know, there is currently no standardized quality assessment for nonrandomized studies. In order to evaluate the risk of bias in our selected studies, we utilized Cochrane’s ACROBAT-NRSI tool as it assesses the studies’ internal validity. To our knowledge, the ACROBAT-NSRI is commonly used in other systematic reviews [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45] and hence we have adopted the same. Additionally, CRS/HIPEC and PIPAC are procedures for distinct indications. The former is a single curative procedure whereas the latter involves multiple cycles of palliative treatment. Direct comparison of patient QoL for these two procedures would present with challenges, such as identifying the appropriate timepoint after surgery for data analysis. We will be looking into more rigorous evaluation of QoL in subsequent studies.

Conclusions

PIPAC is a well-tolerated option for most GC and EOC patients with irresectable PM. Compared to CRS/HIPEC, PIPAC provided better stabilization of QoL in symptom reduction. PIPAC could be a viable option to palliate symptoms with irresectable PM. More carefully conducted randomized studies are needed in this area to validate our findings.


Corresponding author: Assistant Professor, Chin-Ann Johnny Ong, Department of Sarcoma, Peritoneal and Rare Tumours (SPRinT), Division of Surgery and Surgical Oncology, National Cancer Centre Singapore, 11 Hospital Crescent, Singapore 169610, Singapore; Department of Sarcoma, Peritoneal and Rare Tumours (SPRinT), Division of Surgery and Surgical Oncology, Singapore General Hospital, Singapore, Singapore; Laboratory of Applied Human Genetics, Division of Medical Sciences, National Cancer Centre Singapore, Singapore, Singapore; SingHealth Duke-NUS Oncology Academic Clinical Program, Duke NUS Medical School, Singapore, Singapore; SingHealth Duke-NUS Surgery Academic Clinical Program, Duke NUS Medical School, Singapore, Singapore; and Institute of Molecular and Cell Biology, A*STAR Research Entities, Singapore, Singapore, Phone: +(65) 6436 8318, Fax: +(65) 6225 7559, E-mail:
Zhenyue Li and Louis Choon Kit Wong contributed equally to this work.

Funding source: AM-ETHOS Duke-NUS Medical Student Fellowship Award

Funding source: NCCS Cancer Fund (Research)

Funding source: SingHealth Duke-NUS Academic Medicine Centre

Award Identifier / Grant number: CIRG21jun-0038

Acknowledgments

We thank Ines Gockel of the Department of Visceral, Transplant, Thoracic and Vascular Surgery, University Hospital of Leipzig, Leipzig, Germany; Hugo Teixeira Farinha and Martin Hübner of the Department of Visceral Surgery, University Hospital of Lausanne (CHUV), Lausanne, Switzerland for providing clarification and additional data.

  1. Research funding: This work was supported by the AM-ETHOS Duke-NUS Medical Student Fellowship Award, NCCS Cancer Fund (Research) and SingHealth Duke-NUS Academic Medicine Centre, facilitated by Joint Office of Academic Medicine (JOAM). CAJO is funded by the National Medical Research Council Clinician Scientist-Individual Research Grant (CIRG21jun-0038). All the funding sources had no role in the study design, data interpretation or writing of the manuscript.

  2. Author contributions: ZL: Methodology, formal analysis, investigation, data curation, writing – original draft, writing – review and editing; LCKW: investigation, data curation, writing – review and editing; RS: formal analysis, writing – review and editing; HJL: investigation, writing – original draft, writing – review and editing; JWST: investigation, writing – review and editing, visualization, project administration; QXT: investigation, writing – review and editing, visualization JSMW: writing – review and editing, supervision CSC: writing – review and editing, supervision CAJO: conceptualization, methodology, writing – review and editing, supervision. All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Informed consent: Not applicable.

  5. Ethical approval: The local Institutional Review Board deemed the study exempt from review.

References

1. Coccolini, F, Gheza, F, Lotti, M, Virzi, S, Iusco, D, Ghermandi, C, et al.. Peritoneal carcinomatosis. World J Gastroenterol 2013;19:6979–94. https://doi.org/10.3748/wjg.v19.i41.6979.Search in Google Scholar

2. de Bree, E, Michelakis, D, Stamatiou, D, Romanos, J, Zoras, O. Pharmacological principles of intraperitoneal and bidirectional chemotherapy. Pleura Peritoneum 2017;2:47–62. https://doi.org/10.1515/pp-2017-0010.Search in Google Scholar

3. Sugarbaker, PH, Stuart, OA, Vidal-Jove, J, Pessagno, AM, DeBruijn, EA. Pharmacokinetics of the peritoneal-plasma barrier after systemic mitomycin C administration. Cancer Treat Res 1996;82:41–52. https://doi.org/10.1007/978-1-4613-1247-5_3.Search in Google Scholar

4. Reymond, M, Solass, W. Pressurized intraperitoneal aerosol chemotherapy – cancer under pressure. Walter de Gruyter GmbH; 2014.10.1515/9783110366617Search in Google Scholar

5. Desai, JP, Moustarah, F. Cancer, peritoneal metastasis. Treasure Island (FL): StatPearls; 2020.Search in Google Scholar

6. Sadeghi, B, Arvieux, C, Glehen, O, Beaujard, AC, Rivoire, M, Baulieux, J, et al.. Peritoneal carcinomatosis from non-gynecologic malignancies: results of the EVOCAPE 1 multicentric prospective study. Cancer 2000;88:358–63. https://doi.org/10.1002/(sici)1097-0142(20000115)88:2<358::aid-cncr16>3.0.co;2-o.10.1002/(SICI)1097-0142(20000115)88:2<358::AID-CNCR16>3.0.CO;2-OSearch in Google Scholar

7. Bhatt, A. Management of peritoneal metastases – cytoreductive surgery, HIPEC and beyond. Springer; 2018.10.1007/978-981-10-7053-2Search in Google Scholar

8. Prigerson, HG, Bao, Y, Shah, MA, Paulk, ME, LeBlanc, TW, Schneider, BJ, et al.. Chemotherapy use, performance status, and quality of life at the end of life. JAMA Oncol 2015;1:778–84. https://doi.org/10.1001/jamaoncol.2015.2378.Search in Google Scholar

9. Patil, JS, Sarasija, S. Pulmonary drug delivery strategies: a concise, systematic review. Lung India 2012;29:44–9. https://doi.org/10.4103/0970-2113.92361.Search in Google Scholar

10. Staquet, M, Berzon, R, Osoba, D, Machin, D. Guidelines for reporting results of quality of life assessments in clinical trials. Qual Life Res 1996;5:496–502. https://doi.org/10.1007/bf00540022.Search in Google Scholar

11. Bratt, EL, Moons, P. Forty years of quality-of-life research in congenital heart disease: temporal trends in conceptual and methodological rigor. Int J Cardiol 2015;195:1–6. https://doi.org/10.1016/j.ijcard.2015.05.070.Search in Google Scholar

12. Haraldstad, K, Wahl, A, Andenaes, R, Andersen, JR, Andersen, MH, Beisland, E, et al.. A systematic review of quality of life research in medicine and health sciences. Qual Life Res 2019;28:2641–50. https://doi.org/10.1007/s11136-019-02214-9.Search in Google Scholar PubMed PubMed Central

13. Moher, DLA, Tetzlaff, J, Altman, DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 2010;8:336–41. https://doi.org/10.1016/j.ijsu.2010.02.007.Search in Google Scholar PubMed

14. IOM (Institute of Medicine). Finding what works in health care: standards for systematic reviews. Washington, DC: The National Academies Press; 2011.Search in Google Scholar

15. Gockel, I, Jansen-Winkeln, B, Haase, L, Rhode, P, Mehdorn, M, Niebisch, S, et al.. Pressurized intraperitoneal aerosol chemotherapy (PIPAC) in gastric cancer patients with peritoneal metastasis (PM): results of a single-center experience and register study. J Gastric Cancer 2018;18:379–91. https://doi.org/10.5230/jgc.2018.18.e37.Search in Google Scholar PubMed PubMed Central

16. Struller, F, Horvath, P, Solass, W, Weinreich, FJ, Strumberg, D, Kokkalis, MK, et al.. Pressurized intraperitoneal aerosol chemotherapy with low-dose cisplatin and doxorubicin (PIPAC C/D) in patients with gastric cancer and peritoneal metastasis: a phase II study. Ther Adv Med Oncol 2019;11:1758835919846402. https://doi.org/10.1177/1758835919846402.Search in Google Scholar PubMed PubMed Central

17. Tempfer, CB, Rezniczek, GA, Ende, P, Solass, W, Reymond, MA. Pressurized intraperitoneal aerosol chemotherapy with cisplatin and doxorubicin in women with peritoneal carcinomatosis: a cohort study. Anticancer Res 2015;35:6723–9. https://doi.org/10.1055/s-0035-1560004.Search in Google Scholar PubMed

18. Tempfer, CB, Winnekendonk, G, Solass, W, Horvat, R, Giger-Pabst, U, Zieren, J, et al.. Pressurized intraperitoneal aerosol chemotherapy in women with recurrent ovarian cancer: a phase 2 study. Gynecol Oncol 2015;137:223–8. https://doi.org/10.1016/j.ygyno.2015.02.009.Search in Google Scholar PubMed

19. Tan, WJ, Wong, JF, Chia, CS, Tan, GH, Soo, KC, Teo, MC. Quality of life after cytoreductive surgery and hyperthermic intraperitoneal chemotherapy: an Asian perspective. Ann Surg Oncol 2013;20:4219–23. https://doi.org/10.1245/s10434-013-3133-0.Search in Google Scholar PubMed

20. Sterne, J, Higgins, J, Reeves, B. A Cochrane risk of bias assessment tool: for non-randomized studies of interventions (ACROBAT-NRSI). Version 1.0.0; 2014. Available from: http://www.riskofbias.info.2014.Search in Google Scholar

21. Odendahl, K, Solass, W, Demtroder, C, Giger-Pabst, U, Zieren, J, Tempfer, C, et al.. Quality of life of patients with end-stage peritoneal metastasis treated with Pressurized IntraPeritoneal Aerosol Chemotherapy (PIPAC). Eur J Surg Oncol 2015;41:1379–85. https://doi.org/10.1016/j.ejso.2015.06.001.Search in Google Scholar PubMed

22. Farinha, HT, Grass, F, Kefleyesus, A, Achtari, C, Romain, B, Montemurro, M, et al.. Impact of pressurized intraperitoneal aerosol chemotherapy on quality of life and symptoms in patients with peritoneal carcinomatosis: a retrospective cohort study. Gastroenterol Res Pract 2017;2017:4596176. https://doi.org/10.1155/2017/4596176.Search in Google Scholar PubMed PubMed Central

23. Graversen, M, Detlefsen, S, Bjerregaard, JK, Fristrup, CW, Pfeiffer, P, Mortensen, MB. Prospective, single-center implementation and response evaluation of pressurized intraperitoneal aerosol chemotherapy (PIPAC) for peritoneal metastasis. Ther Adv Med Oncol 2018;10:1758835918777036. https://doi.org/10.1177/1758835918777036.Search in Google Scholar PubMed PubMed Central

24. Giger-Pabst, U, Demtroder, C, Falkenstein, TA, Ouaissi, M, Gotze, TO, Rezniczek, GA, et al.. Pressurized IntraPeritoneal aerosol chemotherapy (PIPAC) for the treatment of malignant mesothelioma. BMC Cancer 2018;18:442. https://doi.org/10.1186/s12885-018-4363-0.Search in Google Scholar PubMed PubMed Central

25. Robella, M, Vaira, M, De Simone, M. Safety and feasibility of pressurized intraperitoneal aerosol chemotherapy (PIPAC) associated with systemic chemotherapy: an innovative approach to treat peritoneal carcinomatosis. World J Surg Oncol 2016;14:128. https://doi.org/10.1186/s12957-016-0892-7.Search in Google Scholar PubMed PubMed Central

26. Demtröder, C, Solass, W, Zieren, J, Strumberg, D, Giger-Pabst, U, Reymond, MA. Pressurized intraperitoneal aerosol chemotherapy with oxaliplatin in colorectal peritoneal metastasis. Colorectal Dis 2016;18:364–71. https://doi.org/10.1111/codi.13130.Search in Google Scholar PubMed

27. Nadiradze, G, Giger-Pabst, U, Zieren, J, Strumberg, D, Solass, W, Reymond, MA. Pressurized intraperitoneal aerosol chemotherapy (PIPAC) with low-dose cisplatin and doxorubicin in gastric peritoneal metastasis. J Gastrointest Surg 2016;20:367–73. https://doi.org/10.1007/s11605-015-2995-9.Search in Google Scholar PubMed PubMed Central

28. Yeung, CH, Santesso, N, Pai, M, Kessler, C, Key, NS, Makris, M, et al.. Care models in the management of haemophilia: a systematic review. Haemophilia 2016;22(Suppl 3):31–40. https://doi.org/10.1111/hae.13000.Search in Google Scholar PubMed PubMed Central

29. Vollam, S, Dutton, S, Lamb, S, Petrinic, T, Young, JD, Watkinson, P. Out-of-hours discharge from intensive care, in-hospital mortality and intensive care readmission rates: a systematic review and meta-analysis. Intensive Care Med 2018;44:1115–29. https://doi.org/10.1007/s00134-018-5245-2.Search in Google Scholar PubMed PubMed Central

30. Synnot, A, Karlsson, A, Brichko, L, Chee, M, Fitzgerald, M, Misra, MC, et al.. Prehospital notification for major trauma patients requiring emergency hospital transport: a systematic review. J Evid Based Med 2017;10:212–21. https://doi.org/10.1111/jebm.12256.Search in Google Scholar PubMed

31. Saramago, P, Yang, H, Llewellyn, A, Palmer, S, Simmonds, M, Griffin, S. High-throughput, non-invasive prenatal testing for fetal Rhesus D genotype to guide antenatal prophylaxis with anti-D immunoglobulin: a cost-effectiveness analysis. BJOG 2018;125:1414–22. https://doi.org/10.1111/1471-0528.15152.Search in Google Scholar PubMed

32. Ryan, V, Brown, H, Minns Lowe, CJ, Lewis, JS. The pathophysiology associated with primary (idiopathic) frozen shoulder: a systematic review. BMC Muscoskel Disord 2016;17:340. https://doi.org/10.1186/s12891-016-1190-9.Search in Google Scholar PubMed PubMed Central

33. Rocha, BDD, Zamberlan, C, Pivetta, HMF, Santos, BZ, Antunes, BS. Upright positions in childbirth and the prevention of perineal lacerations: a systematic review and meta-analysis. Rev Esc Enferm USP 2020;54:e03610. https://doi.org/10.1590/S1980-220X2018027503610.Search in Google Scholar PubMed

34. Pollock, M, Somerville, L, Firth, A, Lanting, B. Outpatient total hip arthroplasty, total knee arthroplasty, and unicompartmental knee arthroplasty: a systematic review of the literature. JBJS Rev 2016;4. https://doi.org/10.2106/JBJS.RVW.16.00002.Search in Google Scholar

35. le Roux, SM, Abrams, EJ, Nguyen, K, Myer, L. Clinical outcomes of HIV-exposed, HIV-uninfected children in sub-Saharan Africa. Trop Med Int Health 2016;21:829–45. https://doi.org/10.1111/tmi.12716.Search in Google Scholar

36. Kotronias, RA, Scarsini, R, Rajasundaram, S, De Maria, GL, Ciofani, JL, Ribichini, F, et al.. Transcatheter aortic valve replacement influence on coronary hemodynamics: a quantitative meta-analysis and proposed decision-making algorithm. J Invasive Cardiol 2020;32:37–40.Search in Google Scholar

37. Kalkhoran, S, Glantz, SA. E-cigarettes and smoking cessation in real-world and clinical settings: a systematic review and meta-analysis. Lancet Respir Med 2016;4:116–28. https://doi.org/10.1016/s2213-2600(15)00521-4.Search in Google Scholar

38. Jiang, J, Kang, N, Li, B, Wu, AS, Xue, FS. Comparison of adverse events between video and direct laryngoscopes for tracheal intubations in emergency department and ICU patients-a systematic review and meta-analysis. Scand J Trauma Resusc Emerg Med 2020;28:10. https://doi.org/10.1186/s13049-020-0702-7.Search in Google Scholar PubMed PubMed Central

39. Hoechter, DJ, Shen, YM, Kammerer, T, Günther, S, Weig, T, Schramm, R, et al.. Extracorporeal circulation during lung transplantation procedures: a meta-analysis. ASAIO J 2017;63:551–61. https://doi.org/10.1097/MAT.0000000000000549.Search in Google Scholar PubMed

40. Heidemann, CH, Lous, J, Berg, J, Christensen, JJ, Håkonsen, SJ, Jakobsen, M, et al.. Danish guidelines on management of otitis media in preschool children. Int J Pediatr Otorhinolaryngol 2016;87:154–63. https://doi.org/10.1016/j.ijporl.2016.06.003.Search in Google Scholar PubMed

41. Guarnieri, R, Bertoldo, S, Cassetta, M, Altieri, F, Grenga, C, Vichi, M, et al.. Periodontal results of different therapeutic approaches (open vs. closed technique) and timing evaluation (<2 year vs. >2 year) of palatal impacted canines: a systematic review. BMC Oral Health 2021;21:574. https://doi.org/10.1186/s12903-021-01937-x.Search in Google Scholar PubMed PubMed Central

42. Fu, L, Chen, Y, Yang, X, Yang, Z, Liu, S, Pei, L, et al.. The associations of air pollution exposure during pregnancy with fetal growth and anthropometric measurements at birth: a systematic review and meta-analysis. Environ Sci Pollut Res Int 2019;26:20137–47. https://doi.org/10.1007/s11356-019-05338-0.Search in Google Scholar PubMed

43. Benton, JS, Anderson, J, Hunter, RF, French, DP. The effect of changing the built environment on physical activity: a quantitative review of the risk of bias in natural experiments. Int J Behav Nutr Phys Activ 2016;13:107. https://doi.org/10.1186/s12966-016-0433-3.Search in Google Scholar PubMed PubMed Central

44. Behzadifar, M, Sohrabi, R, Mohammadibakhsh, R, Salemi, M, Moghadam, ST, Taheri Mirghaedm, M, et al.. General health status in Iranian diabetic patients assessed by short-form-36 questionnaire: a systematic review and meta-analysis. BMC Endocr Disord 2018;18:34. https://doi.org/10.1186/s12902-018-0262-2.Search in Google Scholar PubMed PubMed Central

45. Agrawal, K, Anwar, M, Gupta, C, Chand, P, Singh, S. Association of interleukin-1 gene polymorphism and early crestal bone loss around submerged dental implants: a systematic review and meta-analysis. J Indian Prosthodont Soc 2021;21:116–24. https://doi.org/10.4103/jips.jips_511_20.Search in Google Scholar PubMed PubMed Central

46. Bilandzic, A, Fitzpatrick, T, Rosella, L, Henry, D. Risk of bias in systematic reviews of non-randomized studies of adverse cardiovascular effects of thiazolidinediones and cyclooxygenase-2 inhibitors: application of a new Cochrane risk of bias tool. PLoS Med 2016;13:e1001987. https://doi.org/10.1371/journal.pmed.1001987.Search in Google Scholar PubMed PubMed Central


Supplementary Material

The online version of this article offers supplementary material (https://doi.org/10.1515/pp-2021-0154).


Received: 2021-12-06
Accepted: 2022-03-31
Published Online: 2022-04-21

© 2022 Zhenyue Li et al., published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution 4.0 International License.

Downloaded on 17.5.2024 from https://www.degruyter.com/document/doi/10.1515/pp-2021-0154/html
Scroll to top button