FormalPara Key Summary Points

The authors of the network meta-analysis did not address the issue of heterogeneity between the studies resulting from differences in baseline patient characteristics.

There were inappropriate inclusion criteria of trials, with lack of adjustment for differences in key treatment effect modifiers such as prior biologic disease modifying anti-rheumatic drugs (DMARDs) use and methotrexate dose at baseline. In addition, there was no assessment of the impact of the differences in the number of conventional synthetic DMARDs failures allowed between studies.

There was lack of adjustment for placebo arm response across trials.

These major limitations of the analysis make the findings, and consequently the conclusions, inaccurate and therefore biased.

LETTER


Dear Editor,


We read the manuscript authored by Pope et al. (2020) on “Comparative Efficacy of JAK Inhibitors for Moderate-To-Severe Rheumatoid Arthritis: A Network Meta-Analysis” with great interest.

The network meta-analysis (NMA) found that upadacitinib 15 mg once daily had numerically higher efficacy in terms of American College of Rheumatology (ACR) response and clinical remission among approved Janus kinase (JAK) inhibitor combination therapies and monotherapies for patients with inadequate response to conventional synthetic DMARD (csDMARD) treatments. While the authors acknowledged the limitations of their analyses, there is uncertainty that, if they had appropriately addressed limitations that may have been able to be evaluated, the conclusions of these analyses would have remained the same.

While focusing on phase III randomized controlled trials may partially address the issue of heterogeneity resulting from differences in trial design, it does not address the issue of heterogeneity between the studies resulting from major differences in baseline patient characteristics. Most notably, the authors did not address differences in the trials related to key treatment effect modifiers.

Firstly, the authors’ networks included baricitinib trials where patients enrolled in the trials had no prior biologic DMARD (bDMARD) use and upadacitinib and tofacitinib trials where enrolled patients had prior bDMARD use (≤ 20%). We believe this approach is not appropriate for primary analyses, as different previous exposure to bDMARD might constitute a source of bias. A more methodologically sound approach would have been either to only include baricitinib trials in a sensitivity analysis or to investigate the influence of prior biologic exposure through meta-regression techniques [1].

Secondly, differences between trials related to the number of patients with csDMARD failures between studies should have been mentioned and addressed. For example, upadacitinib SELECT-NEXT trial [2] allowed a maximum of two csDMARD failures while baricitinib RA-BUILD trial [3] had no restriction on the previous number of csDMARDs (no maximum numbers).

Thirdly, inclusion of trials allowing for lower background dose of methotrexate (MTX) in the network might also constitute another source of heterogeneity, the impact of which on treatment response has not been reported. We specifically refer here to the inclusion of RA-BALANCE [4] and SELECT-SUNRISE [5], in some of the networks, where the dosage of MTX was much lower than that reported in other included trials in the networks. Trials including Asians-Japanese patients with lower MTX dose should have been excluded in a sensitivity analysis and impact on results compared.

While the authors acknowledge that differences in placebo arm response rates across trials could be a potential confounder of the estimated treatment, they did not adjust the analyses for these differences. This is an issue that might have an impact on the results. Various statistical techniques exist (including adjustment on baseline risk) that could have been used to control for the differences in placebo response rate across trials. An assessment is needed, and missing in this NMA, on how differences in placebo rate response associated with differences in treatment outcomes.

As such, we believe the reported NMA has key limitations, some of which could have been addressed in the analyses, for the results to be valid. While the authors acknowledged some of the limitations, these were not properly addressed with their methodology. We also believe “numerically better results” in an NMA are very inconclusive results to be used for clinical relevance extrapolation and should be avoided as part of the key conclusions.

Digital Features

This article is published with digital features to facilitate understanding of the article. To view digital features for this article go to https://doi.org/10.6084/m9.figshare.13655846.