5-azacitidine prolongs overall survival in patients with myelodysplastic syndrome - systematic review and meta-analysis (Reply to Kumar et al. and to Herbst et al.)

We read with interest the meta-analysis of Kumar et al.1 and the correspondence of Herbst et al.2 The former supports our meta-analysis, recently published in this journal.3 Although both Kumar’s and our meta-analysis show overall survival benefit with 5-azacitidine in patients with myelodysplastic syndrome (MDS), they differ with respect to several aspects.

First, the pooled effect of all hypomethylating agents on overall survival in our study was statistically significant [HR of 0.72, 95% CI 0.60 – 0.85] as opposed to Kumar et al. [HR of 0.82, 95% CI 0.64 – 1.06]. This difference may be related to different statistical methods used to pool the results, and the additional inclusion of survival data obtained from the study published by Kantarjian et al.4 in the analysis of Kumar et al. Since we expected to retrieve only few primary studies with a small sample size, we selected to use the fixed effect model for estimation of the overall effect on survival as the most appropriate method. Second, while according to Kumar et al. there was no statistically significant increase in treatment related mortality (TRM)[RR of 2.47, CI 0.48–12.73], we found a significant relative risk of 7.27 with 95% CI 1.67 –31.64. This disparity stems from different definitions of this outcome. Since we restricted our TRM data to fatal events directly related to hypomethylating agents in the primary studies and did not include in this analysis early mortality or on trial mortality data, we could not include mortality data from Kantarjian et al. who did not report on TRM separately. Another distinction between our meta-analysis and Kumar’s is that we included only analyses of randomized controlled trials and avoided indirect comparisons which compare between observational findings across trials and thus suffer from the pitfalls of observational studies. Indirect comparisons should be interpreted cautiously unlike those of well balanced groups. This is especially true considering the four trials included in the meta-analyses where there were variations in case definitions of the population and in the response rate of the control groups. We thank Kumar et al. for pointing out an error in one of the primary studies5 which we corrected, resulting in a change of the HR for overall survival from HR of 0.66, 95% CI 0.55–0.80 to HR 0.72, 95% CI 0.60–0.85. Despite the slight differences between the two meta-analyses, the resemblance between them further strengthens our results and supports the role of hypomethylating agents in MDS.

We agree with the correspondence of Herbst et al.2 that systematic reviews and meta-analyses should be prepared using explicit methods to increase their accuracy and should be read critically, and we would like to respond to the issues raised.

Regarding the main outcome of overall survival, despite our efforts to get additional information from authors, we did not have sufficient data to include the data from Kantarjian et al. in this analysis.3,4 This point, as well as the possible heterogeneity between trials, is thoroughly discussed in both the Results and the Discussion sections.3 Yet, the benefit of 5-azacytidine for patients with myelodysplastic syndrome (MDS) shown in our meta-analysis is reproducible as supported by the report of Kumar et al.1 Crossover between trial arms after randomization was observed only in the study published by Silverman et al.5 In this case, intention to treat analysis may lead to acceptance of the null hypothesis despite true differences in the effect of studied interventions; if unplanned it may decrease the power of the study. The statistical significant effect that was demonstrated despite crossover of patients does not weaken the results. Systematic reviews are not devoid of errors despite their rigorous approach to identify and summarize the data. The random error in the data retrieval process in the Silverman trial was noted and a corrected version has already been submitted.

Some of the issues raised by Herbst et al. refer to the methods used. MEDLINE was not the sole electronic database searched. Search in CENTRAL was used to ascertain the comprehensiveness of the search as well as search of references of major articles and the included studies, search of conference proceedings and of ongoing trials, as well as attempts to contact principal investigators in the field, as described in the Methods section. Unfortunately, as stated by Herbst et al. themselves, they have not assessed whether a wider search would have increased the number of included studies. We do regret this, since it could add a true value to their correspondence. As used in our protocol and reported in the Methods section, we used Peto’s method to analyze time to event analysis. In the time to event analysis we used the two accepted methods, i.e. Peto and the generic inverse variance methods. Since results were similar with both methods and for technical reasons, we chose to present the inverse variance method results.

The different response definitions used by Silverman et al. as compared to the other primary trials do not support the introduction of selective reporting bias as suggested by Herbst et al. but they reflect the changing definitions over time.5 A sensitivity analysis (not reported) which excluded the trial of Silverman et al. did not affect the results. The definition of transformation to AML is well known amongst professional hematologists. Three out of the four trials included in the meta-analysis defined AML according to the stringent FAB classification (e.g., the presence of more than 30% blasts in the bone marrow).4–6 The fourth trial, which did not report it, was published as an abstract only.7

Although AML is a softer outcome than mortality, its clinical importance in the context of MDS makes it an essential outcome measure.

To conclude, we regret to say that although we appreciate the critical review by Herbst et al., in contrast to the report of Kumar et al., this correspondence could not contribute to our understanding of the role of hypomethylating agents and especially of 5-azacitidine in MDS patients.