Meta-analysis on hypomethylating agents in myelodysplastic syndromes

The addition of hypomethylating agents into the armatorium against myelodysplastic syndromes (MDS) is commonly accepted as a promising new therapeutic option in this otherwise frustrating field. However, due to methodological aspects, it is still the subject of debate whether these compounds prolong survival when compared to other treatment modalities in this population.

Meta-analysis is an important tool for summarizing scientific data from different sources and may be of great help in clinical decision making, guideline development and the conception of new trials. In order to obtain meaningful and valid results from such investigations, it is of utmost importance that a high level of unbiased analytical accuracy is ensured.

We have read the recent work of Gurion et al.1 with great interest and would like to express our concerns with respect to the applied methodology which, in our eyes, does not support the conclusion given in the title.

Using the sources cited in the above publication, we were unable to reconstruct the meta-analysis for overall survival (OS). For the trial of Silverman and co-workers2 we estimated the hazard ratio (HR) to be 0.80 (95% confidence interval 0.57–1.12) for the intention-to-treat (ITT)-analysis and not 0.52 (0.32–0.86) as given by Gurion et al. Our result is in line with the original publication which states that median survival had a P value of 0.10. Using our estimate, the final result of the meta-analysis for OS is 0.74 (0.57–0.96) instead of 0.66 (0.55–0.80).

For the interpretation of these calculations, the cited study of Kantarjian et al. must be considered3 where it reads: “The ITT-analysis […] indicates that median survival was not significantly different […] P=0.636.” While there is insufficient information to calculate a HR, it seems plausible that the addition of this trial may change the result of the meta-analysis to a non-significant result. In our opinion, the problem of missing data is a shortcoming of the presented OS meta-analysis and we regret that this important point has not been discussed by the authors.

Instead, in the Discussion section, the authors mention only a few limitations of the data. This again might be a consequence of the fact that the quality assessment in this analysis was limited to randomization (sequence generation, allocation concealment) and blinding. As a crossover between trial arms after randomization as in the Silverman study may be an important source of bias, this is especially relevant. More generally, the completeness of the outcome data was not discussed. In addition, another important source of bias called selective outcome reporting may be a reason for the differences in the choice of response criteria. The choice of the outcome time to transformation to acute myeloid leukemia (AML) is potentially problematic due to its dependence on AML definitions used. For example, 32% of patients in the trial of Fenaux and co-workers had AML according to the WHO criteria at the beginning of the study.4 Lastly, the high heterogeneity observed (even if a random effects analysis was performed) also limits the strength of the evidence and should, therefore, be given more consideration.

Aside from these issues, there are several errors in the Methods section. Based on the Results section, we conclude that the authors accepted any response definition used by the authors and not only the criteria of the International Working Group (IWG).5 The method of meta-analysis used was the “generic inverse variance method” and not Peto’s method. Moreover, the very brief quality assessment appears to be based on the 4 version of the Cochrane Handbook and not on version 5 which was implemented in 2008.6 Thus, for example, the new handbook requires a citation in order to justify the judgment of each quality item in a risk-of-bias assessment. Furthermore, the electronic search given in the paper of Gurion et al. was not carried out according to general recommendations. Among others, MeSH terms were not used and the EMBASE was not searched. We have not assessed whether a wider search would have increased the number of included studies.

In summary, even a rough review of the presented meta-analysis and the included trials reveals many methodological shortcomings. A correction of the major errors in the meta-analysis of overall survival and its interpretation should change the conclusion of the meta-analysis into: “Currently it is unclear whether newer hypomethylating agents improve overall survival in MDS.” We, therefore, strongly recommend that this meta-analysis should not be used as a basis for clinical decision making or guideline development.