Reproducibility of the World Health Organization 2008 criteria for myelodysplastic syndromes

Abstract

The reproducibility of the World Health Organization 2008 classification for myelodysplastic syndromes is uncertain and its assessment was the major aim of this study. The different peripheral blood and bone marrow variables required for an adequate morphological classification were blindly evaluated by four cytomorphologists in samples from 50 patients with myelodysplastic syndromes. The degree of agreement among observers was calculated using intraclass correlation coefficient and the generalized kappa statistic for multiple raters. The degree of agreement for the percentages of blasts in bone marrow and peripheral blood, ring sideroblasts in bone marrow, and erythroid, granulocytic and megakaryocytic dysplastic cells was strong (P<0.001 in all instances). After stratifying the percentages according to the categories required for the assignment of World Health Organization subtypes, the degree of agreement was not statistically significant for cases with 5-9% blasts in bone marrow (P=0.07), 0.1-1% blasts in peripheral blood (P=0.47), or percentage of erythroid dysplastic cells (P=0.49). Finally, the interobserver concordance for World Health Organization-defined subtypes showed a moderate overall agreement (P<0.001), the reproducibility being lower for cases with refractory anemia with excess of blasts type 1 (P=0.05) and refractory anemia with ring sideroblasts (P=0.09). In conclusion, the reproducibility of the World Health Organization 2008 classification for myelodysplastic syndromes is acceptable but the defining criteria for blast cells and features of erythroid dysplasia need to be refined.

Introduction

For more than 20 years the French-American-British morphological classification was the base for the diagnosis of myelodysplastic syndromes (MDS), a group of acquired clonal hematopoietic stem cell disorders with very heterogeneous outcomes and characterized by ineffective hematopoiesis, cytopenias, dysplastic morphological features, and an increased risk of development of acute myeloid leukemia (AML).1,2 In an attempt to improve its prognostic value, to incorporate other relevant morphological and biological prognostic characteristics, such as grade of myelodysplasia and cytogenetics, and to redefine the border between MDS and AML, an expert panel of the World Health Organization (WHO) proposed, in 2001, a new classification system for MDS3 that was refined in 2008.4 Taking into account the type and number of cytopenias, percentage of cells with dysplastic changes in the different myeloid cell lineages, percentage of blasts in peripheral blood and bone marrow, percentage of ring sideroblasts in bone marrow, absolute monocyte count and conventional cytogenetics, the revised 2008 WHO classification recognizes seven subcategories of MDS that are shown in Table 1 (modified from Vardiman JW et al.5). Although the usefulness of the WHO classification was initially criticized,6,7 it has gained widespread acceptance.1,8 The prognostic value of the 2001 WHO classification is clearly superior to that of the French-American-British classification9,10 and has been incorporated into the recently defined WHO classification-based Prognostic Scoring System (WPSS).11,12 However, the reproducibility of the WHO classification is uncertain. The only published study on interobserver agreement according to the 2001 WHO criteria showed an acceptable reproducibility.13 A recent study evidenced a 20% discrepancy in the WHO 2008 classification-based diagnosis between referring and tertiary care centers.14 The concordance between observers of the 2008 WHO criteria has never been addressed. This issue is not only academically interesting, but also clinically relevant.

Table 1.WHO 2008 classification of myelodysplastic syndromes.

The aim of this study was to assess the reproducibility of the WHO 2008 morphological classification. For this purpose peripheral blood and bone marrow samples from 50 patients with MDS were blindly and independently reviewed by four cytomorphologists from three different referral centers with expertise in the diagnosis of MDS. The interobserver concordance in the quantification of the three morphological characteristics required for the assignment of patients to the appropriate WHO morphological subtype (the number of blast cells in peripheral blood and bone marrow, percentage of ring sideroblasts in bone marrow and myelodysplastic features) was evaluated.

Design and Methods

Patients and samples

Samples of bone marrow aspirates and peripheral blood from 50 patients with a clearly established diagnosis of MDS according to WHO 2008 criteria and diagnosed at two of the participating centers (Hospital del Mar, Barcelona and Hospital Universitari i Politecnic La Fe, Valencia) were included in this retrospective analysis. In all instances the analyzed samples had been obtained at initial evaluation. The number of cases of the different WHO 2008 morphological subtypes selected for review was prefixed according to their expected incidence in previous studies.4 Preference for inclusion in the study was given to cases diagnosed in more recent years and with good quality samples available. Table 2 summarizes the main characteristics of the patients.

Table 2.Main characteristics of the patients.

Morphological studies

Four smears from each patient included in the study were available for blind and independent microscopical review by four experienced cytologists from three centers. Two bone marrow and one peripheral blood May-Grünwald-Giemsa-stained smears were used for assessing percentages of blasts in peripheral blood and bone marrow and percentages of dysplastic cells of the three myeloid cell lines. An additional Prussian blue-stained bone marrow smear was used for assessing the percentage of ring sideroblasts. The cytologists had a meeting to discuss the evaluation of dysplasia and diagnosis using training slides. Blasts and ring sideroblasts were defined according to the recent consensus proposals of the International Working Group on Morphology of Myelodysplastic Syndromes (IWGM-MDS).15 The WHO 2008 recommendations for evaluating the morphological diagnosis of MDS were followed strictly. Thus, bone marrow blast counts were calculated as percentages of all bone marrow nucleated cells. Peripheral blood and bone marrow differential counts were performed on at least 200 and 500 cells, respectively. The evaluation of dysplasia was based on morphological criteria described in the 2008 WHO publication.4 Briefly, the following morphological features of dysplasia were evaluated: (i) dyserythropoiesis: nuclear budding, internuclear bridging, karyorrhexis, multinuclearity, nuclear hyperlobation, megaloblastic changes, basophil stippling, ring sideroblasts and vacuolization; (ii) dysgranulopoiesis: nuclear hypolobation (pseudo Pelger-Huët), irregular hypersegmentation, agranularity, pseudo Chediak-Higashi granules, Auer rods and Döhle bodies; and (iii) dysmegakaryocytopoiesis: micromegakaryocytes, nuclear hypolobation, and multinucleation. As defined in the WHO 2008 classification, the threshold used for considering a myeloid cell line as dysplastic was the presence of ≥10% abnormal cells in the corresponding myeloid lineage. To assess dysplasia at least 200 neutrophils, 200 erythroid precursors and 30 megakaryocytes were evaluated in bone marrow. Information on hemoglobin level, and absolute neutrophil and platelet counts was available for observers when performing the morphological review. In contrast, the observers were blinded to the clinical and cytogenetic data. Consequently, for the purpose of this study, cases of MDS associated with isolated 5q deletion were classified into other MDS morphological subtypes. All the morphological characteristics analyzed were recorded in specific forms designed for this purpose by the Spanish Group on Myelodysplastic Syndromes (GESMD) and transferred into a specific database. Once the review had been finished, no attempt was made to reach a consensus agreement on cases with discrepant results in any of the variables analyzed. The local ethics committees approved the studies which were conducted in accordance with the 2000 revision of the Helsinki Declaration.

Statistical analysis

Agreement between the four observers for continuous quantitative variables (percentages of blasts in peripheral blood and bone marrow, percentages of ring sideroblasts in bone marrow and percentages of dysplastic cells of erythroid, granulocytic and megakaryocytic lineages) was evaluated using the intraclass correlation coefficient (ICC).16 The ICC has advantages over Spearman's correlation coefficient, because it is adjusted for the effects of the scale of measurements, and allows the assessment of agreement when there are more than two observers. Quantitative variables (percentages of blast cells in peripheral blood and bone marrow, percentages of ring sideroblasts in bone marrow and percentage of dysplastic cells in each myeloid lineage) were categorized and also evaluated as categorical variables (Table 3). For this purpose, we used the cutoff levels defined by the WHO classification and an additional cutoff level of 40% of dysplastic cells according to previous data.17 To evaluate the concordance between observers in qualitative and categorized quantitative variables, the generalized kappa statistic for multiple raters (κ) was calculated. An ICC or generalized κ statistic value of 1 denotes complete agreement between the different observers, while an ICC value of 0 denotes agreement equivalent to chance. Both the ICC and the generalized κ statistic can be interpreted as follows: 0-0.2 indicates poor agreement, 0.3-0.4 indicates fair agreement, 0.5-0.6 indicates moderate agreement; 0.7-0.8 indicates strong agreement, and >0.8 indicates almost perfect agreement.18 The statistical package SPSS, version 19.0 (SPSS Inc., Chicago, IL, USA) was used to calculate the ICC and a Microsoft Excel Template to calculate the κ statistic.

Table 3.

Results

The degree of concordance between observers for the different morphological characteristics is summarized in Table 3.

Interobserver concordance regarding blast cell count

There was a strong agreement in the percentage of blast cells in bone marrow considered as a continuous variable. The ICC for this parameter was 0.95 [95% confidence interval (CI), 0.92-0.97; P<0.001)]. When its degree of concordance was assessed stratifying the variable into three categories (<5%, 5-9%, and ≥10%), according to the thresholds used in WHO classification subtypes, the interobserver concordance was moderate (overall κ, 0.57; P<0.001). The degree of agreement was higher and significant when the bone marrow blast percentage was less than 5% (κ, 0.72; P<0.001) or equal or greater to 10% (κ, 0.65; P<0.001), but it was lower for cases with an intermediate percentage of bone marrow blast cells (5-9%) for which only a fair agreement was reached (κ, 0.29; P=0.07). When this variable was further stratified into four categories, adding a cutoff point of 2% blast cells in bone marrow, the interobserver concordance was fair (overall κ 0.42; P<0.001). The κ values were 0.50 (P=0.002) for cases with less or equal to 2%, 0.28 (P=0.065) for cases with more than 2% and less than 5%, 0.29 (P=0.048) for cases with 5% to less than 10% and 0.60 (P<0.001) for cases with more or equal than 10%.

Interobserver agreement for the percentage of blast cells in peripheral blood showed a very good agreement. The ICC for this parameter was 0.82 (95% CI: 0.72-0.89). When the variable was evaluated according to the subcategories used in the WHO classification (absent, less or equal to 1% and more than 1%), the overall kappa score was 0.30 (P<0.002). The agreement between observers was significant in the condition with more than 1% blasts (κ, 0.37; P=0.009), but there was no significant agreement between observers in the condition without blasts (κ, 0.37; P=0.20) and in the intermediate category of less than or equal to 1% blasts (κ, 0.09; P=0.47).

Interobserver concordance regarding ring sideroblast count

The agreement between observers on the percentage of ring sideroblasts in bone marrow was nearly perfect analyzed both as a continuous variable (ICC, 0.96; 95% CI: 0.93-0.98; P<0.001) and with the 15% cutoff point used in the WHO criteria (κ, 0.82; P<0.001).

Interobserver concordance regarding the assessment of dysplasia

When the degree of dysplasia of the three different hematopoietic cell lines was studied as a continuous variable, the degree of concordance between observers was strong and almost perfect for the megakaryocytic lineage with an ICC of 0.91 (95% CI: 0.85-0.95; P<0.001) and for the granulocytic lineage with an ICC of 0.89 (95% CI: 0.83-0.94; P<0.001). A substantial agreement was observed for the erythroid lineage with an ICC of 0.75 (95% CI: 0.60-0.85; P<0.001). When those variables were stratified according to the 10% cutoff point required by the WHO criteria to define a hematopoietic cell lineage as dysplastic, the interobserver agreement was statistically significant for the granulocytic (κ, 0.40; P=0.04) and megakaryocytic (κ, 0.49; P<0.001) lineages. There was poor agreement regarding the erythroid lineage (κ, 0.19; P=0.49). When a cutoff of 40% dysplastic cells was used, the concordance between raters improved for the megakaryocytic and granulocytic lineages but did not improve for the erythroid lineage.

Reproducibility of World Health Organization-defined subtypes of myelodysplastic syndromes

The overall interobserver concordance for WHO-defined MDS subtypes showed a moderate overall agreement (κ, 0.43; P<0.001) (Table 3 and Figure 1). A greater reproducibility was found for patients with refractory anemia with excess blasts-2 (κ, 0.60, P<0.001), refractory cytopenias with unilineage dysplasia (κ, 0.5; P<0.001) and refractory cytopenia with multilineage dyplasia (κ, 0.46; P<0.01). Concordance was lower for refractory anemia with ring sideroblasts (κ, 0.26, P=0.09) and refractory anemia with excess blasts-1 (κ, 0.29; P=0.05).

Figure 1.The presence of granulated blast cells (arrows) makes the distinction between blast cells and promyelocytes (discontinous arrow) difficult so that the number of blast cells may differ and the same patient may be classified as having MDS with or without excess of blasts. regarding morphological features.

Discussion

Despite major advances in the diagnosis of hematologic diseases, cytomorphological criteria remain the cornerstone of the diagnosis of MDS. The current study was designed to evaluate interobserver variability in assigning a diagnosis of MDS according to the WHO 2008 classification criteria and to define potential morphological difficulties. In our study, we observed a moderate reproducibility of the WHO 2008 classification.

In 2008, Muffi et al.15 pointed out the difficulty of morphological diagnosis of blast cells, although the percentage of blasts in bone marrow is one of the main known prognostic factors7,19-20 and has been included in the most commonly used prognostic scoring systems for MDS, such as the International Prognostic Scoring System (IPSS) and the WHO classification-based Prognostic Socring System (WPSS). In our work we found an almost perfect agreement regarding bone marrow blast cell count in cases with <5% or with ≥10%. In those cases with a blast cell count ≥5% and <10%, the rate of concordance showed a moderate agreement. This implies that one patient could be classified as having refractory cytopenia with multilineage dysplasia, refractory anemia with excess blasts-1 or refractory anemia with excess blasts-2 depending on the observer. We decided to evaluate the interobserver concordance for an additional cutoff point of 2% blasts in bone marrow for cases without excess of blasts because this threshold seems to portray prognostic relevance in the revised version of the IPSS21. The degree of agreement was adequate for cases with ≤2% blast cells but, again, it was not as good for cases with blast cell counts between 2% and 5%. Discrepancies in the blast count in bone marrow of patients with myeloid malignancies, including MDS, is partly due to the difficulty in distinguishing between granular blast cells and promyelocytes and the irregular distribution of blast cells in bone marrow. Although a good correlation between the percentage of blasts determined by morphological examination and percentage of CD34 cells determined by flow cytometry is usually observed, blast enumeration by morphology is the gold-standard method.4,22

The correct assignment of the percentage of blasts in peripheral blood is also crucial for a correct diagnosis and classification of patients.23 In our study, we found a fair agreement in peripheral blood blast cell count. This result may be due to the low level of blast cells present in peripheral blood. Interobserver discrepancies may best be resolved by increasing the number of cells in the differential counts.

The recognition of dysplastic signs has a crucial value not only for the diagnosis and classification of MDS patients, but also has a prognostic role in low-risk MDS patients. In this regard, Pseudo-Pelger-Huët anomaly in neutrophils and micromegakaryocytes has been correlated with overall survival.17,24 Besides, several investigators consider that cases with multilineage dysplasia have a less favorable prognosis than those with only dyserythropoietic dysplasia.10,25-26 The WHO classification therefore separated cases with refractory anemia in the previous French-American-British classification into two categories depending on the presence or absence of multilineage dysplastic features. This distinction has been criticized by some groups,6,7 because the assessment of the features of dysplasia is not always easy in clinical practice because of the lack of definition of objective parameters. Poor technical quality of the specimen could also be an obstacle to an accurate diagnosis of dysplasia. In our work we found a moderate but significant interobserver agreement for megakaryocytic and granulocytic dysplasia and a poor agreement for erythroid dysplasia. This is probably because features of dysgranulopoiesis (pseudo-Pelger-Huët, hypogranularity) and dysmegakaryopoiesis (micromegakaryocytes, non-lobulated nuclei and multiple widely separated nuclei) are less subjective and more reproducible than features of erythroid dysplasia.

Figure 2.Evaluation of dysplastic features in erythropoiesis such as megaloblastoid changes (arrows) and cytoplasmic changes (discontinous arrow) is poorly reproducible explaining why the agreement between observers in the evaluation of dyserythropoiesis is not good.

The WHO classification includes a uniform threshold of 10% for dysplasia in each myeloid lineage; however, as discussed by Parmentier et al.,27 this level of dysplasia is highly questionable and is particularly low in the megakaryocytic lineage in which the number of cells analyzed is smaller than in the other series. We analyzed the interobserver concordance with a cutoff point of 40% dysplastic cells and found that the agreement improved in the megakaryocytic and granulocytic lineages but not in the erythroid lineage. These results agree with those of Matsuda et al.17 and Germing et al.28 who proposed raising the threshold of dysmegakaryopoiesis from 10 to 40%.

The prognostic value of the WHO classification is already known.7,9,10,12 Howe et al.13 analyzed the reproducibility of the 2001 WHO classification showing a 92% of agreement among three reviewers. Their discrepancies were related to the identification and enumeration of dyspoiesis in neutrophils and megakaryocytes and in those cases with borderline blast percentages.

Recently, Naqvi et al.14 analyzed the discrepancies in morphological diagnosis of MDS between referral and tertiary centers showing differences in 12% of the patients. They did not, however, analyze the causes of the discrepancy.

The current work, although reviewing a rather limited number of samples, is the first to analyze the correlation between observers of the 2008 WHO morphological criteria. We found a nearly moderate and significant concordance regarding the definition of 2008 WHO MDS subtypes (κ, 0.43; P<0.001). Most differences concerned the distinction of unilineage and multilineage dysplasia; consequently some patients were classified as having refractory anemia with ring sideroblasts or refractory cytopenia with multilineage dysplasia depending on the recognition of dysplasia in one or more myeloid lineages. As previously described by Howe et al.,13 we also had difficulties in assigning MDS subtypes in those cases with borderline blast cell percentages. In fact, a substantial agreement was obtained only in cases with less than 2% or more than 10% of blast cells in the bone marrow.

To sum up, the WHO 2008 classification can be applied with a moderate interobserver concordance. Discrepancies are frequent and may have a potential negative impact on the assignment of prognosis and therapy planning in the individual patient. The degree of agreement could be improved if the criteria for features for dyserythropoiesis were to be refined. Future studies should evaluate the potential increment in the threshold for considering a cell lineage as dysplastic in order to enhance the recognition of multilineage dysplasia. Finally, the diagnosis of MDS is complex, requires an accurate application of the WHO criteria, and should be performed by experienced morphologists. Despite all those measures, it must be highlighted that the value of cytomorphology alone for the classification of MDS is limited. In this regard, the development, standardization, and incorporation into our daily practice of other techniques, such as flow cytometry and molecular studies,29-31 will likely allow us to diagnose, characterize, and classify this heterogeneous group of myeloid neoplasms better in the near future.

Acknowledgments

The authors wish to thank Dr. Ignacio Lorenzo for his helpful comments on statistical issues and Luis Benlloch for data management.

Funding: This work was supported in part by research funding from “RTICC” (Red Temática de Investigación Cooperativa en Cáncer, RD06/0020/0031, RD07/0020/2004), Instituto de Salud Carlos III, Ministerio de Sanidad y Consumo, Spain (FI07/00107; PI07/1009 and PI11/02010); SGR 541 (“Agència de Gestió d'Ajuts Universitaris i de Recerca”, “Departament d'Innovació, Universitats i Empresa”) y COST-European Genetic and Epigenetic Study on AML and MDS (BM080).

Footnotes

• Authorship and Disclosures:Information on authorship, contributions, and financial & other disclosures was provided by the authors and is available with the online version of this article at www.haematologica.org.

• Received June 3, 2012.
• Accepted September 17, 2012.