European Bone Marrow Working Group trial on reproducibility of World Health Organization criteria to discriminate essential thrombocythemia from prefibrotic primary myelofibrosis. Haematologica 2012;97(3):360–5 - Comment

Any study of myeloproliferative neoplasms (MPNs) that lacks adequate clinical input is doomed to cause diagnostic uncertainty and increased controversy. In the paper by Buhr et al. published in Haematologica,1 the authors studied 102 cases of essential thrombocythemia (ET) and early primary myelofibrosis (PMF) by comparing clinical criteria with the histopathological evaluation independently performed by 6 hematopathologists. According to the World Health Organization (WHO) classification, the minor clinical criteria for PMF include leukoerythroblastosis, increase in serum lactate dehydrogenase (LDH) level, anemia, and splenomegaly.2,3 As explicitly stated in the WHO guidelines, the degree of abnormalities can be borderline or marked, and the presence of at least two of these criteria is required for diagnosis.2,4 In the work Buhr et al., “Design and Methods” and Table 1 confirm that the authors failed to define “borderline values” (e.g. presence of circulating myeloid precursors and erythroblasts > 1%, borderline elevated LDH according to local threshold values, anemia defined as hemoglobin <13 g/dL in men or <12 g/dL in women, as well as palpable splenomegaly as > 1 cm below costal margin). Moreover, as shown in Table 2 in the same paper, a considerable number of cases lacked clinical data altogether. Therefore, we would not reach the conclusion “that more than 50% of PMF cases became MPNs unclassifiable when WHO criteria were applied“. Since no ranges of hemoglobin or hematocrit results were provided by the authors, a significant proportion of cases within their ET cohort may in fact represent initial polycythemia vera (PV), clinically mimicking ET.5,6 The inadvertent inclusion of early PV cases would have seriously compromised the analysis and be responsible for unclassifiable cases.

The inadequacy of the study becomes even more evident when its results are compared and contrasted with those obtained in a much larger cohort of patients with ET or PMF, in which clinical and morphological parameters were independently collected by a combined clinicopathological team.7 The authors’ conclusion that none of the 646 cases with early PMF described in the latter study had leukoerythroblastosis is incorrect, because only median values for myeloid precursors and erythroblasts were reported.7 In fact, the range for myeloblasts was up to 4% (mean 0.05%) and that for erythroblasts up to 14% (mean 0.21%), respectively.

In the same study, no information is provided in relation to the length of disease history antedating the examined biopsy. If the biopsy was not obtained at disease onset or in close proximity to such a time, it might show changes related to disease progression that may render the case truly unclassifiable. In addition, WHO criteria can only be applied to untreated patients.2,3 However, no corresponding information is provided in this paper.

Concerning previously published studies, a totally blinded evaluation of 295 patients, clinically considered to represent either ET or possibly early PMF, performed in two distinct pathology departments revealed an overall concordance of 78% with an interobserver reliability of kappa=0.626 (P=0.001), a finding which finds substantial consensus.7 The high concordance in that study, in contrast with what is claimed in the paper by Buhr et al.,1 was obtained in the absence of any prior consensus conference or training. On the other hand, after such a consensus meeting between pathologists and clinicians, the concordance in the final clinical diagnosis was 82% for the total cohort and 93% for the prevalent group of ET versus early PMF patients (kappa=0.740; P=0.001), i.e. substantial agreement.7 Moreover, the authors cited but failed to recognize the importance of a clinicopathological study on 1,104 treatment-naive patients diagnosed independently by seven international centers as being consistent with ET according to the WHO guidelines.8 A review of the initial bone marrow specimens confirmed the ET diagnosis in 81% (890 patients) and was revised to early PMF in only 16% (180 patients). A critical analysis regarding the shortcomings of the study by Wilkins and co-workers,9 as previously stressed,10 is also missing. In another cited study,11 the authors claimed to diagnose ET strictly according to the WHO guidelines but described minor to moderate reticulin fibrosis in more than 50% of their patients, which is definitively inconsistent with the WHO criteria.2,3

The erroneous proposal to replace the WHO classification by a scheme focused on fiber grading (ET grades 1 to 3) includes as first end point “true” ET without increase in reticulin but fails to recognize early (initial) PMF with fiber grade 0. This classification groups together ET, early PMF and overt PMF, and is not compatible with a previous study of the first and last author speaking in favor of pre-fibrotic PMF.12

Regarding the estimated intra- (reproducibility) and inter- (consensus) observer concordance, striking differences between the panelists are outlined in Table 4. In particular, a significant variability in intra-rater reliability ranging from 0.25 to 0.78 is observed which might probably reflect the varying degrees of experience with the WHO classification by some of the panelists (panelists P1, P2, P5, and P6 with kappa values between 0.60 and 0.78 (mean 0.71) compatible with substantial reproducibility contrasting with panelists P3 with a kappa of 0.25 and P4 with 0.55 indicating only a poor intra-rater repeatability). Furthermore, the presented approach to measure the interobserver consensus does not make full use of the original data, and will usually not yield the same values as that obtained from a true multi-rater measure of agreement. To allow comparison with previous studies, unweighted and weighted values of kappa, together with the 95% confidence intervals of the values, should have been estimated.

Finally, the title of the paper by Buhr et al. would suggest to the non-informed reader that it may represent the opinion of the European Bone Marrow Working Group (EBMWG). In reality, the study design and methodology was never fully discussed at EBMWG meetings. The results obtained on a limited database of heterogeneously selected cases lacking adequate clinical correlates most definitely represent a non-consensus based conclusion.