Comment on “Fecal occult blood versus DNA testing: indirect comparison in a colorectal cancer screening population”

Dear editor

Challenges to medical innovation are welcome and essential elements in the dialectic toward practice optimization but must be based on scientifically sound criticism. A multi-target stool DNA test (MT-sDNA) has emerged as a new approach to colorectal cancer (CRC) screening, and in the US, it is approved by federal regulatory bodies, endorsed in societal screening guidelines, and covered by government and private insurers. In their recent paper,¹ Brenner and Chen concluded that CRC screening sensitivity by fecal immunochemical test (FIT) for occult blood is equal to or better than that of MT-sDNA when tests are compared at matched specificity. However, concerns can be raised regarding their methodology and data.

1. Major differences exist across study populations: a cardinal scientific principle dictates that tests are best compared directly within the same population. In this study from Germany,¹ observed FIT results were compared with reported MT-sDNA results from a large multicenter US study.² Their indirect comparison is confounded by between-population differences in patient, lesion, and other variables. For example, patients were substantially younger and CRCs were more advanced in the German study; 67% of German patients versus 37% of US patients were younger than 65 years, and 46% versus 14% of CRCs were Stages III–IV, respectively. These factors may significantly influence test performance.
2. Direct sensitivity comparisons at matched specificities done within a single test population contradict conclusions in the present indirect comparison: in the US study report,² MT-sDNA and FIT sensitivities were directly compared in area under the curve plots across the full range of specificities. At matched specificity of 86.6%, CRC sensitivity was 92.3% for MT-sDNA and 76.9% for FIT, p=0.015. This direct within-study comparison holds greater validity than indirect between-study comparisons.
3. Observed FIT sensitivity is atypical: in the authors’ own prior studies from the same German consortium, FIT sensitivities for CRC were 53%–73%.^3,4 In prior studies using FOB Gold (the FIT used in the present study), CRC screen sensitivities were reported to be 50%–75%.^5,6 These FIT sensitivities are much lower than the 96.7% observed in the present study, which appears to be an outlier.
4. MT-sDNA specificity used is not representative: the comparator US multicenter study was intentionally skewed toward older patients to enrich for pathology and to meet Medicare data requisites, which lowered specificity;² specificity was 94% in patients aged 50–64 years with normal colonoscopy. In a second blinded screen-setting study from Alaska,⁷ specificity for MT-sDNA was 93% and sensitivity for CRC was 100%. Using conservative point specificities, we have estimated that MT-sDNA done at the recommended frequency of every 3 years generates fewer false-positives over a longitudinal program than does FIT done annually. Thus, program specificity may be functionally higher with MT-sDNA than with FIT. In alignment with this, modeling of CRC screening approaches by the US Preventive Services Task Force showed that MT-sDNA done every 3 years yielded the highest benefit-to-harm ratio and fewest unnecessary colonoscopies.⁸

Given the aforementioned scientific concerns, caution must be taken in interpreting the recent report by Brenner and Chen.

Disclosure

Mayo Clinic is an equity investor in Exact Sciences. Dr. Ahlquist is a co-developer of a stool DNA test commercialized by Exact Sciences and serves as a scientific advisor to and research collaborator with Exact Sciences. Mayo Clinic receives royalties from Exact Sciences and, according to policy, shares those with Dr. Ahlquist. The author reports no other conflicts of interest in this communication.

References

1.		Brenner H, Chen H. Fecal occult blood versus DNA testing: indirect comparison in a colorectal cancer screening population. Clin Epidemiol. 2017;9:377–384.
2.		Imperiale TF, Ransohoff DF, Itzkowitz SH, et al. Multitarget stool DNA testing for colorectal-cancer screening. N Engl J Med. 2014;370(14):1287–1297.
3.		Brenner H, Tao S. Superior diagnostic performance of faecal immunochemical tests for haemoglobin in a head-to-head comparison with guaiac based faecal occult blood test among 2235 participants of screening colonoscopy. Eur J Cancer. 2013;49(14):3049–3054.
4.		Hundt S, Haug U, Brenner H. Comparative evaluation of immunochemical fecal occult blood tests for colorectal adenoma detection. Ann Intern Med. 2009;150(3):162–169.
5.		Rubeca T, Rapi S, Confortini M, et al. Evaluation of diagnostic accuracy of screening by fecal occult blood testing (FOBT). Comparison of FOB gold and OC sensor assays in a consecutive prospective screening setting. Int J Biol Markers. 2006;21(3):157–161.
6.		Faivre J, Dancourt V, Denis B, et al. Comparison between a guaiac and three immunochemical faecal occult blood tests in screening for colorectal cancer. Eur J Cancer. 2012;48(16):2969–2976.
7.		Redwood DG, Asay ED, Blake ID, et al. Stool DNA testing for screening detection of colorectal neoplasia in Alaska native people. Mayo Clin Proc. 2016;91(1):61–70.
8.		Knudsen AB, Zauber AG, Rutter CM, et al. Estimation of benefits, burden, and harms of colorectal cancer screening strategies: modeling study for the US preventive services task force. JAMA. 2016;315(23):2595–2609.

Authors’ reply

Hermann Brenner^1–3, Hongda Chen^1,4

¹Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), ²Division of Preventive Oncology, German Cancer Research Center (DKFZ), National Center for Tumor Diseases (NCT), ³German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany; ⁴Program Office for Cancer Screening in Urban China, National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Chaoyang District, Beijing, People’s Republic of China

Correspondence: Hermann Brenner, Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 581, 69120 Heidelberg, Germany
Tel +49 6221 421301
Fax +49 6221 421302
Email [email protected]

Dear editor

We appreciate Dr. Ahlquist’s interest in our article¹ that challenges his previously disseminated claim that the multitarget stool DNA test (MSDT) represents “the new high bar benchmark for noninvasive CRC screening”.² Expectations on the diagnostic performance of this test should in fact be high, given the following facts not mentioned by Dr. Ahlquist:

•The costs of MSDT exceed the costs of fecal immunochemical test (FIT) for hemoglobin, the noninvasive test of choice in an increasing number of countries,³ ~20-fold.⁴

•MSDT requires substantially more complex logistics for stool sampling and shipping (collection and shipping of a whole bowel movement compared to a few milligrams of feces needed for FIT).

•MSDT was ~20 times more often not evaluable than FIT in the study referred to and co-authored by Dr. Ahlquist and sponsored by the manufacturer that had reported on a direct comparison of both tests.⁵

Given the different background and involvement in the MSDT development, evaluation, and promotion of the research community, we are not surprised that the results of our study, which found the diagnostic performance of a quantitative FIT to be essentially equivalent to the previously reported diagnostic performance of MSDT,⁵ would provoke rather controversial reactions.

Although all of the specific concerns raised by Dr. Ahlquist were carefully and critically addressed in our article,¹ we would like to briefly add the following few comments to each of his concerns:

1. Major differences across study populations: it is correct that the MSDT study by Dr. Ahlquist, as a result of intentional oversampling of adults ≥65 years of age, had included a much larger proportion (63%) of adults ≥65 years of age than our study (33%). Due to intentional oversampling of adults ≥65 years of age, the MSDT study was not representative of a true screening population. Nevertheless, in order to address any potential impact of the differences in age structure we reported results of additional analyses in which we adjusted our findings to the age structure of the MSDT study. As reported in our article,¹ this age adjustment did not alter any of our conclusions.
2. Selective reporting of results favoring MSDT: in his letter, Dr. Ahlquist selectively reports a single statistic from the MSDT study that apparently favors the author’s view, while not addressing other, less favorable statistics, such as the 20-fold higher rate of non-evaluable results of MSDT compared to FIT. The single statistic reported by Dr. Ahlquist is based on an exceptionally poor result for FIT obtained in the MSDT study, which is a true outlier compared to the large number of FIT studies reported to date (for details see the following paragraph).
3. Selective referencing of FIT results: Dr. Ahlquist cites FIT sensitivities for CRC from selected previous studies ranging from 50% to 75%. According to a systematic literature review by Lee et al,⁶ sensitivities of FITs for detecting CRC ranged from 25% to 100%. Furthermore, analogous comparative analyses using another widely used FIT had come to essentially the same conclusion.⁷ Dr. Ahlquist’s ascribing our results to “atypical” FIT results is therefore not justified. Quite the contrary is true: the meta-analysis by Lee et al yielded an area under the curve (AUC) of the hierarchical summary receiver-operating characteristic curve of 0.95 with a 95% confidence interval of 0.93–0.97 for 19 studies evaluating diagnostic performance of FIT for CRC detection.⁶ The AUC of 0.89 reported for FIT in the MSDT study by Ahlquist,⁵ which is well below the lower end of the 95% confidence interval of the AUC for the 19 studies included in the meta-analysis,⁶ demonstrates that the true outlier is the poor FIT performance in the MSDT study whereas the AUC of the FIT in our study was well within the range observed by other studies.
4. Unsubstantiated claims and biased comparisons: Dr. Ahlquist claims: “Using conservative point-specificities, we have estimated that MT-sDNA [note that this corresponds to MSDT in our terminology] done at the recommended frequency of every 3 years generates fewer false-positives over a longitudinal program than does FIT done annually”. The assumptions and calculations made to support this claim are neither provided nor referenced by Dr. Ahlquist. Furthermore, it is most obvious that more frequent testing decreases program specificity while increasing program sensitivity. Thus, comparing MSDT every 3 years with annual FIT and exclusively referring to specificity inevitably results in biased comparisons.

Dr. Ahlquist summarizes his remarks by concluding that caution must be taken in interpreting our recent report. While we certainly support suggestions to take care and caution in the interpretation of any scientific work, we feel that particular caution is needed when interpreting Dr. Ahlquist’s statements.

Disclosure

The authors report no conflicts of interest in this communication.

References

1.		Brenner H, Chen H. Fecal occult blood versus DNA testing: indirect comparison in a colorectal cancer screening population. Clin Epidemiol. 2017;9:377–384.
2.		Ahlquist DA. Multi-target stool DNA test: a new high bar for noninvasive screening. Dig Dis Sci. 2015;60(3):623–633.
3.		Schreuders EH, Ruco A, Rabeneck L, et al. Colorectal cancer screening: a global overview of existing programmes. Gut. 2015;64(10):1637–1649.
4.		Lin JS, Piper MA, Perdue LA, et al. Screening for colorectal cancer: updated evidence report and systematic review for the US Preventive Services Task Force. JAMA. 2016;315(23):2576–2594.
5.		Imperiale TF, Ransohoff DF, Itzkowitz SH, et al. Multitarget stool DNA testing for colorectal-cancer screening. N Engl J Med. 2014;370(14):1287–1297.
6.		Lee JK, Liles EG, Bent S, Levin TR, Corley DA. Accuracy of fecal immunochemical tests for colorectal cancer: systematic review and meta-analysis. Ann Intern Med. 2014;160(3):171.
7.		Brenner H, Werner S, Chen H. Multitarget stool DNA testing for colorectal-cancer screening. N Engl J Med. 2014;371:184–185.
		Dove Medical Press encourages responsible, free and frank academic debate. The content of the Clinical Epidemiology ‘letters to the editor’ section does not necessarily represent the views of Dove Medical Press, its officers, agents, employees, related entities or the Clinical Epidemiology editors. While all reasonable steps have been taken to confirm the content of each letter, Dove Medical Press accepts no liability in respect of the content of any letter, nor is it responsible for the content and accuracy of any letter to the editor.