Google AI assisted breast cancer screening misdiagnosis rate is lower than human!

Wuxi apptec AI / Report At the beginning of the new year, Google Health has brought us good news Its latest people Google and deepmind, Cancer Research UK, Northwestern University and royal Surrey County Hospital) is working closely to find out if AI can help radiologists more accurately detect signs of breast cancer, so as to find the best way to improve breast cancer screening technology Recently, the research team published its preliminary results in the journal Nature They found that AI models were able to correctly screen breast cancer signs in screening images at a similar level to experts, thus reducing the false negative (also known as missed diagnosis rate) and false positive (also known as misdiagnosis rate) results of screening Doctors' reading mistakes will lead to false negative or false positive reports of breast cancer patients Such misdiagnosis will not only lead to patients not receiving timely and effective treatment, but also bring unnecessary psychological pressure to them, and bring more workload to radiologists Using artificial intelligence technology, it is possible to improve the occurrence of these problems The AI system consists of three deep learning models, each of which will be used for different levels of analysis, respectively for the analysis of the following three cases: single lesion analysis, single breast analysis, and overall case analysis Each model will produce a cancer risk score between 0 and 1 for the pathology of breast X-ray image, and the prediction accuracy of the overall AI system is generated by combining the average values predicted by these three models Two large datasets for training AI models are from the UK and the US The UK data set was collected from three NHSBSP breast screening sites, including more than 76000 women's mammograms; the US data set collected about 15000 patients' mammograms from northwest Memorial Hospital in Chicago between 2001 and 2018 Later, they used a separate data set to evaluate and verify the reliability of the above training results, which consisted of more than 25000 patient images (25000 in the UK and 3000 in the US) After the evaluation and validation, the missed diagnosis and misdiagnosis of the predicted results of the model were all reduced Compared with the clinical practice data of the United States, AI reduced 5.7% false positive reports and 9.4% false negative reports; compared with the clinical practice data of the United Kingdom, AI reduced 1.2% false positive reports and 2.7% false negative reports This is a "leap" for the current rate of 20% of breast cancer missed detection ▲ breast cancer prediction performance: artificial intelligence system vs clinician (picture source: reference [1]) In addition, the researchers also verified whether the AI model can be used in other medical systems First, they trained AI only according to the data set of British patients, and then used the data set of American patients as the evaluation and verification set of the model In this separate experiment, ai not only predicted breast cancer more accurately than human experts, but also reduced the false positive and false negative reporting rates by 3.5% and 8.1% respectively, which means that the AI system has been applied to other medical rings The potential of environment Finally, in order to further verify the reliability of the system, the researchers called six radiologists to "read the map PK" with AI By interpreting 500 cases from the United States, AI's recognition accuracy was better At the same time, according to the researchers, the essence of this system is to better assist doctors rather than replace doctors, so the current achievements of the research team are enough to prove that human doctors and AI doctors are complementary in the future medical environment AI reduces the rate of missed tests, and human doctors can also point out cases of diseases that AI does not recognize ▲ (a) sample cases that were not identified by 6 experts but successfully identified by AI; (b) sample cases that were successfully identified by 6 experts but missed by AI (picture source: reference [1]) In addition, it should be noted that AI can still recognize breast cancer more accurately with less historical information Compared with human doctors who have patient's medical history and historical data to trace when making decisions, AI only processes and recognizes the X-ray image in front of it, and there is no historical information for reference From this point of view, AI's decision-making process is "more independent" Looking forward to the future, the preliminary results of the study are gratifying This model will help to improve the accuracy and efficiency of breast cancer screening procedures, and reduce the time and pressure for patients to wait for the diagnosis results However, clinical medicine is complex Doctors' decision-making is not a simple binary decision (between the existence of cancer or not), and other signs and symptoms must be considered Acquiring other data is also more complicated from a technical point of view, so in the future, it may be better to use a system that can query electronic medical records to identify and annotate specific cases in combination with the latest AI reference material: [1] McKinney, S.M., Sieniek, M., Godbole, V et al International evaluation of an AI system for breast cancer screening Nature 577, 89–94 (2020) doi:10.1038/s41586-019-1799-6 [2] Google just beat humans at spotting breast cancer — but it won’t replace them Retrieved Jan 2, 2020 from https://www.theverge.com/2020/1/1/21045635/google-ai-detect-breast-cancer-mammograms-healthcare [3] Google Health’s AI identifies breast cancer in mammogram imagery with fewer false positives Retrieved Jan 2, 2020 from https://venturebeat.com/2020/01/01/google-healths-ai-identifies-breast-cancer-in-mammogram-imagery-with-fewer-false-positives/ [4] Using AI to improve breast cancer screening Retrieved Jan 2, 2020 from https://www.blog.google/technology/health/improving-breast-cancer-screening/