Journal of Artificial Intelligence for Medical Sciences
Home > TMRGM: A Template-Based Multi-Attention Model for X-Ray Imaging Report Generation
974 views 1893 downloads
Research Articie May 05,2021
TMRGM: A Template-Based Multi-Attention Model for X-Ray Imaging Report Generation
Xuwen Wang 1 ,  Yu Zhang 2 ,  Zhen Guo 3 ,  Jiao Li 4 hide author's information
Keywords: Chest X-ray; Deep learning; Thoracic abnormality recognition; Medical imaging report generation; Attention mechanism; Medical imaging report template
Cite this article: Wang X, Zhang Y, Guo Z, Li J. TMRGM: A Template-Based Multi-Attention Model for X-Ray Imaging Report Generation. JAIMS [Internet]. 2021;2(1-2):21-32.
Full Text PDF
Download Citation
Abstract


The rapid growth of medical imaging data brings heavy pressure to radiologists for imaging diagnosis and report writing. This paper aims to extract valuable information automatically from medical images to assist doctors in chest X-ray image interpretation. Considering the different linguistic and visual characteristics in reports of different crowds, we proposed a template-based multi-attention report generation model (TMRGM) for the healthy individuals and abnormal ones respectively. In this study, we developed an experimental dataset based on the IU X-ray collection to validate the effectiveness of TMRGM model. Specifically, our method achieves the BLEU-1 of 0.419, the METEOR of 0.183, the ROUGE score of 0.280, and the CIDEr of 0.359, which are comparable with the SOTA models. The experimental results indicate that the proposed TMRGM model is able to simulate the reporting process, and there is still much room for improvement in clinical application.