Back to Publications

Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting

Chantal Pellegrini, Matthias Keicher, Ege Özsoy, Nassir Navab

MICCAI 2023 · arXiv:2307.05766

arXiv Publisher (Springer) Code (GitHub)

Abstract

We introduce Rad-ReStruct, a new benchmark dataset that provides fine-grained, hierarchically ordered annotations in the form of structured reports for X-Ray images. Our dataset provides structured labels for the images in the IU-XRay dataset available on OpenI. We model the structured reporting task as hierarchical visual question answering (VQA) and propose hi-VQA, a novel method that considers prior context in the form of previously asked questions and answers for populating a structured radiology report. The repository includes the structured reports as well as the list of questions to fill this report for every image.

Rad-ReStruct overview

Citation

@article{pellegrini2023radrestruct,
  title={Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting},
  author={Pellegrini, Chantal and Keicher, Matthias and {\"O}zsoy, Ege and Navab, Nassir},
  journal={MICCAI},
  year={2023}
}