VasculAR - Integration of Deep Learning into automatic volumetric cardiovascular dissection and reconstruction in simulated 3D space for medical practice

Short title: 3D Cardiovascular Reconstruction via Deep Learning

Accurate and efficient analysis of cardiac images is essential for diagnosing and treating cardiovascular disease, a major cause of mortality globally, including Vietnam. However, this task is time-consuming, labor-intensive, and prone to errors. The 2D data complicates analysis due to the anatomical complexity and diverse cardiovascular pathologies. To address these challenges, we developed a comprehensive software solution tailored to cardiac data of patients. We assembled an intricate Vietnamese Heart Segmentation and Cardiovascular Disease Dataset (VHSCDD) from local hospitals, ensuring practical application. Our research explored advanced neural network architectures, including CNN-based and Transformer-based models, with different novel attention mechanisms and loss functions. Specifically, by redesigning the feature extraction layers, attention mechanism, and upsampling method out of TransUNet, we introduced RotCAtt-TransUNet++, which outperforms current state-of-the-art models for cardiac segmentation. Furthermore, we optimized the Marching Cubes algorithm for rapid, detailed 3D reconstruction of cardiac structures. To enhance diagnostic capabilities, we developed innovative algorithms integrating multivariable calculus and 3D Fenwick trees for quantitative post-reconstruction analysis. Furthermore, our software features a virtual reality simulation enabling experts to multidimensionally interact with 3D cardiac models, perform measurements, and simulate surgical scenarios. This comprehensive system automates segmentation and disease prediction while facilitating data storage on cloud databases. Expert evaluations confirm that our software significantly improves cardiac defect observation, making it highly applicable in medical practice and anatomy education.

ISEF Virutal Display

Quad chart
Slide Presentation
Video Presentation

Project's Repos

VHSCDD: Newly-established dataset (paper)
RotCAtt-TransUNet++: Novel Deep Learing Architecture (paper with code)
Post-analysis procedures: Volume Computation Algorithm (paper with code)

Demo Videos

Old demo video
New demo video 1
New demo video 2