基于扩散的多模态医学图像融合

Diffusion-based multimodal medical image fusion

  • 摘要: 近年来,随着医学影像学技术的持续发展,图像融合技术在医学图像分析中得到了广泛应用。传统的融合技术受限于人工设计的特征提取过程,导致对图像语义信息的理解和匹配精度不高,且无法充分利用多模态图像的信息。通过研究一种基于扩散模型的多模态图像融合方法,利用扩散模型逐步学习多通道图像在潜在空间的联合特征,克服单一端到端网络学习能力有限的问题,生成高质量的融合图像,并对逆向去噪过程针对多模态医学图像融合任务做出了改进。引入了两个模态判别器加强去噪网络对于模态特征的理解,充分利用不同成像模态之间的互补性信息。在AANLIB数据集上的实验表明,该方法实现了令人满意的融合结果。

     

    Abstract: In recent years, with the continuous development of medical imaging technology, image fusion techniques have been widely applied in medical image analysis. Traditional fusion methods are limited by manually designed feature extraction processes, resulting in limited accuracy in understanding and matching of image semantic information, and inability to fully utilize the information from multimodal images. A diffusion-based multimodal image fusion method is investigated in this paper. This method progressively learns the joint features of multi-channel images in the latent space using a diffusion model to overcome the limited learning capability of single end-to-end networks. And it generates high-quality fused images and improve the reverse denoising process specifically for the task of multimodal medical image fusion. Two modal discriminators are incorporated to enhance the denoising network’s understanding of modality-specific features, fully leveraging the complementary information between different imaging modalities. Experiments on the AANLIB dataset demonstrate that the proposed method achieves satisfactory fusion results.

     

/

返回文章
返回