MIRFuse: an infrared and visible image fusion model based on disentanglement representation via mutual information regularization - Université d'Évry
Article Dans Une Revue Journal of Electronic Imaging Année : 2024

MIRFuse: an infrared and visible image fusion model based on disentanglement representation via mutual information regularization

Résumé

In the domain of image fusion, integrating infrared and visible images provides a more complete scene description by merging the unique strengths of each modality. Existing methods struggle with handling the differences between modalities, which is caused by the inherent entanglement of scene-common information and modality-specific information within each modality. In response, we propose MIRFuse, a model for infrared and visible image fusion based on disentanglement representation via mutual information regularization. The process of disentanglement, in which scene-common information and modality-specific information are separated, forms the basis for identifying both shared and exclusive features. First, mutual information maximization is used as consistency constraint, enabling scene-common encoders to effectively extract shared features. Second, the Hilbert–Schmidt independence criterion is employed as heterogeneity constraint, promoting modality-specific encoders to extract exclusive features. Finally, both shared and exclusive features are identified and combined using various fusion strategies to produce a fused image. The resulting fused image provides a comprehensive representation of the entire scene, allowing for more effective utilization of information from multiple modalities. Our experiments have validated the advanced nature and effectiveness of our method.
Fichier non déposé

Dates et versions

hal-04588319 , version 1 (26-05-2024)

Identifiants

Citer

Shiliang Zhou, Jun Kong, Min Jiang, Danfeng Zhuang. MIRFuse: an infrared and visible image fusion model based on disentanglement representation via mutual information regularization. Journal of Electronic Imaging, 2024, 33 (02), ⟨10.1117/1.JEI.33.2.023005⟩. ⟨hal-04588319⟩
103 Consultations
0 Téléchargements

Altmetric

Partager

More