Surveys in Mathematics and its Applications

ISSN 1842-6298 (electronic), 1843-7265 (print)
 Volume 19 (2024), 179 -- 195

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

NOVEL ALGORITHM FOR MULTIFOCUS IMAGE FUSION: INTEGRATION OF CONVOLUTIONAL NEURAL NETWORK AND PARTIAL DIFFERENTIAL EQUATION

Gargi J Trivedi and Rajesh Sanghvi

Abstract. This paper presents a novel method for Multifocus image fusion that combines anisotropic diffusion PDE filtering and convolutional neural network (CNN) feature extraction. The proposed method aims to preserve image edges and details while reducing noise through the utilization of anisotropic diffusion PDE filtering. Additionally, a CNN architecture with ReLU activation function is employed for feature extraction. The method is evaluated on a dataset of Multifocus images and compared with traditional and CNN-based approaches, demonstrating superior performance in terms of visual quality and quantitative metrics, such as Normalized Mutual Information, Phase Congruency-based metric, and Structural Similarity-based metric. Furthermore, we aim to enhance our approach by incorporating machine learning techniques to optimize the parameters of the image fusion algorithm. By automatically adjusting these parameters, we strive to achieve the most reliable and accurate outcomes.

2020 Mathematics Subject Classification: 34A08; 54H30; 47H10; 47H20; 68U10; 94A08
Keywords: Partial Differential Equation (PDE), Machine learning (ML), Convolutional neural network (CNN), Image fusion (IF), Multifocus images (MF).

Full text

References

  1. U. Ali, I.H. Lee, M.T. Mahmood, Incorporating structural prior for depth regularization in shape from focus, Computer Vision and Image Understanding, 227 (2023), article 103619. https://doi.org/10.1016/j.cviu.2022.103619.

  2. Yin Chen, Rick S. Blum, A new automated quality assessment algorithm for image fusion, Image and Vision Computing, 27, no. 10 (2009), 1421-1432. DOI: 10.1016/j.imavis.2007.12.002.

  3. C. Cheng, T. Xu, X.-J. Wu, MUFusion: A general unsupervised image fusion network based on memory unit, Information Fusion, 92 (2023), 80–92. 10.1016/j.inffus.2022.11.010.

  4. D. Cui, Image segmentation algorithm based on partial differential equation, Journal of Intelligent and Fuzzy Systems, 40 (4) (2021), 5945--5952. 10.14708/ma.v51i2.7204. MR2076335. Zbl 1058.11001.

  5. R.C. Gonzalez, R.E. Woods, S.L. Eddins, Digital image processing using MATLAB, Gatesmark Publishing, 2020.

  6. L. Guo, L. Liu, A Perceptual-Based Robust Measure of Image Focus, IEEE Signal Processing Letters, 29 (2022), 2717–21. 10.1109/lsp.2023.3235647.

  7. M. Hossny, S. Nahavandi, D. Creighton, A. Bhatti, Image fusion performance metric based on mutual information and entropy driven quadtree decomposition, Electronics Letters, 46, no. 18 (2010), 1266–1268. Print ISSN 0013-5194, Online ISSN 1350-911X. DOI: 10.1049/el.2010.1778.

  8. C.-G. Im, D.-M. Son, H.-J. Kwon, S.-H. Lee, Tone Image Classification and Weighted Learning for Visible and NIR Image Fusion, Entropy, 24, no. 10 (2022), article 1435. 10.3390/e24101435.

  9. H. Kaur, D. Koundal, V. Kadyan, Image Fusion Techniques: A Survey, Archives of Computational Methods in Engineering, 28, no. 7 (2021), 4425–4447. 10.1007/s11831-021-09540-7.

  10. L. Li, C. Li, X. Lu, H. Wang, D. Zhou, Multi-focus image fusion with convolutional neural network based on Dempster-Shafer theory, Optik, 272 (2023) article 170223. 10.1016/j.ijleo.2022.170223.

  11. S. Liu, W. Peng, W.g Jiang, Y.Yang, J. Zhao, Y.Su, Multi-focus image fusion dataset and algorithm test in real environment, Frontiers in Neurorobotics, 16 (2022). doi.org/10.3389/fnbot.2022.1024742.

  12. Yu Liu, Lei Wang, Juan Cheng, Chang Li, Xun Chen, Multi-focus image fusion: A survey of the state of the art, Information Fusion, 24 (2020), 71--91.

  13. M. Nejati, Lytro Multi-focus Image Dataset, ResearchGate, 2016, January.

  14. Y. Niu, L. Shen, X. Huo, G. Liang, Multi-Objective Wavelet-Based Pixel-Level Image Fusion Using Multi-Objective Constriction Particle Swarm Optimization, Studies in Computational Intelligence, (2010), 151–78. 10.1007/978-3-642-05165-4_7.

  15. X. Pan, Q. Zhao, and J. Liu, Edge extraction and reconstruction of terahertz image using simulation evolutionary with the symmetric fourth order partial differential equation, Optoelectronics Letters, 17 (3) (2023), 187--192. doi.org/10.3389/fnbot.2022.1024742.

  16. P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, Proceedings of IEEE Computer Society Workshop on Computer Vision, November 1987, 16--22.

  17. P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, no. 7 (1990), 629--639, 10.1109/34.56205.

  18. V. Rajinikanth, S.C. Satapathy, N. Dey, R. Vijayarajan, DWT-PCA Image Fusion Technique to Improve Segmentation Accuracy in Brain Tumor Analysis, in Lecture Notes in Electrical Engineering, 2018, 453–62. 10.1007/978-981-10-7329-8_46.

  19. J. Sliz and J. Mikulka, Advanced image segmentation methods using partial differential equations: A concise comparison, 2016 Progress in Electromagnetic Research Symposium (PIERS), IEEE, 2016.

  20. G.J. Trivedi, R. Sanghvi, Medical Image Fusion Using CNN with Automated Pooling, Indian Journal Of Science And Technology, 15, no. 42 (2022), 2267–74. 10.17485/ijst/v15i42.1812.

  21. G. Trivedi and R. Sanghvi, Fusesharp: A multi-image focus fusion method using discrete wavelet transform and unsharp masking, Journal of Applied Mathematics & Informatics, 41 (5) (2023), 1115--1128. DOI: 10.14317/jami.2023.1115. Zbl 07793714.

  22. G. Trivedi and R. Sanghvi, Optimizing image fusion using modified principal component analysis algorithm and adaptive weighting scheme, International Journal of Advanced Networking and Applications, 15, no. 01 (2023), 5769--5774. 10.35444/ijana.2023.15103.

  23. G. Trivedi and R. Sanghvi, Hybrid Model for Infrared and Visible Image Fusion, Annals of the Faculty of Engineering Hunedoara, 21, no. 3 (2023), 167--173. https://www.proquest.com/scholarly-journals/hybrid-model-infrared-visible-image-fusion/docview/2867370935/se-2.

  24. G. Trivedi and R. Sanghvi, Novel approach to multi-modal image fusion using modified convolutional layers, Journal of Innovative Image Processing, 5 (3) (2023), p. 229. DOI: 10.36548/jiip.2023.3.002.

  25. G. Trivedi and R. Sanghvi, MOSAICFUSION: Merging modalities with Partial differential equation and Discrete cosine transformation, Journal of Applied & Pure Mathematics, 5, no. 5--6 (2023), 389--406. DOI: 10.23091/japm.2023.3892.

  26. G. Trivedi and R. Sanghvi, Automated multimodal fusion with PDE preprocessing and learnable convolutional pools, ADBU-Journal of Engineering Technology, 13, no. 1 (2024), p. 0130104066.

  27. G. J. Trivedi and R. Sanghavi, MSCNN-Multi-Sensor Image Fusion Using Dual channel CNN, Mathematica Applicanda (Matematyka Stosowana), 51, no. 2 (2023), 165--182. doi.org/10.14708/ma.v51i2.7204. MR4713481.

  28. G. T. Vasu, P. Palanisamy, Gradient-based multi-focus image fusion using foreground and background pattern recognition with weighted anisotropic diffusion filter, Signal, Image and Video Processing, 2023. 10.1007/s11760-022-02470-2.

  29. G. Xiao, D.P. Bavirisetti, G. Liu, X. Zhang, Image Fusion, Springer, 2020. 10.1007/978-981-15-4867-3.

  30. G. Zhang, R. Nie, J. Cao, L. Chen, Y. Zhu, FDGNet: A pair feature difference guided network for multimodal medical image fusion, Biomedical Signal Processing and Control, 81 (2023), article 104545. 10.1016/j.bspc.2022.104545.

  31. L. Zhou, A Gradient-based Multi-focus Image Fusion Method Using Multiwavelets Transform, in 2012 International Conference on Industrial Control and Electronics Engineering, 2012. 10.1109/icicee.2012.110.

  32. T. Zhou, Q. Li, H. Lu, Q. Cheng, X. Zhang, GAN review: Models and medical image fusion applications, Information Fusion, 91 (2023), 134–148. 10.1016/j.inffus.2022.10.017.



Gargi J Trivedi
The Charutar Vidyamandal University,
Department of Applied Science & Humanities,
G H Patel College of Engineering & Technology,
Vallabh Vidhyanagar-388120, India.
e-mail: gargi1488@gmail.com

Rajesh Sanghvi
The Charutar Vidyamandal University,
Department of Applied Science & Humanities,
G H Patel College of Engineering & Technology,
Vallabh Vidhyanagar-388120, India.
e-mail: rajeshsanghvi@gcet.ac.in

https://www.utgjiu.ro/math/sma