Multi-scale Blind Image Restoration Algorithm Based on Salient Region Detection

doi:10.12382/bgxb.2023.0992

Abstract

Abstract:

The blind image deblurring algorithm based on priority takes a long time for image deblurring,and has aunidealsalient edge structure extraction capablity. A multi-scale blind image restoration algorithm based on salient region detection is proposed.In order to restore more clear images, an image pyramid is constructed by using a rough-to-finemulti-scale iterative framework. In the aspect of image single scale, asalient region with strong edge structure is extracted first, and the l₀ norm constraint is applied to it. Asalient mapping prior is proposed. Then the salient mapping prior and the maximum posteriori probability are introduced into the traditional image deblurring model to construct a point spread function estimation model, and the semi-quadratic splitting algorithm is usedto solve the non-convex problem of the model. When restoring the point spread function, the change of point spread function similarity is used to limit the transition iteration in each scale. Finally, the fuzzy image and the final estimated point spread function are deconvolved to obtain the restored image. The experimental results show that, compared with the existing mainstream deblurring algorithms, the proposed algorithm can effectively suppress the ringing and artifact phenomena in both synthetic and real data setsand get a good visual experience, the evaluation indexes are better than those of the comparison algorithm, and the image restoration time is greatly reduced.

Key words: multi-scaleimage, salient edge structure, point spread function, semi-quadratic splitting algorithm, point spread function similarity

CLC Number:

TP391

ZHAO Xiaoqiang, WANG Tao, SONG Zhaoyang, JIANG Hongmei. Multi-scale Blind Image Restoration Algorithm Based on Salient Region Detection[J]. Acta Armamentarii, 2024, 45(11): 4020-4030.

Figures/Tables 14

References 29

[1]	陈雄锋, 阮驰. 多参数最优重构水下偏振成像复原方法[J]. 兵工学报, 2023, 44(7):2122-2131. doi: 10.12382/bgxb.2022.0343
	CHEN X F, RUAN C. Multi-parameter optimal reconstruction of underwater polarization imaging restoration method[J]. Acta Armamentarii, 2023, 44(7):2122-2131. (in Chinese)
[2]	JIA J Y. Single image motion deblurring using transparency[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Minneapolis, MN, US:IEEE, 2007:1-8.
[3]	CHO S H, LEE S Y. Fast motion deblurring[J]. ACM Transactions on Graphics, 2009, 28(5): 1-8.
[4]	SUN L B, CHO S H, WANG J, et al. Edge-based blur kernel estimation using patch priors[C]// Proceedings of IEEE International Conference on Computational Photography.Cambridge, MA, US: IEEE, 2013:1-8.
[5]	XU L, JIA J Y. Two-phase kernel estimation for robust motion deblurring[C]// Proceedings of European Conference on Computer Vision.Berlin, Heidelberg, Germany: Springer, 2010:157-170.
[6]	FERGUS R, SINGH B, HERTZMANN A, et al. Removing camera shake from a single photograph[J]. ACM Transactions on Graphics, 2006, 25(25):787-794.
[7]	LEVIN A, WEISS Y, DURAND F, et al. Understanding and evaluating blind deconvolution algorithms[C]// Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition.Miami, FL, US:IEEE, 2009:1964-1971.
[8]	KRISHNAN D, FERGUS R. Fast image deconvolution using hyper-laplacian priors[C]// Proceedings of the 22nd International Conference on Neural Information Processing Systems. Red Hook, NY, US: Curran Associates Inc.,2009:1033-1041.
[9]	XU L, ZHENG S C, JIA J Y. Unnatural L0 Sparse representation for natural image deblurring[C]// Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US: IEEE,2013:1107-1114.
[10]	KOTERA J, FILIP S, MILANFAR P. Blind deconvolution using alternating maximum a posteriori estimation with heavy-tailed priors[C]// Proceedings of International Conference on Computer Analysis of Images and Patterns. Berlin, Heidelberg, Germany:Springer, 2013:59-66.
[11]	PAN J S, HU Z, SU Z X, et al. Deblurring text images via L0-regularized intensity and gradient prior[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, US:IEEE, 2014:2901-2908.
[12]	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(12):2341-2353.
[13]	PAN J S, SUN D Q, PFISTER H, et al. Deblurring images via dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(10):2315-2328. doi: 10.1109/TPAMI.2017.2753804 pmid: 28952935
[14]	YAN Y Y, REN W Q, GUO Y F, et al. Image deblurring via extreme channels prior[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu, HI, US:IEEE, 2017:4003-4011.
[15]	CHEN L, FANG F M, WANG T T, et al. Blind image deblurring with local maximum gradient prior[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, US: IEEE, 2019: 1742-1750.
[16]	WEN F, YING R D, LIU Y P, et al. A simple local minimal intensity prior and an improved algorithm for blind image deblurring[J]. IEEE Transactions on Circuits and for Video Technology, 2021, 31(8):2923-2937.
[17]	HU D D, TAN J Q, ZHANG L, et al. Image deblurring based on enhanced salient edge selection[J]. The Visual Computer, 2023,39:281-296.
[18]	GE X Y, TAN J Q, ZHANG L, et al. Blind image deblurring with Gaussian curvature of the image surface[J]. Signal Processing: Image Communication, 2022, 100:116531.
[19]	GE X Y, TAN J Q, ZHANG L. Blind Image deblurring using a non-linear channel prior based on dark and bright channels[J]. IEEE Transactions on Image Processing, 2021, 30: 6970-6984.
[20]	LIU J, TAN J Q, GE X Y, et al. Blind deblurring with fractional-order calculus and local minimal pixel prior[J]. Journal of Visual Communication and Image Representation, 2022, 89: 103645.
[21]	HU D D, TAN J Q, GE X Y. Image deconvolution using mixed-order salient edge selection[J]. Circuits, Systems, and Signal Processing, 2023,42:3902-3925.
[22]	FENG X, TAN J Q, GE X Y, et al. Blind image deblurring via weighted dark channel prior[J]. Circuits, Systems, and Signal Processing, 2023,42: 5478-5499.
[23]	ZHAI Y, SHAH M. Visual attention detection in video sequences using spatiotemporal cues[C]// Proceedings of the 14th ACM International Conference on Multimedia.New York, NY, US:ACM, 2006:815-824.
[24]	XU L, LU C W, XU Y, et al. Image smoothing via L0 gradient minimization[C]// Proceedings of the 2011 SIGGRAPH Asia conference.New York, NY, US:ACM, 2011:1-12.
[25]	PAN J S, HU Z, SU Z X, et al. L0-regularized intensity and gradient prior for deblurring text Images and beyond[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(2): 342-355.
[26]	HU D D, TAN J Q, ZHANG L, et al. Image deblurring via enhanced local maximum intensity prior[J]. Signal Processing: Image Communication, 2021, 96:116311.
[27]	LIU J, TAN J Q, ZHANG L, et al. Blind deblurring with patch-wise second-order gradient prior[J]. Signal Processing: Image Communication, 2022, 107:116781.
[28]	KOEHLER R, HIRSCH M, MOHLER B, et al. Recording and playback of camera shake: benchmarking blind deconvolution with a real-world database[C]// Proceedings of the 12nd European Conference on Computer Vision. Cham, Germany: Springer, 2012:27-40.
[29]	陈晨, 许金鑫, 危才华, 等. 基于显著性强度和梯度先验的多尺度图像盲去模糊[J]. 激光与光电子学进展, 2020, 57(4):271-277.
	CHEN C, XU J X, WEI C H, et al. Blind deblurring of multi-scale images based on significance intensity and gradient prior[J]. Advances in Laser and Optoelectronics, 2020, 57(4):271-277. (in Chinese)

算法	小孩图片		卡通图片		房子图片		女孩图片		平均 PSNR/dB	平均 SSIM
算法	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	平均 PSNR/dB	平均 SSIM
文献[8]算法	22.1419	0.6464	19.9132	0.5324	19.7796	0.4962	21.2178	0.6374	20.7631	0.5781
文献[10]算法	21.4388	0.6156	19.6025	0.5571	18.5714	0.4653	21.8618	0.6762	20.3685	0.5785
文献[11]算法	16.1998	0.3732	19.0508	0.5231	16.0607	0.3656	19.6773	0.5633	17.7471	0.4563
文献[13]算法	22.0268	0.6651	22.5046	0.6871	20.5868	0.5636	24.3199	0.7867	22.3595	0.6756
文献[14]算法	22.6196	0.6941	22.9667	0.7406	20.9179	0.5844	23.7872	0.7776	22.5729	0.6991
文献[15]算法	22.4674	0.6855	23.2551	0.7464	21.5309	0.6038	23.5795	0.7710	22.7072	0.7017
文献[16]算法	22.9345	0.7062	23.2829	0.7403	21.5023	0.6023	24.1505	0.7836	22.9675	0.7081
文献[29]算法	23.1677	0.7105	22.2148	0.7061	21.2597	0.5545	24.1857	0.7776	22.7069	0.6871
本文算法	23.3789	0.7114	22.7877	0.7196	22.1602	0.6149	24.4595	0.7948	23.1816	0.7107

算法	小孩图片		卡通图片		房子图片		女孩图片		平均 PSNR/dB	平均 SSIM
算法	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	平均 PSNR/dB	平均 SSIM
文献[8]算法	22.1419	0.6464	19.9132	0.5324	19.7796	0.4962	21.2178	0.6374	20.7631	0.5781
文献[10]算法	21.4388	0.6156	19.6025	0.5571	18.5714	0.4653	21.8618	0.6762	20.3685	0.5785
文献[11]算法	16.1998	0.3732	19.0508	0.5231	16.0607	0.3656	19.6773	0.5633	17.7471	0.4563
文献[13]算法	22.0268	0.6651	22.5046	0.6871	20.5868	0.5636	24.3199	0.7867	22.3595	0.6756
文献[14]算法	22.6196	0.6941	22.9667	0.7406	20.9179	0.5844	23.7872	0.7776	22.5729	0.6991
文献[15]算法	22.4674	0.6855	23.2551	0.7464	21.5309	0.6038	23.5795	0.7710	22.7072	0.7017
文献[16]算法	22.9345	0.7062	23.2829	0.7403	21.5023	0.6023	24.1505	0.7836	22.9675	0.7081
文献[29]算法	23.1677	0.7105	22.2148	0.7061	21.2597	0.5545	24.1857	0.7776	22.7069	0.6871
本文算法	23.3789	0.7114	22.7877	0.7196	22.1602	0.6149	24.4595	0.7948	23.1816	0.7107

算法	Blurry3_10		Blurry3_10
算法	PSNR/dB	SSIM	PSNR/dB	SSIM
文献[13]算法	19.9818	0.6549	18.6300	0.6905
文献[14]算法	20.8745	0.6938	18.9163	0.7045
文献[15]算法	20.3550	0.6697	18.3348	0.6773
文献[16]算法	20.9378	0.6925	18.4116	0.6739
文献[29]算法	13.8350	0.3477	18.3762	0.6868
本文算法	20.9735	0.7054	19.0032	0.7104

算法	Blurry3_10		Blurry3_10
算法	PSNR/dB	SSIM	PSNR/dB	SSIM
文献[13]算法	19.9818	0.6549	18.6300	0.6905
文献[14]算法	20.8745	0.6938	18.9163	0.7045
文献[15]算法	20.3550	0.6697	18.3348	0.6773
文献[16]算法	20.9378	0.6925	18.4116	0.6739
文献[29]算法	13.8350	0.3477	18.3762	0.6868
本文算法	20.9735	0.7054	19.0032	0.7104

算法	PSNR/dB	SSIM
文献[13]算法	16.5744	0.6738
文献[14]算法	16.8869	0.6742
文献[15]算法	16.3404	0.6649
文献[16]算法	17.1165	0.7101
本文算法	17.3991	0.7323