HE Yan,HE Jia.Research on Oil Painting Restoration of Non-local Attention Mechanism Generative Adversarial Network[J].Journal of Chengdu University of Information Technology,2022,37(01):34-39.[doi:10.16836/j.cnki.jcuit.2022.01.006]
Non-local注意力机制生成对抗网络的油画修复研究
- Title:
- Research on Oil Painting Restoration of Non-local Attention Mechanism Generative Adversarial Network
- 文章编号:
- 2096-1618(2022)01-0034-06
- Keywords:
- generative adversarial network; Non-local; oil painting restoration; dilated convolution; gated convolution
- 分类号:
- TP751
- 文献标志码:
- A
- 摘要:
- 针对部分油画艺术作品存在图像破损的问题，提出一种基于非局部(Non-local)注意力机制生成对抗网络的油画修复方法。首先，在生成器部分，采用扩张卷积和门控卷积替代原网络中的普通卷积层，增强网络的特征提取能力，同时加入Non-local注意力机制，提升生成器的修复能力; 其次，使用马尔科夫判别器，强化网络的判别效果; 最后，在损失函数部分使用感知损失、GAN损失和L1损失，使整个网络的训练更加稳定。网络在开源的Gallerix油画数据集上进行了验证，实验结果表明:与Global&Local、ParticalConvGAN和Deepfillv2的修复方法相比，PSNR和SSIM指标均超过3种优秀的生成对抗网络，对油画艺术品的修复的效果进行了更好的提升。
- Abstract:
- Aiming at the problem of image damage in some oil painting artworks, a method of oil painting restoration based on Non-local attention mechanism generative adversarial networksgeneration adversarial network is proposed. First, in the part of the generator part, dilated convolution and gated convolution are used to replace the ordinary convolution layer in the original network, to strengthen the judgment effect of the network, and a non-local attention mechanism is added to improve the repair abilityrestoration effect of the generator; secondly, Markov discrimination is used to strengthen the discriminative effect of the network; finally, the loss function uses perceptual loss, GAN loss, and L1 loss are used in the loss function to make, which makes the training of the entire network more stable. The network of this article is verified on the open source Gallerix oil painting dataset. The experimental results show that the PSNR and SSIM indicators surpass the three excellent generative adversarial networks, and improve the effect of oil painting artwork restoration.
参考文献/References:
[1] 郭军胜.大数据背景下的油画破损区域修复算法设计[J].现代电子技术,2020,43(19):31-34.
[2] Bertalmio M,Sapiro G,Caselles V,et al.Image Inpainting[C].Proceedings of ACM SIGGRAPH. New Orleans,2000:417-424.
[3] Yu J,Lin Z,Yang J,et al.Generative Image Inpainting with Contextual Attention[C].IEEE Conference on Computer Vision and Pattern Recognition.United States:IEEE,2018:5505-5514.
[4] 程颖.基于生成对抗网络图像补全的研究与实现[D].成都:电子科技大学,2020.
[5] 赵然.基于深度学习的图像修复方法综述[J].科技风,2020(18):130-137.
[6] Satoshi Iizuka,Edgar Simo-Serra,and Hiroshi Ishikawa.Globally and locally consistent image completion[J].ACM Transactions on Graphics(TOG),2017,36(4):1-14.
[7] Guilin Liu,Fitsum A.Reda,Kevin J.Shih, et al. Image Inpainting for Irregular Holes Using Partial Convolutions[C].European Conference on Computer Vision.Germany:ECCV,2018:85-100.
[8] Yu J,Lin Z,Yang J,et al.Free-Form Image Inpainting with Gated Convolution[C].IEEE International Conference on Computer Vision.Korea:ICCV,2019:4470-4479.
[9] GoodfellowI J,Pouget-Abadie J,Mirza M,et al.Generative adversarial nets[C].International Conference on Neural Information Processing Systems.Canada,2014:2672-2680.
[10] F Yu,Koltun V.Multi-Scale Context Aggregation by Dilated Convolutions[OL].https://arxiv.org/abs/arXiv:1511.07122,2016.
[11] Wang X,Girshick R,Gupta A,et al.Non-local Neural Networks[C].IEEE Conference on Computer Vision and Pattern Recognition.United States:IEEE,2018:7794-7803.
[12] Dauphin Y N,Fan A,Auli M,et al.Language Modeling with Gated Convolutional Networks[C].International Conference on Machine Learning.China,2017:933-941.
[13] Chen L C,Papandreou G,Kokkinos I,et al.Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs[J].Computer Science,2014(4):357-361.
[14] Kingma D,Ba J.Adam:A Method for Stochastic Optimization[EB/OL].https://arxiv.org/abs/arXiv:1412.6980,2014.
[15] Zhou W,Bovik A C,Sheikh H R,et al. Image quality assessment: from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004:600-612.
相似文献/References:
[1]王 蕾,李媛茜.基于生成对抗网络的CT图像生成[J].成都信息工程大学学报,2021,36(03):286.[doi:10.16836/j.cnki.jcuit.2021.03.008]
WANG Lei,LI Yuanqian.CT Image Generation based on Generative Adversarial Network[J].Journal of Chengdu University of Information Technology,2021,36(01):286.[doi:10.16836/j.cnki.jcuit.2021.03.008]
备注/Memo
收稿日期:2021-07-06
基金项目:四川省科技厅苗子工程资助项目(2019Z118); 四川省科技厅应用基础重点资助项目(2017JY0011)