降雨会导致获取的图像出现细节丢失、纹理模糊等问题,严重影响后续计算机视觉任务的分析和研究。为去除图像中的雨条纹,获得干净的背景图像,提出一种基于注意力机制的混合CNN-Transformer单幅图像去雨网络。首先利用Resnet18的前5层进行浅层特征提取,然后利用高低频雨条纹检测模块,分别采用拉普拉斯算子和全局处理器得到高频雨条纹注意力图和低频背景注意力图,生成雨条纹注意力图,促使后续网络对雨条纹重点关注。在传统Transformer结构中加入通道和空间双重注意力,形成改进的混合CNN-Transformer模块以充分提取图像特征,最后通过像素上采样实现特征重构,得到去雨图像。与其他主流去雨方法在公用数据集上的比较结果表明,所提出的网络取得了更好的量化指标与视觉效果,证实本文方法的有效性。
Abstract
Rain in the acquired images can cause problems such as loss of details and blurry textures, which is not conducive to the analysis and research of subsequent computer vision tasks. To remove rain stripe from images and obtain clean background images, a hybrid CNN-Transformer single image deraining network based on attention mechanism is proposed. Firstly, shallow feature extraction is performed by the first 5 layers of Resnet18. Then, the high and low frequency rain stripe detection module is introduced to generate rain stripe attention maps to promote the subsequent network focus on rain stripes by Laplacian operator and global processor. Besides, an improved hybrid CNN-Transformer module is formed to fully extract image features by incorporating channel and spatial attention into the traditional Transformer structure. Finally, feature reconstruction is achieved through pixel sampling to obtain deraining images. Compared with other mainstream deraining methods on common datasets, the results show that the proposed network achieves better quantitative indicators and visual effects, confirming the effectiveness of the proposed method.
关键词
图像去雨 /
高低频雨条纹检测 /
混合CNN-Transformer
Key words
image deraining /
high and low frequency rain stripe detection /
hybrid CNN-Transformer
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 刘经伟. 基于深度学习的无人机目标检测与跟踪系统设计[J].信息与电脑,2025,37(6):11-13.
[2] 李建锋,熊明强,陈园琼,等.基于深度学习的轻量级实时图像分割方法研究[J].通信学报,2025,46(2):176-190.
[3] Fu X,Huang J,Zeng D,et al.Removing Rain from Single Images via a Deep Detail Network[A].In Proceedings of the 2017 IEEE Conference on Computer Vision & Pattern Recognition[C].Hawaii,USA,2017:1715-1723.
[4] Zhang H,Patel V M.Density-Aware Single Image De-raining Using a Multi-stream Dense Network[A].In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition[C].Salt Lake City,UT,USA,2018:695-704.
[5] Chang Y,Chen M,Yu C,et al.Direction and Residual Awareness Curriculum Learning Network for Rain Streaks Removal[J].In IEEE Transactions on Neural Networks and Learning Systems,2024,35(6):8414-8428.
[6] Yang W,Tan R T,Feng J,et al.Deep Joint Rain Detection and Removal from a Single Image[A].In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C].Honolulu,HI,USA,2017:1685-1694.
[7] Luo Y,Xu Y,Ji H.Removing Rain from a Single Image via Discriminative Sparse Coding[A].In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV)[C].Santiago,Chile,2015:3397-3405.
[8] Wang H,Yue Z.Xie Q,et al.From Rain Generation to Rain Removal[A].In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)[C].Nashville,TN,USA,2021:14786-14796.
[9] Zamir S W,Arora A,Khan S,et al.Multi-Stage Progressive Image Restoration[A].In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)[C].Nashville,TN,USA,2021:14816-14826.
[10] Zou S,Zou Y,Li J C,et al.Cross Paradigm Representation and Alignment Transformer for Image Deraining.arXiv2025, arXiv:2504.16455.
[11] Zheng S,Lu C,Wu Y,et al.SAPNet:Segmentation-Aware Progressive Network for Perceptual Contrastive Deraining[A].In Proceedings of the 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW)[C].Waikoloa,HI,USA,2022:52-62.
基金
国家自然科学基金,项目编号：62201333; 山西省研究生科研创新项目,项目编号：2024XSY58
