最近,基于深度学习的方法在各种像增强任务上取得了相当大的成功,包括像去噪[1,39]、像去马赛克和超分辨率[15,18,22,35]。与通常需要自然像统计先验知识的传统像处理算法不同,数据驱动的方法可以隐式地学习这些信息。由于这个事实,基于DL的方法非常适合映射问题[5,37,42]。在这里,学习像ISP可以看作是一个像到像的转换问题,可以通过基于DL的方法很好地解决。在的ZRR数据集中,RAW像可以分解为4个通道,分别是拜耳模式的红色、绿色、蓝色和绿色,如1所示.备注4个通道中的2个记录来自绿色传感器的辐射信息。与RGB像相比,需要额外的操作,例如去马赛克和颜色校正来处理RAW像。此外,由于拜耳滤波器的性质,这4个通道的大小被下采样了2倍。为了使预测和地面实况像的大小一致,需要进行上采样操作。这可以看作是一个恢复问题,应该考虑高频信息的恢复。在我们的观察中,即使作者采用了SIFT[21]和RANSAC[33]算法来减轻这种影响,DSLR和移动拍摄的像对之间的错位仍然很严重。值得一提的是,输入RAW像和真实RGB像之间的微小偏差会导致性能显着下降。
为了解决上述问题,我们引入了一种利用注意力机制和小波变换的新型可训练管道。更具体地说,我们提出的方法的输入是RAW像及其去马赛克对应物的组合作为补充,其中双分支设计旨在强调不同的训练任务,即RAW模型的噪声去除和细节恢复以及去马赛克模型上的颜色映射;采用离散小波变换从原始像中恢复精细的上下文细节,同时保留训练过程中特征的信息量;至于色彩校正和色调映射,则利用res-dense连接和注意力机制来鼓励网络将精力放在重点区域上。
总之,我们的主要贡献是:探索小波变换和非局部注意力机制在像ISP管道中的有效性。采用双分支设计来获取原始像及其去马赛克对应物,这使我们提出的方法能够将RAW像转换为RGB像。轻量级且完全卷积的编码器-解码器设计,在不同的输入大小上具有时间效率和灵活性。
Abdelhamed,,Afifi,M.,Timofte,R.,Brown,M.S.:NTIRE2020challengeonrealimagedenoising:dataset,methodsandresults.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognitionWorkshops,pp.496–497Bruhn,,Weickert,J.,Schn¨orr,:Lucas/KanademeetsHorn/Schunck:combininglocalandglobalopticflowmethods.Int.J.Comput.Vision61,211–23https://doi.org/1023/B:VISI.000004532431943Cao,Y.,Xu,J.,Lin,S.,Wei,,Hu,:GCNet:non-localnetworksmeetsqueezeexcitationnetworksandbeyond.In:ProceedingsoftheIEEEInternationalConferenceonComputerVisionWorkshopsCarreira,J.,Zisserman,:Quovadis,actionrecognition?Anewmodelandthekineticsdataset.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.6299–6308Chen,,Chen,Q.,Xu,J.,Koltun,V.:Learningtoseeinthedark.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.3291–3300Chen,L.,Zhu,Y.,Papandreou,,Schroff,,Adam,:Encoder-decoderwithatrousseparableconvolutionforsemanticimagesegmentation.In:ProceedingsoftheIEEEEuropeanConferenceonComputerVision,pp.801–818Cheng,,Price,,Cohen,S.,Brown,M.S.:Beyondwhite:groundtruthcolorsforcolorconstancycorrection.In:ProceedingsoftheIEEEInternationalConferenceonComputerVision,pp.298–306Dabov,K.,Foi,,Katkovnik,V.,Egiazarian,K.:Imagedenoisingbysparse3-Dtransform-domaincollaborativefiltering.IEEETrans.ImageProcess.16,2080–2095Feichtenhofer,,Pinz,,Zisserman,:Convolutionaltwo-streamnetworkfusionforvideoactionrecognition.In:ProceedingsoftheIEEEEuropeanConferenceonComputerVision,pp.1933–1941Gharbi,M.,Chaurasia,,Paris,S.,Durand,:Deepjointdemosaickinganddenoising.ACMTrans.Graph.35,1–12He,,Wang,,Shi,,Duan,L.Y.:MopmoirepatternsusingMopNet.In:ProceedingsoftheIEEEInternationalConferenceonComputerVision,pp.2424–2432Hu,J.,Shen,L.,Sun,:Squeeze-and-excitationnetworks.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.7132–7141Ignatov,,Timofte,R.,etal.:AIM2020challengeonlearnedimagesignalprocessingpipeline.In:Bartoli,,Fusiello,ECCV2020.LNCS,vol.12537,pp.152–170.Springer,ChamIgnatov,,VanGool,L.,Timofte,R.:ReplacingmobilecameraISPwithasingledeeplearningmodel.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognitionWorkshops,pp.536–537Kim,J.,KwonLee,J.,MuLee,K.:Accurateimagesuper-resolutionusingverydeepconvolutionalnetworks.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.1646–1654Kingma,P.,Ba,J.:Adam:amethodforstochasticoptimization.arXivpreprintarXiv:1416980Kwok,N.M.,Shi,,Ha,Q.P.,Fang,,Chen,S.,Jia,X.:Simultaneousimagecolorcorrectionandenhancementusingparticleswarmoptimization.Eng.Appl.Artif.Intell.26,2356–2371Ledig,,etal.:Photo-realisticsingleimagesuper-resolutionusingagenerativeadversarialnetwork.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.4681–4690Liu,P.,Zhang,,Zhang,K.,Lin,L.,Zuo,W.:Multi-levelwavelet-CNNforimagerestoration.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognitionWorkshops,pp.773–782Liu,X.,Ma,Y.,Shi,Z.,Chen,J.:GridDehazeNet:attention-basedmulti-scalenetworkforimagedehazing.In:ProceedingsoftheIEEEInternationalConferenceonComputerVision,pp.7314–7323Lowe,:Distinctiveimagefeaturesfromscale-invariantkeypoints.Int.J.Comput.Vis.60,91–1https://doi.org/1023/B:VISI.000002966996194Lugmayr,,Danelljan,M.,Timofte,R.:NTIRE2020challengeonreal-worldimagesuper-resolution:methodsandresults.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognitionWorkshops,pp.494–495Mallat,S.:Atheoryformultiresolutionsignaldecomposition:thewaveletrepresentation.IEEETrans.PatternAnal.Mach.Intell.11,674–693Mei,K.,Li,J.,Zhang,J.,Wu,,Li,J.,Huang,R.:Higher-resolutionnetworkforimagedemosaicingandenhancing.In:ProceedingsoftheIEEEInternationalConferenceonComputerVisionWorkshops,pp.3441–344IEEEQian,R.,Tan,R.T.,Yang,W.,Su,J.,Liu,J.:Attentivegenerativeadversarialnetworkforraindropremovalfromasingleimage.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.2482–2491Rana,,Singh,P.,Valenzise,,Dufaux,,Komodakis,N.,Smolic,:Deeptonemappingoperatorforhighdynamicrangeimages.IEEETrans.ImageProcess.29,1285–1298Ratnasingam,S.:Deepcamera:afullyconvolutionalneuralnetworkforimagesignalprocessing.In:ProceedingsoftheIEEEInternationalConferenceonComputerVisionWorkshopsRizzi,,Gatta,,Marini,:Anewalgorithmforunsupervisedglobalandlocalcolorcorrection.PatternRecogn.Lett.24,1663–1677Schwartz,,Giryes,R.,Bronstein,M.:DeepISP:towardlearninganend-to-endimageprocessingpipeline.IEEETrans.ImageProcess.28,912–923Tao,X.,Gao,,Shen,X.,Wang,J.,Jia,J.:Scale-recurrentnetworkfordeepimagedeblurring.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.8174–8182Timofte,R.,Rothe,R.,VanGool,L.:Sevenwaystoimproveexample-basedsingleimagesuperresolution.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.1865–1873Uhm,K.,Kim,S.W.,Ji,S.W.,Cho,S.J.,Hong,J.P.,Ko,S.J.:W-Net:twostageU-Netwithmisaligneddataforraw-to-RGBmapping.In:ProceedingsoftheIEEEInternationalConferenceonComputerVisionWorkshop,pp.3636–364IEEEVedaldi,,Fulkerson,:VLFeat:anopenandportablelibraryofcomputervisionalgorithms.In:Proceedingsofthe18thACMInternationalConferenceonMultimedia,pp.1469–1472Wang,X.,Girshick,R.,Gupta,,He,K.:Non-localneuralnetworks.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.7794–7803Wang,X.,Chan,K.,Yu,K.,Dong,,ChangeLoy,:EDVR:videorestorationwithenhanceddeformableconvolutionalnetworks.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognitionWorkshopsWang,Z.,Simoncelli,P.,Bovik,:Multiscalestructuralsimilarityforimagequalityassessment.In:TheThrity-SeventhAsilomarConferenceonSignals,Systems&Computers,vol.2,pp.1398–140IEEEXu,X.,Ma,Y.,Sun,W.:Towardsrealscenesuper-resolutionwithrawimages.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.1723–1731Yuan,L.,Sun,J.:Automaticexposurecorrectionofconsumerphotographs.In:Fitzgibbon,,Lazebnik,S.,Perona,P.,Sato,Y.,Schmid,ECCV201LNCS,vol.7575,pp.771–78Springer,Heidelberg.https://doi.org/1007/978-3-642-33765-955Zhang,K.,Zuo,W.,Chen,Y.,Meng,,Zhang,L.:BeyondaGaussiandenoiser:residuallearningofdeepCNNforimagedenoising.IEEETrans.ImageProcess.26,3142–3155Zhang,Y.,Li,K.,Li,K.,Wang,L.,Zhong,,Fu,Y.:Imagesuper-resolutionusingverydeepresidualchannelattentionnetworks.In:ProceedingsoftheIEEEEuropeanConferenceonComputerVision.pp.286–301Zhang,Y.,Tian,Y.,Kong,Y.,Zhong,,Fu,Y.:Residualdensenetworkforimagesuper-resolution.In:ProceedingsoftheIEEEConferenceonComputerVisionandPatternRecognition,pp.2472–2481Zhu,J.Y.,Park,T.,Isola,P.,Efros,:Unpairedimage-to-imagetranslationusingcycle-consistentadversarialnetworks.In:ProceedingsoftheIEEEInternationalConferenceonComputerVision,pp.2223–2232
文章为作者独立观点,不代表 股票程序化软件自动交易接口观点