【项目实现】ResShift：通过残差位移实现图像超分辨率的高效扩散模型（NeurIPS 2023，聚焦）

????????基于扩散的图像超分辨率（SR）方法主要受到数百甚至数千个采样步骤要求导致推理速度低的限制。现有的加速度采样技术不可避免地在一定程度上牺牲了性能，导致SR结果过于模糊。为了解决这个问题，我们提出了一种新颖而高效的SR扩散模型，该模型显着减少了扩散步骤的数量，从而消除了推理过程中对后加速的需要及其相关的性能下降。该方法通过移动高分辨率图像和低分辨率图像之间的残差来构建一个马尔可夫链，大大提高了转换效率。此外，还制定了详细的噪声计划，以灵活控制扩散过程中的变速和噪声强度。大量实验表明，即使只有 15 个采样步骤，所提出的方法在合成和真实世界数据集上都获得了优于或至少与当前最先进的方法相当的性能。

论文地址：https://arxiv.org/abs/2307.12348

项目地址：https://github.com/zsyOAOA/ResShift

主机和显卡：【Intel? Xeon(R) CPU E5-2623 v3 @ 3.00GHz × 16】【NVIDIA Corporation GP102 [TITAN Xp]?】

项目效果图：

?

可能遇见的问题：

????????AttributeError: module ‘gradio‘ has no attribute ‘outputs‘

????????AttributeError: module ‘gradio‘ has no attribute ‘image’

????????RuntimeError: CUDA out of memory.

解决办法

?????????AttributeError: module ‘gradio‘ has no attribute ‘outputs‘

????????AttributeError: module ‘gradio‘ has no attribute ‘image’

????????这两个问题属于一类 ，安装正确的gradio版本就可以了

pip install gradio==3.50.2

?????????RuntimeError: CUDA out of memory.

Traceback (most recent call last):
? File "/media/visionx/monica/project/ResShift/inference_resshift.py", line 107, in <module>
? ? main()
? File "/media/visionx/monica/project/ResShift/inference_resshift.py", line 104, in main
? ? resshift_sampler.inference(args.in_path, args.out_path, bs=1, noise_repeat=False)
? File "/media/visionx/monica/project/ResShift/sampler.py", line 234, in inference
? ? im_sr_tensor = _process_per_image(im_lq_tensor)
? File "/media/visionx/monica/project/ResShift/sampler.py", line 186, in _process_per_image
? ? im_sr_tensor = self.sample_func(
? File "/media/visionx/monica/project/ResShift/sampler.py", line 136, in sample_func
? ? results = self.base_diffusion.p_sample_loop(
? File "/media/visionx/monica/project/ResShift/models/gaussian_diffusion.py", line 410, in p_sample_loop
? ? return self.decode_first_stage(final, first_stage_model)
? File "/media/visionx/monica/project/ResShift/models/gaussian_diffusion.py", line 473, in decode_first_stage
? ? out = first_stage_model.decode(z_sample)
? File "/media/visionx/monica/project/ResShift/ldm/models/autoencoder.py", line 35, in decode
? ? quant, emb_loss, info = self.quantize(h)
? File "/home/visionx/anaconda3/envs/ResShift/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
? ? return forward_call(*input, **kwargs)
? File "/media/visionx/monica/project/ResShift/ldm/modules/vqvae/quantize.py", line 281, in forward
? ? torch.sum(self.embedding.weight**2, dim=1) - 2 * \
RuntimeError: CUDA out of memory. Tried to allocate 4.00 GiB (GPU 0; 11.90 GiB total capacity; 8.35 GiB already allocated; 2.66 GiB free; 8.50 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. ?See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

? ? ? ?解决办法：

CUDA_VISIBLE_DEVICES=0 python inference_resshift.py -i ./image/1.jpg -o ./result --task realsrx4 --chop_size 512

改为

CUDA_VISIBLE_DEVICES=0 python inference_resshift.py -i ./image/1.jpg -o ./result --task realsrx4 --chop_size 256

? ? ? ? 其实就是相当于把batchsize调小点，思想是相通的。?

????????另一个解决办法就是放到更大的显卡上，就不再赘述了。?显然我选择了这个，因为我有，哈哈哈哈～

完结撒花

? ? ? ? 任何会改变现状的举动，都是需要巨大勇气的