OpenVINO 2021r2 C++ 超分辨率重建DBPN_综合

想多看看OpenVINO 2021r2对各种SR模型计算层的支持怎么样，再试一下DBPN, 这个是NTIRE2018超分8倍的冠军，只随便测试一下2X的模型

从网上找了个基于caffe的模型 https://github.com/alterzero/DBPN-caffe caffe是个非常老的框架了 OpenVINO应该支持的非常好了.

关于推理数据的预处理和后处理，从原项目可以看出就是简单的除以255.0和乘以255.0

    %% work on illuminance onlyim_gnd = modcrop(im, up_scale);
输入低分辨率图像就是im_gnd除以255然后缩小up_scale倍im_gnd = single(im_gnd)/255;im_l = imresize(im_gnd, 1/up_scale, 'bicubic');%% bicubic interpolationticim_b = imresize(im_l, up_scale, 'bicubic');t_bic=toc;%% DBPNif flat==1[im_dbpn,t_dbpn] = run_cnn(im_b, model, weights,use_gpu);t_dbpn=t_dbpn+t_bic;elseif ensemble==0[im_dbpn,t_dbpn] = run_cnn(im_l, model, weights,use_gpu);else[im_dbpn,t_dbpn]=selfEnsemble(im_l,model,weights,use_gpu);endend%% remove border
输出数据乘以255im_dbpn = shave(uint8(im_dbpn * 255), [up_scale, up_scale]);im_gnd = shave(uint8(im_gnd * 255), [up_scale, up_scale]);im_b = shave(uint8(im_b * 255), [up_scale, up_scale]);

废话不多说，MO转换大法

python "c:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo_caffe.py" --input_shape=[1,3,480,640] --scale_values=[255.0] --input_model=DBPN_2x.caffemodel --data_type FP16

后处理部分针对推理输出数据做一下乘255.0的处理

				//channel == 3cv::Mat Img(H, W, CV_8UC3);unsigned char *image_ptr = Img.data;for (size_t n = 0; n < num_images; n++) {for (size_t i = 0; i < nPixels; i++) {
//输出数据乘以255.0data_img[i * num_channels] = static_cast<float>(output_data[i + n * nPixels * num_channels]*255.0);data_img[i * num_channels + 1] = static_cast<float>(output_data[(i + nPixels) + n * nPixels * num_channels] * 255.0 );data_img[i * num_channels + 2] = static_cast<float>(output_data[(i + 2 * nPixels) + n * nPixels * num_channels] * 255.0);//switch BGR->RGB, OpenCV doesn't need it, just skip it//float temp = data_img[i * num_channels];//data_img[i * num_channels] = data_img[i * num_channels + 2];//data_img[i * num_channels + 2] = temp;if (data_img[i * num_channels] < 0) data_img[i * num_channels] = 0;if (data_img[i * num_channels] > 255) data_img[i * num_channels] = 255;image_ptr[i * num_channels] = data_img[i * num_channels];if (data_img[i * num_channels + 1] < 0) data_img[i * num_channels + 1] = 0;if (data_img[i * num_channels + 1] > 255) data_img[i * num_channels + 1] = 255;image_ptr[i * num_channels + 1] = data_img[i * num_channels + 1];if (data_img[i * num_channels + 2] < 0) data_img[i * num_channels + 2] = 0;if (data_img[i * num_channels + 2] > 255) data_img[i * num_channels + 2] = 255;image_ptr[i * num_channels + 2] = data_img[i * num_channels + 2];}}imshow("DBPN_2x", Img);

最终得到输出效果

原始图片（测试图片来自网络）

Bicubic的2x放大效果

DBPN 2X效果

肉眼看上去好像比EDSR要好一点 :)

最终照例看一下性能，

调用inferRequest_regular.Infer()推理的时间, 在8665U 4核8线程的CPU和 Gen9 24EU的核显上

CPU: 23069ms (0.043FPS)
GPU: 13147ms (0.076FPS)

看来OpenVINO做超分的主要难度在MO转换上，只要MO能够正常的转换成功，后面的推理代码实现基本就是一个OpenVINO的async_classification_sample例程的代码实现。对于2021年发布的OpenVINO版本来说，转换这些前几年的SR模型，就像用机关枪打冷兵器时代的战争一样，完全属于屠杀，轻松畅快 :)

实现代码，仅供参考

https://gitee.com/tisandman/dbpn_ov2021