当前位置: 代码迷 >> 综合 >> 【论文阅读】13-Multi-View Stereo for Community Photo Collections
  详细解决方案

【论文阅读】13-Multi-View Stereo for Community Photo Collections

热度:84   发布时间:2024-02-13 13:25:13.0

【论文阅读】13-Multi-View Stereo for Community Photo Collections

  • 0 basic information
  • 1 background
  • 2 method
    • 2.0 Calibrating Internet Photos
    • 2.1 Estimate depth maps
      • 2.1.1 Find good matches ----重要!
        • 2.1.1.1 image level---global view selection
        • 2.1.1.2 !!Rescaling Views ---after global selection
        • 2.1.1.3 pixel level ---local view selection A
      • 2.1.2 optimizing for both depth and normal
        • 2.1.2.1 image pixels priori queen Q --initial
        • 2.1.2.2 visit each pixel in queen Q---pixel in R & A ,实现立体匹配
  • 3 other

部分参考:链接

0 basic information

Goesele, Michael, Snaveley, Noah, Curless, Brian,. Multi-View Stereo for Community Photo Collections[J]. Proc.int.conf.on Computer Vision, 2007:1-8.

1 background

CPCs(community photo collections)数据 特点:
在这里插入图片描述

2 method

2.0 Calibrating Internet Photos

  1. geometrically
    radial distortion(径向畸变)
    内参、外参
  2. radiometrically

2.1 Estimate depth maps

2.1.1 Find good matches ----重要!

2.1.1.1 image level—global view selection

  1. 目的:find a set of good neighborhood images N ----(usually |N| = 10)
  2. good :sufficient parallax(视差)
  3. good 影响因素:
  • shared feature points number
  • baseline length
  • scale consistency
  1. 量化形式:
    在这里插入图片描述

考虑 feature number & baseline( angle):
在这里插入图片描述
在这里插入图片描述


考虑 feature number & scale

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

  1. N 的生成方式: greedy approach

grow the neighborhood incrementally by iteratively adding to N the
highest scoring view

2.1.1.2 !!Rescaling Views —after global selection

当对不同的reference view 进行深度估计时,neighbor views 根据计算进行与reference view 相对应的scale操作
原因:CPC图像集本身具有较大的尺度差异

  1. Resolution_lowest — Vmin
    相对尺度\分辨率计算公式如下:
    实质: 平均-Shared feature points–average
    在这里插入图片描述
    在这里插入图片描述

  2. R to Resolution_lowest—R_rescaled

  • threshold t(min the relative scale)!!!
  • rescale R!!!
  1. N-Vmin to R_rescaled

  2. NOTICE: rescale with different reference view
    在这里插入图片描述

2.1.1.3 pixel level —local view selection A

1.基于pixel
每个pixel 对应不同的A,并且neighboring image set = A在立体匹配迭代的过程中会不断更新
2.选择要求及定量化指标:

  • corresponding pixels photometrically consistent --mean-removed normalized cross correlation(NCC)
  • sufficiently wide range of observation directions— span of directions from which a given scene point— angular spread

angular spread: the angle of epipolar lines obtained by projecting
each viewing ray passing through the scene point into the reference
view.(该像素的多个候选neighboring view的极线在reference view 上的投影相互之间形成的夹角 )
将其作为score_local的计算因子

ps:极线—根据F计算

在这里插入图片描述
在这里插入图片描述

  1. 选择的步骤:
  • Highest score LR(V)
  • NCC higher threshold ----add to A
  • Repeat stop: the size of A / N no remain

2.1.2 optimizing for both depth and normal

2.1.2.1 image pixels priori queen Q --initial

1.3D POINMTS :sfm sparse points 
+
addition points: feature points visible in all N views2. project 3D POINTS to reference image R—pixels queen Q

2.1.2.2 visit each pixel in queen Q—pixel in R & A ,实现立体匹配

  • traversal --prioritized–estimated matching confidence --selecting the highest one
  • each selected pixel
  1. Pixel—window—planar patch in scene
  2. Planar patch 参数化表示—center pixel的深度h、邻域像素在s\t方向上的单位深度偏移量hs、ht—(ps:hs ht 可表示normal)
  3. Planar patch —project to A—window’
  4. 基于Window& window’一致性约束—构建方程并优化目标函数求解未知量
  • 约束:根据反射模型(选择简单的Lambertian reflectance),匹配位置处的像素值(RGB三通道)对应的感光强度(入射光线强度)相差一个color factor -ck(不同view、不同通道不同一个ck)

在这里插入图片描述

  • 目标函数:SSD
  • 方程个数: 3(RGB) * n * n(window size)*m(A size)
  • 未知量:(center pixel -h \hs\ ht) 3+ 3*m (ck number)
  1. 计算depth 、normal的confidence:average of mean-remove NCC in neighboring view
  2. 迭代优化计算细节----见原文,此处略,迭代过程中A更新

3 other

MVS benchmark datasets
MVS evaluation method

  相关解决方案