Visual Tracking via Adaptive Structural Local Sparse Appearance Model_综合

Visual Tracking via Adaptive Structural Local Sparse Appearance Model

使用自适应的结构化局部外观模型的视觉跟踪

Abstract——摘要

Sparse representation has been applied to visual tracking by finding the best candidate with minimal reconstruction error using target templates.

稀疏表示已经被应用到视觉跟踪中，稀疏表示在目标模版中通过最小重构误差来寻找到最好的候选目标。

However most sparse representation based trackers only consider the holistic representation and do not make full use of the sparse coefficients to discriminate between the target and the background, and hence may fail with more possibility when there is similar object or occlusion in the scene.

然而大部分基于稀疏表示的跟踪只考虑到整体表示，没有充分利用稀疏系数在目标与背景之间的判别性，因此当场景中有相似的目标和遮挡时跟踪失败的概率更高。

In this paper we develop a simple yet robust tracking method based on the structural local sparse appearance model.

在论文中我们发明了一种基于结构化稀疏外观模型的简单但鲁棒的跟踪方法。

This representation exploits both partial information and spatial information of the target based on a novel alignment-pooling method.

这种表示在一种新颖的pooling校正方法上探索了目标的部分信息和空间信息。

The similarity obtained by pooling across the local patches helps not only locate the target more accurately but also handle occlusion.

通过局部补丁上进行pooling得到的相似性不仅定位目标更准确而且能处理遮挡。

In addition, we employ a template update strategy which combines incremental subspace learning and sparse representation.

此外，我们采用了一种结合增量子空间学习和稀疏表示的模版更新策略。

This strategy adapts the template to the appearance change of the target with less possibility of drifting and reduces the influence of the occluded target template as well.

这种策略使跟踪漂移概率更低并且降低了目标模版遮挡的影响，能根据目标外观变化自适应的更新模版。

Both qualitative and quantitative evaluations on challenging benchmark image sequences demonstrate that the proposed tracking algorithm performs favorably against several state-of-the-art methods.

挑战性实验基准图像序列上定性、定量的评估证明了提出的跟踪算法跟目前的算法相比运行很好。

1、Introduction——引言

段落1——描述视觉跟踪问题

Visual tracking has long been an important topic in computer vision field, especially for application of surveillance, vehicle navigation and human computer interface.

视觉跟踪一直是计算机视觉领域内的重要课题，特别是在监控应用，车辆导航和人机交互中。

Although many tracking methods have been proposed, it remains a challenging problem due to factors such as partial occlusions, illumination changes, pose changes, background clutter and viewpoint variation.

虽然提出了许多跟踪算法，但由于部分遮挡、光照变化、姿态变化、复杂背景和视角变化，跟踪问题仍然是一个具有挑战性的问题。

段落2——讲述跟踪算法中的判别法

Current tracking algorithms can be categorized into either generative or discriminative approaches.

目前的跟踪算法可以被分为生成法和判别法。

Discriminative methods formulate tracking as a classification problem which aims to distinguish the target from the background.

判别法把跟踪看作分类问题，目的是区分背景和目标。

It employs the information from both the target and background.

判别法使用背景和目标的信息。

Avidan [2] combines a set of weak classifiers into a strong one to do ensemble tracking.

Reference——弱分类器组成强分类器：

[2] S. Avidan. Ensemble tracking. PAMI, 29(2):261, 2007.

Avidan 将许多弱分类器组合成一个强分类器来进行整体跟踪。

In [7] Grabner et al. propose an online boosting method to update discriminative features and later in [8] a semi-online boosting algorithm is proposed to handle the drifting problem.

Reference——boosting算法：

[7] H. Grabner and H. Bischof. On-line boosting and vision. In CVPR,2006.

[8] H. Grabner, C. Leistner, and H. Bischof. Semi-supervised on-line boosting for robust tracking. In ECCV, 2008.

在论文7中Grabner 等提出在线boosting算法更新判别特征，在论文8中提出了半监督在线boosting算法处理漂移问题。

Babenko et al. [3] use multiple instance learning (MIL) which puts all ambiguous positive and negative samples into bags to learn a discriminative model for tracking.

Reference——多实例学习算法：

[3] B. Babenko, M.-H. Yang, and S. Belongie. Visual tracking with online multiple instance learning. In CVPR, 2009.

Babenko 等人用将所有模糊的正负样本放到一个包中的多实例学习算法学习跟踪的判别模型。

Kalal et al. [9] propose the P-N learning algorithm to exploit the underlying structure of positive and negative samples to learn effective classifiers for object tracking.

Reference——P-N学习算法：

[9] Z. Kalal, J. Matas, and K. Mikolajczyk. P-N learning: Bootstrapping binary classifiers by structural constraints. In CVPR, 2010.

Kalal等人提出的P-N学习算法利用了正负样本的根本结构来学习目标跟踪的有效分类器。

Wang et al. [20] base the discriminative appearance model on superpixels, which facilitates the tracker to distinguish between the target and background.

Reference——基于超像素的判别外观模型：

[20] S. Wang, H. Lu, F. Yang, and M.-H. Yang. Superpixel tracking. In ICCV, 2011.

王等人的基于超像素的判别外观模型算法有利于区别背景和目标来进行追踪。

段落3——讲述跟踪算法中的生成法

Generative methods formulate the tracking problem as searching for the regions most similar to the target model.

生成式方法把跟踪看成搜索与目标模型最相似区域的问题。

These methods are based on either templates [13, 5, 15, 1, 10] or subspace models [4, 18].

Reference——基于模版 [13, 5, 15, 1, 10]

[13] B. D. Lucas and T. Kanade. An iterative image registration techneque with an application to stereo vision. In IJCAI, 1981.

[5] D. Comaniciu, V. Ramesh, and P. Meer. Kernel-based object tracking.PAMI, 25(5):564–575, 2003.

[15] I. Matthews, T. Ishikawa, and S. Baker. The template update problem.PAMI, 26:810–815, 2004.

[1] A. Adam, E. Rivlin, and I. Shimshoni. Robust fragments-based tracking using the integral histogram. In CVPR, 2006.

[10] J. Kwon and K. M. Lee. Visual tracking decomposition. In CVPR,2010.

或基于子空间模型的[4, 18]

[4] M. J. Black and A. D. Jepson. Eigentracking: Robust matching and tracking of articulated objects using a view-based representation.IJCV, 26(1):163–84, 1998.

[18] D. Ross, J. Lim, R.-S. Lin, and M.-H. Yang. Incremental learning for robust visual tracking. IJCV, 77(1):125–141, 2008.

这些方法都是基于模版或子空间模型的。

To adapt to the target appearance variations caused by pose change and illumination change, the target appearance model is updated dynamically.

为了适应由于姿态和光照变化引起的目标外观模型的变化，目标外观模型是动态更新的。

Matthews et al. [15] develop a template update method which can reduce the drifting problem by aligning with the first template to reduce drifts.

Reference——模版更新算法，校正首模版

[15] I. Matthews, T. Ishikawa, and S. Baker. The template update problem.PAMI, 26:810–815, 2004.

Matthews 等人开发了一种能减少漂移问题的模版更新算法，这种算法通过校正第一个模版来减少漂移。

In [18], the low-dimensional subspace representation is learned incrementally during the tracking process to adapt to the changes of target appearance.

Reference——增量子空间学习

[18] D. Ross, J. Lim, R.-S. Lin, and M.-H. Yang. Incremental learning for robust visual tracking. IJCV, 77(1):125–141, 2008.

在论文18中，在跟踪过程中通过增量学习低维子空间表示来适应目标的外观变化。

Kwon et al. [10] decompose the observation model into multiple basic observation models to cover a wide range of pose and illumination variation.

Reference——观察模型分解

[10] J. Kwon and K. M. Lee. Visual tracking decomposition. In CVPR,2010.

Kwon 等人将观察模型分解为多个基观察模型来覆盖姿态和光照的变化。

Most of these methods use the holistic model to represent the target and hence cannot handle partial occlusion or distracters.

大多数方法用整体模型来表示目标，因此不能处理部分遮挡和误导项问题。

段落4——最新的基于稀疏表示的跟踪算法

Recently, several tracking methods based on sparse representation have been proposed [16, 12, 17, 11].

Reference——最新的基于稀疏表示的跟踪算法

[16] X. Mei and H. Ling. Robust visual tracking using L1 minimization.In ICCV, 2009.

[12] B. Liu, L. Yang, J. Huang, P. Meer, L. Gong, and C. A. Kulikowski.Robust and fast collaborative tracking with two stage sparse optimization.In ECCV, 2010.

[17] X. Mei, H. Ling, Y. Wu, E. Blasch, and L. Bai. Minimum error bounded efficient L1 tracker with occlusion detection. In CVPR,2011.

[11] B. Liu, J. Huang, L. Yang, and C. A. Kulikowski. Robust tracking using local sarse appearance model and k-selection. In CVPR, 2011.

最近提出了一些基于稀疏表示的跟踪算法。

Mei et al. [16, 17] adopt the holistic representation of the object as the appearance model and then track the object by solving the L-1 minimization problem.

Reference——解L-1最小化问题来跟踪目标

[16] X. Mei and H. Ling. Robust visual tracking using L1 minimization.In ICCV, 2009.

[17] X. Mei, H. Ling, Y. Wu, E. Blasch, and L. Bai. Minimum error bounded efficient L1 tracker with occlusion detection. In CVPR,2011.

Mei等人用目标整体表示作为目标的外观模型，通过解L-1最小化问题来跟踪目标。

Liu et al. [11] propose a tracking algorithm based on local sparse model which employs histograms of sparse coefficients and the mean-shift algorithm for object tracking.

Reference——直方图稀疏稀疏和均值漂移算法

[11] B. Liu, J. Huang, L. Yang, and C. A. Kulikowski. Robust tracking using local sarse appearance model and k-selection. In CVPR, 2011.

Liu等人提出了基于局部稀疏模型的跟踪算法，这个算法采用了直方图稀疏稀疏和均值漂移算法来处理目标跟踪。

However, this method is based on a static local sparse dictionary and may fail when there is similar object in the scenes.

然而这个算法是基于统计局部稀疏字典的，当场景中有相似目标时可能跟踪失败。

段落5——介绍本文的算法

In this paper, we propose an efficient tracking algorithm with structural local sparse model and adaptive template update strategy.

在本文中，我们提出了一种使用结构化局部稀疏模型和自适应的模版更新策略的有效跟踪算法。

The proposed method samples overlapped local image patches within the target region.

提出的算法在目标区域内有重叠的采样局部图像块。

We observe that sparse coding of local image patches with a spatial layout contains both spatial and partial information of the target object.

我们注意到局部图像块的稀疏编码和空间布局包含目标对象的空间和局部信息。

The similarity measure is obtained by proposed alignment-pooling method across the local patches within one candidate region.

通过对候选区域局部图像块的进行alignment-pooling可以测的候选区域与目标的相似性。

This helps locate the target more accurately and handle partial occlusion.

这有助于定位目标更准确并且能处理部分遮挡问题。

In addition, the dictionary for local sparse coding is generated from the dynamic templates, which are updated online based on both incremental subspace learning and sparse representation.

此外，局部稀疏编码的字典能从动态模版中得到，字典的在线更新是基于局域增量子空间学习和稀疏表示的。

The update scheme facilitates the tracker to account for appearance changes of the target.

这个考虑到目标外观变化更新方案有利于跟踪。

Due to the simplicity of appearance model and template update strategy, our method can track the target efficiently.

由于外观模型的简洁性和模版更新策略，我们的算法能有效的跟工作能够目标。

段落6——本文算法的贡献

The contributions of this work are summarized as follows.

本文的贡献可总结如下。

First, sparse codes of local image patches with spatial layout in an object are used to model its appearance model.

首先，利用目标局域图像块的稀疏编码和空间布局来建模目标的外观模型。

As for the sparse codes, we propose an alignment-pooling method to improve accuracy of tracking and reduce the influence of occlusion as well.

关于稀疏编码，我们提出了的alignment-pooling方法提高了跟踪的准确性并且降低了遮挡的影响。

Second, both incremental subspace learning and sparse representation are employed to update the templates to handle the drifting problem and partial occlusion.

其次，用增量子空间学习和稀疏表示来更新模版处理了漂移问题和局部遮挡问题。

Experiments on challenging benchmark image sequences demonstrate that the proposed tracking approach performs favorably against several state-of-the-art methods.

挑战性实验基准图像序列上的实验证明了提出的跟踪算法跟目前的算法相比运行很好。

2. Related Work and Context——相关工作

段落1——稀疏表示的应用及各自的特点

Sparse representation has been successfully applied in numerous vision applications [21, 16, 12, 17, 11].

Reference——稀疏表示的应用以及各个应用的特点

[21] J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma. Robust face recognition via sparse representation. PAMI, 31(2):210–227,2009.

[16] X. Mei and H. Ling. Robust visual tracking using L1 minimization.In ICCV, 2009.

[12] B. Liu, L. Yang, J. Huang, P. Meer, L. Gong, and C. A. Kulikowski.Robust and fast collaborative tracking with two stage sparse optimization.In ECCV, 2010.

[17] X. Mei, H. Ling, Y. Wu, E. Blasch, and L. Bai. Minimum error bounded efficient L1 tracker with occlusion detection. In CVPR,2011.

[11] B. Liu, J. Huang, L. Yang, and C. A. Kulikowski. Robust tracking using local sarse appearance model and k-selection. In CVPR, 2011.

稀疏表示已成功的用在许多视觉应用中。

With sparsity constraints, one signal can be represented in the form of linear combination of only a few basis vectors.

通过稀疏性约束，信号可以用一些基向量线性表示。

In [16, 17], the target candidate is sparsely represented as a linear combination of the atoms of a dictionary which is composed of dynamic target templates and trivial templates.

Reference——目标被字典原子线性表示

[12] B. Liu, L. Yang, J. Huang, P. Meer, L. Gong, and C. A. Kulikowski.Robust and fast collaborative tracking with two stage sparse optimization.In ECCV, 2010.

在论文16、17中，候选目标可以用由动态模版和琐碎模版组成的字典的原子线性表示。

By introducing trivial templates, the tracker can handle partial occlusion.

通过引入琐碎模版，跟踪器可以处理部分遮挡问题。

This sparse representation problem is then solved through L-1 minimization with non-negativity constraints.

稀疏表示问题是解非负约束的L-1最小化问题。

In [12], dynamic group sparsity which includes both spatial and temporal adjacency is introduced into the sparse representation to enhance the robustness of the tracker.

Reference——动态组稀疏增强跟踪鲁棒性

[12] B. Liu, L. Yang, J. Huang, P. Meer, L. Gong, and C. A. Kulikowski.Robust and fast collaborative tracking with two stage sparse optimization.In ECCV, 2010.

在论文12中，通过引入包含时空邻接的动态组稀疏到稀疏表示中来增强跟踪的鲁棒性。

In [11], a local sparse representation scheme is employed to model the target appearance and then represent the basis distribution of the target with the sparse coding histogram.

Reference——局部稀疏表示对目标建模

[11] B. Liu, J. Huang, L. Yang, and C. A. Kulikowski. Robust tracking using local sarse appearance model and k-selection. In CVPR, 2011.

在论文11中，用局部稀疏表示方案对目标外观进行建模并用稀疏编码直方图来表示目标的基分布。

Due to the representation of local patches, their method performs well especially in handling the partial occlusion.

由于局部块的表示，他们的算法在处理遮挡问题中表现很好。

However, histograms of local sparse coefficients alone cannot provide enough spatial information.

然而，单纯的直方图的局部稀疏系数不能提供足够的空间信息。

A mean-shift algorithm [5] and a sparse-representation-based voting map are used to better track the target.

Reference——均值漂移算法和基于投票映射的稀疏表示

[5] D. Comaniciu, V. Ramesh, and P. Meer. Kernel-based object tracking.PAMI, 25(5):564–575, 2003.

均值漂移算法和基于投票映射的稀疏表示能更好的跟踪目标。

段落2——本文的工作与其它工作的对比

Our work bears some similarity to [11] in the use of local sparse representations.

Reference——局部稀疏表示

[11] B. Liu, J. Huang, L. Yang, and C. A. Kulikowski. Robust tracking using local sarse appearance model and k-selection. In CVPR, 2011.

我们的工作与论文11中局域稀疏表示的用法具有一些相似性。

However, we sample larger overlapped local image patches with fixed spatial layout where there are more spatial structural information in them.

但是，我们采样更大的有固定空间布局的重叠局部图像块，在空间布局中有更多的空间结构信息。

In addition,we make full use of the sparse coding coefficients with the proposed alignment-pooling method rather than histograms and kernel densities to measure the similarity.

此外，我们充分利用了提出的alignment-pooling方法的稀疏编码系数来度量相似性而没用直方图的核密度。

Instead of using fixed template [1] or dictionary [11] learned from the first frame, we update the dictionary adaptively using dynamic templates.

Reference——固定模版1，固定字典11

[1] A. Adam, E. Rivlin, and I. Shimshoni. Robust fragments-based tracking using the integral histogram. In CVPR, 2006.

[11] B. Liu, J. Huang, L. Yang, and C. A. Kulikowski. Robust tracking using local sparse appearance model and k-selection. In CVPR, 2011.

我们用动态模版自适应的更新字典来代替从第一帧得到的固定的模版或字典。

Object tracking with a static template is likely to fail in dynamic scenes due to large appearance change.

在动态场景中目标外观有较大变化时用静态模版进行目标跟踪可能跟踪失败。

In [16, 17], the template is updated according to both the weights assigned to each template and the similarity between templates and current estimation of target candidate.

Reference——根据模版分配权重和相似性来更新模版

[16] X. Mei and H. Ling. Robust visual tracking using L1 minimization.In ICCV, 2009.

[17] X. Mei, H. Ling, Y. Wu, E. Blasch, and L. Bai. Minimum error bounded efficient L1 tracker with occlusion detection. In CVPR,2011.

在论文16、17中，根据每个模版分配的权重和模版与当前估计的候选目标相似性对模版进行更新。

Different from that template update scheme, we employ both incremental subspace learning and sparse representation to update the templates adaptively.

与上面的模版更新方案不同，我们用增量子空间学习和稀疏表示自适应的更新模版。

This template update method reduces the drifting problem and puts more weights on the important parts of the target.

这种模版更新算法减少了漂移问题并且对目标的重要部分赋予了更大的权重。

In addition, it reduces the influence of the template with partial occlusion.

此外，它降低了模版部分遮挡的影响。

3. Structural Local Sparse Appearance Model——结构化局部稀疏外观模型

段落1——跟踪目标的稀疏表示形式

Given the image set of the target templates T= [T1,T2,..., Tn], we sample a set of overlapped local image patches inside the target region with a spatial layout.

给定目标模版的图像序列T= [T1,T2,..., Tn]，我们用同一空间布局在目标区域内采样一系列的重叠的局部图像块。

These local patches are used as the dictionary to encode the local patches inside the possible candidate regions, i.e. D= d1, d2,..., d(n×N) ∈ Rd×(n×N), where d is the dimension of the image patch vector, n is the number of target templates and N is the number of local patches sampled within the target region.

这些局部块作为编码候选目标区域的局部块的字典，例如D= d1, d2,..., d(n×N) ∈ Rd×(n×N), d是图像块向量的维度，n是模版数目，N是目标区域采样的图像块数目。

Each column in D is obtained by L-2 normalization on the vectorized local image patches extracted from T.

D中的每一列是在从T中提取的向量化局部图像块的L-2归一化得到的。

Each local patch represents one fixed part of the target object, hence the local patches altogether can represent the complete structure of the target.

每一个局部块表示目标对象的一个固定部分，因此所有的局部块表示目标的整体结构。

Since the local patches are collected from many templates, this dictionary captures the commonality of different templates and is able to represent various forms of these parts.

由于局部块是从许多模版收集的，字典能抓住不同模版间的共性并且能表示这些部分的不同形式。

For a target candidate, we extract local patches within it and turn them into vectors in the same way, which are denoted by Y =[y1, y2,..., yN ]∈Rd×N .

对于一个候选目标，我们从候选目标提取局部块，将它们变为向量，这些向量表示为Y =[y1, y2,..., yN ]∈Rd×N。

段落2——公式表示

With the sparsity assumption, the local patches within the target region can be represented as the linear combination of only a few basis elements of the dictionary by solving

where yi denotes the i-th vectorized local image patch, bi∈ R(n×N)×1 is the corresponding sparse code of that local patch, and bi >=0 means all the elements of bi are non-negative.

根据稀疏假设，通过解目标区域的局部块可表示为字典基原子的线性组合，yi表示第i个向量化的局部图像块，bi∈ R(n×N)×1 是对应局部图像块对应的稀疏编码，bi>=0意味着bi的所有元素是非负的。

Note B=[b1, b2,..., bN]represents the sparse codes of one candidate.

注：B=[b1, b2,..., bN]表示候选目标的稀疏编码。

The sparse coefficients of each local patch are divided into several segments, according to the template that each element of the vector corresponds to, i.e.,

wheredenotes the k-th segment of the coefficient vector bi.

根据向量每一个元素对应的模版，每个局部块的稀疏系数分成几个部分，例如，表示系数向量bi的第K个部分。

These segmented coefficients are weighted to obtain Vi for the i-th

patch,

where vector vi corresponds to the i-th local patch and C is a normalization term.

这些分割系数的加权获得第i块的Vi,

Vi对应第i个局部块，C是归一项。

As the templates contain the target object with some appearance variation, the blocks that appear frequently in these templates (as indicated by their sparse codes)should be weighted more than others for more robust representation.

由于模版包含目标对象的外观变化，这些在模版中频繁出现（在稀疏编码中可以看出）的块具有更鲁棒的表示，应该比其它的赋予更高的权重。

This weighting process is carried out by Eq.2 with their sparse codes.

加权过程可以通过方程2的稀疏编码来实现。

All the vectors vi of local patches in a candidate region form a square matrix V and further processed with a novel pooling method.

所有候选区域的局部块的向量Vi形成方阵V，可以用一种新的pooling方法进一步处理。

段落3——创新点alignment-pooling定位目标更准确

Although for a single local patch we lose spatial information by considering only its own coefficient vector as described above, we alleviate this problem by using a novel method to pool the responses of local patches within the candidate region.

虽然像上面描述的那样，只考虑单局部块的系数向量会损失空间信息，但我们用一种新方法聚类候选区域局部块的响应可以缓解这个问题。

We propose an alignment-pooling algorithm rather than max-pooling method [22] to improve the accuracy of location estimation.

Reference——论文22中使用max-pooling算法定位目标

[22] J. Yang, K. Yu, Y. Gong, and T. S. Huang. Linear spatial pyramid matching using sparse coding for image classification. In CVPR,2009.

我们提出了一种alignment-pooling算法而不是论文22中的max-pooling算法来提高定位准确率。

After obtaining vi, each local patch at a certain position of the candidate is represented by patches at different positions of the templates.

在得到Vi后，候选目标确定位置上的每个局部块可由模版上不同位置的块表示。

The local appearance variation of a patch can be best described by the blocks at the same positions of the template (i.e., using the sparse codes with the aligned positions).

块(patch)的局部外观变化可以由模版相同位置上的块(blocks)最佳描述(例如：用校准位置上的稀疏编码)。

For example, the top left corner patch of the target object in Figure 1 should be best described by the first element of v1 as it should have the largest coefficient value (via Eq2 and its block location).

例如，目标对象的左上角块(patch)在图1中可以由V1中的第一个元素最好的表示，而V1中有最大的系数值(通过方程2和它的块(block)位置)。

Figure 1. Illustration of feature formation by alignment-pooling

(darker color elements have larger values).

alignment-pooling的特征表示(插图深颜色元素有更大的值)

Therefore, we take the diagonal elements of the square matrix V as the pooled feature, i.e.,

where f is the vector of pooled features.

因此，我们把方阵V对角线上的元素看作聚类特征，例如f=diag(V)，

f是聚类的特征向量。

Since the weighting operation increase the stability of sparse coding, this pooling method further aligns local patterns between target candidate and the templates based on the locations of structural blocks.

由于加权操作增加了稀疏编码的稳定性，聚类方法进一步校正了候选目标和基于结构块位置的模版之间的局部模式。

The aligned tracking results also facilitate the incremental subspace learning for template update in our algorithm.

校正的跟踪结果也有利于算法的模版更新中增量子空间的学习。

The proposed representation with alignment-pooling process captures structural information of a target object in terms of blocks.

提出的有alignment-pooling处理的表示方法抓住了目标对象关于块(blocks)的结构信息。

In addition, this appearance model is able deal with partial occlusion.

此外，外观模型能处理部分遮挡问题。

When occlusion occurs, the appearance change makes the representation of the occluded local patches dense.

当遮挡发生时，外观变化能表示密集的遮挡的局部块。

However, the local patches which are not occluded still have sparse representations.

然而，没遮挡的局部块仍然能被稀疏表示。

段落4——alignment-pooling与其它pooling方法的对比

After pooling across these local patches, the influence of outliers is reduced and the structural information is retained in the representation to better locate the target.

在聚类过这些局部块后，离群点的影响被降低了并且保留了表示中的结构信息，这样可以更好的定位目标。

Figure 2 shows the vector vi and pooled features obtained by our method for good and bad target candidates.

图2显示了向量Vi和用我们的算法处理得到的好、坏候选目标的聚类特征。

Figure 2. Comparison of the pooled features obtained by alignment-pooling as for good and bad candidates. The upper and lower rows show the pooled features for a good candidate (i.e., a region close to ground-truth tracking result) and a bad candidate (i.e., a region with large tracking error).

图2.通过alignment-pooling算法得到的好坏候选目标聚类特征的比较。上下显示了好(例如，接近实际跟踪目标的区域)坏(例如，有大的跟踪错误的区域)候选目标的聚类特征。

When the target object is partial occlusion, the image patches which are not occluded can still be represented by only few atoms of the dictionary with large coefficients whereas the occluded patches have dense representations (as illustrated in the top row of Figure 2).

当目标对象发生部分遮挡时，没发生遮挡的图像块仍能用字典中的较大系数来表示，而发生遮挡的图像块有压缩的表示(像图2上面的描绘的那样)。

However, for a bad candidate, the local image patches have more dense coefficients, and the pooled features are smaller (as illustrated in the bottom row of Figure 2).

然而，对于一个不好的候选目标来说，局部图像块有更多的密集稀疏，而且聚类特征更小(像图2下面描绘的那样)。

To demonstrate the advantage of the proposed alignment-pooling algorithm, we compare the confidence map obtained by three kinds of pooling methods within a range around target.

为了证明提出的alignment-pooling算法的优点，我们对比了三种pooling方法当目标周围变化时得到的信心图。

Based on these observations as illustrated in Figure 3, accurate localization of the target object can be achieved by the proposed local sparse representation with alignment-pooling.

从图3中可以看出，使用alignment-pooling的局部稀疏表示能准确定位目标对象。

Figure 3. Comparison of the confidence map obtained by three kinds of pooling methods within a range around the target object. Red color blocks denote large coefficient (confidence) values and blue color ones denote low values. The resulting confidence map using our representation indicates the patches near the center are likely to belong to the target as opposed to other ones.

图3.三种pooling方法在目标周围得到的信心图对比。红色块表示大的系数(信心)，蓝色表示小的系数。我们表示的信心图结果意味着越靠近中心的块属于目标的可能性越大。

4. Template Update——模版更新

段落1——跟踪的模版更新问题及现有的方法

Tracking with fixed templates is prone to fail in dynamic scenes as it does not consider inevitable appearance change due to factors such as illumination and pose change.

由于使用固定模版进行跟踪没考虑到光照、姿态变化等因素引起的不可避免的外观变化，因此在动态场景中用固定模版进行跟踪的失败概率更高。

However, if we update the template too frequently with new observations, errors are likely to accumulate and the tracker will drift away from the target.

然而，如果我们频繁更新新观测到的模版，可能发生错误累积从而跟踪从目标上漂移。

Numerous approaches have been proposed for template update [15, 18, 16].

Reference——已有的模版更新算法及缺点

[15] I. Matthews, T. Ishikawa, and S. Baker. The template update problem.PAMI, 26:810–815, 2004.

[18] D. Ross, J. Lim, R.-S. Lin, and M.-H. Yang. Incremental learning for robust visual tracking. IJCV, 77(1):125–141, 2008.

[16] X. Mei and H. Ling. Robust visual tracking using L1 minimization.In ICCV, 2009.bounded efficient L1 tracker with occlusion detection. In CVPR,2011.

许多模版更新算法已经被提出。

Ross et al. [18] extend the sequential Karhunen-Loeve algorithm and propose a new incremental principal component analysis (PCA) algorithm to update both the eigenbasis and the mean as new observations arrive.

Reference——增量PCA

[18] D. Ross, J. Lim, R.-S. Lin, and M.-H. Yang. Incremental learning for robust visual tracking. IJCV, 77(1):125–141, 2008.

Ross等人扩展了连续 Karhunen-Loeve算法并且随着新观测值提出了增量PCA算法来更新正交基和均值。

However the PCA based representation is sensitive to partial occlusion because of the assumption that reconstruction error is Gaussian distributed with small variance.

然而如果重构误差是方差较小的高斯分布，那么基于PCA的表示对部分遮挡较敏感。

Mei and Ling [16, 17] apply sparse representation to visual tracking and employ both target templates and trivial templates to handle outliers and partial occlusion.

Reference——稀疏表示、目标模版和琐碎模版

[16] X. Mei and H. Ling. Robust visual tracking using L1 minimization.In ICCV, 2009.

[17] X. Mei, H. Ling, Y. Wu, E. Blasch, and L. Bai. Minimum error bounded efficient L1 tracker with occlusion detection. In CVPR,2011.

Mei和Ling将稀疏表示应用到视觉跟踪中，并使用目标模版和琐碎模版来处理离群点和部分遮挡。

However, this method is not equipped with any mechanism to handle the drifting problem.

但是这个方法不能处理漂移问题。

In this paper, we introduce subspace learning into sparse representation to adapt templates to the appearance change of the target, and reduce the influence of the occluded target template as well.

在本文中，我们将子空间学习引入到稀疏表示中使模版适应的目标的外观变化，并且减少了目标模版遮挡的影响。

段落2——新旧模版的更新问题

In many tracking methods, the earlier tracking results are more accurate so they should be stored longer than newly acquired results in the template stack.

在许多跟踪算法中，前面的跟踪结果是更准确的，因此与模版栈中新得到的结果相比应该保留更长的时间。

One way to balance between the old and new templates is to assign different update probability to the templates.

一种平衡新旧模版的方式是给模版分配不同的更新概率。

We generate a cumulative probability sequence

and generate a random number r according to uniform distribution on the unit interval [0, 1].

我们生成了一种累积概率序列

，

并根据在单位区间[0,1]之间的联合分布得到了一个随机数r。

By determining which section the random number lies in, we can choose the template to be replaced.

通过判断随机数位于哪一部分，我们可以选择要替换的模版。

This leads to slow update of old templates and quick update of new ones, and thereby alleviating the drifting problem.

这会使旧模版更新慢而新模版更新快，因此可以缓解漂移问题。

段落3——模版更新算法

The strength of both sparse representation and subspace learning is exploited to model the updated template.

稀疏表示和子空间学习的优势是对要更新的模版进行建模。

We collect the tracking results of the target object and then carry out the incremental learning method proposed in [18].

Reference——增量学习

[18] D. Ross, J. Lim, R.-S. Lin, and M.-H. Yang. Incremental learning for robust visual tracking. IJCV, 77(1):125–141, 2008.

我们收集目标对象的跟踪结果然后在跟踪结果上使用增量学习算法。

Not only can this incremental method adapt to the appearance change but also preserve visual information the collected observations have in common.

增量学习不仅能适应外观变化而且能保留收集到的观测共有的视觉信息。

The estimated target can be modeled by a linear combination of the PCA basis vectors and additional trivial templates employed in [16]

where p denotes the observation vector, U is the matrix composed of eigenbasis vectors, q is the coefficients of eigenbasis vectors and e indicates the pixels in p that are corrupted or occluded.

估计目标可以由PCA基向量建模，另外的琐碎模版在论文16中使用

，

p表示观测向量，U是正交基向量构成的矩阵，q是正交基向量的系数，e表示p中的像素损坏或遮挡的像素。

As the error caused by occlusion and noise is arbitrary and sparse, we solve the problem as L-1 regularized least square problem,

where and λ is the regularization parameter.

由于遮挡和噪声引起的错误是任意并且稀疏的，我们把这个问题看成解L-1正则化最小方差问题，

，λ是正则化参数。

The coefficients of trivial templates are employed to account for noise or occlusion and avoid much occlusion to be updated into the template set.

用琐碎模版的稀疏来解决噪声和遮挡，从而避免更大的遮挡被更新到模版集中。

Thus the reconstructed image using only PCA basis vectors is not sensitive to the influence of occlusion.

因此仅用PCA基向量的重构图像对遮挡的影响不敏感。

The reconstructed image is then used for updating the template to be replaced.

重构图像被用来更新要替换的模版。

This process can be viewed as introducing sparsity into subspace representation.

处理过程可被看作将稀疏性引入到子空间表示中。

Some templates obtained from the above-mentioned process are shown in Figure 4.

一些从上面过程中得到的模版如图4所示。

图4 提出的模版更新策略得到的模版例子

We can see that the templates obtained when no occlusion occurs can adapt to the appearance change of the target.

我们可以看出得到的模版当没有遮挡发生时能适应目标的外观变化。

When there is occlusion, the templates focus on the parts which are not contaminated.

当发生遮挡时，模版集中在未被污染的部分。

With this template update strategy, our method can adapt to the appearance change of the target and handle the partial occlusion as well.

用这个模版更新策略，我们的方法能适应目标的外观变化并且能处理部分遮挡问题。

The template update strategy is summarized in Algorithm 1.

模版更新策略被归纳在算法1中。

5. Proposed Tracking Algorithm

段落1——跟踪中的贝叶斯框架

In this paper, object tracking is carried out within the Bayesian inference framework.

在本文中，目标跟踪是在贝叶斯推断框架下实现的。

Given the observation set of targetup to the t-th frame, the target state variable xt can be computed by the maximum a posteriori estimation,

where xi indicates the state of the i-th sample.

给定1-t帧的目标观测集，目标状态变量xt有最大后验概率算出，

xi是第i个样本的状态。

The postetrior probability p(xt|z1:t)can be inferred by the Bayesian

theorem recursively,

where p(xt|xt-1)denotes the dynamic model and p(xt|xt-1) denotes the observation model.

后验概率 p(xt|z1:t)由贝叶斯定理递归推出，

p(xt|xt-1)表示动态模型，p(xt|xt-1)观测模型。

The dynamic model p(xt|xt-1) describes the temporal correlation of the target states between consecutive frames.

动态模型p(xt|xt-1)描述了在连续帧之间的目标状态的时间关系。

We apply the affine transformation with six parameters to model the target motion between two consecutive frames.

我们在两个连续帧之间用有六个参数的仿射变换来对目标运动进行建模。

The state transition is formulated as p(xt|xt-1) = N (xt;xt-1, ∑), where ∑ is a diagonal covariance matrix whose elements are the variances of the affine parameters.

状态变换用公式表示为 p(xt|xt-1) = N (xt;xt-1, ∑), ∑是对角协方差矩阵，∑中的每个元素都是仿射系数的协方差。

段落2——模版更新算法

The observation model p(zt|xt) denotes the likelihood of the observation zt at state xt.

观测模型p(zt|xt)表示观测对象zt 在状态 xt的概率。

It plays an important role in robust tracking.

它在鲁棒跟踪中有重要作用。

In our method, the observation model is constructed by

where the right side of the equation denotes the similarity between the candidate and the target based on the pooled feature f.

在我们的算法中，观测模型构造公式为

方程右边表示基于聚类特征f的候选目标与真实目标间的相似性。

With the template updated incrementally, the observation model is able to adapt to the appearance change of the target.

随着模版的不断更新，观测模型能适应目标的外观变化。

6. Experiments——实验

段落1——实验平台及基本实验知识介绍

The proposed algorithm is implemented in MATLAB and runs at 1.5 frames per second on a Pentium 2.7 GHz Dual Core PC with 2GB memory. The .1 minimization problem is solved with the SPAMS package [14] and the regularization constant λ is set to 0.01 in all experiments.

提出的算法是用Matlab实现的，在内存为2G，CPU为奔腾2.7G赫兹的双核PC机1秒能处理1.5帧。

For each sequence, the location of the target object is manually labeled in the first frame.

对每个序列，目标对象的位置是在第一帧手动标定的。

We resize the target image patch to 32×32 pixels and extract overlapped 16 × 16 local patches within the target region with 8 pixels as step length.

我们将目标图像块修正为为32×32像素的，在目标区域提取了重叠的16×16的局部图像块，步长为8像素。

As for the template update, 8 eigenvectors are used to carry out incremental subspace learning method in all experiments every 5 frames.

考虑到模版的更新，在所有实验中每五帧用8个正交向量实现增量子空间学习。

The MATLAB source codes and datasets are available on our websites(http://ice.dlut.edu.cn/lu/publications.html,

http: //faculty.ucmerced.edu/mhyang/pubs.html).

Matlab源码和数据集可在我们的网站上获得。

段落2——实验的数据集

We evaluate the performance of the proposed algorithm on nine challenging sequences from prior work [1, 3, 10, 18, 19], the CAVIAR data set(http://groups.inf.ed.ac.uk/vision/CAVIAR/CAVIARDATA1/)

and our own.

Reference——数据集

[1] A. Adam, E. Rivlin, and I. Shimshoni. Robust fragments-based tracking using the integral histogram. In CVPR, 2006.

[3] B. Babenko, M.-H. Yang, and S. Belongie. Visual tracking with online multiple instance learning. In CVPR, 2009.

[10] J. Kwon and K. M. Lee. Visual tracking decomposition. In CVPR,2010.

[18] D. Ross, J. Lim, R.-S. Lin, and M.-H. Yang. Incremental learning for robust visual tracking. IJCV, 77(1):125–141, 2008.

[19] J. Santner, C. Leistner, A. Saffari, T. Pock, and H. Bischof. Prost:Parallel robust online simple tracking. In CVPR, 2010.

我们提出的算法性能在先前工作用到九个有挑战性的图像序列上进行了测试，包括CAVIAR数据集和我们自己的数据集。

The challenges of these videos include illumination variation, partial occlusion, pose variation,background clutter and scale change.

这些视频的挑战包括光照变化，部分遮挡，姿态变化，复杂背景和尺度变化。

The proposed approach is compared with six state-of-the-art tracking methods including incremental visual tracking (IVT) method [18], fragment-based (FragTrack) tracking method [1], L-1 tracker(L-1)[16], multiple instance learning (MIL) tracker [3], visual tracking decomposition (VTD) method [10] and P-N learning (PN) tracker [9].

提出的算法与六个目前最新的跟踪算法进行对比，六个算法包括增量视觉跟踪IVT[18]，基于碎片的跟踪算法[1]，L-1跟踪器[16]，多实例学习跟踪器[3]，视觉分解算法[10]和P-N学习跟踪器[9]。

For fair evaluation, we evaluate the proposed tracker against those methods using the source codes provided by the authors.

为了公平的评估，我们评估提出的这些算法的跟踪器使用作者提供的源码。

Each tracker is run with adjusted parameters.

每个跟踪器用调整后的参数运行。

6.1Quantitative Evaluation——定量评价

两个标准：中心位置误差，中心位置误差平均值和PASCAL VOC的检测标准

Two evaluation criteria are employed to quantitatively assess the performance of the trackers.

两个评价标准用来定量评估追踪器的性能。

Figure 5 presents the relative position errors (in pixels) between the center and the tracking results.

图5给出了中心和跟踪结果的相对位置误差(像素级)。

图5跟踪器的定量评估的位置错误比较

Table 1 summarizes the average center location errors in pixels.

表1总结了像素上的平均中心位置误差。

表1 平均中心误差。最好的两个结果用红蓝色字体显示。

In addition, given the tracking result RT and the ground truth RG, we use the detection criterion in the PASCAL VOC [6] challenge, i.e.,to evaluate the success rate.

Reference——PASCAL VOC检测标准

[6] M. Everingham, L. Van Gool, C. Williams, J. Winn, and A. Zisserman.The pascal visual object classes (voc) challenge. IJCV,88(2):303–338, 2010.

此外，给定跟踪结果RT和标准RG，我们用 PASCAL VOC 挑战的检测标准，例如，来评估成功率。

Table 2 gives the average success rates.

图2给出了平均成功率。

图2跟踪算法的成功率。最好的结果用红蓝字体显示。

Overall, the proposed tracker performs favorably against state-of-the-art methods.

总的来说，提出的跟踪器比目前的方法性能更优。

The performance of our approach can be attributed to the efficient pooling methods across sparse codes of local image patches with a spatial layout.

我们算法的性能归功于在有空间布局的局部图像块上的有效pooling算法。

6.2Quanlitive Evaluation——定性评估

遮挡问题

Occlusion: Figure 6 demonstrates how the proposed method performs when the target undergoes heavy occlusion or long-time partial occlusion.

遮挡：图6说明了当发生严重遮挡时或长时间的部分遮挡时提出算法的运行效果。

图6跟踪目标被严重遮挡时的跟踪结果

In the Faceocc2 sequence,numerous trackers drift away from the target or do not scale well when the face is heavily occluded.

在Faceocc2序列中，许多跟踪器从目标上漂移或当脸被严重遮挡时尺度发生变化。

Our tracker is able to track the target accurately because the structural local sparse appearance model has both spatial and partial information of the target.

因为结构化的局部稀疏表示模型有空间信息和目标的部分信息，所以我们的跟踪器能准确跟踪目标。

Those information helps avoid much influence of occlusion and better estimate the target.

这些信息有助于避免遮挡的影响并且更好的估计目标。

In the Caviar sequence, numerous methods fail to track the target because there are similar objects around it when heavy occlusion occurs.

在Caviar序列中，由于当遮挡发生时在目标周围有相似的对象，因此许多方法跟踪目标失败。

Our tracker does not drift away when the target reappears again because it is easier to differentiate the target and similar objects using both holistic and local information.

由于我们的跟踪器使用整体和局部信息因此很容易区分目标跟相似对象，当目标再出现时我们的跟踪器不会发生漂移。

Furthermore, our tracker is not affected much by occlusion owing to the structural local sparse appearance model and robust template update scheme.

此外，由于结构化局部稀疏表示模型和鲁棒的模版更新策略，因此我们的跟踪器受遮挡的影响不大。

In the Woman sequence, the target object undergoes pose variation together with long-time partial occlusion.

在Woman序列，目标对象发生了姿态变化和长时间的部分遮挡。

Based on the local patches and adaptive template update strategy, our tracker focuses more on the upper body which remains almost the same though the lower body changes a lot or is heavily occluded.

基于局部块和自适应的模版更新策略，我们的跟踪器更多的集中在保留几乎相同信息的上半身而下半身变化较多或被严重遮挡。

It can successfully track the target throughout the entire sequence.

它能在整个序列中成功的跟踪目标。

The PN tracker (based on object detection with global search)is able to re-acquire the target when the target object reappears after occlusion.

PN跟踪器(基于全局搜索的目标检测)在遮挡之后目标对象再出现时能重新得到目标。

However, the other trackers lock on a car with similar color to the trousers when the legs of the woman are heavily occluded.

而且，当女人的腿被严重遮挡时其它的跟踪器锁定在有裤子颜色相似的车上。

光照变化

Illumination change: Figure 7 presents the tracking results in the sequences with large illumination variation.

光照变化：图7是序列中发生大的光照变化的跟踪结果。

图7发生大的光照变化的跟踪结果

In the Car 11 sequence, the contrast between the target and the background is low.

在Car11序列中，目标和背景的对比度很低。

The IVT tracker and our method perform well in tracking the vehicle while the other methods drift to the cluttered background or other vehicles when drastic illumination variation occurs.

当发生强烈的光照变化时，IVT跟踪器和我们的算法跟踪车辆表现很好而其它的算法跟踪结果漂移杂乱背景或其它车辆上。

This can be attributed to the use of incremental subspace learning which is able to capture appearance change due to lighting change.

这主要是由于使用了增量子空间学习，增量子空间学习能抓住光照引起的外观变化。

In the David sequence, a person walks out of the dark conference room and into an area with spot lights.

在David序列中，一个人从黑暗的会议室中走到有灯光的区域。

Likewise,in the Singer sequence a woman undergoes large appearance change due to drastic illumination variation and scale change.

同样的，在Singer序列中，由于强光和尺度变化，一个女人发生了大的外观变化。

While a few trackers are able to keep track of the target to the end, the proposed algorithm achieves low tracking error and high success rate.

虽然一些跟踪器能跟踪目标到最后，但提出的算法能取得较小的跟踪误差和高成功率。

复杂背景

Background clutter: Figure 8 presents the tracking results where the target objects appear in background clutters.

杂乱背景：图8是目标对象出现在杂乱背景的跟踪结果。

图8目标出现在杂乱背景的跟踪结果

For Board sequence, most trackers drift away from the target as holistic representations are not effective in handling objects with large shape variations.

对于Board序列，由于整体表示不能有效的处理目标大的形态变化，因此大多数跟踪器从目标上发生了漂移。

The FragTrack and proposed methods are able to track the target better due to the use of local appearance models.

由于局部外观模型的使用FragTrack和提出的算法能更好的跟踪目标。

The Stone sequence is challenging as there there are numerous stones of different shape and color.

由于目标周围有许多不同形状和颜色的石子，因此Stone序列是有挑战性的。

The FragTrack, MIL and VTD trackers drift to stones when the target is occluded whereas the IVT tracker and our method successfully keep track of the target throughout the sequence.

当目标被遮挡时FragTrack，MIL和VTD跟踪器发生了漂移而IVT跟踪器和我们的算法在整个序列中成功的跟踪了目标。

The PN tracker (based on object detection with global search) is able to re-acquire the target again after drifting to the background, but with higher tracking errors and lower success rate.

在漂移到背景后，PN跟踪器(基于全局搜索的目标检测)能重新得到目标，但有很高的跟踪误差和较低的成功率。

7. Conclusion——结论

In this paper, we propose an efficient tracking algorithm based on structural local sparse appearance model and adaptive template update strategy.

在本文中，我们提出了一种基于结构化局部稀疏外观模型和自适应模版更新策略的有效的跟踪算法。

The proposed method exploits both spatial and local information of the target by alignment-pooling across the local patches with a spatial layout.

提出了算法通过在空间布局上的局部块进行alignment-pooling探索了目标的空间和局部信息。

This helps locate the target more accurately and is less insensitive to occlusion.

这有助于定位目标更准确并且对遮挡更不敏感。

In addition, sparse representation is combined with incremental subspace learning for template update.

此外，稀疏表示与增量子空间学习结合用来进行模版更新。

It not only adapts the tracker to account for appearance change of the target but also prevents incorrectly estimated or occluded observations from being put into the template set for update.

它不仅能使跟踪器适应目标的外观变化而且能阻止更新时不正确的估计或遮挡的观测进入到模版集中。

Experimental results compared with several state-of-the-art methods on challenging sequences demonstrate the effectiveness and robustness of the proposed algorithm.

在挑战性序列上与当前的算法的实验结果对比表明了提出的算法的有效性和鲁棒性。

Acknowledgements——致谢

X. Jia and H. Lu are supported by the National Natural Science Foundation of China #61071209. M.-H. Yang is supported by the US National Science Foundation CAREER Grant #1149783 and IIS Grant #1152576.

X. Jia 和H. Lu 有中国国家自然科学基金#61071209支持。M.-H. Yang由美国国家科学基金CAREER Grant #1149783和IIS Grant #1152576支持。

Reference