Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Net_综合

文章目录

Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks
- - Abstract
  - I. INTRODUCTION
  - II. ONE-DIMENSIONAL CNN-BASED HSI FE AND CLASSIFICATION
  - - A. Neural Network and Deep Learning
    - B. CNN (1-D CNN)
  - III. TWO-DIMENSIONAL CNN-BASED HSI FE AND CLASSIFICATION
  - - A. Two-Dimensional CNN
    - B. Fine-Tuning and Classification
  - IV. THREE-DIMENSIONAL CNN-BASED HSI FE AND CLASSIFICATION
  - - A. Three-Dimensional CNN
    - B. Spectral–Spatial FE Framework
    - C. Regularizations Based on Sparse Constraints
  - V. VIRTUAL SAMPLE ENHANCED CNN
  - - A. Changing Radiation-Based Virtual Samples
    - B. Mixture-Based Virtual Samples
  - VI. EXPERIMENTAL RESULTS
  - - A. Data Description and Experiment Design
    - B. Design CNN With Spectral Features
    - C. CNN With Spatial Features
    - D. CNN With Spatial–Spectral Features：
    - E. CNN With Virtual Sample
  - VII. DISCUSSION AND CONCLUSION

Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks

可以说深度学习在高光谱影像分类中应用领域引用量最高的论文了吧，值得做个总结。

Abstract

主要贡献：提出CNN用于提取特征。1DCNN , 2DCNN, 3DCNN。
正则化策略：L2 regularization，dropout。
数据增强策略：virtual samples。
实验数据集：Indian Pines, University of Pavia, and Kennedy Space Center。

I. INTRODUCTION

balabala。。。
对Abstract 进行扩展。

II. ONE-DIMENSIONAL CNN-BASED HSI FE AND CLASSIFICATION

A. Neural Network and Deep Learning

故事性的引出深度学习。

B. CNN (1-D CNN)

1D CNN理论
C. Spectral FE Framework for HSI Classification
1D CNN实践
D. L2 Regularization of CNN
优化：L2 regularization

III. TWO-DIMENSIONAL CNN-BASED HSI FE AND CLASSIFICATION

A. Two-Dimensional CNN

2D CNN理论

B. Fine-Tuning and Classification

2D CNN实践

IV. THREE-DIMENSIONAL CNN-BASED HSI FE AND CLASSIFICATION

A. Three-Dimensional CNN

3D CNN 理论

B. Spectral–Spatial FE Framework

3D CNN实践

C. Regularizations Based on Sparse Constraints

这里有一个很有意思的地方：
The dimensionality of the spectral-based CNN, which is presented in Section II-C, is around a couple of hundreds (the number of bands); the dimensionality of the spatial-based CNN, which is presented in Section III-B, is around several hundreds (K×K,e.g.,K=27); the imensionality of the spectral-and-spatial-based CNN, which is presented in Section IV-B, is around several thousands (K ×K ×B).
这个维度为什么是这样算的？是应该简单的记住还是应该深究？

V. VIRTUAL SAMPLE ENHANCED CNN

A. Changing Radiation-Based Virtual Samples

简单线性变换+随机噪声。

B. Mixture-Based Virtual Samples

简单数据融合+随机噪声。

VI. EXPERIMENTAL RESULTS

A. Data Description and Experiment Design

数据表述
实验设置：训练集的选择标准不清楚，好像是10%，但是每个类别的数量论文有。
试验分为四个部分：1DCNN , 2DCNN, 3DCNN，virtual samples。

B. Design CNN With Spectral Features

Architecture Design of the 1-D CNN:
trial-and-error approach again to determine the parameters of model on the number of nodes.
trial-and-error 这个单词我都要忘记了。

数据标准化[-1, 1]，很多参数设置。。。
2) Visualization and Analysis of the 1-DCNN:
对比了随机初始化前后权重的可视化图。
他们确实不一样，但是我感觉可视化其实并不能看出什么。

使用了一种相似度量方法度量相同类和不同类别的相似度，作为一种定量评估指标。
随着层数的加深，同类的相似度在增加，不同类的相似度在减少。这种评估方法还可以。

以后可以用t-sen。
3) Comparisons With Different FE Methods and Classifiers：
多对多：不同的特征抽取器+不同的分类器组合。
特征抽取器：CNN，PCA，FA，LLE。
分类器：KNN，LR，RBF－ＳＶＭ

各种参数选择的介绍。通过trail-and-error 的方法得到。

C. CNN With Spatial Features

The first principal component is used to create the 2-D input.
这是什么意思？只用一维的输入？如果这样的话，后面选择这么大的neighborhood window 就是合理的。
虽然输入的是几百维的数据，但是这个输入经过了降维，然后又才创建patch传入CNN。

Architecture Design, Visualization, and Analysis:

数据标准化[-0.5,0.5]
a large neighborhood window (27 × 27) for the first principal component.
只输入的一维的高光谱影像，在流程上对上了，但是一维信息够吗？
他说是空间信息，还TM真是空间信息，只有一维信息，没有任何光谱信息了！

上面的内容我为什么写两次呢？因为我在2021年看到这种输入信息的方式，我感觉很奇怪，只用一维。不过考虑到当时的硬件限制，理论限制和后面还有3D-CNN的空谱联合的高维输入，还算合理。

D. CNN With Spatial–Spectral Features：

Architecture Design and Parameter Analysis:

We use 27 × 27 × 200, 27 × 27 × 103, and 27 × 27 × 176 neighbors of each pixel as the input 3-D images. 这个在当时的条件下合理吗？以我现在的硬件，虽然我的硬件很垃圾，但是制作这么大的数据集也很困难。
数据标准化[-0.5,0.5]
三个参数分析：dropout, ReLU, and the size of the，spatial window.

Comparative Experiments With Other Spectral–Spatial Methods:
介绍了extended morphological profile 的实现细节。
Computation Cost of the 3-D CNN:

从训练时间来看又是合理的，这么小的数据集，一次训练就几乎要，一次就要将近半个小时。

E. CNN With Virtual Sample

Classification Results:

αm is a uniformly distributed random number in [0.9, 1.1], and β, which is the weight of noise n, is set to 1/25.

αi and αj are uniformly distributed random numbers on the interval [0, 1]

合成没有直接加噪声好。
但是没有写生成了多样样本，让人捉急。

来自 Hyperspectral Image Classification Based on Parameter-Optimized 3D-CNNs Combined with Transfer Learning and Virtual Samples 的参考这篇论文的 virtual sampes 生成，这里介绍的更加详细一点。

η coefficient value.
n Gaussian noise.
η can be set to a uniformly distributed random number in [0.9, 1.1].
If the number of original training samples selected from among the target data is T, then the number of virtual samples will be P × T where P represents the ratio between the number of virtual samples and the number of original samples, and the noise variance of n in Equation (4) could be set to 0.01 at the beginning.
在这里插入图片描述

the number of virtual samples is equal to the number of orignal samples will reaches the highest value.

As presented in Table 4, the OA value is relatively high when the noise variance σ^2 is less than 0.001.
高斯噪声设置小一点好。

Classification Maps:
看看就好。

VII. DISCUSSION AND CONCLUSION

总的来说，作为早期deep learning 在高光谱影像分类的代表作，思想很重要。以现在的眼光来看，速度，精度还是很低的。
目前我也无法理解，他产生2727103 的pavia 上的patch，有必要吗？这得要多好的设备。
论文细节：计算设备，虚拟样本生成的具体情况都没有仔细介绍，
但这一点瑕疵不妨碍他作为我目前知道的基于深度学习的引用最高的高光谱影像分类论文。

我没有代码，但是github上的DeepHyperX库貌似有代码，有兴趣的可以找一下。

参考文献：Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks，chen，TGRS, 2016
Hyperspectral Image Classification Based on Parameter-Optimized 3D-CNNs Combined with Transfer Learning and Virtual Samples，liu, RS, 2018