tf.train.piecewise_constant() 用法介绍

我们看一些论文中，常常能看到论文的的训练策略可能提到学习率是随着迭代次数变化的。在tensorflow中，在训练过程中更改学习率主要有两种方式，第一个是学习率指数衰减，第二个就是迭代次数在某一范围指定一个学习率。

tf.train.piecewise_constant()就是为第二种学习率变化方式而设计的。先看下函数定义

def piecewise_constant(x, boundaries, values, name=None)

Example: use a learning rate that's 1.0 for the first 100001 steps, 0.5for the next 10000 steps, and 0.1 for any additional steps.```pythonglobal_step = tf.Variable(0, trainable=False)boundaries = [100000, 110000]values = [1.0, 0.5, 0.1]learning_rate = tf.train.piecewise_constant(global_step, boundaries, values)

以上是tf源码的定义和自带一个使用例子。

形参x指的是global_step，其实就是迭代次数，boundaries一个列表，内容指的是迭代次数所在的区间，values是个列表，存放在不同区间该使用的学习率的值。

注意，values中的数目应该比boundaries中的数目大1，原因很简单，无非是两个数可以制定出三个区间嘛，有三个区间自然要用3个学习率。

下面看一下实验代码。

import tensorflow as tfvalues = [0.1,0.01,0.001]
boundaries = [20,30]
global_step = tf.Variable(0,trainable=False)  # 因为此代码没有管理global的优化器，其实此处没有用
learning_rate = tf.train.piecewise_constant(global_step,boundaries,values)init =tf.global_variables_initializer()
with tf.Session() as sess:sess.run(init)for global_step in range(100):            #用局部的global_step代替learning_rate = tf.train.piecewise_constant(global_step, boundaries, values)lr = sess.run(learning_rate)print('the %d steps, lr is %f' %(global_step,lr))

部分结果如下：

.....
the 17 steps, lr is 0.100000
the 18 steps, lr is 0.100000
the 19 steps, lr is 0.100000
the 20 steps, lr is 0.100000
the 21 steps, lr is 0.010000
the 22 steps, lr is 0.010000
......
the 28 steps, lr is 0.010000
the 29 steps, lr is 0.010000
the 30 steps, lr is 0.010000
the 31 steps, lr is 0.001000
the 32 steps, lr is 0.001000
the 33 steps, lr is 0.001000