(12) tensorboard

1.介绍

TensorBoard是Tensorflow自带的一个强大的可视化工具，也是一个web应用程序套件。在众多机器学习库中，Tensorflow是目前唯一自带可视化工具的库。它可以用来展示网络图、张量的指标变化、张量的分布情况等。特别是在训练网络的时候，我们可以设置不同的参数（比如：权重W、偏置B、卷积层数、全连接层数等），使用TensorBoader可以很直观的帮我们进行参数的选择。

2.使用

2.1 显示程序的图，并用with tf.name_scope(‘name’)语句使图更简单易懂，下面程序作为例子

1.程序中先使用 with.tf.name_scope(‘name’) 把程序流程中的各小块放到各自的命名空间下

2.各命名空间下的各操作也可以命名

3.在会话下加上 writer=tf.summary.FileWriter(‘TensorGraph/’,sess.graph) 。该语句第一个参数是路径，即把图放在当前目录下的TensorGraph文件，没
有此文件会自动创建。

4.运行程序，就会生成图。

5.打开图在cmd进行，输入tensorboard --logdir=F:\jupyter_notebook\tensorflow_test\TensorGraph --host=127.0.0.1，运行后会给一个网址，用google打开此网址

6.终止此图：ctrl+c

7.当因为修改了程序而导致图发生变化时，需要先把代表原图的文件删掉，然后重新运行程序，即jupyter notebook—>Kernel---->Restart & Runall，然后再在cmd打开此图，这是就是更新后的

import tensorflow as tf
#导入手写数字相关工具包
from tensorflow.examples.tutorials.mnist import input_data#载入数据集,这个语句会自动下载数据集，若网速慢也可以自己下载
mnist=input_data.read_data_sets("MNIST_data",one_hot=True)#定义每次放入神经网路的图片数量，也就是训练数量
batch_size=50
#计算共有多少批次,mnist.train.num_examples代表训练数据的数量
n_batch=mnist.train.num_examples//batch_sizewith tf.name_scope('input'):#定义两个placeholder#[None,784]：行数不定，但有784列（因为图片像素是28*28=784，数据集表示为60000*784，因此此处列数也为784）x=tf.placeholder(tf.float32,[None,784],name='xInput')#[None,10]:数据集中每个数字可分为0到9十个类y=tf.placeholder(tf.float32,[None,10],name='yInput')with tf.name_scope('compute'):#创建简单的神经网络:输入层784个神经元，输出层10个神经元w=tf.Variable(tf.zeros([784,10]),name="wight") #权值，连接输入和输出b=tf.Variable(tf.zeros([10]),name="bias")   #偏置值prediction=tf.nn.softmax(tf.matmul(x,w)+b,name="softmaxNum")  # 通过tf.matmul(x,w)+b计算出信号总和，再通过softmax转为概率值# #二次代价函数作为损失函数(原版本)
# loss=tf.reduce_mean(tf.square(y-prediction))with tf.name_scope('train'):
#使用对数似然函数作为代价函数loss=tf.nn.softmax_cross_entropy_with_logits_v2(labels=y,logits=prediction)#使用Adam优化器最小化losstrain_step=tf.train.AdamOptimizer(0.001).minimize(loss)#定义准确率，来求训练好的模型的预测值准不准确：
#tf.argmax(y,1)是返回y所有类中概率最大的那个；equal是比较两个参数是否相等，返回bool类型
#因为tf.argmax(y,1)的结果是列表，因此correct_prediction也是列表
correct_prediction=tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))
#接下来求准确率,tf.cast()是将bool类型的值转换为浮点型；再用reduce_mean()求平均值得出概率
accuracy=tf.reduce_mean(tf.cast(correct_prediction,tf.float32))#初始化变量
init=tf.global_variables_initializer()with tf.Session() as sess:sess.run(init)#生成本程序的网络图文件writer=tf.summary.FileWriter('TensorGraph/',sess.graph) for num in range(1):  #因为这个程序是为了演示图，所以不花时间训练了for i in range(n_batch):    # n_batch是训练数据的批次数量#获得当前批次的图片，图片数据保存在batchX，图片标签保存在batchYbatchX,batchY=mnist.train.next_batch(batch_size)sess.run(train_step,feed_dict={
    x:batchX,y:batchY})#每训练完一次，看一次准确率,传的数据是测试集的图片和标签print("第{0}次准确率:".format({
    num}),sess.run(accuracy,feed_dict={
    x:mnist.test.images,y:mnist.test.labels}))

Extracting MNIST_data\train-images-idx3-ubyte.gz
Extracting MNIST_data\train-labels-idx1-ubyte.gz
Extracting MNIST_data\t10k-images-idx3-ubyte.gz
Extracting MNIST_data\t10k-labels-idx1-ubyte.gz
第{0}次准确率: 0.9051

2.2 查看运行时图的数据，权重、偏置、损失值、准确率等值在迭代过程中的显示

1.使用tf.summary.scalar(‘accuracy’, accuracy)来保存值，第一个参数是名字，第二个参数代表值

2.tf.summary.histogram(‘weights’, w)直方图展示

3.最后通过 merge=tf.summary.merge_all() 把所有监测值合并起来

4 让merge随着训练模型一起运行，注意此处要用两个变量接收，要不格式会出错

5.每运行一次结束后，通过 writer.add_summary(summary,num)将运行一次监测值的变化和第几次加入到生成的文件中

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_datamnist=input_data.read_data_sets("MNIST_data",one_hot=True)batch_size=50
n_batch=mnist.train.num_examples//batch_sizewith tf.name_scope('input'):x=tf.placeholder(tf.float32,[None,784],name='xInput')y=tf.placeholder(tf.float32,[None,10],name='yInput')with tf.name_scope('compute'):w=tf.Variable(tf.zeros([784,10]),name="wight") #权值，连接输入和输出#展示权值 w 的均值变化tf.summary.scalar('weightScalar',tf.reduce_mean(w))#用直方图展示 w 变化情况tf.summary.histogram('weightHistogram',w)b=tf.Variable(tf.zeros([10]),name="bias")   #偏置值#展示偏置值 b 的均值变化tf.summary.scalar('biasScalar',tf.reduce_mean(b))prediction=tf.nn.softmax(tf.matmul(x,w)+b,name="softmaxNum")  # 通过tf.matmul(x,w)+b计算出信号总和，再通过softmax转为概率值with tf.name_scope('loss'):loss=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(labels=y,logits=prediction))#展示损失值变化tf.summary.scalar('lossScalar',loss)
with tf.name_scope('train'):#使用Adam优化器训练模型，也就是最小化损失函数train_step=tf.train.AdamOptimizer(0.001).minimize(loss)with tf.name_scope('accuracyCompute'):with tf.name_scope('correct_prediction'):correct_prediction=tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))with tf.name_scope('accuracy'):accuracy=tf.reduce_mean(tf.cast(correct_prediction,tf.float32))#展示准确度的变化tf.summary.scalar('accuracyScalar',accuracy)init=tf.global_variables_initializer()#合并所有的监测的指标的变化
merged= tf.summary.merge_all()with tf.Session() as sess:sess.run(init)#生成本程序的网络图文件writer=tf.summary.FileWriter('TensorGraph/',sess.graph) for num in range(21): for i in range(n_batch):batchX,batchY=mnist.train.next_batch(batch_size)#run第一参数加上merge是为了一边训练一边统计；summary是接受云心那个merge结果的，trainRes是接收train_step的，不过不需要trainRes#summary,trainRes=sess.run([merged,train_step],feed_dict={x:batchX,y:batchY})summary=sess.run(merged,feed_dict={
    x:batchX,y:batchY})sess.run(train_step,feed_dict={
    x:batchX,y:batchY})#将summary和sum加入到生成的文件中，sum是第几次writer.add_summary(summary,num)print("第{}次准确率:".format({
    num}),sess.run(accuracy,feed_dict={
    x:mnist.test.images,y:mnist.test.labels}))

WARNING:tensorflow:From <ipython-input-10-6d2870582292>:4: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use alternatives such as official/mnist/dataset.py from tensorflow/models.
WARNING:tensorflow:From F:\anaconda\lib\site-packages\tensorflow_core\contrib\learn\python\learn\datasets\mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Please write your own downloading logic.
WARNING:tensorflow:From F:\anaconda\lib\site-packages\tensorflow_core\contrib\learn\python\learn\datasets\mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.data to implement this functionality.
Extracting MNIST_data\train-images-idx3-ubyte.gz
WARNING:tensorflow:From F:\anaconda\lib\site-packages\tensorflow_core\contrib\learn\python\learn\datasets\mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.data to implement this functionality.
Extracting MNIST_data\train-labels-idx1-ubyte.gz
WARNING:tensorflow:From F:\anaconda\lib\site-packages\tensorflow_core\contrib\learn\python\learn\datasets\mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.one_hot on tensors.
Extracting MNIST_data\t10k-images-idx3-ubyte.gz
Extracting MNIST_data\t10k-labels-idx1-ubyte.gz
WARNING:tensorflow:From F:\anaconda\lib\site-packages\tensorflow_core\contrib\learn\python\learn\datasets\mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use alternatives such as official/mnist/dataset.py from tensorflow/models.
第{0}次准确率: 0.9081
第{1}次准确率: 0.9172
第{2}次准确率: 0.9231
第{3}次准确率: 0.9248
第{4}次准确率: 0.9235
第{5}次准确率: 0.9273
第{6}次准确率: 0.9278
第{7}次准确率: 0.9287
第{8}次准确率: 0.9286
第{9}次准确率: 0.9306
第{10}次准确率: 0.931
第{11}次准确率: 0.9318
第{12}次准确率: 0.9307
第{13}次准确率: 0.9319
第{14}次准确率: 0.9318
第{15}次准确率: 0.9321
第{16}次准确率: 0.933
第{17}次准确率: 0.9325
第{18}次准确率: 0.9324
第{19}次准确率: 0.9325
第{20}次准确率: 0.9325

原先运行时一直报错！！！！各种错误！！！

第一种错误:tags and values not the same shape: [] != [50] (tag ‘loss/lossScalar’)[[{ {node loss/lossScalar}}]]

错误代码:loss=tf.nn.softmax_cross_entropy_with_logits_v2(labels=y,logits=prediction)

改正:loss=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(labels=y,logits=prediction))

第二种错误:提示placeholder没有传数据？？？这他妈是放屁

改正:重启jupyter notebook;重启在Kernel下，所以每次运行都要重启服务！！！！！

2.3 分类过程可视化

import tensorflow  as tf
from tensorflow.examples.tutorials.mnist import input_data
from tensorflow.contrib.tensorboard.plugins import projector#载入数据集
mnist = input_data.read_data_sets("MNIST_data/",one_hot=True)
#运行次数
max_steps = 50
#图片数量
image_num = 3000
#文件路径
DIR = "F:/jupyter_notebook/tensorflow_test"#定乂会活
sess = tf.Session()
#载入囹片
embedding = tf.Variable(tf.stack(mnist.test.images[:image_num]),trainable=False,name='embedding' )#参数概要
def variable_summaries(var):with tf.name_scope('summaries'):mean = tf.reduce_mean(var)tf.summary.scalar( 'mean',mean) #平均値with tf.name_scope('stddev'):stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))tf.summary.scalar('stddev',stddev) #标准差tf.summary.scalar('max',tf.reduce_max(var))#最大値tf.summary.scalar('min', tf.reduce_min(var)) #最小値tf.summary.histogram('histogram',var) #直方图#命名空间
with tf.name_scope('input'):#这里的none表示第一个维度可以是任意的长度x = tf.placeholder(tf.float32,[None, 784],name='x-input')#正确的标签y = tf.placeholder(tf.float32,[None, 10],name='y-input')#显示图片
with tf.name_scope('input.reshape'):image_shaped_input = tf.reshape(x,[-1,28,28,1])tf.summary.image('input',image_shaped_input,10)with tf.name_scope('layer'):#创建一一个简单神经网络with tf.name_scope('weights'):W = tf.Variable(tf.zeros([784,10]),name='W')variable_summaries (W)with tf.name_scope('biases'):b = tf.Variable(tf.zeros([10]),name='b' )variable_summaries (b)with tf.name_scope('wx_plus_b') :wx_plus_b = tf.matmul(x,W) + bwith tf.name_scope('softmax'):prediction = tf.nn.softmax(wx_plus_b)with tf.name_scope('loss'):#交叉熵代价函数loss=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(labels=y,logits=prediction))tf.summary.scalar('1oss',loss)
with tf.name_scope('train'):#使用梯度下降法train_step=tf.train.AdamOptimizer(0.001).minimize(loss)#初始化变量
sess.run(tf.global_variables_initializer())with tf.name_scope('accuracy'):with tf.name_scope('correct_prediction') :#结果存放在一个布尔型列表中correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回-维张量中最大的值所在的位置with tf.name_scope('accuracy'):#求准确率accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) #把correct_prediction变为float32类型tf.summary.scalar('accuracy',accuracy)#产生metadata文件
if tf.gfile.Exists(DIR + '/projector/projector/metadata.tsv'):tf.gfile.DeleteRecursively(DIR +'/projector/projector/metadata.tsv')
with open(DIR+'/projector/projector/metadata.tsv','w') as f:labels=sess.run(tf.argmax(mnist.test.labels[:],1))for i in range(image_num):f.write(str(labels[i])+'\n')#合并所有的summary
merged = tf.summary.merge_all()projector_writer =tf.summary.FileWriter(DIR +'/projector/projector',sess.graph)
saver = tf.train.Saver()
config = projector.ProjectorConfig()
embed = config.embeddings.add()
embed.tensor_name = embedding.name
embed.metadata_path = DIR +'/projector/projector/metadata.tsv'
embed.sprite.image_path = DIR +'projector/data/mnist_10k_sprite.png'
embed.sprite.single_image_dim.extend([28, 28])
projector.visualize_embeddings(projector_writer,config)for i in range (max_steps):#每个批次100个样本batch_xs,batch_ys = mnist.train.next_batch(100)run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)run_metadata = tf.RunMetadata()summary,_=sess.run([merged,train_step],feed_dict={
    x:batch_xs,y:batch_ys},options=run_options,run_metadata=run_metadata)projector_writer.add_run_metadata(run_metadata,'step%03d' % i)projector_writer.add_summary(summary,i)if i%5 == 0:acc = sess.run(accuracy,feed_dict={
    x:mnist.test.images,y:mnist.test.labels})print ("Iter " + str(i) +" , Testing Accuracy= " + str (acc))saver.save(sess,DIR +'/projector/projector/a_model.ckpt',global_step=max_steps)
projector_writer.close()
sess.close()

WARNING:tensorflow:From <ipython-input-1-69a118b0aa4e>:6: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use alternatives such as official/mnist/dataset.py from tensorflow/models.
WARNING:tensorflow:From F:\anaconda\lib\site-packages\tensorflow_core\contrib\learn\python\learn\datasets\mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Please write your own downloading logic.
WARNING:tensorflow:From F:\anaconda\lib\site-packages\tensorflow_core\contrib\learn\python\learn\datasets\mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.data to implement this functionality.
Extracting MNIST_data/train-images-idx3-ubyte.gz
WARNING:tensorflow:From F:\anaconda\lib\site-packages\tensorflow_core\contrib\learn\python\learn\datasets\mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.data to implement this functionality.
Extracting MNIST_data/train-labels-idx1-ubyte.gz
WARNING:tensorflow:From F:\anaconda\lib\site-packages\tensorflow_core\contrib\learn\python\learn\datasets\mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.one_hot on tensors.
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
WARNING:tensorflow:From F:\anaconda\lib\site-packages\tensorflow_core\contrib\learn\python\learn\datasets\mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use alternatives such as official/mnist/dataset.py from tensorflow/models.
Iter 0 , Testing Accuracy= 0.4341
Iter 5 , Testing Accuracy= 0.5381
Iter 10 , Testing Accuracy= 0.5527
Iter 15 , Testing Accuracy= 0.6098
Iter 20 , Testing Accuracy= 0.6489
Iter 25 , Testing Accuracy= 0.6812
Iter 30 , Testing Accuracy= 0.6982
Iter 35 , Testing Accuracy= 0.7
Iter 40 , Testing Accuracy= 0.7144
Iter 45 , Testing Accuracy= 0.7246