基于MNIST数据集实现2层CNN神经网络案例实战-大数据ML样本集案例实战

564 阅读5分钟

版权声明:本套技术专栏是作者(秦凯新)平时工作的总结和升华,通过从真实商业环境抽取案例进行总结和分享,并给出商业应用的调优建议和集群环境容量规划等内容,请持续关注本套博客。QQ邮箱地址:1120746959@qq.com,如有任何学术交流,可随时联系。

1 神经网络基本结构定义

  • 28*28=784个像素点,第一层神经元256,第二层神经元128

2 基本神经网络构建

  • 变量初始化

      import numpy as np
      import tensorflow as tf
      import matplotlib.pyplot as plt
      import input_data
      mnist = input_data.read_data_sets('data/', one_hot=True)
      Extracting data/train-images-idx3-ubyte.gz
      Extracting data/train-labels-idx1-ubyte.gz
      Extracting data/t10k-images-idx3-ubyte.gz
      Extracting data/t10k-labels-idx1-ubyte.gz
    
      # NETWORK TOPOLOGIES
      #第一层神经元
      n_hidden_1 = 256 
      #第二层神经元
      n_hidden_2 = 128
      #28*28 784像素点
      n_input    = 784 
      # 类别10
      n_classes  = 10  
      
      # INPUTS AND OUTPUTS
      x = tf.placeholder("float", [None, n_input])
      y = tf.placeholder("float", [None, n_classes])
          
      # NETWORK PARAMETERS
      stddev = 0.1
      #初始化
      weights = {
          'w1': tf.Variable(tf.random_normal([n_input, n_hidden_1], stddev=stddev)),
          'w2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2], stddev=stddev)),
          'out': tf.Variable(tf.random_normal([n_hidden_2, n_classes], stddev=stddev))
      }
      #初始化
      biases = {
          'b1': tf.Variable(tf.random_normal([n_hidden_1])),
          'b2': tf.Variable(tf.random_normal([n_hidden_2])),
          'out': tf.Variable(tf.random_normal([n_classes]))
      }
      print ("NETWORK READY")
    
  • 前向传播(每一层增加激活函数sigmoid,最后一层不加sigmoid)

       def multilayer_perceptron(_X, _weights, _biases):
          layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(_X, _weights['w1']), _biases['b1'])) 
          layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, _weights['w2']), _biases['b2']))
          return (tf.matmul(layer_2, _weights['out']) + _biases['out'])
    
  • 损失变量和优化器定义

  • softmax_cross_entropy_with_logits交叉熵损失函数(参数pred预测值),reduce_mean除以样本总数。

  • GradientDescentOptimizer采用梯度下降优化求解

      # PREDICTION
      pred = multilayer_perceptron(x, weights, biases)
      
      # LOSS AND OPTIMIZER
      cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y)) 
      optm = tf.train.GradientDescentOptimizer(learning_rate=0.001).minimize(cost) 
      
      #准确率求解
      corr = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))    
      accr = tf.reduce_mean(tf.cast(corr, "float"))
      
      # INITIALIZER
      init = tf.global_variables_initializer()
      print ("FUNCTIONS READY")
    
  • 按照Batch迭代

      training_epochs = 20
      batch_size      = 100
      display_step    = 4
      # LAUNCH THE GRAPH
      sess = tf.Session()
      sess.run(init)
      # OPTIMIZE
      for epoch in range(training_epochs):
          avg_cost = 0.
          total_batch = int(mnist.train.num_examples/batch_size)
          
          # ITERATION(按照Batch迭代,每一次迭代100)
          for i in range(total_batch):
              batch_xs, batch_ys = mnist.train.next_batch(batch_size)
              #填充值
              feeds = {x: batch_xs, y: batch_ys}
              #sess.run(模型训练)
              sess.run(optm, feed_dict=feeds)
              avg_cost += sess.run(cost, feed_dict=feeds)
          avg_cost = avg_cost / total_batch
          # DISPLAY
          if (epoch+1) % display_step == 0:
              print ("Epoch: %03d/%03d cost: %.9f" % (epoch, training_epochs, avg_cost))
              feeds = {x: batch_xs, y: batch_ys}
              
              #sess.run(准确率求解)
              train_acc = sess.run(accr, feed_dict=feeds)
              print ("TRAIN ACCURACY: %.3f" % (train_acc))
              feeds = {x: mnist.test.images, y: mnist.test.labels}
              test_acc = sess.run(accr, feed_dict=feeds)
              print ("TEST ACCURACY: %.3f" % (test_acc))
      print ("OPTIMIZATION FINISHED")
    

3 CNN神经网络

  • 变量初始化

      import numpy as np
      import tensorflow as tf
      import matplotlib.pyplot as plt
      import input_data
      mnist = input_data.read_data_sets('data/', one_hot=True)
      trainimg   = mnist.train.images
      trainlabel = mnist.train.labels
      testimg    = mnist.test.images
      testlabel  = mnist.test.labels
      print ("MNIST ready")
      n_input  = 784
      
      
      n_output = 10
      ##wc1  [3, 3, 1, 64]   中3表示Filter宽度和深度,1表示深度,64表示outchannl最后得到64张特征图。  14*14*128
      ##wc2 [3, 3, 64, 128] 中3表示Filter宽度和深度,1表示深度,64表示输入64张特征图,输出128张特征图。7*7*128   输出1024向量
      ## 卷积层没有减少挺像的大小。
      ## polling层把图像减少到一半
      ## wd1 输入7*7*128 输出1024向量
      weights  = {
              'wc1': tf.Variable(tf.random_normal([3, 3, 1, 64], stddev=0.1)),
              'wc2': tf.Variable(tf.random_normal([3, 3, 64, 128], stddev=0.1)),
              'wd1': tf.Variable(tf.random_normal([7*7*128, 1024], stddev=0.1)),
              'wd2': tf.Variable(tf.random_normal([1024, n_output], stddev=0.1))
          }
      biases   = {
              'bc1': tf.Variable(tf.random_normal([64], stddev=0.1)),
              'bc2': tf.Variable(tf.random_normal([128], stddev=0.1)),
              'bd1': tf.Variable(tf.random_normal([1024], stddev=0.1)),
              'bd2': tf.Variable(tf.random_normal([n_output], stddev=0.1))
          }
    
  • help方法的使用

  • 前向传播

       def conv_basic(_input, _w, _b, _keepratio):
          # INPUT(转换格式,转换成4维 【n,h,w,c】 -1 batchSize大小,可以让TF推断 ,输出通道深度为1)
          _input_r = tf.reshape(_input, shape=[-1, 28, 28, 1])
          
          # 第一层(nn模块CNN, RNN)(conv2 中 strides ->【n,h,w,c】表示在各个上面滑动窗的大小 
          # padding 两种选择 SAME=>滑动窗不够时填充,Valid不填充)。
          _conv1 = tf.nn.conv2d(_input_r, _w['wc1'], strides=[1, 1, 1, 1], padding='SAME')
          
          #_mean, _var = tf.nn.moments(_conv1, [0, 1, 2])
          #_conv1 = tf.nn.batch_normalization(_conv1, _mean, _var, 0, 1, 0.0001)
          # 激活函数relu
          _conv1 = tf.nn.relu(tf.nn.bias_add(_conv1, _b['bc1']))
          
          # max_pool层,ksize表示Window -1 batchSize大小,2*2窗口 1表示,输出通道深度为1
          _pool1 = tf.nn.max_pool(_conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
          
          # dropout不让所有的神经元参与计算比例
          _pool_dr1 = tf.nn.dropout(_pool1, _keepratio)
          
          # 第二层
          _conv2 = tf.nn.conv2d(_pool_dr1, _w['wc2'], strides=[1, 1, 1, 1], padding='SAME')
          #_mean, _var = tf.nn.moments(_conv2, [0, 1, 2])
          #_conv2 = tf.nn.batch_normalization(_conv2, _mean, _var, 0, 1, 0.0001)
          
          _conv2 = tf.nn.relu(tf.nn.bias_add(_conv2, _b['bc2']))
          _pool2 = tf.nn.max_pool(_conv2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
          _pool_dr2 = tf.nn.dropout(_pool2, _keepratio)
          
          #全连接层
          # VECTORIZE
          _dense1 = tf.reshape(_pool_dr2, [-1, _w['wd1'].get_shape().as_list()[0]])
          
          # FULLY CONNECTED LAYER 1
          _fc1 = tf.nn.relu(tf.add(tf.matmul(_dense1, _w['wd1']), _b['bd1']))
          _fc_dr1 = tf.nn.dropout(_fc1, _keepratio)
          
          # FULLY CONNECTED LAYER 2
          _out = tf.add(tf.matmul(_fc_dr1, _w['wd2']), _b['bd2'])
          # RETURN
          out = { 'input_r': _input_r, 'conv1': _conv1, 'pool1': _pool1, 'pool1_dr1': _pool_dr1,
              'conv2': _conv2, 'pool2': _pool2, 'pool_dr2': _pool_dr2, 'dense1': _dense1,
              'fc1': _fc1, 'fc_dr1': _fc_dr1, 'out': _out
          }
          return out
      print ("CNN READY")
    
  • 模型训练和评估

      a = tf.Variable(tf.random_normal([3, 3, 1, 64], stddev=0.1))
      print (a)
      a = tf.Print(a, [a], "a: ")
      init = tf.global_variables_initializer()
      sess = tf.Session()
      sess.run(init)
      
      Tensor("Variable_28/read:0", shape=(3, 3, 1, 64), dtype=float32)
      
      #print (help(tf.nn.conv2d))
      print (help(tf.nn.max_pool))
      
      x = tf.placeholder(tf.float32, [None, n_input])
      y = tf.placeholder(tf.float32, [None, n_output])
      keepratio = tf.placeholder(tf.float32)
      
      # FUNCTIONS
      
      _pred = conv_basic(x, weights, biases, keepratio)['out']
      cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(_pred, y))
      optm = tf.train.AdamOptimizer(learning_rate=0.001).minimize(cost)
      _corr = tf.equal(tf.argmax(_pred,1), tf.argmax(y,1)) 
      accr = tf.reduce_mean(tf.cast(_corr, tf.float32)) 
      init = tf.global_variables_initializer()
          
      # SAVER
      print ("GRAPH READY")
      
      sess = tf.Session()
      sess.run(init)
      
      training_epochs = 15
      batch_size      = 16
      display_step    = 1
      for epoch in range(training_epochs):
          avg_cost = 0.
          #total_batch = int(mnist.train.num_examples/batch_size)
          total_batch = 10
          # Loop over all batches
          for i in range(total_batch):
              batch_xs, batch_ys = mnist.train.next_batch(batch_size)
              # Fit training using batch data
              sess.run(optm, feed_dict={x: batch_xs, y: batch_ys, keepratio:0.7})
              # Compute average loss
              avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keepratio:1.})/total_batch
      
          # Display logs per epoch step
          if epoch % display_step == 0: 
              print ("Epoch: %03d/%03d cost: %.9f" % (epoch, training_epochs, avg_cost))
              train_acc = sess.run(accr, feed_dict={x: batch_xs, y: batch_ys, keepratio:1.})
              print (" Training accuracy: %.3f" % (train_acc))
              #test_acc = sess.run(accr, feed_dict={x: testimg, y: testlabel, keepratio:1.})
              #print (" Test accuracy: %.3f" % (test_acc))
      
      print ("OPTIMIZATION FINISHED")
    
  • 结果展示

       Epoch: 000/015 cost: 30.928401661
       Training accuracy: 0.500
      Epoch: 001/015 cost: 12.954609606
       Training accuracy: 0.700
      Epoch: 002/015 cost: 10.392489696
       Training accuracy: 0.700
      Epoch: 003/015 cost: 7.254891634
       Training accuracy: 0.800
      Epoch: 004/015 cost: 4.977767670
       Training accuracy: 0.900
      Epoch: 005/015 cost: 5.414173813
       Training accuracy: 0.600
      Epoch: 006/015 cost: 3.057567777
       Training accuracy: 0.700
      Epoch: 007/015 cost: 4.929724103
       Training accuracy: 0.600
      Epoch: 008/015 cost: 3.192437538
       Training accuracy: 0.600
      Epoch: 009/015 cost: 3.224479928
       Training accuracy: 0.800
      Epoch: 010/015 cost: 2.720530389
       Training accuracy: 0.400
      Epoch: 011/015 cost: 3.000342276
       Training accuracy: 0.800
      Epoch: 012/015 cost: 0.639763238
       Training accuracy: 1.000
      Epoch: 013/015 cost: 1.897303332
       Training accuracy: 0.900
      Epoch: 014/015 cost: 2.295500937
       Training accuracy: 0.800
      OPTIMIZATION FINISHED
    

4 模型持久化与加载

import tensorflow as tf

v1 = tf.Variable(tf.random_normal([1,2]), name="v1")
v2 = tf.Variable(tf.random_normal([2,3]), name="v2")
init_op = tf.global_variables_initializer()
saver = tf.train.Saver()
with tf.Session() as sess:
    sess.run(init_op)
    print ("V1:",sess.run(v1))  
    print ("V2:",sess.run(v2))
    saver_path = saver.save(sess, "save/model.ckpt")
    print ("Model saved in file: ", saver_path) 

V1: [[-0.61912751  0.10767912]]
V2: [[ 0.10039134 -1.51745009 -0.61548245]
 [ 0.6146487   0.66980863 -1.00977123]]
Model saved in file:  save/model.ckpt

import tensorflow as tf
v1 = tf.Variable(tf.random_normal([1,2]), name="v1")
v2 = tf.Variable(tf.random_normal([2,3]), name="v2")
saver = tf.train.Saver()

with tf.Session() as sess:
    saver.restore(sess, "save/model.ckpt")
    print ("V1:",sess.run(v1))  
    print ("V2:",sess.run(v2))
    print ("Model restored")

V1: [[-0.61912751  0.10767912]]
V2: [[ 0.10039134 -1.51745009 -0.61548245]
 [ 0.6146487   0.66980863 -1.00977123]]
Model restored

总结

基本的神经网络案例,在于真正的入门神经网络的构建。

版权声明:本套技术专栏是作者(秦凯新)平时工作的总结和升华,通过从真实商业环境抽取案例进行总结和分享,并给出商业应用的调优建议和集群环境容量规划等内容,请持续关注本套博客。QQ邮箱地址:1120746959@qq.com,如有任何学术交流,可随时联 秦凯新 于深圳 2018120892153