1 Caracter recognition using a trained MLPh
The purpose of this problem is to have you program, using numpy, the feedforward processing for a multilayer
perceptron (MLP) deep neural network. This processing is dened on slide 33 of the \introduction” lecture
slides. The grading of this homework will give you feedback on your FF program which is valuable since in
a future homework, you will be building on this to program the back-propagation as well.
The MNIST dataset of handwritten digits is widely used as a beginner dataset for benchmarking machine
learning classiers. It has 784 input features (pixel values in each image) and 10 output classes representing
numbers 0{9. We have trained a MLP on MNIST, with a 784-neuron input layer, 2 hidden layers of 200 and
100 neurons, and a 10-neuron output layer. The activation functions used are ReLU for the hidden layers
and softmax for the output layer.
(a) Extract the weights and biases of the trained network from mnist_network_params.hdf5. The le
has 6 keys corresponding to numpy arrays W1, b1, W2, b2, W3, b3. You may want to check their
dimensions by using the `shape’ attribute of a numpy array.
(b) The le mnist_testdata.hdf5 contains 10,000 images in the key `xdata’ and their corresponding labels
in the key `ydata’. Extract these. Note that each image is 784-dimensional and each label is one-hot
10-dimensional. So if the label for an image is [0; 0; 0; 1; 0; 0; 0; 0; 0; 0], it means the image depicts a 3.
(c) Write functions for calculating ReLU and softmax. These are given as:
ReLU(x) = max (0; x)
Softmax(x) =
ex1
Pn
i=1 exi
;
ex2
Pn
i=1 exi
; ;
exn
Pn
i=1 exn
. Note that the softmax function oper-
ates on a vector of size n and returns another vector of size n which is a probability distribution.
For example, Softmax ([0; 1; 2]) = [0:09; 0:24; 0:67]. This indicates that the 3rd element is the most
likely outcome. For the MNIST case, n = 10.
(d) Using numpy, program a MLP that takes a 784-dimensional input image and calculates its 10-dimensional
output. Use ReLU activation for the 2 hidden layers and softmax for the output layer.
(e) Compare the output with the true label from `ydata’. The input is correctly classied if the position
of the maximum element in the MLP output matches with the position of the 1 in ydata. Find the
total number of correctly classied images in the whole set of 10,000. You should get 9790 correct.
(f) Sample some cases with correct classication and some with incorrect classication. Visually inspect
them. For those that are incorrect, is the correct class obvious to you? You can use matplotlib for
visualization:
1
c K.M. Chugg, B. Franzke – February 8, 2020{ EE599 Deep Learning 2
hn <latexit sha1_base64=”(null)”>(null)</latexit><latexit>
xn <latexit sha1_base64=”(null)”>(null)</latexit><latexit> yn <latexit sha1_base64=”(null)”>(null)</latexit><latexit>
qn <latexit sha1_base64=”(null)”>(null)</latexit><latexit>
zn <latexit sha1_base64=”(null)”>(null)</latexit><latexit>
xn <latexit sha1_base64=”(null)”>(null)</latexit><latexit>
ˆwn <latexit sha1_base64=”(null)”>(null)</latexit><latexit>
ˆyn <latexit sha1_base64=”(null)”>(null)</latexit><latexit>
–
ˆwn+1 = ˆwn + ⌘(yn − ˆwtn
xn)xn
ˆyn =
LX
