Description
i. Download and install TensorFlow from https://www.tensorflow.org/install/install_sources or using command sudo pip install tensorflow alternatively the Keras library can be used.
ii. Download MNIST dataset (contains class labels for digits 0-9). using the command:
import tensorflow as tf
data = tf.contrib.learn.datasets.mnist.load_mnist()
or
from keras.datasets import mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
iii. Reduce the training size by 1/10 if computation resources are limited. Define radial basis function (RBF) as
def RBF(x, c, s):
return np.exp(-np.sum((x-c)**2, axis=1)/(2*s**2))
where, x is the actual value, c is centre (assumed as mean) and s is the standard deviation.
Converted 28*28 image into 32*32 using rbf and store the new dataset with the labels. Split the dataset as 80% training and 10% validation and 10% test.
iv. Now run the fully connected network after flattening the data by changing the number the hyper-parameters use adam optimizer(learning rate = 0.001) and categorical cross-entropy loss:
Hidden Layers Activation Function Hidden Neurons
1 Sigmoid [ 16]
2 Sigmoid [16, 32]
3 Sigmoid [16, 32, 64]
Try all the possible combinations.
v. Now run the network by changing the number the Activation Function hyper-parameters:
Hidden Layers Activation Function Hidden Neurons
3 Sigmoid [ 16, 32 , 64]
3 Tanh [ 16, 32 , 64]
3 Relu [ 16, 32 , 64]
vi. Now run the network by changing the number the Dropout hyperparameters:
Hidden Layers Activation Function Hidden Neurons Dropout
3 Relu [ 16, 32 , 64] 0.9
3 Relu [ 16, 32 , 64] 0.75
3 Relu [ 16, 32 , 64] 0.5
3 Relu [ 16, 32 , 64] 0.25
3 Relu [ 16, 32 , 64] 0.10
vii. Plot the graph for loss vs epoch and accuracy(train, validation accuracy) vs epoch for all the above cases. Point out the logic in the report.
viii. With the best set hyperparameter from above run vary the Adam Optimizer learning rate [0.01, 0.001, 0.005, 0.0001, 0.0005]. Print the time to achieve the best validation accuracy (as reported before from all run) for all these five run .
ix. Create five image(size 28*28) containing a digit of your won handwriting and test whether your trained classifier is able to predict it or not.
Submit a report with results.
