zip

[SOLVED] DLP Lab1-Back-propagation

30.00 $

Category:
Click Category Button to View Your Next Assignment | Homework

You will receive the following solution file(s) instantly after successful payment:

zip file icon lab1-gujofn.zip (8508.2 KB)
Assignment Instructions Updated Recently? Submit Below and we will provide new Solution!
Submit New Instructions
πŸ”’ Securely Powered by:
Secure Checkout
5/5 - (1 vote)

Β Lab Objective:

In this lab, you will need to understand and implement simple neural networks with forwarding pass and backpropagation using two hidden layers. Notice that you can only use Numpy and the python standard libraries, any other frameworks (ex : Tensorflow、PyTorch) are not allowed in this lab.

Figure 1. Two-layer neural network Important Date:

  1. Experiment Report Submission Deadline: 3/30 (Tue) 12:00 p.m.
  2. Demo date: 3/30 (Thu)

Turn in: Β 

  1. Experiment Report (.pdf)
  2. Source code

Notice:Β  zip all files in one file and name it likeγ€ŒDLP_LAB1_your studentID_name.zip」, ex: γ€ŒDLP_LAB1_309551009_ι™³η’½ε­˜.zip」

Β 

Requirements: Β 

  1. Implement simple neural networks with two hidden layers.
  2. You must use backpropagation in this neural network and can only use Numpy and other python standard libraries to implement.
  3. Plot your comparison figure that show the predicted results and the groundtruth.

 

Implementation Details:Β 

Β 

Figure 2. Forward pass

  • In the figure 2, we use the following definitions for the notations:
    1. π‘₯1, π‘₯2 ∢ π‘›π‘’π‘Ÿπ‘’π‘Žπ‘™ π‘›π‘’π‘‘π‘€π‘œπ‘Ÿπ‘˜ 𝑖𝑛𝑝𝑒𝑑𝑠
    2. 𝑋 ∢ [π‘₯1, π‘₯2]
    3. 𝑦 ∢ π‘›π‘’π‘Ÿπ‘’π‘Žπ‘™ π‘›π‘’π‘‘π‘€π‘œπ‘Ÿπ‘˜ π‘œπ‘’π‘‘π‘π‘’π‘‘π‘ 
    4. 𝑦̂ ∢ π‘”π‘Ÿπ‘œπ‘’π‘›π‘‘ π‘‘π‘Ÿπ‘’π‘‘β„Ž
    5. L(ΞΈ) ∢ π‘™π‘œπ‘ π‘  π‘“π‘’π‘›π‘π‘‘π‘–π‘œπ‘›
    6. π‘Š1, π‘Š2, π‘Š3 ∢ π‘€π‘’π‘–π‘”β„Žπ‘‘ π‘šπ‘Žπ‘‘π‘Ÿπ‘–π‘₯ π‘œπ‘“ π‘›π‘’π‘‘π‘€π‘œπ‘Ÿπ‘˜ π‘™π‘Žπ‘¦π‘’π‘Ÿπ‘ 
  • Here are the computations represented:

π’πŸ = 𝝈(π‘Ώπ‘Š1)Β Β Β Β Β Β Β Β Β Β Β Β Β Β  π’πŸ = 𝝈(π’πŸπ‘Š2)Β Β Β Β Β Β Β Β Β Β Β Β  π’š = 𝝈(π’πŸπ‘Š3)Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β  Β 

 

  • In the equations, the 𝝈 is sigmoid function that refers to the special case of the logistic function and defined by the formula:
  • Input / Test:

The inputs are two kinds which showing at below. Β 

You need to use the following generate functions to create your inputs x, y.

In the training, you need to print the loss values; In the testing, you need to show your predictions as shown below.

Visualize the predictions and ground truth at the end of the training process. The comparison figure should like example as below.

You can refer to the following visualization code x: inputs (2-dimensional array) y: ground truth label (1-dimensional array)

pred_y: outputs of neural network (1-dimensional array)

 

 

  • Sigmoid functions:
    1. A sigmoid function is a mathematical function having a characteristic “S”-shaped curve or sigmoid curve. It is a bounded, differentiable, real function that is defined for all real input values and has a non-negative derivative at each point. In general, a sigmoid function is monotonic, and has a first derivative which is bell shaped.
    2. (hint) You may write the function like this:
  1. (hint) The derivative of sigmoid function
  • Back Propagation (Gradient computation)

Backpropagation is a method used in artificial neural networks to calculate a gradient that is needed in the calculation of the weights to be used in the network. Backpropagation is a generalization of the delta rule to multilayered feedforward networks, made possible by using the chain rule to iteratively compute gradients for each layer. The backpropagation learning algorithm can be divided into two parts; propagation and weight update.

Part 1: Propagation

Each propagation involves the following steps:

  1. Propagation forward through the network to generate the output value
  2. Calculation of the cost L(ΞΈ) (error term)
  3. Propagation of the output activations back through the network using the training pattern target in order to generate the deltas (the difference between the targeted and actual output values) of all output and hidden neurons.

Part 2: Weight update

For each weight-synapse follow the below steps:

  1. Multiply its output delta and input activation to get the gradient of the weight.
  2. Subtract a ratio (percentage) of the gradient from the weight.
  3. This ratio (percentage) influences the speed and quality of learning; it is called the learning rate. The greater the ratio, the faster the neuron trains; the lower the ratio, the more accurate the training is. The sign of the gradient of a weight indicates where the error is increasing, this is why the weight must be updated in the opposite direction.

Repeat part. 1 and 2 until the performance of the network is satisfactory.

Β 

Pseudocode:

 

  • lab1-gujofn.zip