[SOLVED] NCTU_ML Assignment #3 - Linear Regression & Logistic Regression

This programming assignment aims to help you understand Linear Regression and Logistic Regression.

Before we start

You may choose to go to the PC classrooms or finish this HW elsewhere.
It won’t affect a thing. For fairness’ sake, we’ll use discord as the Q&A system.

Join the discord server for TA support

• This is the same one as the program assignment #1 and #2 used.
• Ask questions on it, and we shall reply. (We won’t respond to raised hands.)
• Try not to ask for obvious answers or bug fixes.
• Memes and chit chat welcome

Objective

• Linear Regression – 55% + (10%)
  1. Data Generation – 15%
     • Randomly generate 1000 (xi,yi) pairs which follow the equation (1)
       yi=3x3i+2x2i−3xi+1+ϵi(1)

       where −1.5<xi<1.0, ϵi∼N(0,0.5) and N represents Normal distribution
       

  2. Data Preprocessing – 10%
     • Generate degree-K polynomial features x^ from x
       x^i=⎡⎣⎢⎢⎢⎢⎢⎢1xix2i…xKi⎤⎦⎥⎥⎥⎥⎥⎥
     • You must experiments 4 different K settings, K=1,2,3,4
     • hint
  3. Model Construction – 20%
     • Linear Regression
       • Which makes predictions y^=wx^, s.t.
         w=argminw′||y−w′x^||2
     • You must construct Linear Regression models to fit and predict data generated by (1)
  4. Validation – 0%
     • Due to the simplicity of Linear Regression, you are not required to implement validation methods.
  5. Results – 10% + (10%)
     • Show the fitted weights and the equations
     • Show the predicted y^ for −1.5<x<1.0
     • Bonus – show the results in a single figure – (10%)
       
       • Legend equations must be written in LaTeX
       • Use × instead of ∗ to represent multiplication operations
       • Use xi instead of x
       • Limit the floating-point numeric weights to be 2 decimal places
         • i.e. no 1.54323423456 but 1.54
       • There should be no redundant signs before weights, i.e no 1+−3.36×xi
• Logistic Regression – 45% + (10%)
  1. Data Generation – 15%
     • Randomly generate 1000 (xi0,xi1,yi) triplets which follows (2)
       [xi0xi1]∼N([yiyi],[0.1000.1])(2)

       where yi is randomly assigned as 0 or 1.
       

  2. Model Construction – 20%
     • Logistic Regression
       • Whose divider Mw uses Logistic function L to perform classification
         Mw(xi)=L(w⋅x)=11+e−w⋅x
       • Takes L2-norm as the objective function to optimize weight w
         w=argminw′||y−Mw′(x)||2
     • Construct a Logistic Regression model to predict yi from [xi0xi1]T generated from equation (2)
  3. Validation – 0%
     • Validation methods are not required in this assignment either.
  4. Results – 10% + (10%)
     • Show the model accuracy – 5%
     • Show the model weights and the corresponded terms – 5%
       • e.g.
         yi=L(4.2+7.7×xi0+6.9×xi1)
     • Bonus – show the decision boundary with a figure – (10%)
       
• Finish during class – 20%
  • Submit your report and source codes to the newE3 system before class ends.
  • Finish time will be determined by the submission time.