[SOLVED] NCTU-CS Assignment #3 -Linear Regression & Logistic Regression

Objective

• Linear Regression – 

• 1. Data Generation – 
     • Randomly generate 1000 (xi,yi)(xi,yi) pairs which follow the equation (1)(1)

yi=3x3i+2x2i−3xi+1+ϵi(1)(1)yi=3xi3+2xi2−3xi+1+ϵi

where −1.5<xi<1.0−1.5<xi<1.0, ϵi∼N(0,0.5)ϵi∼N(0,0.5) and NN represents Normal distribution

• 1. Data Preprocessing – 10%
     • Generate degree-KK polynomial features x^x^ from xx

x^i=⎡⎣⎢⎢⎢⎢⎢⎢1xix2i…xKi⎤⎦⎥⎥⎥⎥⎥⎥x^i=[1xixi2…xiK]

• 1. • You must experiments 44 different KK settings, K=1,2,3,4K=1,2,3,4
     • hint
  2. Model Construction – 20%
     • Linear Regression
       • Which makes predictions y^=wx^y^=wx^, s.t.s.t.

w=argminw′||y−w′x^||2w=argminw′||y−w′x^||2

• 1. • You must construct Linear Regression models to fit and predict data generated by (1)(1)
  2. Validation – 0%
     • Due to the simplicity of Linear Regression, you are not required to implement validation methods.
  3. Results – 10% + (10%)
     • Show the fitted weights and the equations
     • Show the predicted y^y^ for −1.5<x<1.0−1.5<x<1.0
     • Bonus – show the results in a single figure – (10%)
       • Legend equations must be written in LaTeXLaTeX
       • Use ×× instead of ∗∗ to represent multiplication operations
       • Use xixi instead of xx
       • Limit the floating-point numeric weights to be 22 decimal places
         • i.e. no 1.543234234561.54323423456 but 1.541.54
       • There should be no redundant signs before weights, i.e no 1+−3.36×xi1+−3.36×xi

• Logistic Regression – 45% + (10%)

• 1. Data Generation – 15%
     • Randomly generate 1000 (xi0,xi1,yi)(xi0,xi1,yi) triplets which follows (2)(2)

[xi0xi1]∼N([yiyi],[0.1000.1])(2)(2)[xi0xi1]∼N([yiyi],[0.1000.1])

where yiyi is randomly assigned as 00 or 11.

• 1. Model Construction – 20%
     • Logistic Regression
       • Whose divider MwMw uses Logistic function LL to perform classification

Mw(xi)=L(w⋅x)=11+e−w⋅xMw(xi)=L(w⋅x)=11+e−w⋅x

• 1. • • Takes L2-norm as the objective function to optimize weight ww

w=argminw′||y−Mw′(x)||2w=argminw′||y−Mw′(x)||2

• 1. • Construct a Logistic Regression model to predict yiyi from [xi0xi1]T[xi0xi1]T generated from equation (2)(2)
  2. Validation – 0%
     • Validation methods are not required in this assignment either.
  3. Results – 10% + (10%)
     • Show the model accuracy – 5%
     • Show the model weights and the corresponded terms – 5%
       • e.g.

yi=L(4.2+7.7×xi0+6.9×xi1)yi=L(4.2+7.7×xi0+6.9×xi1)

• 1. • Bonus – show the decision boundary with a figure – (10%)

• Finish during class – 20%

• • Submit your report and source codes to the newE3 system before class ends.
  • Finish time will be determined by the submission time.