[SOLVED] NYCU Homework 4

Description :
1. Logistic regression

Input:

1. (number of data points)

2. Function:

1. Generate data point: independently sampled from

2. Generate data point: independently sampled from

(

: mean, : variance)
, where and are

and

and

respectively.

, where and are respectively.

3. Use Logistic regression to separate and
and steepest gradient descent method during optimization.

In other words, when the Hessian is singular, use steepest descent for instead. You should come up with a reasonable rule to determine convergence.(a simple run out of the loop should be used as the ultimatum)

Output:

1. The confusion matrix and the sensitivity and specificity of the logistic regression applied to the training data .

2. Visualization
Plot the ground truth

Plot the predict result

Gradient descent Newton’s method

Use the Gaussian random number generator in homework 3.

Sample input & output (for reference only) Case 1:

. You should implement both Newton’s

Case 2:

2. EM algorithm

Input: MNIST training data and label sets. (Same as HW02) Function:

1. Binningthegraylevelvalueintotwobins.Treatingallpixelsasrandomvariables following Bernoulli distributions. Note that each pixel follows a different Binomial distribution independent to others.
2. Use EM algorithm to cluster each image into ten groups. You should come up with a reasonable rule to determine convergence. (a simple run out of the loop should be used as the ultimatum)

Output:

1. For each digit, output a confusion matrix and the sensitivity and specificity of the clustering applied to the training data.
2. Print out the imagination of numbers in your classifier

   Just like before, about the details please refer to HW02

Hint: The algorithm is a kind of unsupervised learning, so the labels are not used during training. But you can use these labels to help you to figure out which class belongs to which number.

In other words, you should find a way to assign label to each class which you classified before you compute the confusion matrix

Sample input & output (for reference only)

4. 4  0000000000000000000000000000
5. 5  0000000000000000000000000000
6. 6  0000000000000000000000000000
7. 7  0000000000000000000000000000
8. 8  0000000000000000000000000000
9. 9  0000000000000000000000000000
10. 10  0000000000000000000000000000
11. 11  0000000000000000000000000000
12. 12  0000000000000000000000000000
13. 13  0000000000000000000000000000
14. 14  0000000000000000100000000000
15. 15  0000000000000001110000000000
16. 16  0000000000000011110000000000
17. 17  0000000000000011100000000000
18. 18  0000000000000011000000000000
19. 19  0000000000000001000000000000
20. 20  0000000000000000000000000000
21. 21  0000000000000010000000000000
22. 22  0000000000000110000000000000
23. 23  0000000000000110000000000000
24. 24  0000000000000000000000000000
25. 25  0000000000000000000000000000
26. 26  0000000000000000000000000000
27. 27  0000000000000000000000000000
28. 28  0000000000000000000000000000
29. 29  0000000000000000000000000000

30

31. 31  class1:
32. 32  0000000000000000000000000000
33. 33  0000000000000000000000000000
34. 34  0000000000000000000000000000
35. 35  0000000000000000000000000000
36. 36  0000000000000000000000000000
37. 37  0000000000000000000000000000
38. 38  0000000000000000000000000000
39. 39  0000000000001100000000000000
40. 40  0000000000000000111000000000
41. 41  0000000000000000111000000000
42. 42  0000000000000000111000000000
43. 43  0000000000000000011000000000
44. 44  0000000000000000011000000000
45. 45  0000000000000000111000000000
46. 46  0000000000000000111000000000
47. 47  0000000000000001110000000000
48. 48  0000000000000111111000000000
49. 49  0000000000001111111000000000
50. 50  0000000000001111111000000000
51. 51  0000000000011100011000000000
52. 52  0000000000000000000000000000

53. 53  0000000000000000000000000000
54. 54  0000000000000000000000000000
55. 55  0000000000000000000000000000
56. 56  0000000000000000000000000000
57. 57  0000000000000000000000000000
58. 58  0000000000000000000000000000
59. 59  0000000000000000000000000000

60

61 … all other unlabeled imagination of numbers goes here … 62

63. 63  class9:
64. 64  0000000000000000000000000000
65. 65  0000000000000000000000000000
66. 66  0000000000000000000000000000
67. 67  0000000000000000000000000000
68. 68  0000000000000011110000000000
69. 69  0000000000001111111000000000
70. 70  0000000000011111111100000000
71. 71  0000000000000000011100000000
72. 72  0000000000000000001000000000
73. 73  0000000000000000001000000000
74. 74  0000000000000011111000000000
75. 75  0000000000000111111100000000
76. 76  0000000000011111111100000000
77. 77  0000000000000000001100000000
78. 78  0000000000000000001100000000
79. 79  0000000001000000000100000000
80. 80  0000000010000000000100000000
81. 81  0000000010000000001100000000
82. 82  0000000110000000011100000000
83. 83  0000000110000000111000000000
84. 84  0000000111000100110000000000
85. 85  0000000011111011000000000000
86. 86  0000000001111110000000000000
87. 87  0000000000000000000000000000
88. 88  0000000000000000000000000000
89. 89  0000000000000000000000000000
90. 90  0000000000000000000000000000
91. 91  0000000000000000000000000000

92

93 No. of Iteration: 1, Difference: 3176.579389514846 94
95
96

97. 97  class 0:
98. 98  0000000000000000000000000000
99. 99  0000000000000000000000000000
100. 100  0000000000000000000000000000
101. 101  0000000000000000000000000000

102. 102  0000000000000000000000000000
103. 103  0000000000000000000000000000
104. 104  0000000000000000000000000000
105. 105  0000000000000000000000000000
106. 106  0000000000000000000000000000
107. 107  0000000000000000000000000000
108. 108  0000000000000000000000000000
109. 109  0000000000000001100000000000
110. 110  0000000000000011100000000000
111. 111  0000000000000011100000000000
112. 112  0000000000000111000000000000
113. 113  0000000000000111000000000000
114. 114  0000000000000111000000000000
115. 115  0000000000000111000000000000
116. 116  0000000000000010000000000000
117. 117  0000000000000010000000000000
118. 118  0000000000000110000000000000
119. 119  0000000000000010000000000000
120. 120  0000000000000000000000000000
121. 121  0000000000000000000000000000
122. 122  0000000000000000000000000000
123. 123  0000000000000000000000000000
124. 124  0000000000000000000000000000
125. 125  0000000000000000000000000000

126

127 … all other iterations goes here … 128

129. 129  class 9:
130. 130  0000000000000000000000000000
131. 131  0000000000000000000000000000
132. 132  0000000000000000000000000000
133. 133  0000000000000000000000000000
134. 134  0000000000000001110000000000
135. 135  0000000000000011110000000000
136. 136  0000000000000011110000000000
137. 137  0000000000000111000000000000
138. 138  0000000000000110000000000000
139. 139  0000000000001100000000000000
140. 140  0000000000001100000000000000
141. 141  0000000000011100000000000000
142. 142  0000000000011100000000000000
143. 143  0000000000011000100000000000
144. 144  0000000000111001111000000000
145. 145  0000000000110000111000000000
146. 146  0000000000110000011000000000
147. 147  0000000000110000011000000000
148. 148  0000000000110000111000000000
149. 149  0000000000110001111000000000
150. 150  0000000000111111110000000000

151. 151  0000000000111111100000000000
152. 152  0000000000011110000000000000
153. 153  0000000000000000000000000000
154. 154  0000000000000000000000000000
155. 155  0000000000000000000000000000
156. 156  0000000000000000000000000000
157. 157  0000000000000000000000000000

158

159 No.ofIteration:10,Difference:19.89546432548733 160
161
162

163

164. 164  labeled class 0:
165. 165  0000000000000000000000000000
166. 166  0000000000000000000000000000
167. 167  0000000000000000000000000000
168. 168  0000000000000000000000000000
169. 169  0000000000000000000000000000
170. 170  0000000000000111110000000000
171. 171  0000000000011111111000000000
172. 172  0000000000111111111110000000
173. 173  0000000001111111111110000000
174. 174  0000000011111000000111000000
175. 175  0000000011110000000011100000
176. 176  0000000111100000000011100000
177. 177  0000000111000000000001100000
178. 178  0000001111000000000001110000
179. 179  0000001110000000000001110000
180. 180  0000001110000000000001110000
181. 181  0000001110000000000011100000
182. 182  0000001110000000000011100000
183. 183  0000001110000000000111000000
184. 184  0000001111000000001111000000
185. 185  0000000111100000111110000000
186. 186  0000000011111111111100000000
187. 187  0000000001111111110000000000
188. 188  0000000000011111000000000000
189. 189  0000000000000000000000000000
190. 190  0000000000000000000000000000
191. 191  0000000000000000000000000000
192. 192  0000000000000000000000000000

193

194. 194  labeled class 1:
195. 195  0000000000000000000000000000
196. 196  0000000000000000000000000000
197. 197  0000000000000000000000000000
198. 198  0000000000000000000000000000
199. 199  0000000000000000000000000000