[SOLVED] Machine-Learning Homework 2

1. Naive Bayes classifier

Create a Naive Bayes classifier for each handwritten digit that support discrete and continuous
features.

Input:
1. TrainingimagedatafromMNIST

You Must download the MNIST from this website and parse the data by yourself. (Please do not use the build in dataset or you’ll not get100.)
Please read the description in the link to understand the format.
Basically, each image is represented by bits (Whole binary file is in big endian format; you need to deal with it), you can use char arrary to store an

a image.
There are some headers you need to deal with as well, please read the link for more details.

2. TraininglabledatafromMNIST. 3. TestingimagefromMNIST
4. TestinglabelfromMNIST
5. Toggleoption

0: discrete mode
1: continuous mode

TRAINING SET IMAGE FILE(train-images-idx3-ubyte)

0000 32 bit integer 0x00000803(2051) 0008 32 bit integer 28
0016 unsigned byte ??
… … …

magic number number of rows pixel
…

TRAINING SET LABEL FILE(train-labels-idx1-ubyte)

0000 32 bit integer 0x00000801(2049) 0008 unsigned byte ??
… … …

The labels values are from 0 to 9. Output:

magic number label
…

Print out the the posterior (in log scale to avoid underflow) of the ten categories (0-9) for each image in INPUT 3. Don’t forget to marginalize them so sum it up will equal to 1.

For each test image, print out your prediction which is the category having the highest posterior, and tally the prediction by comparing with INPUT 4.

Print out the imagination of numbers in your Bayes classifier

For each digit, print a binary image which 0 represents a white pixel, and 1 represents a black pixel.
The pixel is 0 when Bayes classifier expect the pixel in this position should less then 128 in original image, otherwise is 1.

Calculate and report the error rate in the end. Function:

1. InDiscretemode:

Tally the frequency of the values of each pixel into 32 bins. For example, The gray level 0 to 7 should be classified to bin 0, gray level 8 to 15 should be bin 1 … etc. Then perform Naive Bayes classifier. Note that to avoid empty bin, you can use a peudocount (such as the minimum value in other bins) for instead.

2. InContinuousmode:

Use MLE to fit a Gaussian distribution for the value of each pixel. Perform Naive

Bayes classifier.
Sample input & output (for reference only)

2. Online learning

Use online learning to learn the beta distribution of the parameter p (chance to see 1) of the coin tossing trails in batch.

Input:
1. Afilecontainsmanylinesofbinaryoutcomes:

2. parameterafortheinitialbetaprior

3. parameterbfortheinitialbetaprior
Output: Print out the Binomial likelihood (based on MLE, of course), Beta prior and posterior probability (parameters only) for each line.

Function: Use Beta-Binomial conjugation to perform online learning. Sample input & output (for reference only)

Input: A file (here shows the content of the file)

Output
Case 1: a = 0, b = 0

5 6 7 8 9

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

Case 2: a = 10, b = 1

3. Prove Beta-Binomial conjugation
Try to proof Beta-Binomial conjugation and write the process on paper.

※ You should write down the proof process on paper and take a picture. When you hand in HW02, it must contain your code and picture.

l NOTE:

• ¡  Use whatever programming language you prefer.
• ¡  You can’t use numpy.random.beta in HW02. That would be great if you

  implement all distribution by yourself.

• ¡  HW02 must contain your code and proof process (can be .pdf or any image

  format).