[SOLVED] MachineLearning Homework 2-Phoneme Classification

Task Introduction

Task: MulticlassClassification

Framewise phoneme prediction from speech.

What is a phoneme?

Machine

M M M AH AH SH SH IH IH IH

N N N

N …

A unit of speech sound in a language that can serve to distinguish one word from the other.

• ●  bat / pat , bad / bed
• ●  Machine Learning → M AH SH IH N L ER N IH NG

Task Introduction

Data Preprocessing

a vector

39-dim MFCC

frame 80-dim filter bank …

Task Introduction

Acoustic Features – MFCCs (Mel Frequency Cepstral Coefficients)

vector

For more details,
please refer to Prof. Lin-Shan Lee’s [Introduction to Digital Speech Processing] Chap.7

shape (11,39)

prev frames flatten

label

More Information About the Data

Since each frame only contains 25 ms of speech, a single frame is unlikely to represent a complete phoneme

● Usually, a phoneme will span several frames ○ Hint: post-processing may help

• ●  Concatenate the neighboring phonemes for training
  • ○  In this HW, we concatenate the past and the future five

    frames for training (total 11 frames)

  • ○  You may reshape the input (1,429) back to (11,39) to get

    separated 11 frames

  • ○  Just remember that the label corresponds to the center

    frame

• ●  Finding testing labels or doing human labeling are strictly prohibited!

  Introduction to Digital Speech Processing

future frames

reshape to (11,39)

shape (1,429) flatten 11 frames 11 * 39 = 429 dim

…

Dataset & Data Format

● Dataset: TIMIT Acoustic-Phonetic Continuous Speech Corpus ○ Phonetically balanced for English

● Data Format (The TAs have already preprocessed the data)

timit_11/

• –  train_11.npy → training data (# of training frames, 11 x feature dim)
• –  train_label_11.npy → framewise phoneme label (0-38)
• –  test_11.npy → testing data (# of testing frames, 11 x feature dim)

● Acoustic features (39-dim MFCC)

• ○  Concatenate the past and the future five frames (feature dim = 11 x 39)
• ○  The phoneme label of each input corresponds to the center frame

● Using additional data is prohibited. Your final grade will be multiplied by 0.9!

Sample Code

Colab Link:

https://colab.research.google.com/github/ga642381/ML2021-Spring/blob/main/HW 02/HW02-1.ipynb

● Simple baseline
○ You should able to pass the simple baseline using the sample code provided.

● Strong baseline

• ○  Model architecture (layers? dimension? activation function?)
• ○  Training (batch size? optimizer? learning rate? epoch?)
• ○  Tips (batch norm? dropout? regularization?)

Grading (8pt/10pt)

• ●  (4pt) Submit code to NTU COOL
• ●  (1pt) Public simple baseline
• ●  (1pt) Public strong baseline
• ●  (1pt) Private simple baseline
• ●  (1pt) Private strong baseline

Grading — Kaggle

Bonus (0.5pt)

• ●  If you get full marks in this part, we will make your code public to the class.
• ●  In this case, if you also submit a PDF report briefly describing your methods (<100 words in English), you get a bonus of 0.5 pt.
  (your report will also be available to all students)
• ●  Report template

Kaggle Submission

Kaggle Link: https://www.kaggle.com/c/ml2021spring-hw2

• ●  Displayed name: <student ID>_<anything>
  • ○  e.g. b06901020_puipui
  • ○  For auditing, don’t put your student ID in your displayed name.
• ●  Submission format: .csv file
• ●  Evaluation metric: accuracy
• ●  Submission deadline:

○ 2021/04/02 23:59 (UTC+8)

Kaggle Submission

• ●  You may submit up to 5 results each day (UTC).
• ●  Up to 2 submissions will be considered for the private leaderboard.

remember to select 2 results for your final scores before the competition ends!

Code Submission

• ●  Compress your code and report, then submit it to NTU COOL. <student ID>_hw2.zip

                            e.g. b06901999_hw2.zip
  

• ●  We can only see your last submission.
• ●  Do not submit your model or dataset.
• ●  If your code is not reasonable, your final grade will be multiplied by 0.9!
• ●  Submission deadline:

○ 2021/04/04 23:59 (UTC+8)

Code Submission

● Your .zip file should include only

• ○  Code: either .py or .ipynb
• ○  Report: .pdf (only for those who got 8 points in part one)

  ● Example:

Task Introduction

Task: Hessian Matrix

Imagine we are training a neural network, and we try to find out whether the model reaches a local minima-like point, saddle point, or none of the above. We can make our decision by calculating the Hessian matrix.

What is Hessian?

Hessian is the second order partial derivatives of a model. It is highly recommended to watch the lecture video before starting this part.

Task Introduction

The target function in this task is a one-variable sinc function.

You will get

• ●  a model checkpoint trained by TA,
• ●  a batch of training data,
• ●  a loss function.

You will calculate the Hessian matrix and make the decision accordingly.

Gradient Norm / Minimum Ratio

1. Gradient Norm

In a normal training process, we rarely have gradients equal to zero. In this homework, we regard those gradient norm less than 1e-3 as zero.

2. Minimum Ratio

For an ideal local minima, all the eigenvalues of the hessian matrix are greater than zero. We define the proportion of positive eigenvalues as minimum ratio.

In this homework, if minimum ratio is greater than 0.5 and gradient norm is less than 1e-3, then we assume that the model is at “local minima like”.

Gradient Norm / Minimal Ratio

In this homework, we assume that

• ●  gradient norm < 1e-3 and minimum ratio > 0.5 => local minima like,
• ●  gradient norm < 1e-3 and minimum ratio <= 0.5 => saddle point,
• ●  gradient norm >= 1e-3 => none of the above.

Important Notice

• ●  You don’t need to and shouldn’t change any part of the code.
• ●  You can only use colab to run the code. Otherwise, your result might

  differ due to environmental issue.

• ●  You will get a different checkpoint according to your student ID, so please

  make sure to fill in your student ID in the sample code correctly.

Sample Code

Colab Link:

https://colab.research.google.com/github/ga642381/ML2021-Spring/blob/main/HW 02/HW02-2.ipynb

• ●  After executing the sample code, you should get a result like this.
• ●  Notice that each student will get a different answer, so your answer may

  differ from the example.

• ●  Choose your answer from local minima like, saddle point, or none of the above.

Grading (2pt/10pt)

● (2pt) Correct answer.

NTU COOL Submission

• ●  After you choose your answer, submit it to NTU COOL.
• ●  You can change your answer multiple times before the deadline.
• ●  Submission deadline:

○ 2021/04/02 23:59 (UTC+8)

Deadlines

● 2-1

• ○  Kaggle: 2021/04/02 23:59 (UTC+8)
• ○  NTU COOL: 2021/04/04 23:59 (UTC+8)

  ● 2-2

• ○  NTU COOL: 2021/04/02 23:59 (UTC+8)