Description
Question 1: find, read and summarize a paper
Find a paper (related to Perception/Vision) that you are interested in from:
https://openaccess.thecvf.com/CVPR2021?day=all https://openaccess.thecvf.com/ICCV2021?day=all https://paperswithcode.com/
Requirement: up to 1 page with below info:
(You should never literately copy any sentences from any digital sources for your report)
– The link of this paper.
– In the structure of this paper, where can you find below info?
* Background
* Motivations
* Short Summary of the proposed work * Contribution highlights
* Problem formulation
* Conclusion
– What problem it is resolving?
– What are the input and output of the proposed method?
– The challenges of this problem?
– What are the purposes for the first and second figures?
– What new concept its method introduced?
– What counterparts it compared to and links of typical counterparts? – What aspect are included in the Experiment sections?
– What results it concludes?
Option 1: Python
Question 2: train an instance segmentation model with PyTorch tutorial and Google Colab; or you can deploy it in Palmetto Cluster.
(1) Show screenshots of successful setup, training, and inference on the colab. (15 points) (2) Inference different images in the test set and show screenshots
(3) Inference on your own image
(4) Plot the segmentation metric AP@[IoU=0.50] against the number of training Epochs on a Graph.
(TIPS: if you cannot save accuracy during training, you can manually collect it from outputs.)
(5) Change the batch size, optimizer, learning rate etc… Plot and analyze its influence on accuracy.
(6) Could you improve the network model, train it for better accuracy? (Optional, 5 points) (This question is optional. Extra 5 points until reach the cap of 100)
Option 2: Matlab
Document your results and discussion in your homework submission report.
Your Clemson credential has fully license to Matlab, for computing platform: – You can use ‘MATLAB Online’ https://matlab.mathworks.com/
– Or you use Matlab in your local computer.
Reference:
Deep Learning Toolbox
https://www.mathworks.com/help/deeplearning/index.html
Deep Learning Onramp
https://matlabacademy.mathworks.com/details/deep-learning-onramp/deeplearning
Question 2: ‘on pretrained models’
(1) Select a few (like three) of Pretrained models; Explain what is the input/output, and which paper (if any) it is introduced? Use deepNetworkDesigner to visualize the models.
Reference:
Pretrained Deep Neural Networks https://www.mathworks.com/help/deeplearning/ug/pretrained-convolutional-neural- networks.html
(2) For the selected pretrained models; Test it using some of the images that you download from the web;
Reference:
Classify Webcam Images Using Deep Learning
https://www.mathworks.com/help/deeplearning/ug/classify-images-from-webcam-using-deep- learning.html
Show and discuss your accuracy: like you choose same type of objects with different background, and see their accuracy difference, etc.
Question 3: ‘further training a pretrained models’ (40 pts)
(1) Apply transfer learning to retrain a model to classify a new set of images.
Demo this example, reference:
Train Deep Learning Network to Classify New Images https://www.mathworks.com/help/deeplearning/ug/train-deep-learning-network-to-classify-new- images.html
Explain its input and output; Explain what are ‘Batch’, ‘Epoch’, ‘Iteration’? Try a few different ‘miniBatchSize’, and discuss its accuracy difference.
(2) For the demo of a selected parameters, select a partial of the testing/validation dataset statistically summarize its results for:
Explain what are: True positive (TP) True negative (TN) False positive (FP) False negative (FN)
Provide its result of TP, TN, FP, FN and Accuracy, Precision, Recall, F1-score
