CPSC 585 – Artificial Neural Networks Solved

Description

Platforms

For this project (and, in general, for most machine learning or data science projects) you will need a Jupyter notebook with Python 3. Jupyter allows you to create documents mixing text, equations, code, and visualizations.

This project does not require a GPU to train efficiently, so you may wish to run Jupyter locally. The Jupyter project itself recommends Anaconda if you intend to run notebooks locally on a laptop or desktop computer.

Alternatively you may use the Google Colab cloud service, which is what the instructor will use to evaluate your project.

Libraries and Code

This project must be implemented in pure Python with the Python Standard Library, without recourse to third-party libraries.

Code from A Whirlwind Tour of Python may be reused. All other code and the results of experiments should be your own original work or the original work of other members of your team.

Dataset

The module dataset.py contains capital letters A through Z in three different dot matrix fonts. Each font is 5×7 (7 rows of 5 columns each). Letters are represented as 2-tuples of (image, char), where image is a letter’s image represented as a one-dimensional bitmap of 0’s and 1’s and char is the character corresponding to the letter shown in the image.

The dataset module exports three values:

• TRAINING_SET – contains example tuples of letter images from two different dot matrix fonts
• TEST_SET – contains tuples of letter images from a third dot matrix font
• MESSAGE – contains a list of letter images without corresponding character targets, to be recognized by your trained perceptrons. These letter images are drawn from a mix of the three fonts, with additional noise such as might be produced by an optical scanner.

Experiments

Run the following experiments in a Jupyter notebook, performing each action in a code cell and answering each question in a Markdown cell.

1. Use from dataset import * to load the module, then examine TRAINING_SET, TEST_SET, and MESSAGE.
2. Implement a Python function show(image) to print() a letter image row-by-row using a hash mark (‘#’) for 1 and a space (‘ ‘) for 0. Test this function on some of the images in TRAINING_SET.
3. Implement a perceptron to recognize the letter A by producing output +1 for images of the letter A and -1 for images of all other letters.

Begin with weights initialized to random values and a learning rate of 𝛼 = 0.01. Train the perceptron by applying the update rule of Equation 1.4 in the textbook. Continue training until all items in TRAINING_SET are correctly classified.

1. Use your trained perceptron to classify each image in. Does the perceptron correctly distinguish the letter A from the other letters?
2. Repeat experiment (3) for the rest of the letters, for a total of 26 trained perceptrons. If any perceptron fails to converge, implement the “pocket algorithm” as described at the end of Section 1.2.1.1 of the textbook on p. 8.
3. Repeat experiment (4) for each trained perceptron. What is the accuracy on TEST_SET?
4. Now run each trained perceptron against each image in MESSAGE. Does any image cause more than one perceptron to return +1? Does any image fail to return +1 for at least one one perceptron? If so, run your show() function on these images, and describe what you think might have gone wrong.
5. Use your trained perceptrons to identify the letters in MESSAGE. What does it say in English? (Note that there are no spaces between words.) Why do you suppose this was chosen as the message?