Description
Objectives
This is a warm up assignment to familiarize you with programming skills that will be useful for later assignments.
Grading
We accept submission of Python 3 (3.7 in particular), Java, or C/C++ programs. We recommend you to use Python 3 for your assignments though, as all sample codes or skeleton programs are provided in Python 3. Programming languages other than Python 3, Java, and C/C++ are not allowed.
In addition, we will test and grade your programs on the sunfire server. Please make sure that your programs run properly on sunfire and not only on your own system. By default, we use the python3 program installed in folder /usr/local/Python-3.7/bin on sunfire for grading. Unless stated otherwise for individual tasks, you are allowed to use libraries installed in public folders of sunfire (e.g. /usr/lib) only.
If you choose to use Java or C/C++, we will compile and run your program for grading using the default compilers on sunfire (java 9.0.4 installed in /usr/local/java/jdk/ bin, or gcc 4.8.4 installed in /usr/local/gcc-4.8/bin). The programming language is inferred from the extension name of your source code during grading. Therefore, please use extension names to indicate the language of your programs. For example, if a task requires submission of Checksum.py, you should provide Checksum.java, Checksum.c, or Checksum.cpp, respectively, depending on your programming language. For a Java program, the class name should be consistent with the source file name, and please implement the static main() method so that the program can be executed as a standalone process after compilation.
We will deduct 1 mark for failure to follow instructions. Please take note of the following common issues:
• Make sure that your programs have correct names.
• Make sure that every line of your output is properly ended with a line break (i.e. “ ” character). In particular, your program should end the output with a line break, unless otherwise stated.
• Do not output irrelevant messages that are not shown in sample runs (for example, debugging messages).
Testing Your Programs
To test your programs, please use your SoC UNIX ID and password to log on to sunfire first:
1. If you don’t have your SoC UNIX account, please create it here: https://mysoc.nus.edu.sg/~newacct
2. If you forget your SoC password, please reset it here using your NUSNET ID and password: https://mysoc.nus.edu.sg/~myacct/resetpass.cgi
ssh <SoC-UNIX-ID>@sunfire.comp.nus.edu.sg
4. To copy files to sunfire on command line, use scp. The usage is as following:
scp -r <source-folder> <SoC-UNIX-ID>@sunfire.comp.nus.edu.sg:<target-folder>
5. After logging on to sunfire, you can run a Python 3 script with the following command:
/usr/local/Python-3.7/bin/python3 <path-to-script> For convenience, you can set a shortcut to python3 as follows: alias python3=/usr/local/Python-3.7/bin/python3
This shortcut is temporary, and it lasts for the current SSH session only. To make it permanent, you can run the following command once to store the shortcut into the configuration file: echo alias python3=/usr/local/Python-3.7/bin/python3 >> ~/.bash_profile
With the shortcut, you can then run a Python 3 script simply with
python3 <path-to-script>
Before running the test scripts, please upload your programs along with the test folder from the package to sunfire. Make sure that your programs and the test folder are in the same directory. Then, you can run bash test/<Exercise Name>.sh for testing. For example, to test your program for Exercise 1, run the following command: bash test/Checksum.sh
By default, the script runs through all test cases. You can also choose to run a certain test case by specifying the case number in the command: bash test/Checksum.sh 3
To stop a test, press and optionally hold Ctrl-c if pressing once does not exit the test.
If you use Java or C/C++, we will compile your program each time you run the testing script. The compiled code is stored in a temporary folder and removed after testing, in order to avoid affecting files under your own folders.
Program Submission
Please create a zip file containing your source files and submit it to the
Assignment_0_student_submission folder of LumiNUS Files. The file name should be <Matric Number>.zip where <Matric Number> is your matriculation number which starts with letter A. An example file name would be A0165432X.zip. All file names, including the zip file and all source files within the zip, are case-sensitive. In addition, your zip file should not contain any folders or subfolders or irrelevant files such as test.jpg, although we always try to find your actual source files during grading. You are encouraged to submit only once, or otherwise only your latest submission will be graded.
You are not allowed to post your solutions to any publicly accessible site on the
Internet.
Question & Answer
If you have any doubts on this assignment, please post your questions on LumiNUS forum or consult the teaching team. We are not supposed to debug programs for you, and we provide support for language-specific questions on a best-effort basis only. The intention of Q&A is to help clarify misconceptions and give you necessary directions.
Exercise 1 – Checksum
Checksum can be used to detect if data is corrupted during network transmission (e.g. a bit flips from 0 to 1). Write a program Checksum.py to calculate the CRC-32 checksum for a file <src> entered as command-line argument. File <src> should be placed in the same folder as Checksum.py.
An example code snippet is given below.
with open(“test.jpg”, “rb”) as f:
bytes = f.read()
checksum = zlib.crc32(bytes)
Sample run:
$ python3 Checksum.py test/test.jpg
3237218320
Exercise 2 – PacketExtr
In this exercise, you are going to write a program, PacketExtr.py, to read consecutive “packets” from the stdin stream, extract their data payloads in a responsive manner and output to stdout.
We define a custom format of packets for this exercise. A packet consists of a textbased header and a binary data payload following the header. The header is a string formatted as “Size:␣<size>B”, where ␣ represents a white-space character, and <size> is a decimal integer representing the number of bytes of the binary data following the header. The header is ended with the “B” character, and the payload immediately follows the header without any byte (such as “ ”) in between. For example, “Size:␣2105B” is a complete and valid header.
The binary payload can contain any byte, not limited to printable characters. Therefore, the payload should not be treated as string data in your program. This packet format clearly defines the boundary between header and binary data within a packet, and also the boundary between consecutive packets. It ensures that all information can be parsed correctly over a data stream.
Packets are fed to your program sequentially through stdin, until End-Of-File (EOF) is encountered. It is guaranteed that all packets have correct formats and correct payload sizes. Your program should be responsive to the input in the sense that upon receiving a full packet, it outputs the payload of the packet to stdout without any extra characters including newline. Unlike Exercise 1, the program cannot read all data once and process them in batch.
3/library/io.html#io.BufferedReader and https://docs.python.org/3/library/ io.html#io.BufferedWriter. The following code shows how to do binary I/O on stdin and stdout:
import sys
# read **at most** 5 bytes from stdin data = sys.stdin.buffer.read1(5)
# write data to stdout and flush immediately sys.stdout.buffer.write(data) sys.stdout.buffer.flush()
Here, the data object is of bytes class. Python 3 programs can operate on binary data using bytes objects. This class is very similar to str, the string class. For example, both classes have functions find(), split(), and also slice operators for range access (e.g. a[0:10]). Details about this class can be found at https://docs.python.org/3/ library/stdtypes.html#bytes-and-bytearray-operations.
The following shows how to manipulate bytes objects:
# prepend b to the ’…’ expression to form a bytes object
# instead of str pos = data.find(b’x’) if pos >= 0:
# if byte ’x’ is found in data
part1 = data[0:pos+1] # this is similar to str slicing part2 = data[pos+1:] # slice until the end of data
To convert a bytes object to str, your program can call the decode() method of bytes. In this exercise, there is no need to worry about text encoding, as we only use basic ASCII characters in headers.
Finally, to detect EOF on stdin, your program can check the length of the bytes object read from stdin. If your program expects to read more data from stdin but receives a zero-length bytes object, it means that there is no more data on stdin and EOF is encountered.
When testing your program on command line, you can feed the contents of a file to stdin of your program using file redirection (<), instead of typing into the terminal. You can also use another type of redirection (>) to save the output of your program to a file rather than let it print to the terminal. For example, the following line feeds the file input.data to the program and save its output to output.data: python3 PacketExtr.py < input.data > output.data
You can then compare binary contents of output.data and the given reference output by running: cmp output.data ref-output.data
By default, cmp outputs all differences found between the two files. Hence, no output means that the two files are identical.
During grading, we limit the size of each packet to 1MB and the size of all packets to 10MB. In addition to testing correctness, we will test responsiveness by setting a one-second timeout after feeding your program a full packet. That is, if your program does not output the packet payload on time, we will deem your program as unresponsive. This timeout is sufficient for our test data sizes. We reiterate that your program should not do batch processing, as interactive processing is one of the key points of this exercise.
Sample run:
$ python3 PacketExtr.py < test/packets-a.in > run-a.out
$ cmp test/packets-a.out run-a.out
