Contents
Β
Goals of the Project
βΒ Β Β Β Β Β The goal of this project is to introduce students to machine learning techniques and methodologies that help to differentiate between malicious and legitimate network traffic.
βΒ Β Β Β Β Β In summary, students are introduced to:
βΒ Β Β Β Β Β how to build a machine-learning model based on normal network traffic.
βΒ Β Β Β Β Β how to conduct a blending attack producing artificial network traffic that resembles the normal one, and bypass the learned model.
Suggestion
We recommend that you use whatever is comfortable for you β either local set-up or the Linux VM provided. In the past, students faced no difficulty in setting up the project and working on either Windows or Macintosh OS.
Β
Section 1: Readings and Resources
This project relies on the following readings:
βΒ Β Β Β Β Β βAnomalous Payload-based Worm Detection and Signature Generationβ, Ke Wang and Salvatore J. Stolfo, RAID 2004
βΒ Β Β Β Β Β βPolymorphic Blending Attacksβ, Prahlad Fogla, Monirul Sharif, Roberto Perdisci, Oleg Kolesnikov, Wenke Lee, Usenix Security 2006
βΒ Β Β Β Β Β βSensitivity and specificityβ
Section 2: Project Quiz (10 points)
The quiz covers the readings and resources in Section 1. Its purpose is to help you become familiar with the background knowledge needed to implement the project.
The quiz consists of 10 questions worth 1 point each. It is open-book, non-proctored, and you may attempt it an unlimited number of times. We highly recommend that you finish Section 1βs readings and ML related lectures in this course before you proceed with the quiz.
You can find the quiz on Canvas under Project 5 β Project Five Quiz.
Section 3: Project Setup
You can either use the provided VM to complete the project OR you can set up your own environment locally.
3.1: VM Setup
1. Download the VM using one of the links provided in the project description on canvas. (The password for the VM is cs6262).
2. All the required packages/dependencies are already installed in the VM. The project files are under:
Desktop -> project 5
HOWEVER: please use project5.zip in Canvas for the most updated version.
3.2: Local Setup
1. Download the project zip file from the link provided in the project description on Canvas.
2. Please refer SETUP.txt in the PAYL directory to install the dependencies, using the same versions specified in the SETUP.txt.
TIP: Even if you are using the provided VM, please check SETUP.txt to understand how the project is set up and to get an overview of the various code components in the project. This might help in debugging any issues you might face later.
Section 4: Project Tasks (90 points)
4.1: Task A β Training an ML model to detect normal traffic (18 points)
4.1.1: Preliminary Reading
Please refer to the reference readings to learn about how PAYL model works, in particular,
1)Β Β Β Β how to extract byte frequency from the data
2)Β Β Β Β how to train the model
3)Β Β Β Β the definitions of parameters, threshold and smoothing factor
4.1.2: Code and data provided
The PAYL directory provides the PAYL code and data for model training.
4.1.3: PAYL Code Workflow
Here is the workflow of the provided PAYL code:
βΒ Β Β Β Β Β Operates in 2 modes:
1)Β Β Β Β training mode: It reads pcap files provided in the βdataβ directory, and tests parameters and reports the True Positive rates.
2)Β Β Β Β testing mode: It first builds a model using parameters and data specified in the directory. Then it will test a specific packet and decide whether the test packet fits the model.
Training mode
βΒ Β Β Β Β Β Read the normal traffic data and divide it into two parts, 75% of the data for training and the rest 25% for testing (NOTE: You will NOT change these portions in the code).
βΒ Β Β Β Β Β Sort the payload strings by length and generate a model for each length.
βΒ Β Β Β Β Β Each model per length is based on [mean frequency of each ascii, standard deviation of frequencies for each ascii].
To run PAYL in training mode
$ python3 wrapper.py
Testing mode
βΒ Β Β Β Β Β Read the normal traffic data from the directory, and train a model using specific parameters. Then test the specific packet (fed from the command line) using the trained model.
βΒ Β Β Β Β Β Compute the Mahalanobis distance between each test payload and the model (of the same length)
βΒ Β Β Β Β Β Label the payload: If the Mahalanobis distance is below the specified threshold, label the payload as normal traffic. Otherwise, label the packet as attack traffic.
To run PAYL in testing mode
$ python3 wrapper.py [FILE.pcap]
where FILE.pcap is the data you will test.
4.1.4: Tasks β Conduct experiments to select parameters
βΒ Β Β Β Β Β You are provided with artificial payloads (normal network traffic) to train a PAYL model.
βΒ Β Β Β Β Β After reading the reference papers, it should make sense that you cannot train the PAYL model on the entire traffic of different protocols. So first you need to select a protocol: a) HTTP or b) DNS by changing the hard-coded option in wrapper.py.
βΒ Β Β Β Β Β The next step is to select a proper pair of parameters for the model. For the selection process, you will provide a range for both parameters (by modifying the threshold and smoothing factor in wrapper.py). Then run wrapper.py on training mode and make sure the normal traffic (artificial payloads stored in the default data folder) is fed while training.
βΒ Β Β Β Β Β The code will output the statistics for the parameters in the range. As shown in the figure below, for each pair of parameters, you will observe a True Positive Rate. You need to report a pair of parameters (smoothing factor (mSF) and Mahalanobis distance threshold (mTmd)) output by the code that achieves True Positive rate of 96% or more (and this True Positive rate (mTP) should be reported as well). More than 99% true positive rate is possible and you may find multiple pairs of parameters that can achieve that.
The figure above shows a sample output from the wrapper.py. You will find mSF and mTMD values which make mTP > 96% for both HTTP and DNS protocols respectively. The parameters can be different for the two protocols.
4.1.5: Deliverables
Please report for each protocol that you used, the parameters that you found (output by wrapper.py) in a file named parameters.txt. Please report a decimal with rounded 2-digit accuracy for each parameter.
NOTE: You are given a sample parameters.txt with dummy values in the PAYL directory. Please update the relevant values with your own answer. Check section 4 for more details.
NOTE: The value for βDISTANCEβ in parameters.txt will be obtained in the next task (section 3.2).
TIP: You can set lower and upper bound values of both parameters in wrapper.py as the values you found in training mode to avoid multiple iterations during the testing mode.
4.2: Task B β Evaluating packets on the ML model (18 points)
Download your unique attack payload [YOUR_GTUSERNAME.pcap] from Files in Canvas, and place it in the PAYL directory of the Project 5 folder.
Find your pcap file in the project5.zip file in the project5/student_pcaps directory. It the one with your GT Login (username) like gburdell3.
For this part, you will verify that the model is effective for detection. You will ensure that the attack data does not fit the model, while the normal network traffic fits. Make sure you have completed Task A and that the parameters.txt file is set. Completing Task A and Task B will be essential to demonstrating the polymorphic blending attack in Task C.
Use PAYL in testing mode
βΒ Β Β Β Β Β You will first change wrapper.py to use the parameters recorded in parameters.txt. Open wrapper.py and change the None values of the following parameters:
http_smoothing_factor = None
dns_smoothing_factor = None
http_threshold_for_mahalanobis = None
dns_threshold_for_mahalanobis = None
For BOTH the DNS protocol and the HTTP protocol, verify that your attack payload gets rejected. By βrejectedβ, we mean that you will get a βIt doesnβt fit the modelβ message on your test screen as seen in the following figure, which indicates that your attack sample has been detected as anomalous.
$ python wrapper.py <YOUR GT ID>.pcap
Now verify the artificial payloads (normal traffic). We provide two artificial payloads; one for HTTP (http_artificial_profile.pcap) and one for DNS (dns_artificial_profile.pcap). Both are in the PAYL folder. Test each artificial payload against your model.
Note: We use python3 and the command will either be python or python3 depending on your local environment, on the VM itβs python3.
nΒ set the training_protocol parameter in wrapper.py
$ python3 wrapper.py http_artificial_profile.pcap
nΒ set the training_protocol parameter in wrapper.py
$ python3 wrapper.py dns_artificial_profile.pcap
NOTE: DO NOT forget to change parameters according to each protocol while testing relevant payload, e.g., DNS parameters to test dns_artificial_profile.pcap.) These should be accepted by the model. That is, you should get an output message that says βIt fits the modelβ as presented in the following figure, which indicates that the sample is not detected as anomalous. The output will show your own parameters, not βmSFβ, βmTMDβ, or βmTPβ.
4.2.1: Deliverables
Please report the calculated distance of (where mDISTANCE is in the above figures) in parameters.txt for each protocol with the values of the attack payload (<your GT_Login>.pcap) after completing Task B.
4.3: Task C β Crafting a packet to evade the ML model (54 points)
We assume that the attacker has a specific payload (the attack payload) that they would like to blend in with the normal traffic. Also, we assume that the attacker has access to one packet (artificial profile payload) that is normal and is accepted as normal by the PAYL model.
Preliminary reading: Please refer to the βPolymorphic Blending Attacksβ paper. In particular section 4.2 describes how to evade 1-gram and the model implementation. More specifically we are focusing on the case where m <= n and the substitution is ONE-TO-MANY.
The attackerβs goal is to transform the byte frequency of the attack traffic so that it matches the byte frequency of the normal traffic, and thus bypass the PAYL model.
NOTE: Complete this task ONLY for the HTTP protocol.
Code provided: Please look at the Polymorphic_blend directory. All files (including attack payload) for this task should be in this directory. Hence, copy your unique attack payload also in this directory. Rename ATTACKBODY_PATH in task1.py with your unique attack payload name (YOUR_GTUSERNAME.pcap).
How to run the code
$ python3 task1.py
NOTE: You need to complete Task C before running task1.py.
Main function
task1.py contains all the functions that are called to transform the byte frequency of the attack traffic.
Output
βΒ Β Β Β Β Β The code should generate a new payload (filename: output) that can successfully bypass the PAYL model that you have found above (using your selected parameters in Task A and B).
βΒ Β Β Β Β Β The new payload (output) should include:
βΒ Β Β Β Β Β shellcode.bin (provided)
βΒ Β Β Β Β Β Encrypted attack body
βΒ Β Β Β Β Β XOR table
βΒ Β Β Β Β Β Padding
For detailed definitions of these components, please refer to the βPolymorphic Blending Attacksβ paper.
Substitution table
We provide the skeleton for the code needed to generate a substitution table, based on the byte frequency of attack payload and artificial profile payload. For the purpose of implementation, the substitution table can be e.g. a python dictionary table. We ask that you complete the code for the substitution function. The substitution is one-to-many. Skeleton code prints the substitution table to the console. You will deliver your substitution table in substitution_table.txt file in the following format:
NOTE: This is just an example showing the format of the table. Please ignore the frequency values.
NOTE: The substitution table should have the frequencies as observed in the normal payload. Please do NOT normalize these values in substitution_table.txt. You can normalize the values later during substitution in substitution.py.
Padding
Similarly, we have provided a skeleton for the padding function and we are asking you to complete the rest.
Main tasks
Please complete the code for the substitution.py and padding.py and then run task1.py to generate the new payload (output).
4.3.1: Deliverables
βΒ Β Β Β Β Β You will deliver substitution.py, padding.py, your new payload (output) and substitution_table.txt for this task.
βΒ Β Β Β Β Β Youβre expected to write comments for your code. Otherwise, you will lose 5 points.
Test your output
Test your new payload (output) against the PAYL model and verify that it is accepted. FP should be 100% indicating that the payload got accepted as legit, even though it is malicious. You should run as follows and observe the following output, and get the output message that says, βIt fits the modelβ.
TIP: Check the relevant FAQs in section 6.
IMPORTANT: Please check section 6.6 to understand how you can verify your code.
Section 5: Final dedsliverables and Rubric
|
Tasks |
Deliverable Files |
|
A & B |
parameters.txt |
|
C |
substitution.pyΒ Β padding.pyΒ Β substitution_table.txtΒ Β output |
Β
Total: 5 files
Total points: 100
Please make sure to submit all of the files to Gradescope. Do NOT ZIP your deliverable files.
Note: Gradescope scores all project tasks except the quiz out of 100 points. When calculating your final project grade, we will scale your Gradescope score to 90 points and add your quiz score (out of 10 points). For example, a Gradescope score of 20 points contributes 18 points to your final project grade.
5.1: Project Quiz β 10 points
5.2: Task A β 18 points
Please report (for each protocol) the parameters that you found in a file named parameters.txt. Please report a decimal with 2 digit accuracy for each parameter.
5.3: Task B β 18 points
Please report your calculated distance (mDISTANCE in the above figures) in parameters.txt for each protocol with the values of the attack payload after completing Task B.
parameters.txt format:
|Protocol:HTTP|
|Threshold:1111.00|
|SmoothingFactor:0.01|
|TruePositiveRate:50.00|
|Distance:2000.00|
|Protocol:DNS|
|Threshold:2222.00|
|SmoothingFactor:0.00|
|TruePositiveRate:50.00|
|Distance:2000.00|
NOTE: You are given a sample parameters.txt with dummy values under PAYL directory. Please update each value with your own answer. Those values should only come from the PAYL scriptβs output to the console. (not from the values modified inside the script).
Description of fields in parameters.txt
|Protocol:HTTP|
|Threshold:1111.00|Β Β Β Β Β Β Β // Part A
|SmoothingFactor 0.01|Β Β Β Β // Part A
|TruePositiveRate:50.00|Β Β // Part A
|Distance:20020.00|Β Β Β Β Β Β Β // Part B, mDISTANCE from your unique pcap (python wrapper.py <yourunique.pcap>)
|Protocol:DNS|
|Threshold:2222.00|Β Β Β Β Β Β Β // Part A
|SmoothingFactor:0.00|Β Β Β Β // Part A
|TruePositiveRate:50.00|Β Β // Part A
|Distance:22000.00|Β Β Β Β Β Β Β // Part B, mDISTANCE from your unique pcap (python wrapper.py <yourunique.pcap>)
NOTE: β0.3β should be entered as β0.30β. β2β should be entered as β2.00β.
5.4: Task C β 54 points
Please submit the following files: substitution.py, and padding.py, and your substitution_table.txt, and the output of Task C (generated as a new file after running task1.py). This output is very important and represents a significant portion of the points for Task C.
We grant partial credits for the following conditions:
1. PAYL Model Acceptance (18 points): When testing output against the PAYL model, it is accepted.
2. Output Reproducibility (36 points): output can be recreated with the submitted substitution.py and padding.py files and passes the PAYL testing. In case that output cannot be successfully recreated, partial credits is awarded for individual components as below:
1. Substitution Table (18 points): A substitution table can be correctly generated with substitution.py. Note that our implementation starts with mapping the highest-frequency attack character to the highest-frequency normal character. If you instead use alternative mapping, you will generate a different substitution table (It does not mean that your implementation is incorrect).
2. XOR Table (4.5 points): A XOR table can be correctly generated with substitution.py. Note that we use np.random.rand() to select substitution letters. If you instead use a different random number generator, you will generate a different XOR table (It does not mean that your implementation is incorrect).
3. Padding (9 points): Padding can be correctly generated with padding.py.
Note that full credit will be granted, if your output passes the test against the PAYL model. If the test fails, the autograder will then check for partial credit by verifying whether the generated tables/paddings are byte-for-byte identical to the reference solutions. This is a black-box comparison, not a white-box review of the code logic. Consequently, if an implementation uses a different approachβsuch as an alternative random number generator or a different frequency mapping orderβthe resulting tables and padding will differ from the reference. In such cases, NO partial credit can be awarded.
5.5: Submission / Submitting: Important Notes
βΒ Β Β Β Β Β There are unlimited submissions allowed on this project.
βΒ Β Β Β Β Β Every file name with the wrong name and/or extension will be penalized with 5 points.
βΒ Β Β Β Β Β Do NOT ZIP your deliverable files. You will lose 5 points for zipped files.
βΒ Β Β Β Β Β Follow the format for substitution_table.txt above (section 4.3), for the wrong format you will lose 5 points.
βΒ Β Β Β Β Β Follow the format for parameters.txt above, for the wrong format you will lose 5 points.
βΒ Β Β Β Β Β Do not forget to update your parameters.txt If you submit sample parameters.txt with dummy values, you will get 0/36 for Task A and B. We wonβt accept resubmission after the due date.
βΒ Β Β Β Β Β You should complete Task A, B and C with the same set of parameters. If your output doesnβt pass the model with parameters in your parameters.txt you will get 0 for output.
βΒ Β Β Β Β Β For Task B, in parameters.txt you will report calculated distance after you provide your own attack payload. (i.e. YOUR_GTUSERNAME.pcap)
βΒ Β Β Β Β Β For Task C, You are allowed to import and use βrandomβ or βnumpyβ. Do NOT import any other libraries.
βΒ Β Β Β Β Β If you donβt implement the correct algorithm for Task C, even if your output passes the model you wonβt get full credit. We will not accept the implementation of ONE-ONE mapping. (Check section 5.2)
βΒ Β Β Β Β Β Donβt forget to put comments for your code. Otherwise, you will lose 5 points.
Section 6: FAQs
6.1: Task C clarifications
βΒ Β Β Β Β Β As we saw in part A+B, your unique attack payload does not fit the model. In Task C we want to make our pcap fit.
βΒ Β Β Β Β Β We care about our attack payload being turned into what a normal payload would look like.
βΒ Β Β Β Β Β Weβre simply performing substitution and padding to match the character frequency and packet size.
βΒ Β Β Β Β Β We want to modify the various python functions to perform this substitution and padding for us.
βΒ Β Β Β Β Β After the functions have been crafted, copy the file it generates (output) over to your PAYL directory and test it (python3 wrapper.py output). You want it to fit the model.
6.2: How to implement Substitution Table & Substitute?
First Refer to the βPolymorphic Blending Attacksβ paper. In particular, section 4.2 describes how to evade 1-gram and the model implementation. More specifically we are focusing on the case where m <= n and the substitution is ONE-TO-MANY.
NOTE: We will not accept the implementation of ONE-ONE mapping.
Refer to the example provided in the write-up (section 6.5).
After reading the paper and example, it should be obvious how to implement a substitution table. If you still have any specific questions you can post your questions on Ed Discussion.
Substitution
Given an attack byte, find the mapping in your Substitution Table. You will have multiple choices because of how we constructed the table. Pick one based on the ratio of the bytes frequency to the sum of all frequencies. You have to normalize the frequencies to sum up to 1.
NOTE: You are allowed to import and use βrandom or numpyβ for this task. Do NOT import any other libraries.
6.3: How to implement Padding?
Find the byte with the largest byte frequency difference, say βaβ, and append βaβ to the raw_payload (padding.py). Padding function is called repeatedly when
len(raw_payload) < len(artificial_payload)Β Β (as in task1.py).
So each time you only need to pad one byte when the padding function is called.
6.4: XOR and result Clarification in Task C
Both are lists of characters, where βresultβ keeps the replacement chars and βxorβ keeps the XOR of replacement and corresponding attack value.
From the sample in the write-up. Assume βtβ is replaced by βZβ then your result will include βZβ and xor table will include XOR(βtβ,βZβ)=β.β
NOTE: Be careful while XORing the chars
NOTE: You substitution_table.txt should have the format we mentioned in the writeup.
NOTE: You need to verify your Task C and see your original packet content. (check section 6.6)
6.5: Simple example for substitution
Please refer to the βPolymorphic Blending Attackβ: Substitution part
Example
# normal traffic (x) and attack traffic (y):
x = abbcccdddd
y = rrsss
# distinct characters in normal traffic (n) and attack body (m):
n = 4
m = 2
# frequency of characters in normal traffic f(x) and attack body g(y):
f(x) = [(βdβ, 0.4), (βcβ, 0.3), (βbβ, 0.2), (βaβ, 0.1)]
g(y) = [(βsβ, 0.6), (βrβ, 0.4)]
For the first m characters in (x), create a one-to-one mapping in both sets:
Let: t^f(y_j) = f(x_i), where t^f(y_j) refers to βt Γ fΜ(y_j)β in LaTeX notation (the carrot does NOT refer to exponentiation/power).
Then:
S(s) = [(βdβ, 0.4)]
S(r) = [(βcβ, 0.3)]
t^f(s) = 0.4
t^f(r) = 0.3
For the (m+1)th char, first find the attack character with max ratio of g(y_j)/t^f(y_j):
g(s)/(t^f(s)) = 0.6/0.4 = 1.5
g(r)/(t^f(r)) = 0.4/0.3 = 1.33
So, the next attack character is βsβ. Then, your substitution table at this step is:
S(s) = [(βdβ, 0.4), (βbβ, 0.2)]
S(r) = [(βcβ, 0.3)]
and update:
t^f(s) = 0.4 + 0.2 = 0.6
Repeat to find the next attack character and so on.
g(s)/(t^f(s)) = 0.6/0.6 = 1
g(r)/(t^f(r)) = 0.4/0.3 = 1.33
Now, the next attack character is βrβ. Then, your substitution table at this step is
S(s) = [(βdβ, 0.4), (βbβ, 0.2)]
S(r) = [(βcβ, 0.3), (βaβ, 0.1)]
and update:
t^f(r) = 0.3 + 0.1 = 0.4
After you finish the substitution, you are done with t^f(y_j)βs and you will make a substitution with the frequency weight of each character in the table.
s is substituted with:
d with a probability of 0.4/(0.4+0.2)
b with a probability of 0.2/(0.4+0.2)
r is substituted with:
c with a probability of 0.3/(0.3+0.1)
a with a probability of 0.1/(0.3+0.1)
It is up to you how you implement a weighted random assignment, but it is a trivial step.
6.6: How to verify Task C?
Before you submit, check that your encrypted attack body and XOR table decode back to your original attack payload.
Step 1: Create payload.bin
Near the end of task1.py, add (or uncomment) the following line to write out payload.bin:
payload_file.write(bytearray(ββ.join(adjusted_attack_body + xor_table), βutf8β))
Then run task1.py to generate payload.bin in the Polymorphic_blend directory.
Step 2: Decode payload.bin with verify.py
verify.py is included in the Polymorphic_blend directory. It works on any system with Python 3 β the course VM (Intel or Apple Silicon), Windows, macOS and Linux. In the Polymorphic_blend directory, run:
$ python3 verify.py
(or python3 verify.py OTHER_FILE.bin to decode a different file)
If all is well, you will see your original packet contents. If not, you will see a bunch of garbled characters.
Β








