role: “Mirai Botnet Specialist”
goal: “Identify Mirai infection attempts and DDoS attacks in network traffic by analyzing protocol patterns, exploit signatures, and volumetric anomalies”
backstory: |
You are specialized in analyzing Mirai botnet activity. You understand that:

Mirai infection attempts exhibit these characteristics:
– Telnet (port 23) or HTTP exploitation attempts
Specific CVE exploitation patterns like CVE-2023-1389 (TP-Link)

Default credential attacks against IoT devices

Example exploitation patterns:

1. TP-Link Archer command injection (CVE-2023-1389):

– POST to /cgi-bin/luci/;stok=/locale?form=country

– country= parameter contains URL-encoded shell metacharacters (%3B ;, %24 $, %7C |) followed by curl/wget/busybox

– A legitimate write to the same endpoint has a plain two-letter country code and no shell characters

2. Hikvision web-language injection (CVE-2021-36260):

– PUT to /SDK/webLanguage with an XML body containing $(…) command substitution

Mirai HTTP DoS exhibits:

– Bursts of HTTP requests to one host and port within seconds

– Repeated identical requests; events share a scenario_id

See the difference? The second approach teaches the agent your specific knowledge.

Note: The prompt structure above is only a simplified example. Your agent does not need to follow this exact role/goal/backstory format. What matters is that your prompts encode specific knowledge from your Part 1 analysis, including Mirai infection patterns, DDoS signatures, IoCs, and expected network behaviors.

Part 2.1 : Understanding Prompt Engineering for Security

Core Principles

Principle #1: Specificity Over Generality

Generic: “Analyze this traffic”

Specific: “Identify if this HTTP POST request matches known Mirai exploitation patterns targeting IoT devices, specifically looking for command injection in CGI scripts”

Principle #2: Examples Are Critical

LLMs learn best from examples (few-shot learning) – Your Part 1 answers become training examples – Create additional examples showing edge cases

Principle #3: Context Is King

· Explain why certain patterns indicate malicious activity

· Provide the logic, not just the facts

· Help the agent reason like you did

Principle #4: Structured Output

· Define exactly what format you want

· Use JSON schemas or structured formats

· This ensures consistent, gradable responses

The Prompt Engineering Process

1. Start Broad -> Test -> Observe Failures
2. Add Specific Examples -> Test -> Observe Improvements
3. Refine Edge Cases -> Test -> Improve Accuracy
4. Iterate Until Satisfactory Performance

Don’t expect perfection on the first try. Budget time for iteration.

Part 2.2 : AI Framework Introduction

What is CrewAI?

CrewAI is a framework for building AI agent systems. For this project, you’ll use it to:

· Define your agent’s role and capabilities

· Provide training examples

· Configure how the agent processes network traffic

· Test and iterate on your prompts

The Architecture You’re Working With

We provide a pre-configured CrewAI multi-agent system. Do not modify the architecture, work within the defined structure. You’ll work with multiple specialized agents that collaborate to analyze network traffic.

Multi-Agent Configuration:

# This is already set up for you in crew.py (do not edit). Your agents.yaml (edit the yaml file) supplies only the role, goal and backstory text for each of these three agents.
# Agents —————————————————————

@agent

def net_traffic_analyst(self) -> Agent:

…

@agent

def threat_attribution_researcher(self) -> Agent:

…

@agent

def final_classifier(self) -> Agent:

…

Important: Agents receive text representations of network events, NOT raw PCAP files. Agents then analyze this text.
You’ll use Wireshark/tshark to extract relevant information from PCAPs and format it as text (protocol, method, URI, payload patterns, etc.) for your analysis on how to instruct your agents.

Why Multiple Agents?

Multi-agent systems offer several advantages for security analysis:

Specialization:

· Each agent can focus on specific aspects

· Deeper expertise in their domain

· Clearer separation of concerns

Modularity:

· Easier to debug specific components

· Can update one agent without affecting others

· Can test agents independently

Mirrors Real Analysis:

· Security teams have specialists (network analysts, threat intelligence, incident responders)

· Multiple perspectives improve accuracy

· Collaboration catches edge cases

Your Tasks Summarized

· Understand the provided agent architecture

· Configure each agent’s YAML prompts with specialized knowledge

· Design how agents work together (using the given pre-defined roles)

· Distribute knowledge appropriately across agents and RAG

· Create training examples (optional but recommended)

· Test the multi-agent system against practice questions

· Iterate to improve accuracy

Understanding RAG (Retrieval-Augmented Generation)

What is RAG?

RAG is a technique that enhances AI agents by giving them access to a searchable knowledge base. Think of it like giving your agent a library it can consult:

Without RAG:

User Question -> Agent (with only its prompt) -> Answer

With RAG:

User Question -> Retrieve Relevant Knowledge -> Agent (with prompt + retrieved context) -> Answer

Why RAG Matters for This Project

Your agent’s prompt has token limits – you can’t include everything about Mirai in the backstory. RAG solves this by:

1. Storing detailed knowledge separately in documents

2. Retrieving only relevant pieces for each question

3. Providing context dynamically based on the specific query

Example:

· Question about F5 BIG-IP -> RAG retrieves your document on CVE-2020-5902

· Question about HTTP DDoSs -> RAG retrieves your DDoS attack documentation
(NOTE this used to say UDP floods, UDP attacks are not in scope, fixed)

· Agent gets relevant context for each unique question

How CrewAI’s RAG Works

The system uses ChromaDB, a local vector database that helps the agent search your knowledge files based on meaning, not just exact keyword matches.

· Local vector database that stores your knowledge

· Open-source embeddings

· Converts text into mathematical vectors

· Automatic retrieval – Agent queries the RAG when analyzing events

· Semantic search – Finds relevant content based on meaning, not just keywords

At a high level, the process works like this:

1. You write markdown documents in knowledge/ directory

2. CrewAI breaks those documents into smaller chunks.

3. Each chunk is converted into a searchable representation and stored locally as vectors.

4. When the agent analyzes network traffic, CrewAI searches those chunks for the most relevant information.

5. Most relevant chunks are retrieved and added to the agent’s context

6. Agent uses this context + its prompt to make decisions

Understanding Chunking

Chunking is breaking your documents into smaller pieces for better retrieval.

Why chunk?

· Documents are too long to use entirely

· Agents need specific, relevant information

· Better retrieval accuracy with focused chunks

· How CrewAI chunks: – Fixed-length slices of 4,000 characters (roughly 700–1,000 words) with a 200-character overlap – Boundaries ignore paragraphs and headings – A 300–800 word document is therefore one or two chunks

What this means for you: Keep each document to one topic and under ~700 words so it is retrieved as a single, complete chunk. Put the identifying facts (endpoint, parameter, what distinguishes malicious from benign) in the first few lines.

Good chunking example:

TP-Link Archer AX21 Command Injection (CVE-2023-1389)

Exploitation is a POST to /cgi-bin/luci/;stok=/locale?form=country whose country= value contains URL-encoded shell metacharacters (%3B, %24, %7C) followed by curl, wget or busybox and a chmod/execute step.

Classify as http_exploit with cve CVE-2023-1389. A legitimate request to the same endpoint has a two-letter country code (e.g. country=US) and no shell characters; classify that as benign.

This paragraph above is complete and self-contained – works well as a chunk.

Bad chunking example:

## Exploitation Details

This vulnerability is bad. See the next section for details.

### More Info

The vulnerable endpoint is tmshCmd.jsp. It allows commands.

Problem: Information is fragmented. If only one chunk is retrieved, it lacks context.

Part 2.3 Creating Your Knowledge Base

Directory Structure

Here is a suggested structure for your RAG, it doesn’t need to be this complicated, we estimate a minimum of seven files is needed:

knowledge/
├── exploitation_patterns/
│   ├── f5_bigip_cve_2020_5902.txt
│   ├── tplink_command_injection.txt
│   ├── telnet_brute_force.txt
│   ├── netgear_rce.txt
│   └── generic_cgi_exploits.txt
├── ddos_attacks/
│   ├── udp_generic_flood.txt
│   ├── tcp_syn_flood.txt
│   ├── http_flood.txt
│   └── gre_flood.txt
├── network_indicators/
│   ├── port_scanning_patterns.txt
│   ├── payload_signatures.txt
│   └── traffic_characteristics.txt
├── device_profiles/
│   ├── iot_cameras.txt
│   ├── routers.txt
│   └── dvrs.txt
└── analysis_principles/
    ├── distinguishing_malicious_from_benign.txt
    └── confidence_assessment.txt

Document Template

Use this template for each knowledge document:

# [Title: Specific and Descriptive]

## Overview
[2-3 sentence summary of what this document covers]

## Network Indicators
[Specific, observable patterns in network traffic]
– Protocol: [HTTP/TCP/UDP/etc.]
– Port(s): [Specific ports]
– Distinctive Features: [What makes this recognizable]

## Detailed Pattern Description
[Comprehensive explanation of the pattern, including:]
– What the traffic looks like
– Why this pattern indicates malicious activity
– How to distinguish from legitimate traffic

## Examples
[Concrete examples with actual values]

## Detection Logic
[Step-by-step reasoning for identifying this pattern]

## Classification Guidelines
– Activity Type: [Infection/DDoS/Reconnaissance/Benign]
– Subtype: [Specific technique]
– Confidence Level: [When to be High/Medium/Low confidence]
– Related CVE: [If applicable]
– Target Devices: [What devices are affected]

## Related Patterns
[Links to similar or related attack patterns]

## Important Distinctions
[How to avoid false positives – what looks similar but isn’t this]

Writing for Effective Retrieval

Principle 1: One Topic Per Document

Bad: A single document covering “All Mirai Attacks”

Good: Separate documents for each attack type

Why: Focused documents mean focused chunks. Better retrieval accuracy.

Principle 2: Front-Load Key Information

Put the most important information at the beginning of each section:

## UDP Generic Flood Detection

UDP Generic Flood is characterized by high-volume UDP traffic with fixed 512-byte
payloads and randomized source IPs, targeting a single destination. This is Mirai’s
most common DDoS technique.

[Details follow…]

Why: If only the first chunk is retrieved, it contains the essentials.

Principle 3: Use Clear, Specific Language

Vague: “This attack is bad and targets vulnerable systems”

Specific: “CVE-2020-5902 targets F5 BIG-IP load balancers via POST requests to /tmui/locallb/workspace/tmshCmd.jsp”

Why: Specific language matches query terms better during retrieval.

Principle 4: Include Synonyms and Variations

## Telnet Brute Force (Credential Stuffing / Dictionary Attack)

Mirai scans for exposed telnet services (port 23 or alternate port 2323) and attempts default credentials. This brute force attack (also called credential stuffing or dictionary attack) tries common username/password combinations.

Why: Different queries might use different terms. Coverage helps retrieval.

Principle 5: Make Sections Self-Contained

Each section should make sense without reading other sections:

## Detection Logic for F5 BIG-IP Exploitation

When analyzing HTTP traffic for F5 BIG-IP exploitation (CVE-2020-5902), check:
1. URI contains “/tmui/locallb/workspace/tmshCmd.jsp”
2. Method is POST
3. Body contains “command=” parameter
If all three are present, classify as Infection (Remote Code Execution) with High confidence.

Why: Chunks are retrieved independently. Each must be complete.

Content Strategy: What to Include

Suggested Minimum Requirements:

· 5+ exploitation patterns with detailed indicators

· HTTP DDoS attack and Telnet brute force with traffic characteristics

· Device profiles explaining typical vulnerabilities

· 1 document on analysis principles (how to think about classification)

Recommended Additions:

· Edge cases and tricky scenarios

· False positive examples (benign traffic that looks suspicious)

· Attack variations (e.g., “HTTP DDoS with different payload sizes”)

· Protocol-specific analysis guides

Building Your Knowledge Base is an iterative process

Part 1: Core Exploitation Patterns (Start Here)

From your Part 1 analysis, create documents for:

1. Each CVE or vulnerability you identified in the PCAP

2. The telnet brute force pattern (if present)

3. Any HTTP-based exploits you observed

Part 2: DDoS Attack Patterns

Document each DDoS type, currently only HTTP DOS is in scope.

Part 3: Supporting Documentation

Add documents that help with edge cases:

1. How to distinguish high traffic volumes (DDoS vs. legitimate)

2. How to assess confidence levels

3. Common false positives and how to avoid them

Part 4: Test and Refine

As you test your agent:

1. Note which questions it struggles with

2. Check what gets retrieved (see testing section below)

3. Add or refine documents to improve retrieval

Implementing Your Multi-Agent Architecture

You’ll be working with three specialized agents that collaborate to analyze network traffic. We provide a base structure with the three agents, but you have flexibility in how you configure and specialize them.

Critical Understanding: Your agents will NOT directly analyze PCAP files. Instead:

· You use Wireshark/tshark to extract information from PCAPs

· You format this as text (protocol, URIs, payload patterns, volumetric data)

· Your agents analyze these text representations

Example of what agents receive:

Protocol: HTTP
Method: POST
URI: /tmui/locallb/workspace/tmshCmd.jsp
Host: 192.168.1.5
Body: command=cat /etc/passwd
Rate: Single request

Configuring Your Agents

The YAML Configuration File

Open the src/malware_analysis_agent/config/agents.yaml file and you’ll see multiple agent definitions. For example:

agent_name_1:
role: >
    [YOUR ROLE DEFINITION FOR AGENT 1. For example net_traffic_analyst ]

goal: >
    [YOUR GOAL DEFINITION FOR AGENT 1]

backstory: >
    [YOUR BACKSTORY WITH KNOWLEDGE FOR AGENT 1]

Important Notes:

Each agent is defined in the same file, separated by the agent name key. Agent names should be as given and should not be changed:

1. net_traffic_analyst

2. threat_attribution_researcher

3. final_classifier – do not modify this, it enforces the output schema for grading

You will only modify the net_traffic_analyst and threat_attribution_researcher agents, please do not modify the final_classifier unless you are an expert user who can test the results, we will not support questions about changing the final classifer.

– Each agent has its own role, goal, and backstory

– Agents can have overlapping knowledge, but specialization is more effective

Key Mirai Patterns to Recognize

Mirai infection typically involves:

· Telnet/HTTP exploitation of IoT devices

· Specific CVE patterns in your knowledge base

· Default credential attacks

Mirai DDoS attacks exhibit:

· High-volume traffic floods

· Fixed packet sizes (often 512 bytes for UDP)

· Spoofed source IPs

When you encounter specific patterns (URIs, port combinations, packet characteristics), retrieve detailed information from your knowledge base to make accurate classifications.

Creating Training Examples (Optional but Recommended)

Recommended resource:

Understanding CrewAI Training

CrewAI’s training feature allows you to provide labeled examples that get added to the agent’s context. This is essentially few-shot learning:

• Zero-shot: Agent has only its backstory (no examples)

• Few-shot: Agent has backstory + your training examples

• Result: Significantly better performance with good examples

Use the instructions the crew documentation to use a CLI session on events you are interested in. Make sure the input dataset is a small subset of representative data you want to train examples on.

It is important that you create additional examples beyond that which we provide to cover benign data flows, and realistic exploit payloads.

Creating training/examples.json

Your training file should include diverse examples:

[
{
    “input”: “Protocol: HTTP\nMethod: POST\nURI: /tmui/locallb/workspace/tmshCmd.jsp\nHost: 192.168.1.5\nBody: command=cat /etc/passwd”,
    “output”: {
      “activity_type”: “Infection”,
      “subtype”: “Remote Code Execution”,
      “cve”: “CVE-2020-5902”,
      “target_device”: “F5 BIG-IP”,
      “confidence”: “High”,
      “reasoning”: “POST request to BIG-IP management interface with command injection in tmshCmd.jsp endpoint. This is the signature exploitation pattern for CVE-2020-5902.”
    }
},
{
    “input”: “Protocol: UDP\nSource: 203.0.113.15\nDest: 198.51.100.20:53\nPayload Size: 512 bytes\nRate: 1500 packets/second”,
    “output”: {
      “activity_type”: “DDoS”,
      “subtype”: “UDP Generic Flood”,
      “cve”: null,
      “target_device”: “DNS Server”,
      “confidence”: “High”,
      “reasoning”: “High-rate TCP traffic with fixed payload size typical of Mirai HTTP DDoS. The 512-byte fixed payload and volumetric characteristics match Mirai’s UDP attack module.”
    }
},
{
    “input”: “Protocol: HTTP\nMethod: GET\nURI: /api/users\nHost: example.com\nUser-Agent: Mozilla/5.0\nStatus: 200 OK”,
    “output”: {
      “activity_type”: “Benign”,
      “subtype”: “Normal API Request”,
      “cve”: null,
      “target_device”: “Web Application”,
      “confidence”: “High”,
      “reasoning”: “Standard HTTP GET request to legitimate API endpoint with normal user agent and successful response. No indicators of malicious activity.”
    }
}
]

Sourcing Training Examples

Create Variations:

· Modify exploitation patterns slightly (different URIs, different payloads)

· Show similar attacks against different devices

· Include edge cases you discovered.

Include Negative Examples:

· Benign traffic that might look suspicious

· Failed exploitation attempts

· Legitimate high-volume traffic (not DDoS)

How Many Examples?

• Minimum: 10 examples

• Recommended: 15-20 examples covering diverse scenarios

• Diminishing Returns: Beyond 25 examples, you get less improvement

Quality over Quantity: Five excellent, diverse examples beat 20 mediocre ones.

Understanding LLM Limitations

Why Your Agent Might Fail

Insufficient Context: The agent doesn’t have the domain knowledge you gained in Part 1.

Solution: Add more detailed explanation in backstory.

Ambiguous Instructions: Vague prompts lead to unreliable outputs.

Solution: Be extremely specific about what indicators matter.

Hallucination: – LLMs can make up plausible-sounding but false information.

Solution: Ground the agent with concrete examples and patterns.

Over-Generalization: The agent might apply patterns too broadly.

Solution: Include negative examples and edge cases.

Best Practices for Reliability

1. Validate with Concrete Indicators:

# Don’t:
“Look for suspicious HTTP requests”

# Do:
“Identify HTTP requests to /cgi-bin/*, /tmui/*, or paths containing ‘../’
as these indicate command injection or path traversal attempts”

2. Specify Confidence Levels:

“When multiple strong indicators are present (specific CVE pattern +
known vulnerable endpoint + exploit payload), confidence should be High.
When only one indicator is present, confidence should be Medium or Low.”

3. Include “I Don’t Know” Conditions:

“If the traffic pattern doesn’t match any known Mirai exploitation or
attack patterns, and there are no clear indicators of malicious activity,
classify as Benign with Low confidence and note the ambiguity.”

Part 2.5 Deliverable / Submission

Number of Submissions

You get ten submissions to Gradescope. Use them wisely!

What to Submit

Submit via Gradescope (submit as a ZIP file). There is no Canvas submission:

4. Your zip must have this layout at its top level (no wrapping folder

1. agents.yaml – Your configured multi-agent system

2. knowledge/ directory – Your complete knowledge base (all .txt files).

3. trained_agents_data.pkl – From the optional CrewAI training mode with human labeling

Important: The auto-grader WILL use your knowledge base and all your agents. When grading:

4. Your knowledge files are loaded into the RAG system

5. Your multi-agent system is initialized with your configurations

6. Test questions are run through your agent pipeline

7. Agents can query your knowledge base via RAG

8. Accuracy is measured against the holdout dataset

This means both your agent configuration AND knowledge base are critical:

· Well-organized knowledge = better retrieval = better performance

· Well-designed agents = better specialization = more accurate analysis

· YAML prompts + Knowledge Base work together for optimal results

Strategic Distribution for Multi-Agent Systems:

Put in the agent.yaml prompts:

· Each agent’s specific role and responsibilities

· How agents should interact with each other

· High-level reasoning frameworks for each agent

· Agent-specific analysis approaches

· General classification logic and confidence assessment

Put in your knowledge base (RAG):

· Specific exploitation patterns with detailed indicators

· CVE details and attack signatures

· Device-specific vulnerability information

· Detailed examples of traffic patterns

· Edge cases and false positive scenarios

· Technical details that would clutter agent prompts

Grading Rubric

Component	Points	Description
Costs Quiz	5 points	3.3% weight
Phase 1 Quiz	45 points	Manual Malware Analysis – 30% weight
Phase 2	100 points	Automated Agent Analysis – 66.7% weight
Total	150 points

Gradescope Rubric

For accessibility needs, please see the spreadsheet of the full rubric for the Phase 2 Gradescope portion of the project. Or see the image below:

Grading Notes

• Multi-agent design matters – clear specialization and coordination score higher

• Quality of individual agent prompts matters – well-structured, focused agents perform better

• Knowledge-base organization matters – well-chunked, focused documents improve retrieval

• Innovation is rewarded – creative approaches to agent architecture and knowledge encoding

• The baseline will be established using minimal-effort prompts to determine 0%. There is a minimum quality threshold to get credit.

• You do not need 100% accuracy for full credit. Just because you got 100% credit for a category does not mean your agentic system correctly labeled all events

• Multi-agent configuration assessed by: clear agent specialization, effective coordination strategy, appropriate knowledge distribution across agents

• Knowledge base quality is assessed by: document organization, chunk effectiveness, coverage of patterns, and whether retrieval provides relevant context during testing

Tips for Success

General Strategy

5. Start Early – The iteration cycle takes time, especially with multiple agents

6. Master Part 1 First – Your agents are only as good as your understanding

7. Design agent specialization clearly – Each agent should have a distinct, focused role

8. Build RAG incrementally – Start with core patterns, expand as you learn

9. Test agents individually first – Make sure each agent works before testing the full system

10. Test the full pipeline – Ensure agents work together effectively

11. Use the Practice Questions – They’re there to help you improve

12. Iterate across all layers – Agent coordination, RAG, YAML prompts, and training examples

13. Read Error Messages – They often tell you exactly what’s wrong

14. Test retrieval carefully – Use verbose mode to see what context each agent receives

Common Pitfalls to Avoid

“I’ll just paste the source code into my prompt”:

· The agent won’t learn from raw code

· Token limits will be exceeded

· This shows you didn’t understand the assignment

“I’ll use super generic prompts and hope the LLM knows Mirai”:

· It won’t work

· The LLM needs YOUR specific knowledge

· You’ll score at the baseline (0%)

“I’ll skip the training examples”:

· While optional, they significantly improve performance

· Few-shot learning is powerful

· Your Part 1 answers are already examples – use them!

“I’ll put everything in the RAG and make my YAML prompts too generic”:

· Your YAML provides the reasoning framework and instructions

· RAG provides specific knowledge, but the agent needs guidance on HOW to use it

· Balance is key: YAML = “how to think”, RAG = “what to know”

· Without good prompts, even perfect knowledge won’t be used effectively

“I’ll test only on practice questions similar to my training examples”:

· The Gradescope holdout set will include variants you haven’t seen

· Your agent needs to generalize

· Test on diverse scenarios

“I’ll make all my agents do the same thing”:

· Defeats the purpose of multi-agent architecture

· No benefit over single agent if they’re not specialized

· Each agent should have a distinct, focused role

“My agents don’t need to know about each other”:

· Agents work better when they understand their role in the pipeline

· Tell agents what they receive from others and what they should provide

· Clear expectations improve coordination

Frequently Asked Questions

Technical Questions

Q: Can I use a different LLM besides Azure gpt-4o-nano?

A: Locally for testing and iteration, yes. But your final submission will be graded using Azure gpt-5-nano.

Q: How do I know if my YAML syntax is correct?

A: Run the practice test suite. It will catch syntax errors immediately.

Q: Can I modify the CrewAI framework code?

A: No. You should only modify the YAML configuration, training examples, and knowledge base documents. The framework itself is off-limits.

Q: What if I run out of Azure credits?

A: Very unlikely with normal usage (~$10-20 total). Any credits you use beyond free credits are your responsibility.

Q: How large should each knowledge document be?

A: Aim for 300-800 words per document. Each should be focused on one topic. Too short and it lacks detail; too long and chunking becomes problematic.

Conceptual Questions

Q: Why can’t I just include the PCAP file as context?

A: Token limits / cost is one factor, and more importantly, raw data isn’t knowledge. You need to extract the meaningful patterns and teach them to the agent.

LLMs can’t read PCAP files because they are binary.

Q: What if my agent works great on practice questions but fails on the holdout set?

A: This is overfitting. Your agent memorized specific examples instead of learning general patterns. Add more diverse examples and focus on teaching concepts, not memorizing.

Q: How should I balance knowledge between my YAML prompt and my RAG documents?

A: Think of it this way:

· YAML = HOW to think – Reasoning frameworks, analysis approach, classification logic

· RAG = WHAT to know

· Specific exploitation patterns, CVE details, detailed indicators

· Example: Agent YAML says “Compare traffic to known exploitation patterns and assess confidence based on indicator matches.”

· RAG provides the actual exploitation patterns to match against.

Q: Can I organize my knowledge base / RAG however I want?

A: Yes, as long as it retrieves effectively. The suggested structure exploitation_patterns/, dos_attacks/, etc.) is a starting point, but you can adapt it based on what works best for your retrieval testing.

Grading Questions

Q: How many test questions will the holdout set have? A: 50 events in six categories of questions, including exploitation attempts, DDoS attacks, Mirai variants, and benign traffic.

Q: Will the holdout set include malware other than Mirai? A: Yes, 2-3 questions will involve malware similar to Mirai (same techniques, different implementation). This tests generalization.

Q: What determines the baseline (0%)? A: We’ll run minimal-effort configurations through the grader to establish what “random guessing” or “generic prompts” achieve. Anything above that earns points.

Q: Can I submit multiple times? A: You have a limited number of submissions to Gradescope. Use your practice testing to get it right before submitting.

Academic Integrity

What You Can Do

· Discuss high-level strategies with classmates

· Share resources about Mirai, CVEs, or prompt engineering

· Help each other with Wireshark or technical setup

· Discuss general prompt engineering techniques

What You Cannot Do

· Share your specific YAML configurations or training examples

· Copy another student’s prompts or backstory text

· Share answers to the Part 1 graded quiz questions

· Use someone else’s configuration as a template

· Submit work that is not your own

AI Tool Usage Policy

You may use AI / LLM tools (ChatGPT, Claude, etc.) to:

· Help understand Mirai malware behavior

· Learn about agentic AI and agent design principles.

· Learn about specific CVEs or vulnerabilities

· Brainstorm prompt engineering strategies

You may NOT use AI tools to:

· Generate your YAML configuration wholesale

· Debug syntax errors (Don’t upload your agents to LLMs.)

· Answer the Part 1 graded questions

· Create your training examples without understanding them

Getting Help

Office Hours

Check Ed Discussion for TA office hours. Come prepared with:

· Specific questions about your approach

· Examples of where your agent is failing

· What you’ve already tried

Don’t ask TAs to debug your entire configuration. Ask targeted questions.

Acknowledgements / Funding

We are incurring significant costs to run the autograder. We are grateful for these costs being approved by Dr. David Joyner.

Conclusion

This project is designed to challenge you to think deeply about how AI can augment security analysis while understanding its limitations. The most successful students will:

15. Develop genuine expertise in Mirai’s behavior through careful analysis

16. Think carefully about how to encode that expertise into prompts

17. Iterate relentlessly to improve their agent’s performance

18. Reflect critically on the role of AI in cybersecurity

Resources and Links

Mirai Botnet

• Usenix writeup: Understanding the Mirai Botnet

• Mirai Source Code: GitHub Repository

Malware Analysis Resources / CVEs / Exploits

• Mirai Technical Analysis: Cloudflare Blog

• CVE Database: https://cve.mitre.org/

• Exploit Database: https://www.exploit-db.com/

• Cyberchef:

Ghidra (NSA software)

· Ghidra Website: https://github.com/NationalSecurityAgency/ghidra

· Ghidra Cheatsheet: https://htmlpreview.github.io/?https://github.com/NationalSecurityAgency/ghidra/blob/stable/GhidraDocs/CheatSheet.html

· Wikipedia Page; https://en.wikipedia.org/wiki/Ghidra

Wireshark & PCAP Analysis

• Wireshark User Guide: https://www.wireshark.org/docs/wsug_html_chunked/

• tshark Man Page: https://www.wireshark.org/docs/man-pages/tshark.html

• PCAP Analysis Tutorial: Malware Traffic Analysis

Prompt Engineering

• OpenAI Prompt Engineering Guide: https://platform.openai.com/docs/guides/prompt-engineering

• Anthropic Prompt Engineering: https://docs.anthropic.com/claude/docs/prompt-engineering

• Few-Shot Learning: Research Paper

CrewAI

• Official Documentation: https://docs.crewai.com/

[SOLVED] CS6262 SWARM Project 4 Fall2026

Part 2: Building Your AI-Augmented Analysis Agent

Philosophy: Teaching, Not Prompting

Part 2.1 : Understanding Prompt Engineering for Security

Core Principles

Principle #1: Specificity Over Generality

Principle #2: Examples Are Critical

Principle #3: Context Is King

Principle #4: Structured Output

The Prompt Engineering Process

Part 2.2 : AI Framework Introduction

What is CrewAI?

The Architecture You’re Working With

Why Multiple Agents?

Your Tasks Summarized

Understanding RAG (Retrieval-Augmented Generation)

What is RAG?

Why RAG Matters for This Project

How CrewAI’s RAG Works

Understanding Chunking

Part 2.3 Creating Your Knowledge Base

Directory Structure

Document Template

Writing for Effective Retrieval

Content Strategy: What to Include

Building Your Knowledge Base is an iterative process

Implementing Your Multi-Agent Architecture

Configuring Your Agents

The YAML Configuration File

Key Mirai Patterns to Recognize

Creating Training Examples (Optional but Recommended)

Creating training/examples.json

Sourcing Training Examples

How Many Examples?

Understanding LLM Limitations

Why Your Agent Might Fail

Best Practices for Reliability

Part 2.5 Deliverable / Submission

Number of Submissions

What to Submit

Grading Rubric

Gradescope Rubric

Grading Notes

Tips for Success

General Strategy

Common Pitfalls to Avoid

Frequently Asked Questions

Technical Questions

Conceptual Questions

Grading Questions

Academic Integrity

What You Can Do

What You Cannot Do

AI Tool Usage Policy

Getting Help

Office Hours

Acknowledgements / Funding

Conclusion

Resources and Links

Mirai Botnet

Malware Analysis Resources / CVEs / Exploits

Ghidra (NSA software)

Wireshark & PCAP Analysis

Prompt Engineering

CrewAI

Related products

CS6262 Project 5

CS6262 – Project 4

CS6262 Project 1-Penetration Testing Spring 2025

Related in this category

More in this category

CS6262 Project 5- Machine Learning for Security

CS6262 Project 4 -Network Monitoring 2025

CS6262 Project 4 Network Monitoring Fall2025

CS6262 Project 1-Penetration Testing Spring 2025

CS6262 Project 2 -Network Security, Fall2025

CS6262 – Project 3: Malware Analysis

Want to See Past Work First?