CSCE 665: Advanced Networking and Security

Spring 2019

Instructor:        Dr. Guofei Gu (guofei@cse, 502C HRBB)

Lectures:         MWF 3:00 pm- 3:50 pm, Rm 126 HRBB

Office Hour:    Monday 4-5pm or by appointment

Course Description

Prerequisites: Operating Systems, Computer Networks, and C/C++ and/or Java.

This course will introduce various modern topics in computer and network security. It will provide a thorough grounding in computer and network security suitable for those interested in conducting research in this area, as well as students more broadly interested in real-world security issues/techniques. Topics may span (but not limited to):

Learning objectives & outcomes:

Textbook (s): There is no required textbook. Most readings will be from research papers in top security conferences and journals (listed below). In addition to research papers, you may also read the following textbooks for more security background.

Three hours of lecture per week. (3 units)


Paper presentation/mini-review and class participation: 20%
Homework: 40% 
Mini research project: 40%

There is no mid-term or final exam.

All late submissions within one day after the deadline will automatically lose 40% points. Submissions two days after the deadline will NOT be accepted (unless you get permission from the instructor).

There will be bonus points for EXCELLENT homework and mini research projects.

Paper mini-review

You are expected to write several mini-reviews (actual numbers to be announced in class). This mini-review should at least include the following five items:

Details to be discussed in class.


There will be several homework to help you better understand security principles and techniques. Please finish instructed assignments/labs by yourself and submit necessary reports/materials in google classroom.

Mini Research Project

There will be a semester-long mini research project. You will do research in a team (up to 2 people) or individually. You can choose any interesting topic in computer/network security (not necessary a topic discussed in class, and tied with your current research is encouraged). Be ambitious and start thinking of project topics early!.

During this project, you need to submit a project proposal, a progress report, and the final report. A project proposal should contain introduction (motivation, problem statement), proposed technique/solution, survey of related work (and comparison), project plan (tasks, timeline, job split in the team). A project progress report is essentially close to the final report with some experiments/evaluation to be filled. The final report is expected to be a workshop quality paper. You will also present your project in the final mini-workshop.

For the project report, please use the IEEE proceeding template available here. Reports should have a minimum length of 8 pages (excluding references). Reports must be formatted for US letter size (not A4) paper in a two-column layout, with columns no more than 9.25 in. high and 3.5 in. wide. The text must be in Times font, 10-point or larger, with 11-point or larger line spacing.

The grade of projects will be based on the following factors: novelty, depth, correctness, clarity of presentation, and effort.
Please submit the proposal/report in Google classroom. Each group/team only needs to send one.

Ethics & Academic Integrity

We will study/discuss threats and attacks in class. You should be fully aware of ethics when studying these techniques. If in any context you are not sure about where to draw the line, come talk to me first.

"An Aggie does not lie, cheat, or steal or tolerate those who do."

Upon accepting admission to Texas A&M University, a student immediately assumes a commitment to uphold the Honor Code, to accept responsibility for learning, and to follow the philosophy and rules of the Honor System. Students will be required to state their commitment on examinations, research papers, and other academic work. Ignorance of the rules does not exclude any member of the TAMU community from the requirements or the processes of the Honor System.

Schedule (tentative)

This tentative schedule will be updated as the course progresses and it is subject to change! Please check back for most recent update!

Week Topic Readings Note 
Course overview & logistics none
Computer security overview KPS, PP, SB  
2 Web and Social Network Security CSS explained, CSRF, SQL Injection
Die Free or Live Hard? Empirical Evaluation and New Design for Fighting Evolving Twitter Spammers. Chao Yang, Robert Harkreader, Guofei Gu. RAID'11
Analyzing Spammers' Social Networks For Fun and Profit -- A Case Study of Cyber Criminal Ecosystem on Twitter. Chao Yang, Robert Harkreader, Jialong Zhang, Suengwon Shin, and Guofei Gu. WWW'12
Homework 1 out
Lab basics
3 Software security Smashing the stack for fun and profit.
Return-to-libc attack.
[Further readings] Blended attacks.
Buffer Overflows: Attacks and Defenses for the Vulnerability of the Decade, Crispin Cowan, et al.
On the effectiveness of address-space randomization.CCS'04

4 Malware: Basics
How to 0wn the Internet in Your Spare Time, Stuart Staniford, Vern Paxson and Nicholas Weaver, Security'02
The Zombie Roundup: Understanding, Detecting, and Disrupting Botnets. Evan Cooke, Farnam Jahanian, and Danny McPherson
Homework 2 out
Know your Enemy:Tracking Botnets Using honeynets to learn more about Bots, Paul Bacher, Thorsten Holz, Markus Kotter, Georg Wichersk
A Multifaceted Approach to Understanding the Botnet Phenomenon. Moheeb Abu Rajab, Jay Zarfoss, Fabian Monrose, Andreas Terzis. IMC'06
5 Intrusion detection  The Base-Rate Fallacy and Its Implications for the Difficulty of Intrusion Detection. S. Axelsson. ACM TISSEC'00
A sense of self for Unix processes. S. Forrest, S. A. Hofmeyr, A. Somayaji, and T. A. Longstaff. S&P'96

Bro: A System for Detecting Network Intruders in Real-Time, Vern Paxson, Computer Networks, 31(23-24), pp. 2435-2463, 14 Dec. 1999.
Anomalous Payload-based Network Intrusion Detection. Ke Wang and Salvatore J. Stolfo. RAID'04
Intrusion detection via static analysis. S&P'01
Project proposal due

Malware: Bonet detection BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection. Guofei Gu, Roberto Perdisci, Junjie Zhang, and Wenke Lee. Security'08
BotHunter: Detecting Malware Infection Through IDS-Driven Dialog Correlation. Guofei Gu, Phillip Porras, Vinod Yegneswaran, Martin Fong, and Wenke Lee. Security'07
CyberProbe: Towards Internet-Scale Active Detection of Malicious Servers. Nappa et al. NDSS'14
AutoProbe: Towards Automatic Active Malicious Server Probing Using Dynamic Binary Analysis. Xu et al. CCS'14

SDN Security Openflow tutorial
FRESCO: Modular Composable Security Services for Software-Defined Networks. Seungwon Shin, Phillip Porras, Vinod Yegneswaran, Martin Fong, Guofei Gu, and Mabry Tyson. NDSS'13
AVANT-GUARD: Scalable and Vigilant Switch Flow Management in Software-Defined Networks. Seungwon Shin, Vinod Yegneswaran, Phil Porras, and Guofei Gu. CCS'13
8 Mobile Security
Android basics.
"Dissecting Android Malware: Characterization and Evolution," Yajin Zhou, Xuxian Jiang,S&P'12.

Homework 3 out 
Student Proposal
Student proposal presentation
Spring break
No class
10 Web security Student presentation, debate, discussion: [Jonathan Grimes, Raj Vardhan, William Bogardus]
Anomaly Detection of Web-based Attacks. CCS'03
EXPOSURE: Finding Malicious Domains Using Passive DNS Analysis. NDSS'11
NAVEX: Precise and Scalable Exploit Generation for Dynamic Web Applications. USENIX Security'18

11 IDS/Malware Student presentation, debate, discussion: [Rishabh Singla, Andrew Meserole, Michael Nelson]
Effective and Efficient Malware Detection at the End Host. Clemens Kolbitsch et al. USENIX Security'09
Your Botnet is My Botnet: Analysis of a Botnet Takeover. Brett Stone-Gross et al. CCS'09

BlackIoT: IoT Botnet of High Wattage Devices Can Disrupt the Power Grid. USENIX Security'18
Student presentation, debate, discussion: [Hao Jin, Manish Patel, Michael Nelson ]
Detecting Credential Spearphishing in Enterprise Settings. USENIX Security’17
Click Trajectories: End-to-End Analysis of the Spam Value Chain. S&P'11
FBS-Radar: Uncovering Fake Base Stations at Scale in the Wild. NDSS’17
Project progress report due
SDN security Student presentation, debate, discussion: [Manish Patel, Aaron Skouby, Jonathan Grimes]
Rosemary: A Robust, Secure, and High-performance Network Operating System. CCS'14
SPHINX: Detecting Security Attacks in Software-Defined Networks. NDSS'15
Towards SDN-Defined Programmable BYOD (Bring Your Own Device) Security. NDSS'16

Mobile & IoT security
Student presentation, debate, discussion: [Andrew Meserole, Rishabh Singla, Raj Vardhan]
On the Feasibility of Large-Scale Infections of iOS Devices. USENIX Security'14
Security Analysis of Emerging Smart Home Applications. S&P 2016 
DolphinAttack: Inaudible Voice Commands. CCS'17

15  Mini-workshop: student project   
16 Mini-workshop (4/29; 4/30)

Final report due TBA


Some course materials may have incorporated those developed by Dr. Nick Feamster (Georgia Tech), Dr. Wenke Lee (Georgia Tech), Dr. Wenliang Du (Syracuse), and Dr. Vitaly Shmatikov (UT Austin).