A Case Study on Spam Detection, Analysis, and Trace Based on Campus Emails

View Statistics Email Alert RSS Feed

Information

Details

Project title

Code/計畫編號

NSC98-2221-E019-014

Translated Name/計畫中文名

垃圾郵件偵測分析與追蹤---大型校園郵件系統實例研究

Project Coordinator/計畫主持人

Chun-Chao Yeh

Funding Organization/主管機關

National Science and Technology Council

Department/Unit

Department of Computer Science and Engineering

Website

https://www.grb.gov.tw/search/planDetail?id=1917060

Year

2009

Start date/計畫起

01-08-2009

Expected Completion/計畫迄

31-07-2010

Bugetid/研究經費

460千元

ResearchField/研究領域

資訊科學--軟體

本計畫希望藉由實際大型（中文）郵件系統的追蹤分析，做實際系統的案例研讀。從系統的角度瞭解異常郵件的行為及其潛在網路安全問題，包含一般的垃圾郵件（spam mail）及在 URL 中夾帶各類攻擊的惡意郵件（malicious mail）。限於研究期限與人力，本研究案預定在未來一年的計畫執行中，做以下的研究觀察： 1.垃圾郵件偵測：垃圾郵件偵測技術是反垃圾郵件主要的核心技術之一。過去到現在也有相當多的技術發展出來。然而大部分以statistics machine-learning 技術為主，利用垃圾郵件大量發送而產生多封相同郵件重複出現的特性相對較少人注意。同時這些技術對於中文郵件的有效性尚須進一步研究觀察。因此，我們在這個子議題上主要有兩個研究觀察重點： 1.URL-based 近似郵件偵測方法。這是過去我們發展出來方法，此方法利用相同垃圾郵件重複出現的特性來過濾垃圾郵件，我們希望這此基礎上再深入探討。這個方法並沒有語言文字的拘限性，可以應用到其他郵件系統。2.一些主要統計分析的垃圾郵件偵測方法（如 bayesian filter 及SVM 等方法）在中文郵件的有效性。這部分主要是因為我們考慮的是中文郵件系統，自然得面對中文垃圾郵件過濾問題，為了計畫的完整性，我們也希望在這個議題上有些探討，初步以目前大多數使用的bayesian filter 為主做深入探討，如果時間允許我們會嘗試如SVM 等方法。 2.異常郵件行為分析：這個部分主要是透過郵件記錄檔（email log）的分析，希望長期追蹤異常郵件行為。例如偽冒（不正常）發信主機或發信人/收信人住址，黑名單，或是其他不正常郵件發送協定（SMTP）的行為。由於一般正常的郵件伺服器皆不會有這些問題，因此這些異常行為通常是垃圾郵件主機的活動。透過這個觀察我們希望瞭解異常郵件的行為模式與趨勢。 3.問題郵件主機追蹤：透過上述垃圾郵件偵測與異常郵件分析以及郵件記錄檔的紀錄，可以反推疑似問題郵件發送主機的IP，如果多封問題郵件指向同一個IP，則此IP 是濫發垃圾郵件主機的可能性就相對增高。透過這樣的機制，除了可以找出大量發送垃圾郵件的問題主機外，我們主要的目的為發現那些可能被入侵的一般使用者主機，例如遭殭屍網路（botnet）控制，對此疑似被入侵主機提出警訊，以避免該主機在使用者/管理者不知情下繼續被操控利用。這個部分也呼籲國際上「終結Spam 殭屍電腦」的行動計畫。 Spam is an old problem while we are still suffering from. According to an international study by Messaging Anti-Abuse Working Group (MAAWG), the ratio of spam over total received emails in this study is around 84%-87% for recent one year. Similar studies conducted by a domestic authority (National Communication Commission, NCC, Taiwan) based on domestic ISP data reveal similar results as well. More than 80% of received emails were reported as spam during past half year. In this research proposal, we plan to conduct a system-wide study on spam detection, analysis, and trace based on campus emails. We’ll focus on the following three issues: 1. spam detection. Most of current anti-spam researches focus on the approaches based on statistics machine-learning schemes, such as Bayesian filter, SVM, K-NN, etc. We instead want to develop a different approach based on near-duplicate mail detection. In our past researches, we have shown that URLs embedded in emails could be served as a good invariant feature for duplicate mail detection. In this research proposal we will continue our efforts to enhance the proposed scheme to against possible attacks on it. Meanwhile, we will pay more attention to Chinese Spam problems. 2. abnormal email analysis. Another main purpose of this research proposal is to understanding spam behavior through email log. We’ll conduct a spam analysis over a long period of email log study. 3. spam IP trace. Based on spam detection and abnormal email analysis, we can find out those suspicious emails. One more step is to know where the spam comes from. We’ll trace those hosts subject to sending spam mails, which could be classified as open relay/open proxy, normal email servers, and user hosts. For user hosts subject to sending spam, they are highly related to a bonet host, a compromised host and controlled by remote attackers. We’ll pay more attention to spam hosts related to bonets.

Keyword(s)

垃圾郵件
近似郵件偵測
異常郵件行為分析
Spam
Dear-duplicate email detection
Spam behavior analysis

DSpace CRIS

A Case Study on Spam Detection, Analysis, and Trace Based on Campus Emails

Details

Description