Adversarial Learning in ML: Combatting Internet Abuse & Spam

Explore the realm of adversarial learning in ML through combating internet abuse and spam. Delve into the motivations of abusers, closed-loop approaches, risks of training on test data, and tactics used by spammers. Understand the challenges and strategies involved in filtering out malicious content and protecting users in the digital landscape.

• Adversarial Learning
• Internet Abuse
• Spam Filtering
• ML Systems
• Cybersecurity

maka_483 Follow

Uploaded on Sep 10, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. ML Design Pattern: Adversarial Learning Internet Abuse & Spam

2. Example: Running an Internet Scale Email Service Filter 100s of Millions of messages per day 10s of Millions of users per day Email Store Junk Store Email Senders User Interface Email Service

3. What Abusers Want Reasons for Abuse Crime, espionage For fun To make money Making Money with Abuse Driving traffic Compromising personal information Compromising computers Boosting content Suppressing content Stealing content Email Spam Advertising Phishing attacks Distribute malware

4. Closed Loop Approach to Abuse Filter New Spam Attack Defeats Spam Defeats Filter Users Encounter Over Time Email Store Junk Store Email Senders Email Service User Interface Deploy New Model Labels trickle in Training Set

5. And what happens? New ML System still running, retraining regularly, no positive impact Go Live Developing new ML System Avg Inbox % Spam Avg Inbox % Spam Avg Inbox % Spam Avg Inbox % Spam P(Spam | Content) Feature engineering insightful Compare to existing model and bounds look great Small scale user testing success Engineering it to scale Take time, be sure 0 0 0 0 10 10 10 10 20 20 20 20 30 30 30 30 40 40 40 40 50 50 50 50 60 60 60 60 70 70 70 70 80 80 80 80 90 90 90 90 100 100 100 100 110 110 110 110 Day Day Day Day Spammers notice, put down Mai Thais, tweak scripts Amazing Success Total Failure

6. One Risk: Training on Test Data Spam Campaigns Total Mail Good Mail Data Train Test Time Randomly Sample Train/Test Data - Works if all data is independent - Leads to training / testing on same spam - Overoptimistic Partition by Time/Sender - Better but not perfect (some overlap) - Time gap measurability for accuracy - Important for accurate operating points Training Set Test Set

7. What Spammers Do Anti- Filter If inbox label 1 If not label 0 Use to craft next spam Spammer Training Set Filter Sends thousands of probes to owned accounts Defeats Filter? Scripted interactions Email Store Junk Store Email Senders Email Service User Interface Deploy New Model Labels trickle in Latency vs Scale Spammers in strong position Training Set

8. Why did Machine Learning Fail? Assumption: Data is Independent and Identically Distributed (I.I.D.) Data at training time = Data at runtime Spam Distribution New Model Spam Distribution New Model Spam Distribution After 9% 9% 9% 8% 8% 8% Avoid Feature Space Unselected Words Token Attacks Content in Images 7% 7% 7% 6% 6% 6% Percent of Spam Percent of Spam Percent of Spam 5% 5% 5% 4% 4% 4% 3% 3% 3% Avoid Model Samples never seen before Mimic good content Change quickly 2% 2% 2% 1% 1% 1% 0% 0% 0% Feature Representation Feature Representation Feature Representation

9. Abuse is a business ?????????????? > ???????????? Expected Return Expected Cost ?????????????? = ????????????????? ?????????????? ????? ???? ???????????? = ?????????????? + ????????????? + ????????????? Varies with Affiliate Marketing (many options) Compromised Accounts (a few bucks) sophistication, say 25% Small Fixed Cost ~Zero Marginal Cost Pretty low, maybe .5% IP Address to send spam from Web site to collect conversion Economics, lots of subtlety, but Net: Abuser makes about .1 cent per email in the inbox Net: Abuser needs ~1,000 in inbox per dollar on IP / Web site

10. So what is the role of machine learning? Content Filtering First obvious place to put machine learning Targets things that spammers can easily change Puts machine learning in a weak position Reputation Target the IPs spammers use to send mail Target the webhosts spammers collect conversions Target the things that cost spammers money

11. Closed Loop for Reputation Reputation Filter New Email Campaign Throttle Known Good? Content Filter Aggressive Users Encounter Over Time Junk Store Email Store Email Senders Email Service User Interface Deploy New Models Labels trickle in Attack things that cost abusers money - Block bad, throttle unknown Free known good senders from potential FPs - Known good bypass filtering - More aggressive filtering for unknown P(will have lots of complaints by tomorrow | history of sender behavior, user feedback) Content Training Set Reputation Training Set Goal: Identify good senders as fast as possible

12. Quick Preview of Intelligence Architectures Reputation Filter Filter Better Filter Known Good? Content Filter Junk Store Junk Store Email Store Email Store Junk Store Email Store P(Spam | Content) P(Spam | Content, Reputation) Some sort of hybrid system Criteria for deciding: Ability to use loophole in one system (content) to beat the other (reputation) Reduction in churn in mistakes (for known senders) as spammers change attacks Ability to partition work across multiple modelers / modeling teams Degree of coupling between models / information Ability to control with heuristics / business logic Requirements for immediate accuracy for long term accuracy (maintainability)

13. Summary of Adversarial Machine Learning Key assumption of ML is I.I.D. Almost always violated in practice Really always violated in adversarial settings Target things abusers have to do (and cost real money), not things they happen to be doing (and can change cheaply) Naively using ML for abuse Works if you re small (not targeted) Totally fails if you re big enough (targeted) General Lesson: The obvious way to use machine learning is not always the best way Abuse is business, real goal is to make abusers expect to lose money