Understanding Causality in News Event Prediction
Learning about the significance of predictions in news events and the process of causality mining for accurate forecasting. The research delves into problem definition, solution representation, algorithms, and evaluation in event prediction. Emphasis is placed on events, time representation, prediction functions, and the extraction of causality relations from news articles for state inference.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Learning Causality for News Events Prediction Kira Radinsky, Sagie Davidovich, Shaul Markovitch Technion - Israel Institute of Technology
What is Prediction? a rigorous, often quantitative, statement, forecasting what will happen under specific conditions. [Wikipedia] A description of what one thinks will take place in the future, based on previous knowledge. [Online Dictionary]
Why is News Event Prediction Important? Event Predicted event (Pundit) Al-Qaida demands hostage exchange A country will refuse the demand Strategic Intelligence Volcano erupts in Democratic Republic of Congo Thousands of people flee from Congo Strategic planning 7.0 magnitude earthquake strikes Haitian coast Tsunami-warning is issued Strategic planning China overtakes Germany as world's biggest exporter Wheat price will fall Financial investments
Outline Motivation Problem definition Solution Representation Algorithm Evaluation
Problem Definition: Events Prediction Ev is a set of events T is discrete representation of time Prediction Function , s.t.: occurred at time occurred at time
Outline Motivation Problem definition Solution Representation Algorithm Evaluation
Causality Mining Process: Overview News Articles acquisition Crawling [NYT 1851-2009] Modeling & Normalization Causality Pattern Classification <Pattern, Constraint, Confidence> Event Extraction Tagging Dependency parsing (Stanford parser) Causality Relations extraction Context inference Thematic roles normalized Base forms URIs assignment (Contextual Disambiguation) Thematic roles assignment Based on VerbNet Index Causality Graph Building Built on 20 machines 300 million nodes 1 billion edges 13 million news articles in total State Inference
Outline Motivation Problem definition Solution Representation Algorithm Evaluation
Modeling an Event Comparison between events (Canonical) 1. (Lexicon & Syntax) Language & wording independent 2. (Semantic) Non ambiguous Generalization / abstraction Reasoning Many philosophies Property Exemplification of Events theory (Kim 1993) Conceptual Dependency theory (Schank 1972)
Time Event & Causality Representation Event Representation Causality Representation 5 Quantifier kill Troops Action Attribute Afghan 1/2/1987 11:15AM +(3h) Time- frame Event2 5 Afghan troops were killed Army bombs Weapon warehouse US US Army Action Time- frame Location 1/2/1987 11:00AM +(2h) Event1 Kabul US Army bombs a weapon warehouse in Kabul with missiles Missiles
Outline Motivation Problem definition Solution Representation Algorithm Causality Mining Process Evaluation
Machine Learning Problem Definition Goal function: Learning algorithm receives a set of examples and produces a hypothesis which is good approximation of
Algorithm Outline Learning Phase 1. Generalize events 2. Causality prediction rule generation Prediction Phase 1. Finding similar generalized event 2. Application of causality prediction rule
Algorithm Outline Learning Phase 1. Generalize events 1. How do we generalize objects? 2. How do we generalize actions? 3. How do we generalize an event? 2. Causality prediction rule generation
Generalizing Objects the Russian Federation Eastern Europe RUS Russian Federation Land border China 185 Russia USSR Rouble (Rub) 643
Generalizing Actions Levin classes (Levin 1993) 270 classes Class Hit-18.1 Roles and Restrictions: Agent[+int_control] Patient[+concrete] Instrument[+concrete] Members: bang, bash, hit, kick, ... Frames: Name Example Syntax Semantics cause(Agent, E)manner(during(E), directedmotion, Agent) !contact(during(E), Agent, Patient) manner(end(E),forceful, Agent) contact(end(E), Agent, Patient) Basic Transitive Paula hit the ball Agent V Patient
Generalizing Events: Putting it all together NATO strikes an army base in Baghdad US NATO Location Present Event 1/2/1987 11:00AM +(2h) Time-frame Baghdad Generalization rule Action Army base Actor: [state of Nato] Property: [Hit1.1] Theme: [Military facility] Location: [Arab City] Country Army strikes Military facility Similar verb City Weapon warehouse bombs US Army Action Location Past Event Time-frame 1/2/1987 11:00AM +(2h) Kabul US Army bombs a weapon warehouse in Kabul with missiles Instrument rdf:type Missiles
Generalizing Events: Event distance metric NATO strikes an army base in Baghdad US NATO Location Present Event 1/2/1987 11:00AM +(2h) Time-frame Baghdad Action Army base Country Army strikes Military facility Similar verb City Weapon warehouse bombs US Army Action Location Past Event Time-frame 1/2/1987 11:00AM +(2h) Kabul US Army bombs a weapon warehouse in Kabul with missiles Instrument rdf:type Missiles
Algorithm Outline Learning Phase 1. Generalize events 2. Causality prediction rule generation
Prediction Rule Generation Time 5 Quantifier kill Troops Action Attribute 5 Afghan troops were killed Afghan Effect Event 1/2/1987 11:15AM +(3h) Time- frame Nationality Afghanistan Army Country Type Type bombs Weapon warehouse Country US US Army Action Cause Event Time- frame 1/2/1987 11:00AM +(2h) Kabul Location Effect Theme Attribute = Cause Location Country Nationality Effect Action=kill Effect Theme=Troops US Army bombs a weapon warehouse in Kabul with missiles Missiles
Algorithm Outline Prediction Phase 1. Finding similar generalized event 2. Application of causality prediction rule
Finding Similar Generalized Event 0.2 Baghdad bombing 0.7 0.3 0.8 0.65 0.2 0.1 0.75
Prediction Rule Application Time kill Troops Action Attribute Predicted Effect Event Time- frame T1+ Nationality bomb Theme1 Country Actor1 Action Input Event Time- frame Location1 T1 Location Effect Theme Attribute = Cause Location Country Nationality Effect Action=kill Effect Theme=Troops Instrument1
Outline Motivation Problem definition Solution Representation Algorithm Evaluation
Prediction Evaluation Human Group 1: Mark events E that can cause other events. Human Group 2: Given: Random sample of events from E , predictions and time of events Search the web and give estimation on the prediction accuracy
Prediction Accuracy Results Highly certain Certain Algorithm 0.58 0.49 Humans 0.4 0.38
Causality Evaluation Human Group 1: Mark events E for test for the second two control groups and the algorithm. Human Group 2: Given: Random sample of events from E. State what you think would happen following this event. Human Group 3: Given: algorithm predictions + human (2nd group) predictions Evaluate the quality of the predictions
Causality Results [0,1) [1-2) [2-3) [3,4] Avg. Rank Avg. Accuracy Algorithm 0 2 19 29 3.08 77% Humans 0 3 24 23 2.86 72% The results are statistically significant
Event Predicted Event (Human) Predicted event (Pundit) Al-Qaida demands hostage exchange Al-Qaida exchanges hostage A country will refuse the demand Volcano erupts in Democratic Republic of Congo Scientists in Republic of Congo investigate lava beds Thousands of people flee from Congo 7.0 magnitude earthquake strikes Haitian coast Tsunami in Haiti effects coast Tsunami-warning is issued 2 Palestinians reportedly shot dead by Israeli troops Israeli citizens protest against Palestinian leaders War will be waged Professor of Tehran University killed in bombing Tehran students remember slain professor in memorial service Professor funeral will be held Alleged drug kingpin arrested in Mexico Mafia kills people with guns in town Kingpin will be sent to prison UK bans Islamist group Islamist group would adopt another name in the UK Group will grow China overtakes Germany as world's biggest exporter German officials suspend tariffs Wheat price will fall
Accuracy of Extraction Extraction Evaluation Action Actor Object Instrument Location Time 93% 74% 76% 79% 79% 100% Entity Ontology Matching Actor Object Instrument Location 84% 83% 79% 89%
Related work Causality Information Extraction Goal: Extract causality relations from a text Techniques: 1. Usage of handcrafted domain-specific patterns [Kaplan and Berry-Rogghe, 1991] 2. Usage of handcrafted linguistic patterns [Garcia 1997],[Khoo, Chan, &Niu 2000], [Girju &Moldovan 2002] 3. Semi-Supervised pattern learning approaches, based on text features [Blanco, Castell, &Moldovan 2008], [Sil & Huang & Yates 2010] 4. Supervised pattern learning approaches based on text features [Riloff 1996],[Riloff & Jones 1999], [Agichtein & Gravano, 2000; Lin & Pantel, 2001]
Related work Temporal Information Extraction Goal: Predicting the temporal order of events or time expressions described in text Technique: learn classifiers that predict a temporal order of a pair of events based on a predefined features of the pair. [Ling & Weld, 2010; Mani, Schiffman, & Zhang, 2003; Lapata & Lascarides,2006; Chambers, Wang, & Jurafsky, 2007; Tatu & Srikanth, 2008; Yoshikawa, Riedel, Asahara, & Matsumoto, 2009]
Future work Going beyond human tagged examples Incorporating time into the equation When will correlation mean causality? Using other sources than news Incorporating real time data (Twitter, Facebook) Incorporating numerical data (Stocks, Weather, Forex) Can we predict general facts? Can a machine predict better than an expert?
Summary Canonical event representation Machine learning algorithm for events prediction Leveraging world knowledge for generalization Using text as human tagged examples Causality mining from text Contribution to machine common-sense understanding The best way to predict the future is to invent it [Alan Kay]
