Enhancing Open Information Extraction with Focused Entailment Graphs

Explore how Focused Entailment Graphs improve Open Information Extraction (Open IE) by structuring propositions and their entailment relations. These graphs help consolidate natural language expressions like "relieve headache" and "treat headache" to organize data hierarchically for better understanding. By adding structure to Open IE, paraphrases can be merged into mutual entailment cliques, facilitating the extraction of valuable information.

• Open IE
• Entailment Graphs
• Structured Data
• Natural Language Processing
• Information Extraction

tanyahr Follow

Uploaded on Sep 19, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

Presentation Transcript

1. Focused Entailment Graphs for Open IE Propositions Omer Levy Ido Dagan Bar-Ilan University, Israel Jacob Goldberger

2. Open IE Extracts propositions from text which makes aspirin relieve headaches. ???????,???????, ????? ? No supervision No pre-defined schema

3. Whats missing in Open IE? Structure Open IE does not consolidate natural language expressions relieveheadache treatheadache

4. Adding Structure to Open IE Which structure? Build a graph of Open IE propositions and their semantic relations

5. Adding Structure to Open IE Which structure? Build a graph of Open IE propositions and their entailment relations Why entailment? Merges paraphrases into mutual entailment cliques aspirin relievesheadache aspirin treatsheadache Organizes information hierarchically from specific to general aspirin relievesheadache painkiller relieves headache

6. aspirin, eliminate, headache aspirin, cure, headache coffee, help, headache drug, relieve, headache headache, control with, aspirin drug, treat, headache tea, soothe, headache analgesic, banish, headache headache, respond to, painkiller headache, treat with, caffeine Original Open IE Output

7. drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache tea, soothe, headache headache, control with, aspirin aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache Consolidated Open IE Output

8. Semantic Applications Example: Structured Queries What relieves headaches?

9. Semantic Applications Example: Structured Queries What relieves headaches? (?,???????, ????? ?)

10. drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache tea, soothe, headache headache, control with, aspirin aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache Structured Query:(?,???????, ????? ?)

11. drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache tea, soothe, headache headache, control with, aspirin aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache Structured Query:(?,???????, ????? ?)

12. drug painkiller caffeine analgesic tea aspirin coffee Structured Query:(?,???????, ????? ?)

13. Our Contributions Structuring Open IE withProposition Entailment Graphs Dataset: 30 gold-standard graphs, 1.5 million entailment annotations Algorithm for constructing Focused Proposition Entailment Graphs Analysis: Predicate entailment is not quite what we thought

14. Proposition Entailment Graphs

15. Related Work: Predicate Predicate Entailment Graphs Berant et al. (2010,2011,2012) We extend Berant et al. s work from predicates to propositions

16. Focused Proposition Entailment Graphs Nodes: Open IE propositions Edges: Textual Entailment

17. Focused Proposition Entailment Graphs Assumptions: Binary Propositions and Common Topic Binary Propositions 1,??,?? 2 ??= ?? Focused on a common topic ? ??= (?,??,??) ??= (??,??,?)

18. Focused Proposition Entailment Graphs Assumptions: Binary Propositions and Common Topic Binary Propositions 1,??,?? 2 ??= ?? Focused on a common topic ? = ????? ? ??= (?,??,??) ????? ?,????? ??? ,?????????? ??= (??,??,?) ???????,???????, ????? ?

19. drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache tea, soothe, headache headache, control with, aspirin aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache

20. drug, relieve, headache drug, treat, headache headache, respond to, painkiller headache, treat with, caffeine analgesic, banish, headache tea, soothe, headache headache, control with, aspirin aspirin, cure, headache aspirin, eliminate, headache coffee, help, headache

21. Focused Proposition Entailment Graphs Edges: Textual Entailment ??= (?,??,??) ??= (??,??,?) Proposition Entailment Simpler than sentence-level entailment More complicated than lexical entailment Enables investigation of inference phenomena in an isolated manner

22. Constructing Proposition Entailment Graphs Task Definition: Given a set of propositions ??, find all their entailment edges.

23. Dataset

24. Dataset: High-Quality Open IE Propositions Google s Syntactic N-grams Based on millions of books Filter for subject-verb-object Including prepositional objects and passive Result: 68 million high-quality propositions

25. Dataset: Annotating Entailment Graphs Select 30 healthcare topics antibiotic, caffeine, insomnia, scurvy, Collect a set of propositions focused on each topic Manually clean noisy extractions Retaining ~200 propositions per graph (average) Efficiently annotate entailment 1.5 million entailment judgments

26. Algorithm

27. How do we recognize proposition entailment? ??? ?????, ?????, ???? . ????? ? ????? ??? ?????,????????? ??? ,???????

28. How do we recognize proposition entailment? ??? ?????, ?????, ???? . ? ?? ? ???? ??? ?????,????????? ??? ,??????? ?? Observation: propositions entail their lexical components entail

29. How do we recognize proposition entailment? ??? ?????, ?????, ???? . ? ?? ? ?? ??? ?????,????????? ??? ,??????? ?? ?? Observation: propositions entail their lexical components entail

30. How do we recognize proposition entailment? ??? ?????, ?????, ???? . ? ?? ? ?? ??? ?????,????????? ??? ,??????? ?? ?? Proposition entailment is reduced to lexical entailment in context

31. Lexical Entailment Lexical Entailment Features ?1 ?2 ?3 Lexical Entailment (Logistic) ? ? = ? ? ?

32. Lexical Entailment Lexical Entailment Features Features WordNet Relations UMLS Distributional Similarity String Edit Distance ?1 ?2 ?3 Lexical Entailment (Logistic) ? ? = ? ? ? Supervision

33. From Lexical to Proposition Entailment Lexical Entailment Features ?1 ?2 ?3 Lexical Entailment (Logistic) ? ? = ? ? ? Supervision

34. From Lexical to Proposition Entailment Predicate Entailment Features Argument Entailment Features ??1 ??2 ??3 ??1 ??2 ??3 Predicate Entailment (Logistic) Argument Entailment (Logistic) ? ? ? = ? ?? ?? ? = ? ?? ?? Supervision Supervision

35. From Lexical to Proposition Entailment Predicate Entailment Features Argument Entailment Features ??1 ??2 ??3 ??1 ??2 ??3 Predicate Entailment (Logistic) Argument Entailment (Logistic) ? ? ? = ? ?? ?? ? = ? ?? ?? Supervision Supervision Proposition Entailment (Conjunction) ? ? = ? ?

36. Distant Supervision (WordNet)? Predicate Entailment Features Argument Entailment Features ??1 ??2 ??3 ??1 ??2 ??3 Predicate Entailment (Logistic) Argument Entailment (Logistic) ? ? ? = ? ?? ?? ? = ? ?? ?? WordNet WordNet Proposition Entailment (Conjunction) ? ? = ? ? Following Snow (2005), Berant (2012)

37. Direct Supervision (30 Annotated Graphs) Predicate Entailment Features Argument Entailment Features ??1 ??2 ??3 ??1 ??2 ??3 Predicate Entailment (Logistic) Argument Entailment (Logistic) ? ? ? = ? ?? ?? ? = ? ?? ?? Proposition Entailment (Conjunction) ? ? = ? ? Annotated Graphs

38. Direct Supervision (30 Annotated Graphs) Predicate Entailment Features Argument Entailment Features ??1 ??2 ??3 ??1 ??2 ??3 ? ? Hidden Layer Proposition Entailment (Conjunction) ? ? = ? ? Annotated Graphs

39. Flat Model Predicate Entailment Features Argument Entailment Features ??1 ??2 ??3 ??1 ??2 ??3 Proposition Entailment (Logistic) ? ? = ? ?? ??+ ?? ?? Annotated Graphs

40. Compared Methods Component-Level Distant Supervision (WordNet) Predicates & Arguments Predicates Only Arguments Only Proposition-Level Direct Supervision (30 Annotated Graphs) Hierarchical (our method) Flat All methods used Berant et al. s Global Optimization method

41. Results

42. Direct Supervision: Flat vs Hierarchical Hierarchal model performs better than flat model 70% 65% Better to model predicate and argument entailment separately Performance (F1) 60% Hierarchical (Our Method) Flat 55% 61.6% 63.7% 50%

43. Distant vs Direct Supervision Direct supervision is better 70% Although WordNet provides more training examples 65% Performance (F1) 60% Hierarchical (Our Method) Best Distant (Arguments Only) Flat 63.7% 55% 61.6% 59.7% 50%

44. Predicate Entailment with Distant Supervision Ignoring predicates improves distant supervision baselines 70% 60% 50% Performance (F1) 40% Arguments Only 30% 59.7% Predicates & Arguments 20% Predicates Only 10% 7.2% 8.0% 0%

45. Are WordNet relations capturing real-world predicate entailments?

46. Predicate Entailment vs WordNet Relations Over a predicate inference subset, how many predicate entailments are covered by WordNet? Positive 12% Negative 15% Positive indicators synonyms, hypernyms, entailment None 74%

47. Predicate Entailment vs WordNet Relations Over a predicate inference subset, how many predicate entailments are covered by WordNet? Positive 12% Negative 15% Positive indicators synonyms, hypernyms, entailment None 74% Negative Indicators antonyms, hyponyms, cohyponyms Why isn t WordNet capturing predicate entailment?

48. Predicate Entailment is Context Context- -Sensitive Sensitive The words do not necessarily entail, but the situations do. ???? ???? ????? ?????? ????? ??????

49. Predicate Entailment is Context Context- -Sensitive Sensitive The words do not necessarily entail, but the situations do. ??????? ?? ??????? ?? ????????? ???????? ??????????? ????????? ?? ??????? ?? ???????????

50. Investigating Context Context- -Sensitive Sensitive Entailment Recent work on context-sensitive lexical inference e.g. (Melamud et al., 2013) Previous datasets Lexical substitution (McCarthy and Navigli, 2007) Predicate inference (Zeichner et al., 2012) We offer a new dataset of real-world lexical entailments in context! Sample: synthetic vs naturally occurring Size: several thousands vs 1.5 million

Load More ...