Entity-based Memory Network for Text Comprehension
Entity-based Memory Network is a model designed for text comprehension, focusing on question-answering tasks in both open and closed domain QA. It incorporates features like distributed representation, feature modeling, and memory models at various levels to generate output features, predict answers, and update entity states. The network utilizes pre-trained word vectors and modular components for sentence reconstruction and entity-related tasks through iterations, aiming to enhance comprehension and information retrieval in natural language processing.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Entity-based Memory Network for Text Comprehension Wang Xun 2016/10/07
Text Comprehension Text Comprehension The ability to read text, process it, and understand its meaning Question answering ?: ????,???????? {?????} Open domain QA / Closed domain QA Formulating NLP tasks as a QA problem POS tagging Q: What are the parts of speech? Co-reference resolution Q: What does XX refer to? Xxx xxx Closed domain QA Open domain QA Closed domain QA
answer Memory Network model Distributed Representation answer Unified Feature Representation feature model feature feature model answer Raw text
Memory Models LSTM/GRU, neuron level memory
Memory Network, sentence level memory (with attentions)
Entity-based Memory Network Generate output feature Predict the answer Extracted entities Answer Update states of entities States of entities (Memories) Response Modular Question Input sentences Generation Modular Retrieve related memories Current Sentence Output Feature Modular Input Modular
Entity-based Memory Network Autoencoder Mary moved to the bathroom . Mary moved to the bathroom . S S Reconstruct Sentence Using Entities Pre-trained Word Vectors Mary bathroom ?:? = ??????????? ? While (1): ? = ? ??? ? , ???? ? = ?(??? ? , ??? ???? ) ???????? ||? ? ||2 Return [ Mary , bathroom ],f
Entity-based Memory Network Mary moved to the bathroom . S Pre-trained Word Vectors Mary bathroom
Entity-based Memory Network ?(??)?= ?( ?,??) Scores mth iteration Scores -- 1st iteration Scores 2nd iteration Entity states ? = ( ,?) Output feature vector Update Update Update = ? = (???( ,? ?)) Question
Full supervision Weak supervision
Experiments bAbI 20 topics Toy data, text advanture Path finding/position reasoning .36/.65 (MNN) .35/.60 (DMNN) .53/.67 (EMNN)
Large Movie Review Dataset 50,000 reviews from IMDB, about 30 reviews per movie. Scores from 1(---) to 10 (+++) FUTZ is the only show preserved from the experimental theatre movement in New York in the 1960s (the origins of Off Off Broadway). Though it s not for everyone, it is a genuinely brilliant, darkly funny, even more often deeply disturbing tale about love, sex, personal liberty, and revenge, a serious morality tale even more relevant now in a time when Congress wants to outlaw gay marriage by trashing our Constitution. The story is not about being gay, though -- it s about love and sex that don t conform to social norms and (-):1-4 / (+):7-10 Q: What does the reviewer think about the movie? A: Positive/Negative. 97.2 (previous work: 89/93.4/95)
Machine Comprehension Test 500 stories and 2000 questions Most options contain one or two words. No world knowledge is required. multiple choices questions as QA options as related entities/full supervision 683 questions Acc.=.73+ (.60)
Once upon a time there was a princess who lived in a high tower and she was not allowed to leave because of her mean mother. One day she chose to leave but her mother would not let her. The princess climbed out the window of the high tower and climbed down the south wall when her mother was sleeping. She wandered out a good ways. Finally she went into the forest where there are no electric poles but where there are some caves. 1: Where did the princess wander to after escaping? A) Mountain *B) Forest C) Cave D) Castle 2: Who escaped from the tower? A) Mother *B) Princess C) Man D) John
bAbI Acc of >95 for 18 of 20 topics Toy data, text adventure game, not convincing Large Movie Review Dataset Closed domain Machine Comprehension Test Children Stories Open domain
