Question Answering System for SQuAD
Zhongjie Li, Xuandong Lei, Lu Bian
Introduction
Question answering (QA) is a system that can pull
answers from an unstructured collection of natural
language documents. Its a crucial task that requires both
natural language understanding and processing.
We are using Stanford Question Answering Dataset
(SQuAD), which predicts an answer span as a starting
and ending indices in context. Results are evaluated with
softmax loss, F1 score and EM score.
Context: .... in adverse weather conditions , ice and
snow clearing equipment can be used to improve
traction on the landing strip...
Question: in adverse weather conditions , what can be
used to improve traction on the landing strip ?
Answer: ice and snow clearing equipment
Dataset
Glove Vectors for Word
Representation
Training: 81387 triplets of
(Question, Context,
Answers)
Validation: 4285 triplets of
(Question, Context,
Answers)
Max length of context: 766
Max length of question: 60
99.9% length of context:
449
99.9% length of question:
30
If want to fix the length of
paragraph and question in
the model, we can pick
450 and 30 as their
lengths
Baseline Model Co-attention Model
The model includes an
encoder that understand
the meaning of question
and paragraph, and a
decoder to predict the
answers starting and
ending indices based on
the understanding.
T h e b a s e l i n e mo d e l
implements a sequence
attention mix model.
Bi-Directional Attention Model
Experiment Evaluation
Predictions: p(answer_start), p(answer_end)
Loss: CE(y_s, y_s) + CE(y_e, y_e)
Bi-directional attention works best, it has
more hidden layers
Future Goals
Dynamic Pointing Decoder: There may exist several answer spans, each corresponding to a local
maxima. Using dynamic pointing decoder which alternates between estimating the start and end of the
Similarity Matrix: S = PWQT, use W to learn the similarity between P and Q answer span, enables model to recover from initial local maxima.
Paragraph-to-Question Attention: which question word are most relevant to each paragraph Encoder: Use natural language sentence matching (NLSM) to assess the relevance between
word, P = softmax(S)P paragraphs and questions
Question-to-Paragraph Attention: which paragraph words have most similarity to each query
word and can be used to answer the question [1]Xiong C. et. al. Dynamic coattention networks for question answering. arXiv preprint 2016.
Q = QTsoftmax(maxcol(S)) [21] Wang. Z. et. al. Bilateral multi-perspective matching for natural language sentences. arXiv preprint
2017.