Christopher Sciavolino will present his MSE Talk "Shortcomings of Dense Retrievers for Open-domain Question Answering" on Wednesday, April 28, 2021 at 1:30PM via Zoom.

Zoom Link: https://princeton.zoom.us/j/99489245546

Committee: Danqi Chen (adviser), Karthik Narasimhan (reader)

All are welcome to attend.

Abstract: Dense retrieval mechanisms have exploded in popularity for open-domain question answering (QA), usurping sparse retrieval mechanisms like TF-IDF and BM25 as the defacto retrieval method. While dense models do an excellent job of understanding the semantic intent of questions, we empirically observe they lack the ability to capture precise lexical information and generalize to unseen datasets. We construct a QA dataset based on T-REx and observe that dense models underperform sparse models on even simple questions like "Where was [X] born?". We also note that dense models degrade in performance when evaluated on new question distributions not seen during training. We finally perform analyses and experiments aimed at closing this dense model generalization gap.