Skip to content

Repository for the paper: "Improving the Question Answering Quality using Answer Candidate Filtering based on Natural-Language Features" @ ISKE 2021

Notifications You must be signed in to change notification settings

WSE-research/answer_validation

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

31 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Improving the Question Answering Quality using Answer Candidate Filtering based on Natural-Language Features

Abstract

Software with natural-language user interfaces has an ever-increasing importance. However, the quality of the included Question Answering (QA) functionality is still not sufficient regarding the number of questions that are answered correctly.

In our work, we address the research problem of how the QA quality of a given system can be improved just by evaluating the natural-language input (i.e., the user's question) and output (i.e., the system's answer).

Our main contribution is an approach capable of identifying wrong answers provided by a QA system. Hence, filtering incorrect answers from a list of answer candidates is leading to a highly improved QA quality. In particular, our approach has shown its potential while removing in many cases the majority of incorrect answers, which increases the QA quality significantly in comparison to the non-filtered output of a system.

Approach

The place of Answer Validation process in Knowledge Graph Question Answering

image

Our approach: to verbalize SPARQL query-candidates in order to use an Answer Validation Classifier

image

Results for Question Answering quality improvement

image

  • P@1: From 0.2476 to 0.4251 (i.e., by 71.7%) while using Approach 2 and to 0.2948 (i.e., by 19.1%) while using Approach 3.
  • P@5: from 0.1036 to 0.1368 (i.e., by 32.0%) while using Approach 2 and to 0.1183 (i.e., by 14.2%) while using Approach 3.
  • NDCG@5: from 0.3249 to 0.4698 (i.e., by 44.6%) while using Approach 2 and to 0.3787 (i.e., by 16.5%) while using Approach 3.

Authors

Aleksandr Gashkov Aleksandr Perevalov Maria Eltsova Andreas Both

Cite

To Be Done.

About

Repository for the paper: "Improving the Question Answering Quality using Answer Candidate Filtering based on Natural-Language Features" @ ISKE 2021

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 3

  •  
  •  
  •