Feature Extraction for Token Based Word Alignment for Question Answering Systems

• Autores: Dilip Kumar Sharma, Namita Mittal, Anubhav Agarwal
• Localización: Computación y Sistemas (CyS), ISSN 1405-5546, ISSN-e 2007-9737, Vol. 22, Nº. 4, 2018, págs. 1359-1366
• Idioma: inglés
• DOI: 10.13053/cys-22-4-3070
• Enlaces
  • Texto completo
• Resumen

  • Abstract: Mapping between the source words and the target words in a set of parallel sentences are a crucial part of Question Answering (QA) systems. If an accurate aligner is used in QA systems then the efficiency of these systems also gets increased. We purpose the aligner which despite using very less lexical resources gives very good results in terms of precision, recall and F1. Previous aligners either uses more lexical resources or uses very less lexical resources. Hence, we have used POS TAG and WordNet as lexical resources. But some words whose meaning we may not know but these occur in a similar distribution and by observing their distribution these words are similar. Consider two sentences ”Lambodar is the son of Parvati” and ”Ganesha is the son of Parvati”. Here we will not find the meaning of Lambodar and Ganesha in Wordnet but since they have similar distributions so they should be aligned. For these words, we used Distribution Similarity Feature in our word aligner. This distributional similarity helps our aligner in broader coverage of words. Previous aligners were having recall in the range of 75-86 but this aligner has recall in the range of 88.4-93.3. Similarly, Exact match of previous aligners was in the range of 21-35.3 but the proposed aligner’s exact match range is 46.1-58.6. Similarly F-measure and precision have also increased.

• Referencias bibliográficas
  • Blunsom, P.,Cohn., T.. (2006). Discriminative word alignment with conditional random fields. Proceedings of ACL.
  • Chambers, N.,Cer, D.,Grenager, T.,Hall, D.,Kiddon, C.,MacCartney, B.,de Marneffe, M. C.,Ramage, D.,Yeh, E.,Manning, C. D.. (2007). Learning...
  • De Marneffe, M.-C.,Manning, C.. (2008). The stanford typed dependencies representation. workshop on Cross-Framework and Cross-Domain Parser...
  • Gimpel, K.,Smith, N. A.. (2010). Soft max margin CRFs: training log-linear models with cost functions. NAACL.
  • Kumar Sharma, L.,Mittal, N.. (2017). Prominent feature extraction for evidence gathering in question answering. Journal of Intelligent and...
  • Lafferty, J. D.,McCallum, A.,Pereira, F. C. N.. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence...
  • MacCartney, B.,Galley, M.,Manning, C. D.. (2008). A phrase-based alignment model for natural language inference. EMNLP.
  • MacCartney, B.,Manning, C.. (2008). Modeling semantic containment and exclusion in natural language inference. ACL.
  • McCallum, A.,Bellare, K.,Pereira, F.. (2005). A conditional random field for discriminatively -trained finite-state string edit distance....
  • Thadani, K.,McKeown, K.. (2011). Optimal and syntactically-informed decoding for monolingual phrase-based alignment. ACL.
  • Wan, S.,Dras, M.,Dale, R.,Paris, C.. (2006). Using dependency-based features to take the para-farce out of paraphrase. Australasian Language...
  • Yao, X.,Van Durme, B.,CallisonBurch, C.,Clark, P.. (2013). A lightweight and high performance monolingual word aligner. ACL.