Exploring NLP Techniques for Duplicate Question Detection to Maximizing Responses on Q&A Websites - Korade, Nilesh B.; Salunke, Mahendra B.; Asalkar, Gayatri G.; Khedkar, Rutuja G.; Bhosale, Ashwini U.; Joshi, Dhanashri M.; Jadhav, Amol C. (International Journal of Intelligent Systems and Applications in Engineering, 2024)
Emerging technologies known as Question Answering Systems (QAS) offer accurate and precise responses to common questions. Duplicate Question Detection (DQD) has demonstrated its capacity to enhance the user experience and drastically decrease response time by utilizing past responses. Word choice and sentence construction might differ significantly, making it difficult to determine if the two questions are asking the same thing. Finding questions on question-and-answer sites such as Quora, Stack Overflow, Blurtit, etc. that are semantically identical is very important to make sure that users receive both high-quality and high-quantity content according to the question’s purpose, improving the user experience entirely. Quora’s dataset of four lacks labelled question pairs used in the presented research. Our research has involved the construction of new features and the demonstration of their ability to improve accuracy. The study examines various vectorization methods and how they affect accuracy, with Word2Vec proving to be a good performer among the methods. In order to identify duplicate questions in the question pair dataset, we explored and used various machine learning and deep learning techniques. The cascaded CNN outperforms other modern algorithms and offers outstanding value over all assessment metrics.