Leveraging Crowdsourcing for Dynamic Answer Validation and Generation in Time-Constrained Conversational AI Systems

While modern AI systems have made substantial progress in addressing factual queries, the ability to handle complex, opinion-based, or situational questions remains a significant hurdle. In the context of conversational agents, realtime question answering (QA) has become a crucial capability. This paper investigates the role of crowdsourcing in improving the efficacy of QA systems, specifically in time-sensitive environments. We explore two approaches: using crowdsourced workers to validate candidate answers rapidly and utilizing crowdsourcing for direct answer generation under strict time constraints. Our experiments demonstrate that crowdsourced workers can effectively assess the quality of answers within a one-minute limit, providing valuable input that helps rank or filter the most appropriate responses. Furthermore, we find that answers generated by crowdsourcing outperform automated systems, presenting a feasible method for hybridizing human intelligence with AI to enhance the overall performance of realtime QA systems. This study contributes to the development of conversational agents capable of providing more reliable and contextually accurate responses in real-world scenarios.