Scientist AI: An agentic AI to evaluate research ideas proposed by commonly available AI chatbots

Scientist AI: An agentic AI to evaluate research ideasproposed by commonly available AI chatbots The growing use of AI-powered chatbots to generate research ideas presents new challenges for the academic ecosystem. While these tools offer rapid ideation and broad knowledge access, the ideas they produce often lack standardized evaluation, transparency, and safeguards against hidden risks such as geopolitical bias, unexplainability, and inconsistency. Academic authorities, journals, and conference publication houses currently have no reliable mechanism to assess the creativity, dataset suitability, or overall quality of AI-generated research ideas before they enter the academic pipeline. This absence of structured validation creates inefficiencies and exposes the academic community to potential reliability and integrity issues. There is a critical need for a systematic, human-centric approach to filter, score, and monitor AI-generated research ideas to ensure both operational efficiency and academic rigor.IntroductionRecent advances in generative AI have transformed the way researchers, students, and practitioners conceptualize new research ideas. AI chatbots are increasingly being used as ideation partners, capable of producing a wide range of research suggestions within seconds. However, this rapid generation of ideas introduces new complexities for the academic and print-media industries that oversee scholarly publishing. Publication houses, journals, and conference committees must evaluate not only human-created submissions but also AI-assisted or AI-generated ideas—yet existing frameworks do not adequately address the quality, creativity, or reliability of such outputs.To address these challenges, we propose ASK, an agentic AI system designed to evaluate research ideas recommended by commonly available AI chatbots. ASK assesses each idea using three key metrics: (i) a creativity index, (ii) recommended dataset suitability, and (iii) user satisfaction score. In addition, ASK provides an analytics dashboard enabling academic authorities and publication houses to compare the creativity of AI-generated ideas and rank them accordingly. By serving as a structured funnel, ASK mitigates catastrophic risks associated with generative AI—such as geopolitical bias, unexplainability, and inconsistent reliability—while enhancing operational efficiency and preserving human-centric oversight.