EVALUASI PROGRAM AI CHATBOT ANSIA DI PERGURUAN TINGGI: ANALISIS MODEL CIPP (CONTEXT, INPUT, PROCESS, PRODUCT)

This study aims to evaluate ANSIA (SIDIA Assistant), an artificial intelligence -based chatbot program at Universitas Negeri Surabaya, using the CIPP (Context, Input, Process, Product) evaluation model. A descriptive-evaluative qualitative approach was employed. Data were collected through in-depth interviews and document analysis involving developers from the Directorate of Digital Learning Innovation (DIPD) at UNESA as key informants, as well as six lecturers and 19 students from the Office Administration Education undergraduate study program, cohorts 2024 and 2025, as primary informants. Data credibility was examined through source triangulation, and the data were analyzed using the Miles and Huberman interactive model. The evaluation revealed four major gaps. In the context component, ANSIA was developed without a formal needs assessment, resulting in a gap in user awareness. In the input component, server infrastructure and WhatsApp integration were not adequately supported by a dedicated budget, formal socialization, or sufficient operational guidelines. In the process component, an increase in aggregate server-log access did not fully reflect student use of ANSIA at the study-program level. Interview findings indicated that preferences for commercial artificial intelligence tools and limitations in the internal knowledge base were factors associated with low utilization. In the product component, a discrepancy emerged between developers’ claims of 24-hour service availability and users’ experiences in accessing and using the system. The study recommends expanding the scientific knowledge base, providing visual user guides, allocating a dedicated budget, and integrating ANSIA into teaching and learning activities. ABSTRAK Penelitian ini bertujuan mengevaluasi program chatbot berbasis artificial intelligence, yaitu ANSIA (Asisten SIDIA), di Universitas Negeri Surabaya menggunakan model evaluasi CIPP (Context, Input, Process, Product). Penelitian menggunakan pendekatan kualitatif dengan desain deskriptif evaluatif. Data dikumpulkan melalui wawancara mendalam dan dokumentasi yang melibatkan pengembang dari Direktorat Inovasi Pembelajaran Digital (DIPD) UNESA sebagai informan kunci serta enam dosen dan 19 mahasiswa Program Studi S1 Pendidikan Administrasi Perkantoran angkatan 2024 dan 2025 sebagai informan utama. Keabsahan data diperiksa melalui triangulasi sumber, sedangkan analisis data menggunakan model interaktif Miles dan Huberman. Hasil evaluasi menunjukkan empat kesenjangan utama. Pada komponen context, pengembangan ANSIA tidak didahului asesmen kebutuhan formal sehingga ditemukan kesenjangan kesadaran (awareness gap) pengguna. Pada komponen input, dukungan infrastruktur server dan integrasi WhatsApp belum diimbangi anggaran khusus, sosialisasi formal, serta panduan operasional yang memadai. Pada komponen process, peningkatan jumlah akses berdasarkan log server agregat tidak sepenuhnya mencerminkan pemanfaatan ANSIA oleh mahasiswa di tingkat program studi. Temuan wawancara menunjukkan bahwa kecenderungan menggunakan artificial intelligence komersial dan keterbatasan basis pengetahuan internal menjadi faktor yang berkaitan dengan rendahnya pemanfaatan tersebut. Pada komponen product, terdapat perbedaan antara klaim pengembang mengenai ketersediaan layanan selama 24 jam dan pengalaman pengguna dalam mengakses serta memanfaatkan sistem. Penelitian ini merekomendasikan perluasan basis pengetahuan ilmiah, penyediaan panduan visual, pengalokasian anggaran khusus, dan integrasi ANSIA dalam kegiatan pembelajaran.