AI Assistance Variants in Software Development Cycles

Aim/Purpose With the technology of artificial intelligence (AI) improving every day it is important to find ways to harness AI in the software development life cycle (SDLC). This research demonstrates how AI tools were incorporated into an upper division Computer Science course to assist with development of various memory games. Background Since ChatGPT’s release in 2022, other companies have released rival chatbots each competing for a piece of the new market. With the plethora of AI options now available, it is important for a developer to learn to use AI as an assistant within the development of a custom project. Methodology The research presented is a multi-case, cross-analysis of four student researchers in a required, senior level Computer Science course. All students were tasked with collecting mixed-methods data on two AI assistants, throughout design and development a unique memory app; then these four students pooled data and conducted a cross-comparative analysis. To prepare for cross analysis, standardized Likert rankings and thematic categories were developed and consistently used during data collections. AI assistants evaluated: Claude, Copilot, ChatGPT Free, and ChatGPT Paid. Throughout the development process, each student provided both of their AI assistants with the same initial queries, the results of which were given a Likert ranking and notes were kept regarding AI accuracy. Individual datasets were examined, then pooled and the combined dataset was used to finalize hypothesis findings. The four student-researchers presented their multi-case, mixed-methods analysis as a snap-shot in time regarding the value of AI as assistants in the development of their projects. Contribution This paper builds on prior research focusing both on student experience and instructional methods in capstone-like courses. This study examines using AIs as assistants as a current trend in Computer Science education. Findings During multi-case analysis, two hypotheses were analyzed against the data of the four student-researchers. The cross examination of data found no statistical significance between the helpfulness of paid vs. free AI as course project assistants; while non-IDE AI assistants performed significantly better than IDE assistants across 7 out of 8 usage type categories. Recommendations for Practitioners Technology instructors can use this research to incorporate AI assistants into advanced courses that focus on building custom software, with cautions that foundational coding skills and knowledge should be in place prior to attempting complex projects. Companies that are researching how AI can be integrated into the software development process can use this research to see preferred strengths of various AI’s, with cautions for use with proprietary data. Recommendations for Researchers Researchers can observe how different AI’s can assist with application development. Further research is encouraged as AI capabilities will continue to evolve. Impact on Society The researchers’ findings show AI in light of its current abilities and limitations in the software development life cycle. While AI assistants excelled in simple to medium complexity debugging tasks, there were many complex tasks where a human coder was preferred over the AI assistants; however, this is expected to change over time. Future Research As future technology strengthens AI some aspects of the study may become historical; however, the core of the research, that of using AI as assistants in development of software projects is expected to remain pertinent to education for some time.