Facilitating Trust in AI-assisted Software Tools

The day to day of a software engineer involves a variety of tasks. While many of these tasks are collaborative, it is not always possible or feasible to engage with other engineers for task completion. Software tools, such as code generators and static analysis tools, aim to fill this gap by providing additional support for developers to effectively complete their tasks. With a steady stream of new tools emerging to support software engineers, including a new breed of tools that rely on artificial intelligence (AI), there are important questions to answer regarding the trust engineers can, and should, put into their software tools and what it means to build a trustworthy tool. To this end, this paper presents findings from a mixed methods investigation of the factors that contribute to trust in traditional and AI-assisted software tools. First, we introduce the PICSE (pronounced “pixie”) framework for trust in software tools that we developed based on a set of 18 interviews with software practitioners internal and external to Microsoft. We then discuss insights from a survey with 368 internal Microsoft responses on the relevance and importance of the factors in our framework with respect to traditional and AI-assisted tools.