Using Large Language Models to Assist with Project Performance Assessment

NASA uses an Independent Assessment Team (IAT) to review the performance of its science missions. Every month, the IAT uses a rubric to assess subjective red, yellow, or green (stoplight) ratings for each mission in cost, schedule, technical, and programmatic domains and includes text comments explaining the rating choices. This study used Large Language Models to analyze the sentiment of historical IAT monthly review text comments in the cost domain. The resulting sentiments were compared to the corresponding stoplight color. It is found that there is some correlation between color and sentiment for a given month, but not always. Overall, there are more negative sentiments than neutral or positive due to providing comments for all yellow and red ratings, but not providing comments for all green ratings. Negative sentiment along with the keyword identification should provide stakeholders and the projects with potential concerns to consider.