WIP: A Preliminary Investigation of Students as Peer Evaluators of Their Team Members

This work-in-progress research paper focuses on the first-year engineering teams' peer evaluation processes. Teamwork has been identified as a necessary skill for professional engineers and thus an ABET learning outcome. Reliable systematic processes to assess teamwork effectiveness are crucial for improving team outcomes and identifying dysfunctional teams in a classroom setting. Evaluating team effectiveness and identifying dysfunctional teams has been traditionally done through peer evaluations. However, there is a lack of evidence on what/how students' perceptions of effective teammates and functional teams impact peer evaluations. Identifying common indicators and behaviors that students consider when evaluating their peers is a first step to exploring the reliability of peers as evaluators, understanding potential biases, and using peer feedback to investigate dysfunctional teams. This paper aims to take this first step and will explore the following question: “What evidence do students provide to describe poor team behaviors?” The study was conducted at a large, public, urban, Midwestern R1 institution. For the first two semesters of the engineering curriculum, students are required to participate actively in team-based projects as part of engineering design thinking courses. The following procedure was used to answer our research question. Students were first asked to respond to a peer evaluation instrument based on a four-factor (Interdependency, Goal setting, Potency, and Trust) team effectiveness model. Then, students were asked to grade their teammates on a cumulative score of 100 for the team and support their distribution choices with open-ended comments on team behaviors. We started with an analysis of students' comments regarding negative behaviors to support their low rating for each teammate. We selected comments that had a high misalignment between the model-based peer evaluation and corresponding distributed ratings, resulting in a sample of 93 students. Next, we conducted inductive coding on the comments to identify frequently mentioned negative behaviors and see if these behaviors were addressed in the model-based peer evaluation. Eight major themes emerged from students' comments.