An Awareness Assessment Model for Collaborative Interfaces

[Context] Awareness has been a valuable concept in Collaborative Systems since its formation, and awareness research has followed the whole field’s evolution over the last decades. We consider awareness the backbone of a collaborative environment; all collaborative concepts are archived through it. [Problem] According to the literature, awareness is a well-known but still not fully reached concept; and researchers should achieve a better understanding of support by conceiving and testing novel technology that provides awareness. [Goal] This work consists of establishing an awareness assessment model for collaborative interfaces from participants’ perspectives by analyzing awareness information provided by the application. [Result] We performed a systematic mapping study to identify awareness support in the collaborative system context and compiled the results into a multidimensional taxonomy of three main awareness dimensions. Then, we present a new assessment method for awareness and collaboration support centered on the participant’s perspective by developing a measurement instrument based on Item Response Theory (IRT). [Conclusion] The methodology allowed us to construct and interpret an awareness quality scale to evaluate the awareness support from the participant’s perspective. The scenarios evaluated demonstrated: i) suitable indicators from the perspective of demographic data and IRT parameterization, ii) the applicability of the awareness scale over the participant’s perspective, and iii) the model replicability for different scenarios and contexts.