Intention Reasoning for User Action Sequences via Fusion of Object Task and Object Action Affordances Based on Dempster–Shafer Theory

To reduce the burden on individuals with disabilities when operating a Wheelchair Mounted Robotic Arm (WMRA), many researchers have focused on inferring users' subsequent task intentions based on their "gazing" or "selecting" of scene objects. In this paper, we propose an innovative intention reasoning method for users' action sequences by fusing object task and object action affordances based on Dempster-Shafer Theory (D-S theory). This method combines the advantages of probabilistic reasoning and visual affordance detection to establish an affordance model for objects and potential tasks or actions based on users' habits and object attributes. This facilitates encoding object task (OT) affordance and object action (OA) affordance using D-S theory to perform action sequence reasoning. Specifically, the method includes three main aspects: (1) inferring task intentions from the object of user focus based on object task affordances encoded with Causal Probabilistic Logic (CP-Logic); (2) inferring action intentions based on object action affordances; and (3) integrating OT and OA affordances through D-S theory. Experimental results demonstrate that the proposed method reduces the number of interactions by an average of 14.085% compared to independent task intention inference and by an average of 52.713% compared to independent action intention inference. This demonstrates that the proposed method can capture the user's real intention more accurately and significantly reduce unnecessary human-computer interaction.