Method for Creating Behavior of Cognitive Agents Based on Multimodal Signal Processing

<p>The paper considers the problem of predicting the agent's activity based on the text description of the task and visual analysis of the environment. An update of the approaches of classical cognitive architecture is proposed, allowing its application in a real environment. An addition to the semiotic method of symbolic designation with the author's neural network mechanism for linking vectors of text and visual spaces is developed. A number of experiments with the obtained model in a complex environment of a car driving emulator are conducted.</p>