Flow constructing and optimizing method for composite web service based on Q-learning

We propose a method of automated flow generation for the web services composition according to the de fined target state based on reinforcement machine learning. An agent that uses Q-learning gradually accu mulates knowledge about the environment to updates the evaluations of the usefulness of its actions (these actions correspond to the existing services). The task is divided into two subtasks: - construction of possible flows represented as sequences of services where the results of the previous service execution change the current environment state and enable the exe cution of the next service; - choice of the optimal flow according to the history of interactions and to QoS criteria that is adapted to environment changes. We determine the main components of reinforcement learning and analyze their specifics for service com position task. Additional approaches that allow avoiding looping and the use of unnecessary services are considered. We propose modification of the Q-learning method developed for automatic generation of flows based on input and output data of web services and for selecting the optimal flow based on the analysis of their qualitative characteristics. This modified method uses approach with memory where the agent expands its knowledge about the environment at each step. We consider characteristics of proposed method based on analysis of its software implementation. Possibilities of proposed method are considered on example of generation an optimal study sequences used for individual educational trajectories in accordance with the personal needs of students. Every learning ob ject (information object used for educational needs described by metadata) is considered as a specific ser vice where inputs and outputs are represented by required and result competencies.