Mobile robot in nonstationary environment
Abstract
Mobile robot in nonstationary environment
Incoming article date: 07.12.2017A research software complex was developed to form the behavior of a mobile robot in a nonstationary environment based on the e-puck robot. The complex includes the method of additional training of the robot, cluster analysis and the modified algorithm Q-learning, which ensures the robot's movement to the target. The recognition algorithm is based on the concept of ""similarity"" between images from the robot's camera and the standard marks of object clusters formed in the learning process. Standards are formed gradually from ""similar"" labels. If the label is ""similar"" to the existing standard, then the object has already met before and the corresponding action of the robot is also defined. Otherwise, a new cluster is created, so there is additional training. The number of objects is almost unlimited. To ensure the movement of the robot to the target, the Q-learning algorithm is used with reinforcement. It is based on the use of the matrix of the expected reward Q. Knowing the current state of the environment, the robot chooses an action that is expected to bring the maximum reward. The reward is given if the target label is in a certain area of the frame of the video camera and has a certain size. Initially, the matrix Q is given randomly. In addition, at each step the matrix values change in such a way that the matrix gives optimal control in any situation. In the case of the robot e-puck, the possible actions are rectilinear motion or turn (left / right). To overcome the problem with several targets, the algorithm uses several reward matrices Q. When a new goal occurs, the algorithm creates a new reward matrix. In this modification, all matrices Q are subjected, regardless of the target the robot is looking for. The final decision on the action of the robot takes on the basis of a matrix corresponding to the current goal. Despite the difficulties and problems that have arisen, it has been possible to implement a complex that can be relatively easily transferred to another physical platform and can be used to create more sophisticated intelligent systems.
Keywords: autonomous mobile robot e-puck, Q-Learning algorithm, cluster analysis, training, recognition, digital methods of processing video images