Detecting human body parts Using confidence maps and landmark maps

<p>This article discusses the problem of finding key points on an object and determining its components using the example of the human body as one of the most pressing computer vision problems. The problem is that for an image with a supposed object (in our case, a person), it is necessary to find the position of all important components/points, for a person these are joints: shoulders, elbows, hands, knees, etc. To solve this problem, it is proposed to use neural networks using “heat maps”, which are presented in the form of a confidence map and a landmark map. The landmark map is a matrix corresponding to the original image, each cell of which contains a value from 0 to 1 with the probability of the location of the desired joint (special point) in the corresponding pixel. The landmark map is a matrix, each cell of which contains a two-dimensional vector with the direction of the next joint. The main feature is that the maps help each other to predict the correct values: thanks to the confidence map, the landmark map knows the approximate location of the landmarks and it is easier for it to determine the desired direction and vice versa, knowing the approximate direction to the next landmark, it is easier to predict the location of the landmark. These maps are calculated in several stages, and at each new stage, information from the previous one is used, so when calculating each subsequent map, the accuracy of the results obtained increases. In this work, the optimal number of stages was 6-7, but this value can change depending on the final architecture. Demonstrations of the work are carried out on the COCO dataset, which includes 18 points for each human body.</p>