Performance Improvement of Blink Types Classification Using a 3D-CNN by Mode Value Correction

We have developed blink measurement methods that can be applied to input interfaces. To use the eye-blinking information as an input trigger, it is necessary to automatically classify blink types into voluntary and involuntary. A method for blink type classification using a three-dimensional convolutional neural network (3D-CNN) has been proposed. This classification method takes a short image sequence of the periocular area and classifies the blink type. We previously reported on several performance-improving methods that can be applied to this 3D-CNN. Since our classification using 3D-CNN outputs classification results in units of video frames, multiple types of classification results could be mixed together during a period of a single blinking motion. To address this problem, we employ a correction method to calculate the mode value as a representative value for the consecutive blink period. This paper proposes a correction method to improve accuracy based on limiting the aggregation range of the mode to a reliable portion. The evaluation experiment resulted in 97.9% accuracy and 94.0% F-score in the classification results for each short image sequence for 10 subjects. Then, 97.5% accuracy and 97.3% F-score were obtained for the accuracy of blink type classification.