Development of CNN-Based Terrain Classifier Using Depth Camera

Humans adjust their lower limb control strategies based on the terrain they are walking on. This principle also applies to users of robotic orthoses or prostheses. However, currently commercialized robotic orthoses and prostheses do not adapt their control strategies according to terrain; instead, they apply the same walking control as if on flat ground. This leads to awkward and uncomfortable walking patterns for the wearer. The root cause of this issue is that the robot is unable to recognize and classify different terrains. To address this, we propose a terrain classification system based on stereo depth cameras. The proposed classifier categorizes the data from the stereo depth camera into five distinct terrain types: level ground, ramp ascent, ramp descent, stair ascent, and stair descent. By leveraging the 3D information provided by the stereo depth camera, the system is able to effectively differentiate between flat ground and slopes, which are challenging to distinguish using an RGB camera. The terrain classification system achieved an accuracy of 87.06%. This demonstrates that effective terrain classification can be accomplished using only the 3D data from the stereo depth camera.