Fabric Flattening and Alignment System Using Real-Time Mesh-Based State Estimation and Visual Servoing

Fabric flattening and alignment are foundational yet challenging operations in automated garment manufacturing. In this paper, we propose a dual-arm manipulator system for flattening a fabric part and aligning it with a desired pose on the flat table using a flinging operation and visual servoing. The dual-arm manipulator holds the fabric part and conducts a flinging operation first to eliminate the wrinkles on the fabric part. After the flinging operation, the dual-arm manipulator pulls the fabric part to align it with the desired pose using visual servoing, while keeping it flat. To achieve this operation, we propose a real-time mesh tracking scheme for the fabric, which consists of a global registration and a fine registration. The global registration registers a canonical mesh to a captured point cloud. The fine registration is based on the As-Rigid-As-Possible (ARAP) deformation framework as a model-free method, which preserves the overall mesh shape while allowing for a fine wrinkle reconstruction. The experimental results demonstrate that the proposed system is capable of real-time mesh registration, effective wrinkle elimination via flinging, and precise fabric alignment through visual servoing.