ITTPD: In-place Tensor Transposition with Permutation Decomposition on GPUs

Tensor transposition is a fundamental operation in tensor computations with broad applications.Most existing algorithms rely on out-of-place transposition, which duplicates the tensor in memory and repositions elements in their transposed order, resulting in high memory demands.On memory-constrained devices such as Graphics Processing Units (GPUs), this approach is often impractical.To address this challenge, we propose ITTPD (In-place Tensor Transposition with Permutation Decomposition), a GPU-based method designed to operate within strict memory constraints.Similarly to the previous work EITHOT, ITTPD utilizes permutation decomposition, breaking down complex permutations into simpler transposition primitives, and determining an execution sequence that fits within available memory.Comparing to existing work, ITTPD partitions the tensor into smaller sections for the tensor size larger than the memory limits and transposes each independently.The implementation is optimized for GPU memory access patterns, improving the execution efficiency for each primitive.Experimental results show that ITTPD significantly outperforms state-of-the-art out-ofplace GPU implementations, offering reduced memory overhead and faster execution times.Moreover, ITTPD can process tensors nearly twice as large as those feasible with out-of-place methods, making it a versatile solution for N-order tensor transpositions.