Analysis of Immersive Mid-Air Sketching Behavior, Sketch Quality, and User Experience in Design Ideation Tasks

Immersive 3D sketching systems empower users with tools to create sketches directly in the air around themselves, in all three dimensions, using only simple hand gestures. These sketching systems have the potential to greatly extend the interactive capabilities of immersive learning environments. The perceptual challenges of Virtual Reality (VR), however, combined with the ergonomic and cognitive challenges of creating mid-air 3D sketches reduce the effectiveness of immersive sketching used for problem-solving, reflection, and to capture fleeting ideas. We contribute to the understanding of the potential challenges of mid-air sketching systems in educational settings, where expression is valued higher than accuracy, and sketches are used to support problem-solving and to explain abstract concepts. We conducted an empirical study with 36 participants with different spatial abilities to investigate if the way that people sketch in mid-air is dependent on the goal of the sketch. We compare the technique, quality, efficiency, and experience of participants as they create 3D mid-air sketches in three different tasks. We examine how users approach mid-air sketching when the sketches they create serve to convey meaning and when sketches are merely reproductions of geometric models created by someone else. We found that in tasks aimed at expressing personal design ideas, between starting and ending strokes, participants moved their heads more and their controllers at higher velocities and created strokes in faster times than in tasks aimed at recreating 3D geometric figures. They reported feeling less time pressure to complete sketches but redacted a larger percentage of strokes. These findings serve to inform the design of creative virtual environments that support reasoning and reflection through mid-air sketching. With this work, we aim to strengthen the power of immersive systems that support midair 3D sketching by exploiting natural user behavior to assist users to more quickly and faithfully convey their meaning in sketches.