HeXA: Haptic-enhanced eXtended reality framework for material-informed Architectural design

Traditional architectural design processes are caught in sequential and isolated planning workflows, starting with pen-and-paper sketching, digitizing, visualizing for stakeholders, and refining. Only after the basic design has been established, structural engineers get involved and modify the structure, such that the design withstands all external loads. This segregated approach is driven by knowledge barriers associated with structural analysis and by lengthy feedback loops regarding structural design decisions involving finite element simulations. This paper presents research on a novel collaborative and digital design framework, labeled as HeXA, that lowers this knowledge barrier and enables all stakeholders to actively participate in structural design decisions. HeXA couples architecture, structural and material mechanics, computer science and graphics, as well as robotics in six modules: (1) a sketching interface for architectural design, (2) a geometric modeling module that translates the sketch to finite element meshes, (3) a material modeling module that predicts the mechanical properties of sustainable bio-composites, (4) a structural analysis module that assesses the mechanical performance using finite element simulations, and (5) an extended reality environment’s immersive visualization that is enhanced by (6) structural haptic feedback provided by a collaborative robot. This allows users to touch and push the envisioned structure in a virtual environment and experience realistic structural deformations provided by the robot and its arm in real-time while resulting stress fields are mapped onto the virtual structure. This way, architectural and structural design processes – particularly at the crucial early design stage – become collaborative, interactive, accessible, and responsive as demonstrated in the paper based on two use cases in theater stage design and building architecture. • A digital, collaborative, immersive, and engaging design experienceis created. • The structural response of architectural designs can be explored in extended reality. • Haptic feedback upon pushing on the structure is provided by a collaborative robot. • Visual feedback of stress fields in virtual reality aids architectural formfinding. • The framework enables material-informed decision-making during early design stages.