A Multimodal Approach to Heritage Preservation in the Context of Climate Change

Cultural heritage sites face accelerating degradation due to climate change, yet tradi-tional monitoring relies on unimodal analysis (visual inspection or environmental sen-sors alone) that fails to capture the complex interplay between environmental stres-sors and material deterioration. We propose a lightweight multimodal architecture thatfuses sensor data (temperature, humidity) with visual imagery to predict degradationseverity at heritage sites.Our approach adapts PerceiverIO with two key innovations: (1) simplified encoders(64D latent space) that prevent overfitting on small datasets (37 samples for training(555 with data augmentation), 13 for validation, and 13 for testing.), and (2) AdaptiveBarlow Twins loss that encourages modality complementarity rather than redundancy.On data from Strasbourg Cathedral, Our model achieves 76.9% accuracy and 77.0%macro-F1 score on the test set, a 43% improvement over standard multimodal archi-tectures (VisualBERT, Transformer) and 25% over vanilla PerceiverIO.Ablation studies reveal that sensor-only achieves 61.5% while image-only reaches46.2%, confirming successful multimodal synergy. A systematic hyperparameterstudy identifies an optimal moderate correlation target (τ =0.3) that balances align-ment and complementarity, achieving 69.2% accuracy compared to other τ values(τ =0.1/0.5/0.7: 53.8%, τ =0.9: 61.5%). This work demonstrates that architectural sim-plicity combined with contrastive regularization enables effective multimodal learningin data-scarce heritage monitoring contexts, providing a foundation for AI-driven con-servation decision support systems.