Multimodal Classification of Urban Trees into Remote-Sensing-Derived Taxonomic Groups Across the United States

Urban trees provide important ecosystem services; however, municipal inventories used to characterize urban-forest composition remain incomplete and inconsistent among cities. We developed a multimodal deep-learning framework that combined municipal inventories with tree-centered National Agriculture Imagery Program aerial imagery, airborne LiDAR canopy height, Sentinel-2 time series and phenology, Google Satellite Embeddings, and daily weather. Beginning with 5.43 million records from 60 U.S. cities, quality-control and data-availability screening retained 1.85 million records representing 579 taxa across 36 cities. Because many species and genera had overlapping remote-sensing signatures, a supervised discrimination model identified 11 intermediate groups that were subsequently collapsed into six remote-sensing-derived taxonomic groups. On an independent test set of 161,508 records, the final pruned ensemble achieved 0.793 accuracy, 0.772 macro-F1, and 0.792 weighted-F1. Group-specific F1 ranged from 0.705 to 0.867 among trees with LiDAR-derived vegetation height ≥ 2 m. Gate weights emphasized high-resolution NAIP-based inputs, whereas ablation and single-modality analyses showed that Sentinel-2 and weather time series supplied important complementary temporal information. Mean accuracy and macro-F1 declined to 0.685 and 0.478 when target cities were withheld but remained substantially higher than for models trained using individual cities. This framework supports scalable classification of individual urban trees into broader remotely distinguishable groups, not species-level identifications or ecological functional types. Sentinel-2 measurements provide contextual information at 10 m resolution rather than individual-crown observations.