A Hybrid CNN-Transformer with Window-Based Attention for Vegetation-Soil Semantic Segmentation in Agricultural Vision Systems

Accurate vegetation-soil segmentation is essential for agricultural vision systems operating under variable field conditions. This study evaluates a hybrid CNN-Transformer architecture combining a convolutional pre-encoder, hierarchical shifted-window attention, and a U-Net-like decoder for segmentation in structured crop-row environments. A 13-channel input representation incorporating ten RGB-derived vegetation descriptors was investigated without requiring additional spectral sensors. Model development systematically examined input representation, optimizer, Transformer depth, loss function, and patch size. The final configuration was evaluated across five independent training runs and compared with conventional CNN, lightweight, Transformer-based, and hybrid segmentation architectures. On the internal test set, Hybrid-13Ch achieved 95.9 ± 0.22% Global Accuracy, 96.3 ± 0.76% Mean Accuracy, 89.1 ± 0.37% mean Intersection over Union (mIoU), and 96.6 ± 0.53% Mean BFScore. On an external test set comprising 300 annotated frames from two previously unseen video sequences, it achieved 87.3 ± 0.39% mIoU and 94.3 ± 0.58% Mean BFScore. Incremental and subtractive ablation experiments assessed the contributions of the convolutional pre-encoder, shifted-window attention, and MLP processing. These findings support the proposed architecture and RGB-derived representation for agricultural semantic segmentation.

Authors

Institutions

Publication Details

Journal
Applied Sciences
Published
2026-11-19
DOI
https://doi.org/10.3390/app162211486
Primary Topic
Smart Agriculture and AI
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

A Hybrid CNN-Transformer with Window-Based Attention for Vegetation-Soil Semantic Segmentation in Agricultural Vision Systems

Gonzalo Pájares, Eva Besada-Portas, Esther Gascó, Clara I. López-González
Applied Sciences
Smart Agriculture and AI
article

A Hybrid CNN-Transformer with Window-Based Attention for Vegetation-Soil Semantic Segmentation in Agricultural Vision Systems

Gonzalo Pájares, Eva Besada-Portas, Esther Gascó, Clara I. López-González
article en

Abstract

Accurate vegetation-soil segmentation is essential for agricultural vision systems operating under variable field conditions. This study evaluates a hybrid CNN-Transformer architecture combining a convolutional pre-encoder, hierarchical shifted-window attention, and a U-Net-like decoder for segmentation in structured crop-row environments. A 13-channel input representation incorporating ten RGB-derived vegetation descriptors was investigated without requiring additional spectral sensors. Model development systematically examined input representation, optimizer, Transformer depth, loss function, and patch size. The final configuration was evaluated across five independent training runs and compared with conventional CNN, lightweight, Transformer-based, and hybrid segmentation architectures. On the internal test set, Hybrid-13Ch achieved 95.9 ± 0.22% Global Accuracy, 96.3 ± 0.76% Mean Accuracy, 89.1 ± 0.37% mean Intersection over Union (mIoU), and 96.6 ± 0.53% Mean BFScore. On an external test set comprising 300 annotated frames from two previously unseen video sequences, it achieved 87.3 ± 0.39% mIoU and 94.3 ± 0.58% Mean BFScore. Incremental and subtractive ablation experiments assessed the contributions of the convolutional pre-encoder, shifted-window attention, and MLP processing. These findings support the proposed architecture and RGB-derived representation for agricultural semantic segmentation.

Applied SciencesVol. 16(22)
Universidad Complutense de Madrid (ES), Technological University Dublin (IE)
Smart Agriculture and AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon