Distribution-Aware Post-Training Quantization for the Segment Anything with Concepts (SAM3) Model

Segment Anything with Concepts (SAM3) enables concept-driven segmentation and tracking across images and videos in various settings. However, the SAM3 Transformer architecture is challenging to quantize at low precision due to substantially different activation distributions across operations. Existing approaches rely on predefined quantization structures, gradient-based reconstruction, or higher precision for sensitive components, without considering distribution variations for accurate fixed-precision post-training quantization (PTQ). We propose DA-QSAM3, a distribution-aware and operator-aligned framework for post-training weight-and-activation quantization. The central principle of our approach is to represent sensitive activations according to their empirical distributions while selecting quantization parameters based on their effects on downstream operations. We propose the Information-Preserving Quantizer (IPQ) that learns head-wise non-uniform quantization levels for post-Softmax probabilities and selects the refinement depth using attention-output distortion. Our Distribution-Aware Post-GELU Quantization (PGQ) refines non-uniform levels using FC2 sensitivity weighting and selects the final quantizer using FC2-output distortion. Tuned LayerNorm Clipping (TLC) determines channel-adaptive pre-MLP LayerNorm activation ranges by balancing clipping and rounding distortions measured after FC1. The proposed DA-QSAM3 framework requires no retraining, backpropagation, or block reconstruction. At W4A4, DA-QSAM3 achieves an average cgF1 of 50.8 ± 0.2 across seven image subsets, compared with 54.1 for full-precision SAM3 and 8.8 for Mix-QSAM3, giving a 3.3-point gap to full precision and a 42.0-point improvement over Mix-QSAM3. At W8A8, DA-QSAM3 achieves 53.6, exceeding Mix-QSAM3 by 1.2 points and remaining 0.5 points below full precision. For video tracking, it preserves more than 93% of full-precision pHOTA on every evaluated subset.

Authors

Institutions

Publication Details

Journal
Sensors
Published
2026-11-18
DOI
https://doi.org/10.3390/s26227288
Primary Topic
Advanced Neural Network Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Distribution-Aware Post-Training Quantization for the Segment Anything with Concepts (SAM3) Model

Andreas E. Savakis, Navin Ranjan
2 links
Sensors
Advanced Neural Network Applications
article

Distribution-Aware Post-Training Quantization for the Segment Anything with Concepts (SAM3) Model

Andreas E. Savakis, Navin Ranjan
article en

Abstract

Segment Anything with Concepts (SAM3) enables concept-driven segmentation and tracking across images and videos in various settings. However, the SAM3 Transformer architecture is challenging to quantize at low precision due to substantially different activation distributions across operations. Existing approaches rely on predefined quantization structures, gradient-based reconstruction, or higher precision for sensitive components, without considering distribution variations for accurate fixed-precision post-training quantization (PTQ). We propose DA-QSAM3, a distribution-aware and operator-aligned framework for post-training weight-and-activation quantization. The central principle of our approach is to represent sensitive activations according to their empirical distributions while selecting quantization parameters based on their effects on downstream operations. We propose the Information-Preserving Quantizer (IPQ) that learns head-wise non-uniform quantization levels for post-Softmax probabilities and selects the refinement depth using attention-output distortion. Our Distribution-Aware Post-GELU Quantization (PGQ) refines non-uniform levels using FC2 sensitivity weighting and selects the final quantizer using FC2-output distortion. Tuned LayerNorm Clipping (TLC) determines channel-adaptive pre-MLP LayerNorm activation ranges by balancing clipping and rounding distortions measured after FC1. The proposed DA-QSAM3 framework requires no retraining, backpropagation, or block reconstruction. At W4A4, DA-QSAM3 achieves an average cgF1 of 50.8 ± 0.2 across seven image subsets, compared with 54.1 for full-precision SAM3 and 8.8 for Mix-QSAM3, giving a 3.3-point gap to full precision and a 42.0-point improvement over Mix-QSAM3. At W8A8, DA-QSAM3 achieves 53.6, exceeding Mix-QSAM3 by 1.2 points and remaining 0.5 points below full precision. For video tracking, it preserves more than 93% of full-precision pHOTA on every evaluated subset.

SensorsVol. 26(22)
Rochester Institute of Technology (US)
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon