Training Indoor and Scene-Specific Semantic Segmentation Models to Assist Blind and Low Vision Users in Activities of Daily Living
<i>Goal:</i> Persons with blindness or low vision (pBLV) face challenges in completing activities of daily living (ADLs/IADLs). Semantic segmentation techniques on smartphones, like DeepLabV3+, can quickly assist in identifying key objects, but their performance across different indoor settings and lighting conditions remains unclear. <i>Methods:</i> Using the MIT ADE20K SceneParse150 dataset, we trained and evaluated AI models for specific indoor scenes (kitchen, bedroom, bathroom, living room) and compared them with a generic indoor model. Performance was assessed using mean accuracy and intersection-over-union metrics. <i>Results:</i> Scene-specific models outperformed the generic model, particularly in identifying ADL/IADL objects. Models focusing on rooms with more unique objects showed the greatest improvements (bedroom, bathroom). Scene-specific models were also more resilient to low-light conditions. <i>Conclusions:</i> These findings highlight how using scene-specific models can boost key performance indicators for assisting pBLV across different functional environments. We suggest that a dynamic selection of the best-performing models on mobile technologies may better facilitate ADLs/IADLs for pBLV.
Authors
- Sahil Faizal
- Todd E. Hudson (ORCID: https://orcid.org/0000-0003-4506-2670)
- Kevin C. Chan (ORCID: https://orcid.org/0000-0003-4012-7084)
- Giles Hamilton-Fletcher (ORCID: https://orcid.org/0000-0001-5903-4334)
- Chen Feng (ORCID: https://orcid.org/0000-0003-3211-1576)
- Ruijie Sun
- John-Ross Rizzo
Institutions
- Courant Institute of Mathematical Sciences (US)
- New York University (US)
Publication Details
- Journal
- IEEE Open Journal of Engineering in Medicine and Biology
- Published
- 2025-01-01
- DOI
- https://doi.org/10.1109/ojemb.2025.3607816
- Primary Topic
- Visual Attention and Saliency Detection
- Type
- article
- Field-Weighted Citation Impact
- 0.00