BodySync AI: A Multimodal Agentic System for Nutrition Label Extraction and Pose-Based Body Analysis

This paper presents BodySync AI, a multimodal calorie-tracking prototype that combines optical character recognition (OCR), object detection, pose estimation, and an agentic large language model (LLM). The system supports two tasks: extracting nutritional values from photographs of food-packaging labels, and analyzing body proportions from full-body images. Nutrition extraction uses a classical image-processing front end (perspective correction, denoising, illumination normalization, and contrast enhancement) followed by EasyOCR and rulebased parsing of calories, fat, carbohydrates, and protein. In parallel, we train a YOLOv8n nutrition-label detector on the Open Food Facts Nutrition Table Detection dataset and study the effect of training epochs, learning rate, optimizer, batch size, and input resolution in 14 controlled runs. The best configuration reaches an mAP@50 of 0.904 and an mAP@50-95 of 0.727 on the validation split. Body analysis uses MediaPipe Pose landmarks to compute normalized shoulder width, hip width, torso length, and shoulder-to-hip ratio, which feed a baseline rule-based bodytype estimate. A Groq-hosted Llama 3.1 8B agent exposes these components as tools through a natural-language interface. We describe the system as built, report the detector experiments, and outline the evaluations and learned body-composition models planned as future work.

Authors

Institutions

Publication Details

Journal
Journal of Machine Learning Advances
Published
2026-11-16
DOI
https://doi.org/10.66879/jmla.2026.83
Primary Topic
Advanced Neural Network Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

BodySync AI: A Multimodal Agentic System for Nutrition Label Extraction and Pose-Based Body Analysis

Abhishek Raj Mishra, Stephanie Roselle Silva, Yuvjeeth Hosur Sreenivas
1 links
Journal of Machine Learning Advances
Advanced Neural Network Applications
article

BodySync AI: A Multimodal Agentic System for Nutrition Label Extraction and Pose-Based Body Analysis

Abhishek Raj Mishra, Stephanie Roselle Silva, Yuvjeeth Hosur Sreenivas
article en

Abstract

This paper presents BodySync AI, a multimodal calorie-tracking prototype that combines optical character recognition (OCR), object detection, pose estimation, and an agentic large language model (LLM). The system supports two tasks: extracting nutritional values from photographs of food-packaging labels, and analyzing body proportions from full-body images. Nutrition extraction uses a classical image-processing front end (perspective correction, denoising, illumination normalization, and contrast enhancement) followed by EasyOCR and rulebased parsing of calories, fat, carbohydrates, and protein. In parallel, we train a YOLOv8n nutrition-label detector on the Open Food Facts Nutrition Table Detection dataset and study the effect of training epochs, learning rate, optimizer, batch size, and input resolution in 14 controlled runs. The best configuration reaches an mAP@50 of 0.904 and an mAP@50-95 of 0.727 on the validation split. Body analysis uses MediaPipe Pose landmarks to compute normalized shoulder width, hip width, torso length, and shoulder-to-hip ratio, which feed a baseline rule-based bodytype estimate. A Groq-hosted Llama 3.1 8B agent exposes these components as tools through a natural-language interface. We describe the system as built, report the detector experiments, and outline the evaluations and learned body-composition models planned as future work.

Journal of Machine Learning AdvancesVol. 2(1)
San Diego State University (US)
Zero hunger
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon