Pupil Tracking Without Goggles: A Solar-Assisted TinyML Brim-Based Pupil and Head Motion Tracking System with Digital Mimic

This work presents a solar-assisted wearable pupil-tracking microsystem that integrates on-device inference, head pose sensing, energy harvesting, and mobile visualization within a single hat-based platform. The device is fabricated using 3D-printed thermoplastic polyurethane (TPU), providing a flexible and durable structure that supports embedded electronics while maintaining user comfort. The brim houses two ESP32-S3 microcontrollers, two grayscale eye-facing cameras, an inertial measurement unit (IMU), a lithium-polymer battery, a custom-shaped flexible amorphous silicon photovoltaic module, and a charging circuit. Each microcontroller runs an INT8-quantized convolutional neural network (CNN) that performs real-time 2D pupil coordinate regression and eye closure classification from 64 × 64 grayscale images. Pupil coordinates, eye state information, and head pose measurements are transmitted via Bluetooth Low Energy (BLE) to a custom iOS application, where a Unity-based 3D avatar mirrors the wearer’s relative eye movement and head motion while enabling lightweight data logging for offline analysis. By relocating sensing, computation, and power electronics from eyewear to the hat brim, the proposed system enables unobtrusive, autonomous pupil and eye state monitoring for naturalistic cognitive and behavioral research without requiring personal point-of-gaze calibration.