An Assistive Activity Classification System to Passively Recognize and Monitor Medicine Intake Using Video Transformer

Recent advances in deep learning models have allowed for solutions to use video classification models to record and track medicine-taking. Some describe wearable devices with cameras attached to them, and others involve using mounted camera systems. Mounted camera systems truly have great potential for use in the real world as they are non-intrusive, require no setup on the user’s part, and are consistent because they do not move with the person. We propose a passive medicine monitoring and advisory system that requires no additional setup other than a camera and a mount. It utilizes a deep learning video classification model to accurately sense and $\log$ when a patient takes medicine. The information that can be gathered is crucial for both patients and doctors in adhering to prescriptions, especially for older people who may have to keep track of several medicines every day. This solution can increase medication effectiveness dramatically, with an average accuracy of $\mathbf{9 3. 6 6 \%}$. A computer-based offline system is much faster and more cost-effective than human-based systems used in the past.