Traditional dietary assessment methods, such as food diaries, are limited by high participant burden and misreporting(1). Digital tools, such as wearable cameras, offer an alternative reducing reliance on self-logging, but concerns remain around accuracy, researcher burden, and privacy(2-3). AI-powered photologging, using vision-based systems for food identification and nutrient analysis via smartphone meal capture, provides a low-burden solution that goes beyond tracking intake. By analysing meal composition and variety, it enables assessment of diet quality and diversity, shifting the focus beyond nutrient quantity to a comprehensive understanding of dietary patterns. The aim of this study was to evaluate the accuracy of an AI-powered photologging system in identifying food groups and estimating nutrient content from meal photographs, compared to traditional weighed food logs.

A total of 10,000 meals from 2,124 individuals were analysed from the PREDICT 1 (NCT03479866) and PREDICT 2 (NCT03983733) cohorts (UK and US). Meal photographs were processed using large language model (LLM)-based image analysis and compared against weighed food logs. For ingredient retrieval, we compared food groups (n=77) of retrieved and suggested ingredients. Model performance was evaluated using precision (correctly retrieved groups), recall (proportion of true groups retrieved), and F1 score (harmonic mean of precision and recall). Metrics are scored on a scale (0 to 1), with higher values indicating better performance. Accuracy of nutrient estimates were compared using the Wilcoxon Signed-Rank Test.

Top performing food groups (excluding alcohol) by F1 score (95% CI) included “tea and coffee” (0.88 [0.86, 0.90]), “vegetables” (0.87 [0.85, 0.89]), and “baked beans” (0.84 [0.61, 1.00]). Lower performance was seen for “plant milk” (0.13 [0.07, 0.18]), “soup” (0.07 [0.00, 0.20]), and “hot chocolate” (0.07 [0.00, 0.19]). In the UK and US cohorts respectively, precision by meal (95% CI) was 0.72 (0.74, 0.73), 0.65 (0.64, 0.66); recall was 0.75 (0.74, 0.76), 0.73 (0.72, 0.74); and F1 score was 0.71 (0.71, 0.75), 0.66 (0.65, 0.67). Small differences (95% CI, effect size r = 0.047 to 0.172) were observed for all macronutrients per meal: calories +139 kcal (95% CI: 134, 144), protein +6.11g (5.83, 6.39), carbohydrates +15.6g (14.9, 16.3), fat +7.89g (7.48, 8.33), fibre +2.30g (2.19, 2.42), sodium +0.23g (0.22, 0.25), and sugar +6.83g (6.49, 7.17).

Preliminary results demonstrate that smartphone-based photologging with LLMs can accurately identify ingredients and estimate nutrients. This provides a valuable solution to dietary assessment, in a real-world setting, in the era of smartphone apps. A key limitation is the sole use of a proprietary dataset and internal food groups; future work will benchmark performance on public datasets for broader validation.