Our Research: AI vs. Dietitians vs. Non-Dietitians in Carb Counting

Presented at the American Diabetes Association (ADA) Scientific Sessions, 2025

We make a photo carb counter, so we have an obvious interest in you believing it works. This page is the actual poster data, including the categories where a dietitian still beats it.

What we presented

Humans vs. AI in Carb Counting: Real-World Accuracy Data from Dietitians, Non-Dietitians, and AI.
Julieta Mazzoldi, Benicio Mazzoldi, Florencio Mazzoldi
Poster 1897-LB · American Diabetes Association (ADA) Scientific Sessions, June 20, 2025

In one sentence: across 12,000+ real Carbsnap meal photos, the AI beat non-dietitians outright and held its own against registered dietitians — ahead of them on mean error, behind them on the typical (median) case.

Why "dietitian consensus" is the benchmark

For a plated, mixed, real-world meal there's rarely a single knowable "true" carbohydrate value — even lab analysis of an identical recipe varies with portioning. The standard that actually matters clinically is the range trained professionals converge on. So for this study, every meal photo's ground truth was the average of at least four registered dietitians' independent carbohydrate estimates for that photo — the same way you'd judge a trainee clinician.

The dataset

Before Carbsnap was a consumer app, it was a carb-counting practice platform where registered dietitians and people with diabetes estimated carbohydrates on real meal photographs. That produced two things: a dietitian-consensus benchmark for evaluation, and published findings on human estimation error:

  • People underestimate carbohydrates in fruits and vegetables — on one plate, dietitian consensus median 135g vs. patient median 50g. Read the analysis →
  • People overestimate carbohydrates in meats — mean error ≈ +2.5g; on one dish, dietitians 0–5g vs. patient median 20g. Read the analysis →
  • People are fairly accurate on pasta — per-image mean error between −2g and +1.5g. Read the analysis →

The pattern: human carb-estimation error is not random noise — it is systematic bias by food category. That's the error profile a photo-AI assistant needs to not share.

Method

We pulled 12,000+ food entries logged through the Carbsnap app, each with a meal photo and a carbohydrate estimate, and split the estimators into non-dietitians (regular app users, no formal training) and dietitians (verified registered clinicians). For each photo:

  1. We showed it to at least 4 dietitians.
  2. The average of their estimates became the ground truth.
  3. We showed the same photo to non-dietitians and to an AI model (OpenAI GPT-4o) and recorded their estimates.

Accuracy was deviation, in grams, from that ground truth, broken out across 12 food categories.

Key results

Box plots of carb-estimation error in grams, by food category, comparing dietitians, non-dietitians, and AI. AI (green) runs the tightest, most centered boxes in most categories; dietitians (blue) are tightest in bread, rice, and vegetables/fruit; non-dietitians (red) run widest and most skewed almost everywhere, especially fried food and vegetables/fruit.
Error distribution by food category, across all 12,000+ Carbsnap entries. Each box is 50% of that group's estimates for that category; whiskers show the full spread.
  • AI beat non-dietitians outright — better 70% of the time, with a mean error 9g lower.
  • Against dietitians it's a split decision, and the same split shows up everywhere in this data: AI won on mean error 54% of the time, but dietitians kept the edge on the typical (median) case.
  • That split holds by food category. AI's weakest ground was bread, rice, and vegetables/fruits — dietitians had the lower median error there. But even in those categories, AI's estimates had less spread and less systematic bias than dietitians' did, the same mean-vs-median pattern as the headline number, just playing out at the category level.
  • AI's strongest ground was dairy, eggs, and desserts — its tightest, most consistent estimates of the three groups.
  • Vegetables/fruits and fried foods were hard for everyone — the largest errors across all three groups — and AI still ran the smallest spread of the three even there, despite trailing dietitians on median error in that category.
  • Non-dietitians had a real safety problem, not just a less-accurate one: nearly 40% of their estimates were off by more than 25g, enough to meaningfully throw off an insulin dose.
Three histograms of absolute carb-estimation error in grams: dietitians, non-dietitians, and AI. Dietitians and AI are both heavily concentrated under 25 grams of error. Non-dietitians have a much longer tail past 25 grams, highlighted with a box, showing far more large misses.
Absolute error distribution by group. The boxed region (>25g error) is visibly heavier for non-dietitians than for dietitians or AI — that's the ~40% figure above.

Where this doesn't hold up

A single photo can't see hidden ingredients (sugar in a sauce, oil absorbed into fried food), struggles with depth and volume, and can't catch dissolved carbs in a liquid. That's why Carbsnap treats its estimate as an editable starting point, not a verdict — you can revise the description and re-estimate, and we'd rather you think of every number as a well-informed second opinion than a dosing instruction. More on where photo AI fails →

Poster & citation

Thumbnail of the full ADA 2025 conference poster: Humans vs. AI in Carb Counting

View the full poster (PDF) View on Diabetes Journals (ADA)

Mazzoldi J, Mazzoldi B, Mazzoldi F. 1897-LB: Humans vs. AI in Carb Counting. Diabetes 2025; 74 (Supplement_1): 1897-LB. Poster presented at: American Diabetes Association (ADA) Scientific Sessions; June 20, 2025.

For researchers and clinicians

Happy to share methodology details and evaluation data with clinical researchers and diabetes educators on request, and we offer free practitioner-dashboard access to dietitians who want to try Carbsnap out with their own clients. Reach us at info@carbsnap.com, or see the practitioner dashboard →

Carbsnap's carb counting is free forever — App Store · Google Play. Carbsnap is an estimation aid, not a medical device; always follow your care team's guidance on insulin dosing.