AI Lead Maya: Inside the Habits AI — How Your Personalised Coach Actually Works

By AI Lead Maya · 2025-05-01

A plain-English tour of the AI under the hood: how the food vision model reads your plate, how we keep your health data private, how the coach 'remembers' you, and why the answers get more personal the longer you use the app.

Hi — Maya here. I lead the AI team at Habits, which means I spend my days designing the systems that turn your data into a coach that actually knows you. There's a lot of mystery (and a fair bit of hype) around AI in health apps, so I wanted to walk you through, in plain English, what's really going on inside Habits.

If you remember one thing: none of this works without your data, and none of your data leaves your control. Both halves of that sentence matter. Let's go.

The four AI systems inside the app

Habits isn't one big "AI". It's four specialised systems working together:

1. The Coach — the chat and voice you talk to.
2. The Vision Model — the one that reads your meal photos.
3. The Pattern Engine — the quiet system that watches your data over time and surfaces what matters.
4. The Planner — the system that writes your daily plan, weekly meal plan and workouts.

Each is good at one thing. Stitched together, they feel like a single coach who knows you well.

1. The Coach (chat + voice)

When you type or talk to Habits, the request doesn't just go to a generic chat model. It goes through what we call your personal context layer: your medication, dose schedule, food log, goals, recent insights, and any relevant clinician-reviewed protocols.

That context is assembled fresh on every message, then handed to a large language model alongside guardrails written by our clinical team. The guardrails are the boring-but-critical bit: they stop the model giving advice outside its lane (e.g. dose changes, anything diagnostic) and route those questions to "talk to your doctor" with a clean summary you can share.

Voice works the same way — the audio is transcribed, the same context layer fires, and you get the same brain just with your voice as the input.

2. The Vision Model (the one that reads your plate)

This is the bit that feels most like magic. You snap a photo of your meal and within a couple of seconds you get calories, protein, carbs, fat and a "processing level" estimate.

How? A multimodal vision model identifies the foods on the plate, estimates portion sizes from visual cues (the plate, your fingers if visible, common reference objects), and looks up nutrition data for each item.

A few honest things to know:

• It's right about 85–90% of the time on common foods, less on unusual or mixed dishes. Always sanity-check the protein number, that's the one we care about most.
• Your edits train your personal model. When you correct a portion or a food, that correction is folded into the model that recognises your meals next time. After 20–30 photos, repeat meals get auto-detected with much higher accuracy.
• On-device where we can. On supported phones, the recognition runs locally — no photo ever leaves your device.

3. The Pattern Engine (the quiet hero)

This is the one nobody sees, but it's the reason your insights start feeling eerily specific around week 4.

Every night, a small batch job looks across your last 14+ weeks of data — sleep, food, weight, mood, workouts, medication adherence, side effects — and runs a set of statistical checks. We're looking for things like:

• Days you sleep < 6 hours followed by overeating the next day
• Side effects clustered around a specific time after dosing
• Weight trend stalling vs. average protein dropping
• Mood dips correlating with specific foods or skipped workouts

When something crosses a confidence threshold, it becomes a "personalised insight" you'll see in the app. Crucially, you'll always see the data we used to find it — never a black-box claim. If we're wrong, you can dismiss the insight and the engine learns to weight that pattern lower for you.

4. The Planner

Your daily plan, weekly meal plan and workout sessions are generated by a planning system that takes inputs like:

• Your protein and calorie targets
• What you ate yesterday (so today's plan complements it)
• Your dose schedule (we lean into protein and hydration on dose days)
• Your training schedule
• Your stated dislikes, allergies and the foods you actually cook

The planner doesn't just generate text. It generates a structured plan that the rest of the app can act on — meals you can log in one tap, workouts that play in the workout player, reminders pre-scheduled for the right time.

How the AI gets smarter the longer you use it

A common question: "Will the AI eventually feel personal, or is it generic forever?" Here's the honest answer:

• Week 1: mostly generic, lightly tuned by your medication and goals.
• Week 2–3: starts knowing your typical meals and your calorie pattern.
• Week 4–6: the pattern engine has enough data to surface insights specific to you.
• Month 2+: the coach answers feel meaningfully personal — it knows your dose curve, your usual meals, your training rhythm, and the patterns from your weekly reviews.

The single biggest accelerator? Logging consistently for the first 14 days and connecting Apple Health or Google Fit. After that, the personalisation compounds.

Privacy: the bit I personally care about most

I'll be blunt — health data is the most sensitive data category there is. Our rules:

• Your data is yours. You can export everything as JSON or CSV at any time, and delete your account permanently in two taps.
• We never sell data. Not to advertisers, insurers, anyone.
• AI training is opt-in, anonymised, and aggregated only. Your individual messages, photos and logs are never used to train third-party general-purpose models.
• On-device wherever possible. Food recognition, on-device. Sensitive transcripts, processed and discarded.
• Encryption end-to-end between the app and our servers, and at rest in the database.

If you ever want the technical detail, our privacy page has the long version. Anything that isn't there, you can ask the team directly.

What we're building next

Three things on the AI roadmap I'm personally excited about:

1. Better long-context memory — the coach will reference things you mentioned weeks ago without you re-explaining.
2. Proactive nudges — instead of waiting for you to ask, the coach will suggest the small course-corrects (more protein at lunch, walk after dinner) before they become a problem.
3. Multi-modal weekly reviews — short, narrated 60-second video summaries of your week, generated for you, so you can listen instead of read.

The takeaway

The AI in Habits isn't trying to replace your doctor, your trainer or your common sense. It's trying to be the always-available, well-informed, slightly nerdy friend who's read all the GLP-1 research, noticed every small pattern in your week, and can tell you what to actually do about it tonight.

The more you put in — meals, mood, sleep, the odd voice question — the more useful it gets. That's the whole loop.

If you ever want to know more about how something works, ping the team in-app. I read most of the AI questions personally. — Maya