AI Engineer World's Fair 2025
My AI Thinks I'm Eating My Feelings (and Other Nutritional Insights)
About this talk
Alma founder Rami Alhamad explains how his team builds a personalized AI nutrition companion, including an evidence-informed Alma Score developed with academic adviser Eric Rimm. He describes improving reliability by constraining large language models to narrow tasks, progressively returning food-recognition results to an iOS client while completing USDA database matching, and learning from beta feedback and tracking streaks. Additional product lessons include persistent user-editable context, proactive engagement, voice-based meal logging, and supporting multiple interaction modalities.
Chapters
- 0:00Why personalized nutrition needs an AI companion
- 2:30Alma Score and lessons from beta feedback
- 3:38Constrained LLM workflows and progressive nutrition lookup
- 4:58Tracking streaks and persistent, editable agent context
- 6:16Proactive coaching and multimodal meal logging
- 8:14Product differentiation, foundation-model data, and closing
Talk transcript
- 0:00
Hey everyone, how's it going? So excited to be here with you guys and to share more of some of the lessons I've got Alma over the last eight months or so.
- 0:07
All right, first off, let me introduce you a little bit more to what we're doing. Now, it's our belief at Alma that eating well shouldn't be hard. Good health starts with simple, personalized nutrition.
- 0:15
For the longest time, it's been super challenging to actually get a good sense of how you're eating. And despite all the different apps that are out there, on a personal level and from talking to so many of my friends and users throughout my journey in wearables and health and wellness space, it's become just really obvious that the
- 0:32
equation is off, you know, in terms of how much information you have to share with these apps, and so how little you get back out. So it's our belief that this shouldn't be this difficult.
- 0:43
With AI, we can finally solve this problem and build something that's truly unique and a true nutrition companion to help you kind of strive towards your-- the best self.
- 0:55
So let me tell you a little bit more about what we're up to at Alma. So we believe that at Alma there's three core pillars to this new vision of an AI nutrition companion.
- 1:03
It really boils down to three things. One is make nutrition tracking dead simple. Keep it super easy, make it natural. Make it feel like you're texting a friend versus having to sit there and search through endless lists of different products.
- 1:16
Or, you know, at best, take a picture and cross your fingers that you're gonna get something remotely close to what you actually had. The other aspect is once you make tracking really easy, how do you take that information and build really strong, powerful context around the user, their flavor profile, their interests, their habits, their hobbies, and help
- 1:33
steer them in the direction that they tell you they wanna go in. And then finally, this is what we're excited about kicking off in the second half of this year, is how do we take all that information and get to a point where we're actually connecting you with products and different restaurants and meals that would actually help
- 1:48
you hit your goals. And that's a vision that we're very excited about. Beyond that, we've also discovered through our conversations with users, we, you know, we're-- me and Adam, uh, are-- my co-founder are really big believers in building in public.
- 2:02
Uh, so we actually started-- We shipped our beta less than two months after incorporation, and it was a closed beta for roughly about four months, and then we shipped in February of this year.
- 2:12
And we immediately started learning that users don't just wanna know what calories and macros they're having, but they also wanna get a rough sense of how holistically well they're eating.
- 2:21
They wanna know about the quality of the food. And we realized that, you know, a thousand calories could be had a million different ways, not all of it is equal in terms of quality.
- 2:30
And this is where we developed, uh, the Alma Score concept with, uh, our academic advisor at Harvard University, Dr. Eric Rimm. Alma Score is very simple. It's meant to be a score out of a hundred that can easily steer you throughout the day to just have more of the things that we know through all the research and
- 2:46
decades of science are gonna be fundamentally good for your health and try to nudge you away from some of the things that we know are not great for you.
- 2:55
Beyond that, let's just start talking about building an AI, though. I think that's what we're ultimately all here for. So one of the biggest lessons we learned during the beta is that it's really important to rely on user feedback.
- 3:06
I know we're all obsessed with evals right now. We're certainly looking at some, and we've experimented with some over the last few months. But fundamentally, nothing beats real-time, in-the-moment feedback from your users.
- 3:17
So one of my favorite features that we built to kinda achieve that is this little drop-down toast that pops up that says, "How did Alma do?" after every single interaction with Alma.
- 3:26
Uh, we obviously plan to tone that down eventually, but in the early days, it's been extremely helpful for us in guiding how we build Alma and how we can measure accuracy and improvements with Alma as you track with it.
- 3:38
Uh, the other thing we realized is that LLMs, if you give them an open-ended, uh, very large task, uh, the error rates are just gonna be through the roof.
- 3:47
It's really, really important to figure out a way to make it so that you can constrain the LLM to very specific tasks. And we, instead of kinda being frustrated at that, we realized that we just need to work with that constraint and find the best way that we can get our desired ultimate user experience going.
- 4:05
And one of the things that we executed on that I'm really happy and proud of the team for doing is we actually broke up the process of how we relay the information, uh, back to the user after every single time they track with Alma and track with Alma through steps.
- 4:18
So at every step, we send down certain aspects. So for example, for this screenshot, I just said to Alma, "I had a banana." Immediately recognized in one of our modules that banana was in the phrase.
- 4:28
It extracted that as a food item. It sent it down to the client, the iOS app. And then from there, now it's busy processing through all the different steps, including, uh, getting involved with our USDA database matching system so that it can actually find the right appropriate caloric content and send that down eventually.
- 4:46
This makes the experience feel so much faster for users, and it's much easier for them to interact with than just waiting for the entire process of every single step of our LLMs to actually be, uh, sent down.
- 4:58
So highly recommend you kinda think about how you could do that as well in your process. How do you break up the steps? The other thing is, uh, you know, we [chuckles] actually last week, one of our engineers broke streaks, and I've never seen our WhatsApp community and our intercom blow up the same way since.
- 5:12
Hopefully never again, but I'm okay with it if it happens. We ended up seeing so many users come back and say, "Don't mess with my streaks. What happened to my streak?"
- 5:19
And we realized that even though we thought of this as kinda like a little cute feature that we were just gonna add towards the end, it's clearly something that users really value, and this is something that we are planning to double down on and just really make sure that we're having fun while we're building Alma for us
- 5:33
and for our users. Beyond that, it's really, really important that when you're building an AI agent, we've discovered that it's very important that you're continuing to build context. Users get really frustrated if they have to reiterate the same things, whether it's when it comes to tracking or when it comes to actually asking Alma questions and engaging.
- 5:53
So we built out features like what you see here at the top, that little orb that says, "The value updated." Whenever you interact with Alma, we actually take whatever piece of information, if it's novel, and we add it to our knowledge data set about you.
- 6:05
So Alma's constantly learning, and you can view those yourself and delete and remove them as you wish, but you could also just continue to add to them, and Alma just gets smarter every single time you interact with it because of that.
- 6:16
Finally, we realized that you can only count on your users opening up Alma to ask questions and engage so often, you really wanna be proactive about reaching out to them.
- 6:27
So one of the features we built that was really cool in that regard is to actually have Alma detect certain insights about what you're eating and surface things you might not know about.
- 6:37
For example, I didn't discover that blueberries are really, really good for certain things. Like, if you pair it with dark chocolate, it's actually increases the absorption of vitamin C.
- 6:44
And there's a whole bunch of different things that I've learned through my engagement with Alma throughout, throughout it just kinda reaching out, pinging me, and letting me know certain things about my food that I didn't know about.
- 6:53
So it's-- feels like this constant process of learning about food, but in a bite-sized format. Pun not intended. The other thing we learned, this was something I obsessed about a lot early on in the days of building Alma, is to think through, you know, which modality is gonna be the winning one.
- 7:09
You know, I personally love voice. I love talking to Alma. I just turn it on at the end of the day if I'm busy, and I'll just track, you know, I'll just track my lunch, breakfast, and dinner all in one shot in under ten seconds.
- 7:20
It used to take me minutes and feel like this really laborious exercise I dreaded previously, and now it's that simple with voice. But I learned from engaging with our users and seeing how they interact with Alma, that they actually love the multimodality.
- 7:32
So I've often heard this from our users that they like the fact that Alma makes it really easy to talk to, to take a photo, or to text, and depending on the context.
- 7:41
So instead of really being bullish on one specific modality, I've learned that you really wanna provide your users with as many different modalities that make sense for them.
- 7:51
In terms of looking forward and where we think the space is going, where we're doubling down on Alma is it's really comes down to these three key learnings from our experience.
- 7:58
One is brand and design count for a lot. Code is becoming a commodity, and no matter how great your code is, if you don't have a product that's really user-centric and has a design that's visually appealing, it's gonna be extremely hard for you to stand out.
- 8:14
So we're doubling down on that. It's in our DNA. My co-founder, Adam, is absolutely focused on it, and I trust him fully in it. And then it's really an important part of how we kind of think about building Alma over time.
- 8:24
And then second pillar is, you know, really trust on partnerships. You earn your users' trust, especially when you're new, by seeing who they trust and partnering with them if there is an aligned mission.
- 8:34
So we're really focusing on that right now. We constantly ask our users where do they get their information from? Who do they trust? And we try to focus on seeing how we can bring some of those into the mix and partner with them i-in a win-win type format.
- 8:46
And then finally, community and new data is something that we're really, uh, shifting our focus towards now as well. We're realizing that as our community grows, there are so many interesting things that people wanna learn about how others are eating.
- 8:59
It's just kind of a very curious aspect of food consumption. Like, food is generally something that you-- people like to have in company, and people like to talk about food.
- 9:08
So people are very curious to know how else-- how other Alma members eat and how could they potentially learn from them or adopt some of their habits. So we're doubling down on features like that.
- 9:17
And, you know, I think the reality is most of the LLM foundation model companies and some of the bigger players are hoovering up data. So if there's something online that you're scraping and you're getting access to, there's a high chance that they've already done that.
- 9:30
So you really wanna think about how can you leverage your users and members to create net new data that's actually valuable and interesting, and this is something that we're doubling down on as well so.
- 9:40
I wish you all the best in this journey. We're just getting started at Alma, and if you wanna give it a shot, uh, shot and try it out, just, uh, give it a go.
- 9:47
And just as a thank you for taking the time to listen to this presentation, I wanted to throw you a, a little promo code here. Use ALMAFAM, chip it Alma.
- 9:54
I hope to see you in the Alma WhatsApp community soon. Take care.