Google Brings Machine Learning To Calorie Counting


Last week, Boston hosted the Rework Deep Learning Summit, a conference focused on the rapidly expanding role that artificial intelligence, neural networks, and deep learning algorithms are playing in the evolution of software. The conference attracted more than 50 speakers from schools like MIT, Harvard, and Princeton, and notable tech companies like Google, Twitter, eBay, and EyeWire. During the conference, Google research scientist Kevin Murphy spoke about a previously undisclosed project he is working on with his team at Google that uses artificial intelligence programming methods to analyze photos of food and then accurately calculate its total calorie count.

Murphy is a respected leader in the AI field. He earned a BA in Computer Science from the University of Cambridge, a PhD in Computer Science from UC Berkeley, and then did his postdoctoral fellowship at MIT’s Artificial Intelligence Lab. He has published over 80 papers on machine learning and is the Editor-in-Chief of the Journal of Machine Learning Research. Now, he is working at Google, applying his knowledge of machine learning to the laborious task of keeping a food journal.

Called Im2Calories, Murphy’s app allows users to take a picture of a plate of food and then algorithms analyze the photo to identify what foods are on the plate, the serving size of each item, and then the total calorie count for the meal. The technology that powers the feature was originally developed by a UK startup called DeepMind that was pioneering artificial intelligence methods until Google acquired it for $400 million in 2014. Google’s intent with the acquisition was to incorporate the advanced methods into its search engine algorithms, which it is currently working on, but a number of side projects have sprung up around Google’s Mountain View campus that are also leveraging the technology.

In his presentation, Murphy demonstrated the app calculating the total calories of a plate of food with two eggs, two pancakes, and three strips of bacon on it. He acknowledges that because the system relies on caloric data from standard nutritional labels, that themselves can be inaccurate, the results are sometimes off, missing by more than 20 percent on occasion. In the long run, if users will engage with the app initially, this problem should correct itself as the very premise of machine learning is that it analyzes the data it receives and self-corrects its own algorithms over time. Murphy explains that while the app is not perfect yet, the potential is promising enough to continue moving forward. He explains, “Ok fine, maybe we get the calories off by 20 percent. It doesn’t matter. We’re going to average over a week or a month or a year. And now we can start to potentially join information from multiple people and start to do population level statistics. I have colleagues in epidemiology and public health, and they really want this stuff.”

Google recently filed a patent for Im2Calories but Murphy would not disclose when the features might become available.

Enjoy HIStalk Connect? Sign up for update alerts, or follow us at @HIStalkConnect.

↑ Back to top

Founding Sponsors

Platinum Sponsors