The Ray-Ban Meta smart glasses are poised for significant enhancements, thanks to upgrades in the social network’s AI assistant. The onboard assistant is now gaining support for real-time information, and Meta is testing innovative “multimodal” capabilities that enable it to respond to questions based on the user’s environment.
Previously, Meta AI had a “knowledge cutoff” up until December 2022, limiting its ability to answer questions about current events, game scores, traffic conditions, and other on-the-go inquiries. However, Meta CTO Andrew Bosworth announced a change, stating that all Meta smart glasses in the United States will now have access to real-time information, partly powered by Bing.
In a separate development, Meta is initiating tests on one of its assistant’s intriguing features known as “multimodal AI.” This capability, previewed during Connect, enables Meta AI to provide contextual answers about the user’s surroundings and respond to queries based on what the user is observing through the glasses.
The updates have the potential to transform Meta AI from a novelty into a more practical and useful tool, addressing a notable concern highlighted in my initial review of the otherwise impressive smart glasses. However, widespread access to the new multimodal functionality for most smart glasses users may still be some time away. Meta CTO Andrew Bosworth mentioned that the early access beta version will initially be limited to a “small number of people who opt in” in the United States, with broader access anticipated in 2024.
Mark Zuckerberg shared several videos showcasing the new capabilities, offering a glimpse of what users might experience. From the clips, it seems that users can activate the feature with commands starting with “Hey Meta, look and tell me.” For instance, Zuckerberg demonstrates asking Meta AI to assess a shirt he’s holding and suggest matching pants. Screenshots also revealed Meta AI identifying an image of a piece of fruit and translating the text of a meme.
In a video posted on Threads, Bosworth added that users will have the ability to inquire about their immediate surroundings and pose creative questions, such as generating captions for recently taken photos.
