Google Lens AI-Powered Feature To Let You Record & Search Any Query

When a query is made, the custom Gemini model, powered by cloud technology, meticulously processes the video frame-by-frame to generate relevant search results.

The "Ask with video" feature in Google Lens is driven by cutting-edge speech models, advanced visual processing, and a custom Gemini AI model hosted on the cloud.

Source : Google

Google showcased a new Lens capability named Google Lens Ask With Video at the Google I/O 2024 event. It is powered by the multi-model Gen AI technology and lets users ask questions using a video. Now the tech giant is finally rolling it out to Android and iOS devices to add to the usefulness of Google Lens. As per Google, Lens’ “Ask with video” feature will be quite useful when it comes to troubleshooting gadgets.

Google also gave a demo in which it showcased how the feature can be used to troubleshoot a faulty old record player without having to go through a number of complex web searches.

How Does Google Lens' New Feature Work?

When users perform a search on Google, they will now receive an AI-generated overview of potential answers along with links to more detailed information. The "Ask with video" feature in Google Lens is driven by cutting-edge speech models, advanced visual processing, and a custom Gemini AI model hosted on the cloud. This allows Google to analyse videos in real-time, frame by frame, providing accurate search results in seconds. This capability enhances user experience by quickly addressing complex queries that would typically take longer with manual searching.

When a query is made, the custom Gemini model, powered by cloud technology, meticulously processes the video frame-by-frame to generate relevant search results. This AI-powered analysis occurs in just seconds, offering a significantly faster alternative to manual searching, which could take several minutes to achieve the same precision. In various test scenarios, Google Lens provided results with remarkable accuracy. This feature is particularly beneficial for DIY enthusiasts, as it simplifies troubleshooting or repairing gadgets by delivering timely and detailed instructions through video analysis.

Google Lens is available as a standalone app on Android devices, allowing users to explore its visual search capabilities directly. However, iPhone users don't have access to a dedicated Lens app; instead, they can use the feature within the Google app. Meanwhile, Apple is rolling out its own competing technology, called Visual Intelligence, as part of Apple Intelligence. This feature enables users to ask questions about anything simply by holding down the camera capture button, similar to Google Lens’ functionality for visual-based queries.