Google’s new Pixel 9 can search your screenshots with AIYour screenshots sometimes contain vital information but retrieving them is not always easy.
https://www.washingtonpost.com/technology/2024/08/13/google-pixel-9-feature-ai-screenshots-photos-gemini/SAN FRANCISCO — Mixed into my phone’s photo library right now are 4,811 screenshots, many of them containing information I thought I’d want to hang on to. But capturing those images was the easy part — finding them when I needed them was the real problem.
Google says it has a solution.
Among other new AI tricks, Google’s new Pixel 9 smartphones — unveiled Tuesday at the company’s Bay View campus in Mountain View, Calif. — can help you find vital info like WiFi passwords, event details and more locked away in the screenshots you’ve snapped in the past.
All you have to do, the company says, is to ask, either in a new Pixel Screenshots app or by interacting with its AI-powered Gemini assistant.
That means the days of scrolling endlessly through photos to find that one crucial image is just about over — if you’re willing to switch to a Pixel, anyway. After chewing on your request, the app (ideally) answers it with the information pulled from the screenshot — and shows you the images that data came from, just in case.
The feature isn’t on by default — you have to specifically opt-in if you want to use it. And thankfully, none of your screenshots are ferried off to far-flung servers to be processed; Google says all of its image analysis happens directly on its Pixel phones, thanks in large part to its new Tensor G4 processors.
If you take a screenshot of an app that contains information you want to remember, like this door code …
… The Pixel Screenshots app can later tell you what the door code is when asked. (Google)Compared to the current crop of AI tools embedded into smartphones — like text rewriters, image manipulators and audio transcribers — this attempt to give phones a kind of visual memory could be surprisingly useful in daily life. After all, who among us hasn’t taken a screenshot of something they wanted to remember?
There are, of course, some caveats. The feature can pull useful tidbits from new screenshots you take, but by default, it can’t look at older ones saved in services like Google Photos. (You can, however, manually add older images to the Pixel Screenshots app if you really wanted to.) The app itself admits that its responses “may be inaccurate,” though it gives you an option to double-check its answers.
For better or worse, this tool is also one that Google plans to keep exclusive to its Pixel 9 smartphones, rather than make available to outside Android phone makers the company has partnered with on AI features in the past.
That’s not great news for owners of Android devices that already have the hardware needed to run AI models without having to reach out to the cloud — and there are going to be plenty of those, according to research from IDC.
Shipments of these “generative AI smartphones” are expected to more than quadruple this year, the firm says. But the decision to keep certain AI features solely on its own devices might be crucial to Google as it seeks to carve out a more substantial share of smartphone shoppers.
Despite continually shipping more Pixel phones each year since 2021, Google accounted for just 0.8 percent of global smartphone market share in the first half of 2024, according to market research firm Canalys.
Other new AI features baked into Google’s new phones include a chattier version of its Gemini AI called Gemini Live, which can now engage in long, meandering conversations with its owner — much like how users can speak naturally to tools like OpenAI’s ChatGPT. The company says Gemini can now also react to, and offer additional information about what’s displayed on the phone’s screen, beating Apple’s upgraded Siri to the punch — and to user’s pockets.
“Google had a first-mover advantage in the market that helped it differentiate, but the competition is catching up,” said Runar Bjorhovde, an analyst at Canalys.
Total Posts: 10261