Google Is Always Listening. Now It’s Watching, Too, With the Nest Hub Max.

With the new Nest Hub Max, a smart speaker with a screen, Google is adding an eye to its talking artificial intelligence. When I flash my palm at the device, a camera spots me and immediately pauses my music. Talk to the hand, robot!

When I walk by a Hub Max, the Google Assistant greets me on its screen, “Good afternoon, Geoffrey.”

This wizardry is made possible by facial recognition. The $230 (roughly Rs. 16,500) Nest Hub Max offers a glimpse of how this controversial tech might be used in our homes – if people aren’t too turned off by the privacy implications.

Living with Google’s latest creation for a few days embodied the cognitive dissonance of being a gadget guy in 2019. You can appreciate the fun and wonder of new technology that you also know brings new concerns. I kept wondering: Do any of these camera functions make it worth bringing face surveillance inside my home? Despite some applaudable privacy protections from Google, my family never got to a yes.

The Hub Max is a larger 10-inch version of Google’s popular Nest Hub countertop computer, which people (including me) use as a digital picture frame, speaker, kitchen TV and smart home controller. It’s a solid upgrade for those functions, with a sharp screen and impressive sound for such a small box.

But it’s the addition of a wide-angle camera that everyone will be talking about. When the smaller Hub debuted in 2018, Google crowed about how it didn’t include one – unlike Facebook’s poorly received Portal and Amazon’s Echo Show, which raised eyebrows with a feature that lets you “drop in” via camera any time on select friends. (Amazon chief executive Jeff Bezos owns The Washington Post.)

The Hub Max takes those camera controversies and says, “hold my beer.” Rival Amazon Alexa can recognize different people’s voices, but not faces. What made Google think now was a good time to go there? It’s a sign both that Google thinks it has better privacy protections – and that consumers are, incorrectly in my estimation, more trusting of Google than of other tech giants.

What do you get out of letting Google watch you through a Hub Max?

First, the camera can be used for basic video calls. Say, “OK, Google, video call James,” and it connects via Google’s Duo chat app on a phone, laptop or another Hub Max. Like the Facebook Portal, the Hub Max’s wide-angle camera pans around to track your movements and tries to keep everyone’s faces in the frame.

But that’s just the beginning of Google’s camera story. If you opt in to the Hub Max’s “face match” features, Google asks each family member to briefly scan his or her mug. Then the Hub Max’s camera stays on, ambiently looking for those faces. When the Assistant finds one, it tailors the content on its screen. In the morning, it spotted me and proactively displayed a rundown of my day and commute – I never issued a command. You can also use it to leave reminders that pop up when the intended recipient passes by.

Understanding context is one of the biggest challenges for computers, and Google’s face technology opens interesting possibilities for personalisation in shared spaces.

Next, the camera powers gesture commands. While it’s looking for faces, the Hub Max is also keeping an eye out for your hands, which can serve as nonverbal, non-touchscreen commands – in other words, sign language. For now, these are limited to “play” and “pause” music by flashing a palm, but Google says it’s exploring more.

Finally, the Hub Max can serve as a security camera, letting you peek in on a stream of what’s going on at home. And like other Google Nest cameras, you can opt in to having the Hub Max send alerts when it identifies familiar faces. With this, you could always know exactly what time the kids got home . . . until they figure this out and unplug the darn thing.

Is all of that cool, or just creepy? The Hub Max reminded me, at times, of a baby HAL 9000 from “2001: A Space Odyssey.” The extra step of the Assistant knowing you’re there even when you don’t interact with it shifts the power balance in a subtle but important way.

Why should we trust Google with any of this? First, all of the camera-sensing features are optional – you can still use the Hub Max with only your voice to play music and watch TV without turning the camera on. And Google Assistant has better default settings for voice privacy than Amazon’s Alexa.

Google also built some visual cues and controls for the camera. There’s a physical button you can press to cut the camera. A green light turns on whenever the security camera is live broadcasting, though not while the other camera-sensing features are running.

For the face-matching functions, which require the camera to be always looking, Google borrowed a page from the Apple playbook. All the processing to match your face happens on the device itself. Those images are deleted immediately, and Google says it does not keep a record of when it identifies faces. (This is similar to how Face ID works on an iPhone – your face never leaves the iPhone.) Google has also pledged to “keep your video footage, audio recordings, and home environment sensor readings separate from advertising.”

As a privacy guy, I’m glad Google is taking those steps. Keeping face data local makes it less likely to be stolen or misused by police, marketers or data spies – if you trust Google to maintain its data standards and security precautions.

But with Google, it’s always important to ask what the company is getting out of the deal. Your face matches may stay on the Hub Max, but any interaction with the Assistant (like tapping on the screen or issuing a voice command) gets added to your Google profile which can, by default, be used to target you with ads. The Hub Max also gets you to use more Google services, like YouTube, search and Duo, all of which also add to its valuable profile of you.

The concerns only multiply from here. Today the face features are only accessible by Google, but what happens when it begins letting other apps and services access your face? Or when the tech also detects emotion in a face?

Ultimately, the Hub Max suffers from the same affliction as many new Google products: It’s frighteningly advanced technology that hasn’t identified the problem in our lives that needs solving. None of the camera functions the Hub Max offers today make it worth bringing surveillance inside my house.

Google and all the other companies pushing face tech are going to have to keep working on uses that cross the chasm from creepy to can’t-live-without-it.

© The Washington Post 2019