Microsoft’s Copilot AI gets a voice and the ability to see websites you browse

Beyond debuting new features for Copilot+ AI PCs and Windows 11’s 2024 update, Microsoft is also giving its Copilot AI a makeover on the web, mobile and desktop. That includes a slightly friendlier interface wherever you access it, along with new capabilities like Copilot Voice, which allows you to talk conversationally with the AI assistant. Ultimately, Microsoft is aiming for Copilot to be seen as more than just a party trick for generative AI search and image creation — it’s trying to make it a core part of your daily workflow.

That starts with a cleaner and simpler UI that makes Copilot look different than a boring old search engine. That new look hits Copilot on the web, iOS, Android and Windows today. You’ll also be able to access Copilot from within Whatsapp, which could be useful if you want to avoid Meta’s AI assistant.

Microsoft also plans to release more experimental AI features via Copilot Labs, similar to the way Google has given users access to early features with its own Labs. There’s Copilot Vision, which lets the AI see everything you’re viewing in Edge, and Think Deeper, a method for letting Copilot work harder on more complex dilemmas. Both features also have the potential to create new privacy and security issues, but Microsoft appears to have learned a bit from its messy rollout for Recall on Copilot+ PCs. Notably, these Copilot Labs offerings will only be available to people paying for Copilot Pro subscriptions.

Microsoft says Copilot Vision sessions are “opt-in and ephemeral,” and nothing is stored for AI training. Any data related to your session is deleted once you’re done. The feature also won’t work on all websites, only a “limited list of popular websites.” And Microsoft notes that Copilot only interprets images and text that you’re seeing on a website; it’s not doing any major content processing. The company also plans to take feedback from early users, refine safety measures and “keep privacy and responsibility at the center of everything we do.”

There’s no doubt that Microsoft has to re-establish trust with its users after the botched rollout for Recall, a feature that was meant to help you find anything you’ve done on your computer, but had some gaping security holes early on. “Ultimately, users want to have confidence with anything happening with sensitive data caches,” Microsoft’s Windows head, Pavan Davuluri, said when we asked him about the lessons the company learned from the Recall fiasco. He added that he believes the company has “gone above and beyond” to create that confidence by establishing new security standards for Recall.

There are less controversial features coming, as well: Copilot Daily uses Copilot Voice to delivery a summary of news, weather and potentially reminders about tasks every day. Based on the demos we’ve seen, it’s a lot like an early morning podcast built just for you. Copilot Discover, meanwhile, will give you tips about AI features to use (pictured in the screenshot above), depending on Microsoft services you’ve used in the past.

Personally, I’m still a bit ambivalent about Microsoft’s near-instant transformation into an AI company. That’s particularly true since we’re still dealing with inaccuracies and other issues from OpenAI’s ChatGPT, which Microsoft relies on to power Copilot. These Copilot features seem a bit more useful than being a souped-up search engine, but the company still needs to prove it can build AI features with bulletproof security and privacy.

As with practically any AI product these days, Microsoft is also staggering the release of these Copilot features. Copilot Voice and Daily launch today, but Voice will only be available in English in the US, Canada, Australia, the UK and New Zealand. Copilot Daily, meanwhile, only works in the US and UK for now. Copilot Vision will head to Labs over the coming days for a “limited number” of Copilot Pro subscribers, while Think Deeper is available for Pro users in the US, Canada, Australia and New Zealand.

This article contains affiliate links; if you click such a link and make a purchase, we may earn a commission.