It seems December is the month for exciting AI announcements. Microsoft announced on Thursday that its Copilot Vision tool is now in preview for select ysers.
The company declared in a blog post that “Vision is the first AI experience of its kind, available only on Microsoft Edge.” The tool is meant to see what you’re seeing on your screen, making it easier for the AI to assist you in whatever task you’re working on.
And if you’ve tried any Copilot-related service in the past and had a bad experience, don’t write Copilot Vision off just yet. Furthermore, the company has acknowledged the criticism from users: “In the last few weeks, we have taken our users’ feedback and worked to improve further the speed, simplicity, and personality of Copilot and its Voice to make it a more helpful AI companion.”
Copilot Vision is described as “having a second set of eyes as you browse,” and that sounds like the ideal AI companion to me. This is an opt-in tool, so you don’t have to enable it if you don’t want to, but the way Microsoft is painting it, it looks like a useful feature in your AI toolbox.
How will Copilot Voice be useful? And is it worth privacy risks?
Microsoft showed off Copilot Vision in action in a few different ways.
In one example, Copilot Vision was used to plan a museum visit. The AI could answer conversational questions, navigate on-screen, and show where to click on the website for more related information.
Copilot Voice can also assist with holiday shopping, although Microsoft clarified that it can only interact with “a select set of websites to start.” In the video, Copilot Voice heads to Target’s website and can find ugly Christmas sweaters and filter options for men specifically, as well as read on-screen details about how to clean the sweater.
Another supported website is GeoGuessr.com, which hosts a browser-based game that Copilot Voice can teach you how to play in real time.
Copilot Voice could prove to be quite useful in the first scenario, which involves a lot of planning, especially for those who love to research before eating at a new restaurant, moving to a new city, or going on vacation.
An AI companion — perhaps Copilot Voice — can help someone moving cross country how much they need to budget for moving costs, when they should start packing, and the best cities to stop in along the drive, based on total trip time. One perk to Copilot Voice versus other options would be the ability to speak their questions and requests rather than type them.
For those worried about privacy or security, Microsoft makes it clear that it is “prioritizing copyright, creators, and [its] users’ privacy and safety.” Copilot Vision is entirely opt-in, and your data and the context Vision gathers during a session isn’t saved.
The company announced in the Thursday blog post that “Only Copilot’s responses are logged to improve our safety systems. “
The company plans to roll out Copilot Vision slowly. At first, only “a limited number of Copilot Pro subscribers in the United States, through Copilot Labs,” will be able to test Vision as a preview, and it’ll only work with a select set of websites. Then, the company “will expand access to more Pro subscribers and more websites over time.”
