Microsoft Introduces Voice, Vision, and Automation to Windows 11 Copilot

Microsoft announced some “intelligence” updates to its Windows 11 OS, which made a number of previously announced AI features more available to Copilot. The company is also enhancing the overall Copilot experience in a manner that they anticipate will make computing more natural and useful for people on a day-to-day basis.

Voice Activation

Users can now tell the Copilot to “activate” itself by simply saying “Hey Copilot” as opposed to typing out their command or requests. This feature is set to work in the background and the assistant is set to come to “life” as soon as it hears the user begin their vocal request. This could enable a new way to use the Copilot to quickly set reminders, compose emails, pull up information, and more with simple voice commands.

Copilot Vision for Windows

Copilot Vision for Windows is a feature that gives the Copilot the ability to see what is on a user’s screen, in order to provide contextually relevant help and suggestions based on what they are looking at on their devices. This could help power tasks like providing insights or suggestions on a spreadsheet the user is working on, for example, or suggesting corrections to a formula. It is currently available to all Windows 11 users.

Copilot Actions

Copilot Actions is a new feature that builds on the earlier-announced capabilities of the Copilot that is set to enable the AI assistant to perform actions directly inside applications based on user prompts. Copilot Actions will allow the assistant to write and edit documents, manage files, compose emails, and much more using natural language. Actions can be activated using voice or text commands. “Actions” are designed to be accessible with a number of different apps and web browsers including Word, Outlook, Teams, Edge, and more.

Copilot Connectors

Microsoft is also enhancing the third-party integrations the Copilot has through Copilot Connectors, which will allow the assistant to help users link their Gmail, Google Calendar, and Google Photos accounts along with Google Drive, OneDrive, and Microsoft Outlook among others. This will allow the assistant to tap into and manage information from across the different services and products a user has a stake in with the company.

Security and Privacy

Microsoft says that security, user control, and privacy are very much a part of the considerations that went into the enhanced capabilities and that features like Copilot Actions is meant to work within isolated workloads with express approval by a user before the assistant has access to specific kinds of data and information.