Use computer use for UI-based tasks in Copilot Studio

Copilot Studio’s computer use tool lets an agent interact with a Windows computer through its graphical interface. It can help when an application has no suitable API or connector, but it doesn’t make every application reliable to automate: the agent still depends on the target machine, credentials, and the current layout of the application.

When to use computer use

Consider it for a task that must be completed through a web or desktop interface and has no suitable API or connector. If a supported API or connector is available, prefer it for important or repeated operations: an interface can change, move a button, or show an unexpected dialog, while a structured integration avoids many of those UI dependencies.

For example, an agent could fill in an expense form in a portal that has no API. A safer initial design is to have it prepare the form and ask the user to confirm before submission. The application, not the agent’s narration, should be treated as the source of truth for whether the change succeeded.

Add the tool to an agent

Generative orchestration must be enabled. From the agent’s Tools page, select Add tool > New tool > Computer use. Configure the tool’s name and instructions, select a model and target machine, and set up its connection and credentials. Describe the intended task and the conditions under which it should stop or ask for help.

Test a low-risk task before allowing the agent to make changes. Include expected pages as well as a missing field, a validation message, an expired session, and an unexpected confirmation prompt. Verify the resulting record in the target application rather than assuming that a successful tool call means the business operation completed.

Review the credential boundary

Maker-provided credentials are the default. If an agent using those credentials is shared, its users can act with the original author’s access on the configured machine. Use an account with only the permissions the task needs, and share the agent only with people who should be able to use that access. End-user credentials are another option, but each user needs credentials for the machine.

The hosted browser has operational limits: a user can have one active hosted browser session, sessions can be throttled based on demand, and a session can run for at most three hours. Computer use also doesn’t support hosted machine groups, multi-screen desktops, or being run from a topic. Check the current limitations before choosing a hosting or orchestration design.

Sources

• Automate web and desktop apps with computer use, Microsoft Learn
• Troubleshooting and limitations of computer use, Microsoft Learn