--- url: "https://www.xcademia.com/news/github-copilot-can-now-interact-with-desktop-apps-using-computer-use" title: GitHub Copilot Can Now Interact With Desktop Apps Using Computer Use description: "GitHub Copilot can now interact with desktop apps on macOS and Windows, expanding AI-assisted automation to GUI-only and legacy software." publishedAt: "2026-10-02T09:48:51.942+00:00" updatedAt: "2026-10-08T00:43:50.60304+00:00" type: news category: "ai-ml" source_name: GitHub Changelog source_url: "https://github.blog/changelog/2026-10-01-github-copilot-can-now-interact-with-desktop-apps/" tags: - "#GitHubCopilot" - "#ComputerUse" - "#AIAgents" - "#DeveloperTools" - "#Automation" - "#GitHub" - "#AgenticAI" - "#AI" --- # GitHub Copilot Can Now Interact With Desktop Apps Using Computer Use > GitHub Copilot's new computer use capability lets it interact with desktop applications on macOS and Windows, including GUI-only and legacy software, while keeping users in control through permissions and approval workflows. Source: **GitHub Changelog** · 2 October 2026 GitHub has introduced computer use for GitHub Copilot, allowing the coding assistant to interact directly with desktop applications. The capability is now available in public preview through GitHub Copilot CLI and the GitHub Copilot app on macOS and Windows. The feature expands Copilot beyond traditional code and command-line workflows. It can interact with graphical applications by reading accessible application content and visual context, clicking controls, entering and editing text, pressing keys, scrolling, dragging items, and navigating workflows across applications. ## From Code Assistance to Desktop Workflows The key change is that Copilot can now work with applications that may not expose an API, command-line interface, or MCP integration. This is particularly relevant to legacy software and GUI-only applications. GitHub gives examples such as reviewing information in a legacy desktop application, updating presentation content, entering information into GUI-based software, and moving information between applications as part of a multi-step workflow. For example, instead of requiring a dedicated integration for every application, a user can describe the desired outcome and the applications involved. Copilot can then use computer-use tools to navigate the graphical interface. ![info-1](https://0a515t3ure77wbvx.public.blob.vercel-storage.com/articles/1790934384623-info1--104-.webp) ## Human Approval Remains Part of the Workflow Computer use is disabled by default. Users must explicitly enable it before Copilot can access the capability. Depending on the configured permissions, Copilot can ask for approval before controlling an application. Users can grant access for a session, save approval for future sessions, or deny access. GitHub also allows users to review applications that have been given persistent approval. Enterprise administrators can disable computer use through managed settings. On macOS, users are guided through the required Accessibility and Screen Recording permissions. The feature can also be interrupted if an operation behaves unexpectedly. ![info-2](https://0a515t3ure77wbvx.public.blob.vercel-storage.com/articles/1790934402099-info2--105-.webp) ## Why GUI Automation Matters The addition addresses a practical limitation of software automation. Many applications still depend heavily on graphical interfaces and do not offer a suitable API, command-line tool, or MCP server. GitHub's documentation notes that direct tools such as APIs, MCP servers, terminal commands, filesystem tools, or dedicated browser tools typically provide more structured and predictable results when they are available. Computer use is intended for situations where interaction with a visual interface is required. This distinction is important for organisations evaluating AI agents. Computer interaction provides broader access to existing software, but it also introduces another layer of operational risk. ## Security and Reliability Considerations GitHub warns that computer use can encounter changing interfaces, dynamic controls, unexpected window states, and other conditions that may result in incorrect actions. An agent could potentially select the wrong control or enter information in the wrong location. The company also warns that ambiguous instructions or unexpected on-screen content can lead to unintended actions affecting devices, data, or connected accounts. GitHub specifically advises users to review permissions, target applications and results, particularly when actions could modify data or affect other people. This means computer use should be viewed as an interactive automation capability rather than a replacement for human oversight. ![info-3](https://0a515t3ure77wbvx.public.blob.vercel-storage.com/articles/1790934421459-info3--104-.webp) ## How to Enable Computer Use In GitHub Copilot CLI, users can run: `/computer on` The status can be checked with: `/computer show` Computer use can be disabled with: `/computer off` In the GitHub Copilot app, users can open **Settings**, select **Computer Use**, and enable the feature. GitHub also documents `/computer on` as an option in the app. Computer use is currently in **public preview**, meaning the capability remains subject to change. ## What This Means for AI-Assisted Workflows The announcement highlights a broader shift from AI assistants that primarily generate information toward agents that can interact with software interfaces. For developers and enterprises, the immediate significance is not simply that Copilot can click buttons. It is that an AI agent can potentially operate across software environments where conventional integrations are unavailable. At the same time, the GitHub documentation makes clear that computer use has different characteristics from structured integrations. Where an API, MCP server or command-line interface is available, those approaches can offer more predictable interaction. The practical value of computer use will therefore depend on the type of software being automated, the permissions granted to the agent, and the level of human oversight applied to the workflow. GitHub Copilot computer use is currently available in public preview on GitHub Copilot CLI and the GitHub Copilot app for macOS and Windows. ## Original source https://github.blog/changelog/2026-10-01-github-copilot-can-now-interact-with-desktop-apps/ ## Tags `#GitHubCopilot` · `#ComputerUse` · `#AIAgents` · `#DeveloperTools` · `#Automation` · `#GitHub` · `#AgenticAI` · `#AI` --- ## About this content This Markdown news article is the citation-grade twin of [GitHub Copilot Can Now Interact With Desktop Apps Using Computer Use](https://www.xcademia.com/news/github-copilot-can-now-interact-with-desktop-apps-using-computer-use). It is published by **Xcademia** (UK Companies House 12322710) and is available for AI search engines and large language models to index, summarise, and cite. When citing or quoting, please attribute *Xcademia* and link back to the source URL above. - Source: https://www.xcademia.com/news/github-copilot-can-now-interact-with-desktop-apps-using-computer-use - Publisher: Xcademia, https://www.xcademia.com - Catalogue index: https://www.xcademia.com/llms-full.txt