Computer Use — Extending Beyond APIs
Put Co-Work in control of native desktop applications — with a principled safety model for every action it takes.
What you'll learn
The Line That Changes Everything
Everything Co-Work does — running skills, executing connectors, reading files, spawning sub-agents — happens inside a virtual machine running on your computer. That VM is sandboxed. It is isolated. If something goes wrong inside, the blast radius is contained.
Computer Use is different. When Co-Work uses Computer Use, it operates directly on your host machine — outside the VM, in the same environment where your banking app, your email client, and your file system live.
This is not a subtle distinction. It is the most important fact in this module, and it has direct implications for how you configure and constrain Computer Use before enabling it.
Computer Use operates OUTSIDE Co-Work's virtual machine sandbox, directly on your host machine. It can access, move the mouse on, and interact with any application you have not explicitly blocked. Configure your blocklist before enabling it. This capability is currently a research preview available on Pro and Max plans on macOS. Windows support is coming.
What Computer Use Actually Does
The mechanism is straightforward once you understand it. Co-Work takes a screenshot of your screen, analyzes what it sees, determines the appropriate action, moves the mouse and/or types, then takes another screenshot to verify the result. It repeats this perception-action loop until the task is complete.
This is how it controls any native application that has no public API. The app does not need to support any integration protocol. Co-Work interacts with it the same way a human would — by looking at the screen and clicking things.
The practical implications:
- Any visual interface is controllable — legacy desktop software, apps without APIs, native macOS apps, custom internal tools
- Speed is slower than API calls — each screenshot-analyze-act cycle takes time; use connectors when they exist
- Brittleness is higher — if the UI changes or a dialog appears unexpectedly, the workflow can fail; design for fallback
- Actions are persistent — Computer Use can send messages, submit forms, make purchases; these are real-world actions that cannot be undone by closing a tab
The Three-Way Decision Framework
Computer Use is the tool of last resort, not the default. Before reaching for it, work down this decision tree:
| Situation | Right Tool | Why |
|---|---|---|
| App has a native Co-Work connector | Connector | Fastest, most reliable, structured API calls — no visual parsing required |
| No connector, but app has a web UI | Chrome (browser automation) | Browser-based interaction is faster and more stable than full desktop control |
| No connector, no web UI — native desktop only | Computer Use | Last resort when there is no other path |
This priority order comes directly from Anthropic's official documentation. Connectors are fastest and most reliable. Chrome browser automation is second. Computer Use is third — reserved for applications that genuinely have no other integration path.
The connector → Chrome → Computer Use priority order is documented in Co-Work's official sources. This is not a community preference — it is the architecture Anthropic designed. Connectors are faster, more reliable, and safer than Computer Use for every task they can handle.
Setup: Three Required Steps
Computer Use requires three setup steps, in this order. Do not run Computer Use tasks until all three are complete.
Step 1: Enable in Settings
Open Co-Work Settings → Computer Use → toggle Enable. This activates the feature but does not grant any permissions yet.
Step 2: Grant Accessibility Permission
Open System Settings → Privacy & Security → Accessibility → find Claude Co-Work → toggle Allow. This permission lets Co-Work move the mouse and simulate keyboard input in other applications.
Step 3: Grant Screen Recording Permission
Open System Settings → Privacy & Security → Screen Recording → find Claude Co-Work → toggle Allow. This permission lets Co-Work take screenshots to see what is on screen.
Both system permissions are required. Accessibility without Screen Recording means Co-Work can click but cannot see. Screen Recording without Accessibility means it can see but cannot act. Both are necessary for the perception-action loop to work.
The Per-Application Blocklist
Before running any Computer Use task, configure your blocklist. This is one of the most important safety controls available and one that no community video has covered in detail.
Co-Work has a default blocked category list that includes banking applications, healthcare portals, cryptocurrency wallets, and investment platforms. But the default is a starting point, not a complete configuration.
To configure the blocklist: Settings → Computer Use → App Permissions → Block [app name].
The principle to apply is straightforward: block any application where an unreviewed action would be irreversible or financially significant. That means:
- Any banking or financial app not already on the default list
- Any app that can send communications on your behalf (Slack, email clients, messaging apps)
- Any app with purchasing or payment capability
- Any app with administrative access to services you depend on
Blocking an app does not prevent you from using Computer Use with other applications. It simply ensures that even if a task goes in an unexpected direction, Co-Work cannot touch the applications you have designated as off-limits.
The "Check Before Irreversible Action" Guardrail
The single most effective safety measure for Computer Use is a single sentence added to your global instructions. Add this now, before you run any Computer Use task involving write operations:
"Before taking any irreversible action via Computer Use, describe what you are about to do and ask for my confirmation."
Irreversible actions include: sending a message, submitting a form, making a purchase, deleting a file, changing account settings, posting anything publicly. Reversible actions include: opening an application, navigating within an app, reading information from the screen.
With this guardrail active, Computer Use tasks pause before crossing the point of no return. You review what Co-Work is about to do, confirm if it looks correct, and then it proceeds. This single instruction converts Computer Use from an autonomous actor into a supervised collaborator for all write operations.
Add the "check before irreversible action" instruction to your global instructions before running any Computer Use task that involves writing, sending, submitting, or purchasing. The guardrail costs nothing and prevents the category of mistake that is hardest to recover from. This is a principle shared by multiple practitioners including Swyx, who recommends always telling Claude to check before irreversible actions in any agentic workflow.
High-Value Use Cases
With setup complete and guardrails in place, Computer Use opens workflows that are genuinely impossible through connectors or browser automation:
AI-Powered Shopping with Approval Flow
Paul (Co-Work practitioner) describes a workflow combining Dispatch and Computer Use: send a purchase request from your phone via Dispatch, Co-Work identifies the item and finds it in the target app, then pauses before checkout and sends a summary back to Dispatch for approval. You approve from your phone, and it completes the purchase.
The critical ingredient: the approval step. Do not remove it. Paul also recommends using Privacy.com virtual cards with per-merchant limits for any automated purchasing workflow — a community practice that limits financial exposure from unexpected behavior.
Controlling Legacy Desktop Software
Enterprise software that predates public APIs — accounting systems, specialized industry tools, internal applications — can be controlled via Computer Use. Instead of manually navigating through multi-step export workflows, Co-Work can execute them on schedule. The same guardrail applies: any action that modifies data or sends output requires confirmation.
Multi-App Workflows Without API Chains
Some workflows span applications that individually have connectors, but the specific cross-app sequence has no native integration path. Computer Use can bridge these gaps — though the guidance is to exhaust connector options first before using screen-based automation for the same task.
Configure Computer Use and Run Your First Low-Stakes Task
Work through these steps in sequence. Do not skip the blocklist step — it comes before any test tasks for a reason.
Success criteria: Computer Use enabled with both system permissions granted. Blocklist configured with at least one blocked app. Guardrail active in global instructions. Two tasks completed — one read-only, one write task that correctly triggered the approval gate.