CosmicTaco

Anthropic Unveils AI Model Capable of Controlling Desktop Apps

CosmicTaco · 2024-10-23T09:13:56.351868+05:30

- Anthropic has released an upgraded version of its **Claude 3.5 Sonnet** model that can interact with any desktop app. - The AI model can emulate keystrokes, button clicks, and mouse gestures, essentially acting like a human using a PC. - Developers can access this feature via Anthropic’s API, Amazon Bedrock, and Google Cloud’s Vertex AI platform. - Despite its capabilities, the model struggles with basic actions like scrolling and zooming and often misses short-lived actions and notifications. - Anthropic has taken steps to deter misuse, such as not training the model on users’ screenshots and prompts, and preventing the model from accessing the web during training. Source: [Techcrunch](https://techcrunch.com/2024/10/22/anthropics-new-ai-can-control-your-pc/)

Anthropic has released an upgraded version of its Claude 3.5 Sonnet model that can interact with any desktop app.
The AI model can emulate keystrokes, button clicks, and mouse gestures, essentially acting like a human using a PC.
Developers can access this feature via Anthropic’s API, Amazon Bedrock, and Google Cloud’s Vertex AI platform.
Despite its capabilities, the model struggles with basic actions like scrolling and zooming and often misses short-lived actions and notifications.
Anthropic has taken steps to deter misuse, such as not training the model on users’ screenshots and prompts, and preventing the model from accessing the web during training.

Source: Techcrunch