A recent study from the Center for AI Safety and Scale AI challenges corporate claims about artificial intelligence’s ability to automate complex remote work tasks. Tested on 240 real-world freelance projects across 23 domains, leading AI agents completed less than 3% of projects at an acceptable quality level, exposing a significant gap between AI hype and actual capabilities.
Among the tested systems, China’s Manus AI led with a 2.5% success rate, followed closely by Elon Musk’s Grok 4 and Anthropic’s Claude Sonnet 4.5 at 2.1%, with OpenAI’s GPT-5 trailing at 1.7%. Researchers identified poor quality and incomplete outputs as the primary causes of failure, affecting approximately 45.6% and 35.7% of AI deliverables respectively. These findings starkly contradict narratives from major tech companies asserting transformative AI impacts.
Despite such limitations, companies like Amazon have accelerated AI-driven layoffs, citing AI advancements as justification. Amazon plans to cut up to 30,000 corporate jobs, framing AI as the most transformative technology since the internet. Experts warn this trend may spur similar moves across industries, although the technology is far from fully automating complex work.
The study highlights critical AI shortcomings, such as inadequate long-term memory, lack of continual learning, and inability to acquire new skills on the job. Examples include AI producing excessively short videos instead of requested longer ones and generating architectural models with inconsistent visual features. This challenges optimistic projections that AI could automate 90% of programming jobs shortly.
While improvements in AI coding and reasoning abilities are noted, the technology struggles with multi-step tasks demanding integration of diverse tools and nuanced understanding. The research contrasts sharply with upbeat statements from industry leaders like OpenAI CEO Sam Altman about AI’s rapid progress toward artificial general intelligence.
Overall, the study offers a sobering perspective on AI’s current role in the workplace, urging a more measured approach to AI adoption rather than headline-driven job cuts justified by overstated automation claims.
Also Read: Indian Govt. to introduce AI & CT into the school curriculum, starting in Grade 3
