25% faster, 40% better: The new math of AI-accelerated work

With proper implementation, AI can create tremendous value. A new study reveals just how much value can be generated, writes Dr Gleb Tsipursky.

Artificial intelligence (AI) has long promised to revolutionise businesses and upgrade productivity. This year, generative AI tools like ChatGPT have significantly disrupted industries and work processes, according to a new McKinsey report. And now we have definitive proof that AI can provide massive boosts to knowledge worker productivity and quality, with a groundbreaking new study from Harvard Business School researchers partnering with Boston Consulting Group (BCG) providing robust evidence of AI’s transformative potential.

So, what are companies doing with generative AI?

McKinsey finds that close to 25 per cent of top-level executives surveyed report they are directly utilising generative AI technologies in their professional roles. Additionally, over a fourth of respondents from AI-adopting firms indicate that generative AI is currently a topic of discussion at board meetings. Intriguingly, 40 per cent of survey participants express that their organisations plan to ramp up overall AI investment due to the advancements in the field of generative AI. Industries especially gaining from AI are marketing, product development, and customer service.

The most successful companies using AI – called AI high performers – are pushing generative AI adoption furthest. These organisations achieve over 20 per cent of earnings from AI before interest and taxes.

AI high performers embed generative AI across more functions than others, especially product development and supply chain management. They pursue new revenue streams with generative AI, while others focus on cost cuts. AI powers their product innovation and new AI-based offerings.

High performers invest over five times more in AI than peers, with over 20 per cent of digital budgets for AI. They adopt AI extensively, embedding AI in four or more functions. Knowledge graphs and natural language abilities supplement generative AI.

But is generative AI actually any good?

The Harvard Business School researchers conducted an extensive randomised controlled field experiment with hundreds of management consultants at BCG. These were highly educated, highly paid professionals engaged in complex tasks integral to their jobs.

The results reveal AI led to major gains in productivity, efficiency, and performance. Consultants completed their work much faster and produced substantially higher quality output with AI assistance.

The researchers partnered with BCG on executing two pre-registered randomised experiments with their consultants. Nearly 800 BCG consultants participated, representing around 7 per cent of the company’s individual contributor consultants globally. The participants had high levels of education, with most possessing MBAs from top programs or PhDs in technical fields. Their annual compensation exceeded $250,000.

The study encompassed two distinct experiments, each with a different type of realistic task commonly encountered by the consultants. Participants were randomly assigned to control and treatment groups. Some had access to AI assistance based on OpenAI’s powerful GPT-4 system, while others completed the tasks without AI.

The careful experimental methodology enables direct causal conclusions about AI’s impacts. By randomly assigning consultants to groups and experimentally varying AI access, the researchers isolated AI’s effects.

Inside AI’s capabilities frontier – boosting productivity

For tasks inside AI’s current capabilities frontier, the results demonstrate tremendous productivity gains. With AI assistance, consultants completed their work 25 per cent faster on average.

Each participant initially completed an assessment task, establishing their baseline performance. In the subsequent experiment, consultants had to brainstorm and develop product prototypes and market launch plans for shoes targeting niche segments.

The 18 sub-tasks covered creativity, analysis, writing, and persuasion. Consultants with AI access finished significantly more sub-tasks, by 12.2 per cent, compared to the control group. AI helped participants reach the final task question 22.5 per cent faster on average. Clearly, for realistic activities within AI’s capabilities, it provides massive productivity and efficiency boosts to highly skilled professionals. Unlike previous automation focused on routine tasks, AI augments complex jobs.

Surging quality – over 40% higher

Remarkably, the researchers found that AI not only enhanced efficiency but also substantially improved output quality. For tasks within its frontier, AI increased quality by over 40 per cent based on randomised human evaluations.

Consultants leveraged AI to generate ideas, conduct analysis, and craft written documents. Subject responses were graded by human evaluators, producing quality scores. The AI groups had dramatically higher averages.

Across numerous statistical models controlling for individual differences, the AI benefits remained very significant. This provides unambiguous proof of AI’s ability to meaningfully amplify human capabilities and performance.

Other research found similar results for individual professional tasks. For instance, researchers at MIT assigned occupation-specific writing tasks to 444 college-educated professionals and gave half of them access to ChatGPT. They found that ChatGPT substantially raises average productivity: time taken decreases by 0.8 SDs, and output quality rises by 0.4 SDs. Inequality between workers decreases as ChatGPT compresses the productivity distribution by benefiting low-ability workers more.

Navigating the jagged frontier crucial

However, Harvard Business School researchers caution that the AI frontier currently has uneven boundaries. Performance gains occur for tasks within capabilities but not outside.

In a second experiment involving business analysis, consultants with AI incorrectly answered questions almost 20 percentage points less often. The data for analysis was designed so AI would make an error if not carefully reviewed.

Many consultants overly relied on flawed AI outputs. However, others adeptly combined their expertise with AI, dividing responsibilities and interrogating results. As AI systems rapidly evolve, professionals must grasp the shifting frontier by developing new collaboration practices. Organisations should promote navigation skills to maximise benefits.

Democratising intelligence – benefits across ability

Importantly, AI conferred advantages regardless of consultants’ baseline abilities. Those below average at the assessment task improved their performance by over 40 per cent with AI support. The top half performers obtained almost 20 per cent gains.

So AI allows the democratisation of intelligence, uplifting employees at all skill levels. Workers weak in certain areas can harness AI to compete with talented peers.

Of course, top performers also achieve substantial benefits. AI increases the ceiling for everyone rather than only reducing the floor.

Navigating AI risks

The transformative promise of generative AI in augmenting human capabilities and optimising various industries is indeed staggering. However, this potential must be balanced against substantial ethical and safety concerns. Issues such as AI misalignment with human values, data security vulnerabilities, and the amplification of human biases are immediate challenges that require ethical frameworks and technological solutions.

Furthermore, as AI systems grow in complexity, the risks of unintended autonomous decision making and even existential threats cannot be ignored. Research into constraining AI’s operational domain and learning human values is essential to mitigate extinction-level risks.

Moreover, according to Anthony Aguirre, the executive director of the Future of Life Institute, business leaders can, at the same time, support generative AI’s benefits for productivity and innovation while also addressing its monumental risks. Doing so requires recognising generative AI’s dual-use nature – that it can be employed for both beneficial and malicious ends – and mandates strong governance policies and oversight mechanisms.

For example, advanced cryptographic solutions like differential privacy can ensure data security, while third-party audits can check against the system’s inadvertent reinforcement of societal biases. Regulatory bodies worldwide must work in concert to establish cohesive safety guidelines and standards, as waiting for a catastrophe to happen before acting is not a viable option.

Navigating the intricate landscape of AI’s promise and peril requires a multifaceted strategy that transcends a mere focus on productivity and efficiency. It is imperative for companies, research bodies, and regulators to collaborate closely to build a future where AI is both beneficial and safe. This necessitates a proactive approach towards integrating ethical frameworks, technological solutions, and global governance in AI development and deployment.

The future of work – prominent role for AI

This rigorous study provides definitive proof of AI’s massive potential to transform knowledge work. Even in jobs demanding advanced education and highly compensated for problem-solving skills, AI delivers major productivity and quality improvements.

Smart integration of humans and AI is crucial for future success. Constructive navigation of the jagged frontier, instead of avoidance or blind reliance, allows maximising benefits. Workflows should seamlessly leverage both human and AI strengths.

As AI capabilities expand, so will its impact on organisations and workers. Company leaders must proactively formulate strategies to capitalise on the promise of AI augmentation while mitigating risks. That’s what I tell my clients when helping guide them in deploying generative AI in their companies.

With proper implementation, AI can create tremendous value. It promises to revolutionise productivity and quality across industries and occupations. The future is bright for humans and AI working together if we address AI risks and leverage its strengths.

Dr Gleb Tsipursky is an author, consultant, and chief executive of Disaster Avoidance Experts.