How AI Helps the Best and Hurts the Rest

Can generative AI serve as an effective adviser for business owners and entrepreneurs? Intuitive chat-based natural language interfaces mean that anyone who can read and write can use GenAI tools for a wide range of tasks, even if they lack technical skills. This has obvious appeal for entrepreneurs and small business owners, many of whom could benefit from an on-demand adviser able to help with marketing, pricing, operations, and strategy.

Improving the performance of entrepreneurs at scale has proved to be challenging. The most effective interventions tend to be high touch, such as hands-on consulting, individualized mentorshipund in-person networking. However, they are expensive to deliver and difficult to scale. In emerging markets specifically, this constraint is often even tighter: High-quality business support can be scarce, and its cost can be prohibitive relative to organizational resources. A low-cost and always-available AI mentor could potentially deliver, at scale, the type of business guidance that has historically been limited by the availability and cost of human experts.

To test whether accessing generative AI can actually help small businesses, we ran a field experiment with hundreds of small business owners in Kenya. We randomly gave half of them access to a WhatsApp contact that connected them to a version of OpenAI’s GPT-4 that we had prompted to act as a Kenyan business adviser, and then we tracked business performance over time. The key factor driving either an increase or decrease in profits and revenues? Whether an entrepreneur had the judgment to distinguish good AI advice from bad.

Testing AI Advice in the Real World

Many previous studies of generative AI have focused on narrow, well-defined tasks, such as drafting emails, developing business strategy, oder generating marketing ads. For such tasks, the tool’s output can often be used with little modification, allowing even less-skilled users to benefit from AI assistance. Consistent with this idea, studies have found that the workers who were struggling the most before using AI benefited the most from using such tools.

Managing a business is not a narrow or well-defined task, though. Entrepreneurs often face vague and ambiguous problems.