How Should CEOs Think About Generative AI? – Delivering Business Value

In our first article of this two-part series, How Should CEOs Think About Generative AI? – Generative AI’s Potential and Limitations, we attempted to place generative AI in the context of disruptive change and understand the potential as well as the limitations of this powerful emerging technology.

In the concluding article of this series, we look at how generative AI can deliver business value to an organization.

Using generative AI to deliver business value

Generative AI can deliver business value if used wisely and with the proper safeguards. Two kinds of tasks that are well-suited to the use of Generative AI. They are as follows:

Automation of linear tasks:  Job tasks can be classified as “Linear” when completion is a function of resource and time and the output is deterministic, meaning, there is one right answer.  Here are 3 examples:

1. A customer-facing example would be when a consumer calls a customer service representative of an airline to plan a multi-leg trip for their family. This call could take over an hour as it involves iterating through a combination of availability, costs, seating, timing, and connecting times.

   With generative AI, a consumer could accomplish this task directly with generative AI using natural language to iterate through the options, resulting in dramatic improvements in customer experience while avoiding excessive labor costs. 

2. An internal-facing example would be a marketing analyst who is asked for a side-by-side comparison of key elements of 12 vendor proposals. To complete the task, the analyst can use generative AI to read, synthesize, and summarize dozens of vendor proposals in minutes instead of doing it manually in hours.

3. A third example would be of a sales manager asking a Structured Query Language (SQL) developer an ad-hoc question about top declines in sales across 100 products and 25 global regions.

   Here, the sales manager can directly generate the answer using generative AI which will create and run a SQL query without the need for the developer.

These three examples illustrate that for linear tasks, it is a matter of the time spent to receive an objective answer.  This approach sidesteps the hallucination risk of the “generative” component.

The solution uses the “chat” component as a conversational interface to an internal “brain” consisting of internal systems or data sources within the firewalls of the enterprise.

Assisting in non-linear tasks.  Those job tasks with no one right answer but clearly wrong answers are “non-linear.” In this case, having more resources or investing more time does not necessarily result in a perfect answer. Getting to a high-quality answer requires searching in a loop, complex reasoning, critical thinking, and creativity. Here are two examples: 

• The creation of a first draft of a business plan for a direct-to-consumer product entry.

• Generating a business strategy document for the next fiscal year given strategic planning documents of the last three years.

In either case, generative AI may generate a first draft with varying quality and will require an expert to iterate with increasing context and instruction to deliver a board-level document.

An expert can drastically improve the quality of first and intermediate outputs can be drastically improved over time by including output templates in the input context.

The quality of the output will be proportional to the skill level of the end-user in the domain as well as in providing instructions to the Generative AI. For this opportunity type, the solution uses “chat” along with the “generative” components but with humans in the loop to get to the final output with guard rails around hallucination. 

Strategic and tactical return on investment of Generative AI

Generative AI has the potential to drastically increase the productivity of certain parts of the business and in some cases, transform an entire business.

Based on our experience of 500+ AI projects and early research from organizations experimenting with generative AI, there are broadly two types of return from investments in generative AI:

1. Tactical returns where existing jobs are improved through a measurable increase in throughput or reduction in headcount. For example, a major travel and hospitality company is seeing up to 50% improvement in the productivity of experienced developers in their first generative AI proof-of-concept.

IBM CEO Arvind Krishna, when asked about non-customer-facing back-office jobs, commented, “I could easily see 30% of that getting replaced by AI and automation over a five-year period”. 

These tactical returns can improve operating margins as well as reinvest resources from “run the business” processes to fund “change the business” initiatives.  Ultimately, the 80-20 rule is a good thumb rule to use - 20% improvement in 80% of the tasks across the business or a given function is possible.

Improvements beyond the 20% gain would require significant investments in skilled domain experts to analyze the output, develop the training and change management, and establish the process re-engineering.

2. The second type, strategic return, is in competitive advantage where generative AI can power new business models or develop new products and markets and enable new operating models.

Erik Brynjolfsson, a leading, technology-focused economist based at Stanford University, conducted a study of what happened to a company and its workers after it incorporated a version of the popular interactive AI chatbot ChatGPT into workflows. 

Brynjolfsson noted, "And what this system did was it took people with just two months of experience and had them perform at the level of people with six months of experience.” Ultimately, the system was able to establish a floor for the performance of lower-skilled customer service reps and boost customer satisfaction.

Recommendations

Generative AI represents an opportunity as well as a challenge. There are important considerations that all CEOs should take into account before getting started. These are:

1. Establish a basic policy and governance structure about using generative AI. This may include banning it for certain tasks but also establishing a policy and governance framework for balancing the risk and reward with Generative AI. It is important to know how the ROI will be measured and how use cases across the business will be prioritized.

2. Decide which configuration is appropriate for the organization - “chat” component with enterprise data sources, full LLMs with enterprise data sources, and open source LLMs trained with enterprise data. This is particularly important for non-linear jobs as generated output must be vetted.

3. Educate business and technical employees on generative AI concepts, tools, and the enterprise policy so there is a minimum level of literacy in the organization before it is adopted widely. 
   

4. Guard against overreliance on generative AI as there is a potential to diminish human intuition and creativity in employees getting their job done. 

5. Build a business case with a list of high-priority use cases that are modeled and sized. It is important to avoid developing a solution that is chasing a problem with generative AI.

Will generative AI disrupt companies and entire industries? Only time will tell, but it is important to understand that generative AI has the potential to revolutionize the way we do business. This comes at the risk of equal potential for good or harm at scale and makes generative AI, unlike previous forms of Artificial Intelligence.

These are questions that all CEOs should think about as they forge ahead in their experiments with this potentially transformative new technology.

About the Authors:

Randy Bean is the author of Fail Fast, Learn Faster: Lessons in Data-Driven Leadership in an Age of Disruption, Big Data, and AI, and a contributor to Harvard Business Review, Forbes, and MIT Sloan Management Review.  He was Founder and CEO of NewVantage Partners, a strategic advisory firm which he founded in 2001 and which was acquired by Paris-based global consultancy Wavestone. He now serves as Innovation Fellow, Data Strategy at Wavestone. You can follow him on LinkedIn. 

Laks Srinivasan is co-founder and managing director of the Return on Artificial Intelligence  Institute.  He was previously co-COO of Opera Solutions and an Associate with Booz Allen Hamilton.  He holds an MBA from The Wharton School and a degree in electrical engineering.  He now serves on the board of Lehigh Valley Public Media.