Chat Me Up: Six Best Practices for Benefiting From ChatGPT

In less than six months since its release, ChatGPT has become a global technology phenomenon. Organizations are attempting to assess the potential impacts that generative language tools might have on all aspects of their operations. This article explores six key practices organizations should apply to get the most out of using this tool. These practices are based on over two decades of work in artificial intelligence, natural language processing and unstructured data analytics. They are designed to maximize  productivity through this technology, rather than be mere entertainment.

1. Begin With the End In Mind

If you have been exploring ChatGPT, you have likely discovered that it can be a tremendous time sink. One can spend an inordinate amount of time interacting with the technology without necessarily creating productive outcomes. 

Engaging with ChatGPT requires having clear, specific goals. While exploration may be fun, there needs to be an end in mind. This is particularly true if you’re seeking to use the technology across the entire business. Many of us are familiar with the analysis-paralysis that can overwhelm an organization’s processes. Productivity is frequently lost in large scale “data democratization” efforts as more people spend more time analyzing and less time actually doing. There is a significant danger of such productivity losses with ChatGPT, given its conversational interface and social interaction. So, when using ChatGPT, try to execute rather than explore.

ChatGPT allows you to complete the first 80% of content-based work nearly instantly. The question you must ask and answer is this: Are you going to use this tool to increase speed and reduce costs, or are you going to use it to improve results? If the goal is to increase speed and lower costs, this 80% answer needs to be “good enough,” and rapidly used. If instead the goal is to improve quality, then the speed of ChatGPT may cause significant issues. The ability to generate partial results rapidly will put even greater pressure on your most talented humans to contribute their additional 20% of high-quality material.

2. Trust But Verify

President Reagan’s famous saying on nuclear disarmament, “trust but verify,” applies to popularity engines like ChatGPT. The system presents the most average and most popular responses to your queries so you aren’t getting the best answers. ChatGPT can improve a below average response, but rarely will it provide an exceptional response. This can prove very helpful to a person who is a neophyte on the question at hand but will not likely generate better results for an expert. 

There is a chance that ChatGPT will continue to improve over time, but only if those better responses happen to also be the most popular. Hence, it is important to not take ChatGPT results at their face value. Any output from the system that you intend to rely upon should be independently verified through other sources. Indeed, any discrepancies that you may find should be back-filtered through ChatGPT to see if it can maintain some referential integrity and to provide the feedback that ChatGPT is designed to use.  

3. Consistent, If Not Common

Not all responses from ChatGPT are created equally. A consensus among a small number of responses is typically more reliable than variation among a large number of responses. This is the experts-versus-wisdom-of-the-crowd issue, which ChatGPT brings to the fore. 

Asking ChatGPT such details behind its responses can give clarity to the quality of its results, and this should be a standard practice. Your queries should ask where ChatGPT sourced its responses. What are the credentials behind those citations? The more that you intend to rely upon ChatGPT’s outputs, the more skeptical you should be of its results until verified.

4. Numbers, Names and Nouns

In the eDiscovery industry, digital information, such as emails and texts, is used as evidence in legal proceedings. There are many best practices that are used to find smoking-gun messages buried in billions of emails. Among these lessons learned is that some elements of language are more useful than others. 

Numbers, Names and Nouns refers to using word types to preferentially search large volumes of data. Numbers are useful as they can be verified easily and unambiguously. Asking ChatGPT how many restaurants there are in Paris will give you a much more accurate response than asking what is the best restaurant in Paris. The number of restaurants is also easily verifiable by other means, thereby providing a good sanity check.

Names refers to using proper nouns for verification. The more ChatGPT presents proper nouns in its responses, the more likely the results are both accurate and meaningful. If you asked ChatGPT about a certain musician and it mentions an article in “Rolling Stone” magazine, it is likely to be a good reference on this topic. 

Finally, the presence of nouns is generally more useful than verbs. Verbs refer to actions, and actions often have multiple meanings. If you mention “jumping” one person might think of a basketball player while another might think of a kangaroo, and a third might think of a child skipping rope. But mention a “rock” and most people will have a similar idea of what is being discussed. 

So, the rule of thumb here is as you assess results coming from ChatGPT, use Numbers, Names and Nouns as opportunities to verify the results you are receiving. 

5. Less Is More

In Shannon’s Information Theorem regarding unstructured data, a critical principle is that, the less often a piece of information appears in a text, the more meaningful that piece of information is. This is the opposite of the “popularity model” of ChatGPT which makes it useful in checking ChatGPT’s results. 

For example, let’s imagine that you asked ChatGPT to tell you about hurricanes. The word “hurricane” might appear in the response 20 or 30 times, but the word “Katrina” might appear only once. Asking ChatGPT deeper questions about “hurricanes” may not be more enlightening. However, asking ChatGPT to tell you more about “Hurricane Katrina” will likely provide substantially more information. Note too, in this example, that “Hurricane Katrina” is a name, or proper noun, thus satisfying the preceding rule. 

A great example of this effect comes from the infamous Enron bankruptcy case. In the millions of emails that were examined by regulators, only a handful of them identified the underlying fraud. These emails were associated with the code word “Jedi,” a proper noun. This word seemed extremely out of place and stood out to examiners. Once they searched the use of this word, the entire fraudulent scheme was rapidly identified. 

So, when using ChatGPT, use words or terms that appear least in the responses to craft your subsequent questions. Dig deep into rarer terms and you will gain stronger insights faster.

6. Response Drift

ChatGPT is designed to use feedback to learn from our queries. It is collecting new information all the time and learns from what we ask of it. As such, the answer it provides to a question today may be different in the future. If you are using this technology for something persistent such as contract drafting or copywriting, it should be standard practice to ask the system to periodically update its prior contributions.

In this way, you can keep such documents up to date more easily and ensure that you don’t fall into copyright or plagiarism situations. As more and more corporations utilize this platform, there will be an ever-growing likelihood that people will start asking similar questions and will likely receive similar answers. In many cases, this will be irrelevant. But, in those cases where such repetition might be a problem, it will likely be significantly so.

Your Friendly Neighborhood Cyborg

Truly effective use of a tool like ChatGPT should cause significant change in how your organization operates. If it doesn’t, it’s not transformational. Such change is inherently disruptive. So, if you are not experiencing some degree of organization and managerial discomfort from adopting artificial intelligence, you’re simply not trying hard enough. Achieving a new operational state where humans and bots work interdependently will not come easily nor cheap. You will know you have succeeded when, despite the effort and pain of change, you wouldn’t want to return to how things used to be. 

About the Author

Christopher Surdak is an award-winning, internationally recognized transformation expert. He leverages leading-edge technologies to drive results-oriented digital transformation for organizations of all sizes, industries, and regions. Most recently, he served as the Acting Chief Technology Officer with The White House Office of Science and Technology Policy. 

Surdak is the author of the books “The Care and Feeding of BOTS," “Jerk: The Digital Transformation Cookbook,” and “Data Crush: How the Information Tidal Wave Is Driving New Business Opportunities.”

Surdak holds a Juris Doctor from Taft University, an Executive Masters In Technology Management and a Moore Fellowship from the Wharton School of Business at the University of Pennsylvania, a Master’s Certificate in Information Security from Villanova University, and a Bachelor of Science in Mechanical Engineering from Pennsylvania State University.