The tech world is agog with the capabilities and potential of artificial intelligence. AI is the hottest trend in decades, and pundits are all over the map predicting the scope of its impact. Every white-collar job, from the call center to the CEO’s office, can possibly be augmented, replicated, or outright replaced by AIs. Depending on who you listen to, the impact of AI could be nothing much, a huge impact on productivity without net job loss, the wholesale displacement of tens of millions of workers, or our destruction as a species. I confidently predict the correct answer is in that range.
In this post, I’ll touch briefly on what AI might be able to do in the realm of language, its current weaknesses in that realm, and the natural advantages technical communicators enjoy over today’s AI systems. (The rest of you are on your own.)
The Promise of AI
AI’s first real introduction to popular culture may have been when IBM’s purpose-built, supercomputer-based Watson defeated the two best human “Jeopardy” champions in 2011. Since then, AI capabilities have expanded with stunning swiftness across a broad front of human endeavor. An AI passed the standard bar exam in 2023. In a recent study, an AI passed the Turing test, fooling humans into thinking it’s also a human. Some people have even claimed that AIs have achieved sentience. The output from AIs seems poised to become indistinguishable from that of humans, to the consternation of essay graders and job seekers everywhere. Today, AIs are on probationary periods as Google and Microsoft search assistants, corporate chatbots, and graphic artists.
AI technology is rapidly advancing. Its greatest promise may be that with the effort and resources being poured into it right now, AI’s future may far outstrip its present, and quickly. But let me tell you, for right now, it had better.
The Pitfalls of AI
Watson parsed natural-language inputs and rapidly searched an offline database to produce correct responses faster than humans. As impressive as today’s AIs appear, they are, in a way, a significant step back from the Watson benchmark. The current generation of AIs is fast but suffers from a fundamental problem: you cannot trust AI answers, no matter how eloquent or convincing they sound. The industry term “hallucination” is a diplomatic way of saying that some AI answers are untethered from reality. Today’s AIs mimic human speech, but they are unable to reason, do any kind of math, or apply elementary logic. Consequently, they are wont to produce answers that are hilarious, horrifying, and sometimes dangerously wrong. Right now, AIs make unsuitable law clerks, software engineers, customer-facing chatbots, and research assistants. To me, today’s AIs are, in the philosophic sense of the word, nothing but bullshit generators.
Given how AIs need to absorb large language models to better predict the next word in a sequence—providing, in other words, highly developed autocompletion—they are a great demonstration of the old expression “garbage in, garbage out.” Today, AI vendors are sucking in everything they can obtain on the internet, even Reddit subgroups, hardly the acme of human thought. Worse, information sources aren’t weighted. AIs literally believe that everything they “read” on the internet is true. A Reddit joke post was apparently regurgitated, undigested, in a top answer on how to keep the cheese from sliding off pizza: mix in glue(!).
In an episode of the original Star Trek, Captain Kirk is about to be duplicated and floods his mind with angry, racist thoughts. The clone curtly dismisses a suggestion from Spock with the jarring, “I’m sick of your half-breed interference.” In response to threats of copyright infringement justified as “fair use,” some image providers are adopting a similar tactic, “poisoning” input data with false metatags, hoping that if their work is stolen for an LLM and used to train an AI, the results the AI produces will be wrong. Other actors are simply malevolent, manipulating training data with malware and phishing attacks.
Finally, current AIs are prone to the worst sin an information provider can commit: making stuff up. Writing for BuiltIn.com, Ellen Glover reported, “ChatGPT can generate URLs, code libraries and even people that do not exist, and it can reference made-up news articles, books and research papers).” It’s all fun and games until someone relies on an incorrect AI statement and dies.
My advice for now is to give AI less credibility than President Reagan once gave the Soviet Union’s word. Don’t trust, and verify. I suggest keeping an eye on Google’s AI Overview: many of the bad examples I cited originate there, and Google is actively trying to clean them up. If any corporation can fix a problem with its core product, Google can. Let’s see how they do.
The Value of Credibility
So, what’s a technical writer to do? Upper management is playing with a new toy that’s faster and cheaper* than we are. If an AI can crank out hundreds of pages of technical material in minutes, we’re going to be out of a job soon, right? Well, hang on. I think there are several big reasons why the profession isn’t doomed just yet.
For one thing, working for a client involves access to the client’s products and subject-matter experts. Your work is inherently more credible than the work of someone or something unaffiliated with the product.
A technical open-source LLM will consist of existing technical documents posted to the web. The technical documents generated from this model will sound like the large corpus of public-facing libraries created by IBM, Oracle, Amazon Web Services, Microsoft, and other major vendors. This isn’t in itself an issue. But if your company tries to generate technical documentation that way, AI will provide sound-alike technical information, which will be incorrect. Of course, your company can and should limit its input to its own proprietary specifications and internal library—you know, from the Confluence pages and SharePoint site that no one has cleaned up in ten years. How will the AI distinguish between correct and incorrect specs, or between current, proposed, and abandoned features? And how will your legal department feel about loading your proprietary information into an LLM that uses your input as training material?
Assuming your company opts for a slim language model consisting only of its own current, vetted functional specifications, an AI might be expected to re-present the information in an acceptable style. (A prompt might be: “Using Simplified English and an eighth-grade reading level, create procedures for system administrators to configure the router.”) But this lacks something technical writers do that I consider something of an unspoken black art. When we turn a specification into customer information, we don’t include everything in it. Information on how the engineer implemented a feature and how its algorithms work is not meant for customers. I can’t articulate how we decide what to filter out; it took me years to acquire the experience and confidence to do so. Without a librarian, a library is just every book that the community can afford to acquire. Librarians are curators. We curate technical information for our clients and customers.
I’ve always had the gift of gab, and I can write a persuasive sentence even if it’s incorrect. As time-consuming and ego-bruising as a technical review can be, it is a strength of our work process. A reviewed and approved document is far more likely to be correct. Fact-checking is required if incorrect information has legal consequences—and technical information always carries legal weight, regardless of where it comes from. Can you correct an AI’s output? So far, programming AIs, which produce straightforward compiles/doesn’t compile output, can’t incorporate feedback. Even if there’s a mechanism to review and correct AI output, implementing my advice to verify everything will significantly reduce AI’s seeming time and cost advantages.
Finally, as professionals, we have ethical standards. One basic definition of quality technical writing is clarity, concision, and correctness. Call it pride in our work. Even absent review, our own sense of ethics drives us toward correctness. We don’t make stuff up.
The challenge of AI is to create correct, credible information. Today’s AIs produce information that sounds credible but is not correct. We’re all about correctness and credibility. I think we can meet this challenge.
*[Update 11 July 2024: I’ve learned that one of my assumptions is wrong. AI is not cheaper than we are. In fact, the CPU and power requirements are very substantial.]
Steven Jong 