Launch on one question. The first release answers only the question customers ask most, and answers it correctly.
Measure before you widen it. Track the number the launch was meant to move, and add the next question only after it moves.
Every reply is yours. You answer for what your chatbot tells a customer, including the wrong answers.
Hold it to the bar customers bring. People who use general chatbots expect yours to understand them and follow a conversation.
Tune the answers before launch. Work through the key questions with the people who know them, one by one.
A free lesson from AI in customer service and experience, one of its 13 lessons. Read another free lesson
The audience starts skeptical
Gartner found in a December 2023 survey that 64% of customers would prefer that companies didn't use AI in their customer service. Your chatbot launches into that audience. An airline lost in small claims court over one wrong answer from its website chatbot, and the way through is a first release small enough to get right.
1. Launch on the top question only
Someone who leads AI at a large professional association tells the story of a national postal service that launched a holiday chatbot answering only its top question, tracking a package. The service measured the drop in call volume, confirmed the first question was working, and only then added the next most frequent question: think big, start small, iterate often. The first release should answer one question, the one your contact center gets most. One question is the whole first release. Hold the bot to answering that question correctly, and to nothing else. A proposal may land for a chatbot that answers every question customers might ask. Your counteroffer is the top question, answered well.
2. Grow one measured step at a time
Each question earns its way in:
Question one ships. The bot answers the top question only. Gartner's surveyed customers named harder access to a person as their top worry about AI in customer service, so keep that route open.
The measurement lands. Read the number chosen before launch, the drop in call volume, against the baseline.
Question two ships. The next most frequent question goes in, but only after the first measurement says the bot worked.
The measurement repeats. Same number, same reading, before anything else is added.
3. You answer for what the bot says
One wrong answer from your chatbot can leave you answering for it in court, as an airline found when its website chatbot's refund answer lost in a small claims ruling (the case that opens Course 1, Lesson 19). Most companies cannot get away with a viral, absurd AI answer. Customer-facing chatbots have also been tricked into bad deals. Bots that try to answer everything at once invite exactly these failures, as a city government chatbot that gave illegal advice showed (Course 5, Lesson 6). Hold the first release to one question.
4. One question, answered to a full bar
Customers who have used the general chatbots expect yours to understand them and hold context through a conversation. The old tolerance for a bot that misunderstands most of the time is gone. A one-question release does not buy you a lower bar. How much the bot takes on and how well it answers are separate decisions, and the first release has to win both. Answer one question, and answer it to the bar customers bring.
5. Tune the answers with the people who know them
Before your chatbot goes live, capture the specific requirements behind the questions it will answer. Then sit with the people who know those answers and tune how the bot answers the key questions, one by one.
Exercise (20 minutes): fill in the one-question launch card
Use the chatbot proposal on your desk, or the bot already live whose next question the team wants to add. Fill in the card for the first release; you won't know every line yourself. Every line you can't fill gets an owner and a date before you close the card. The card download includes a filled-in example of a launch that isn't ready yet, and why.
One-question launch card (PDF)
A blank card and a filled-in example, one page each.
These three prompts go into your company's approved AI assistant. Use the first to fill in the card one row at a time; it won't guess a number for you. Use the second when the card is done, to hear the case against launching yet. The third asks about your industry and your chatbot and turns the lesson into a 30-day plan, with a checklist, rubric or one-page brief for your boss if you want one. Use placeholders for company, system and people names; the help doesn't depend on them.
Prompt 1: Fill in my card with me
Paste this, then add your chatbot in one or two sentences and the card below.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X) and round any figures. The help you get does not depend on them.
You are helping me fill in a one-question launch card for a customer-facing chatbot. The card is pasted at the end.
Rules:
- Ask me about one row at a time, in order, and wait for my answer before moving on.
- Write down only what I tell you. Never suggest, estimate or invent a number, a name, a yes/no answer or a decision for me, even if I ask you to guess. If I ask, say that this row needs a real answer and ask who could give it.
- If I don't know an answer, write "Not known" and ask who should find out and by when. Put that in the Owner / by when column.
- After the last row, show the whole card as a table with three columns: Question, Answer, Owner / by when. Then list every row marked "Not known" with its owner, and remind me that any unanswered row means the chatbot is not ready to launch.
My chatbot in one or two sentences: [DESCRIBE YOUR CHATBOT]
The card:
[PASTE THE CARD]
Prompt 2: Stress-test my card
Paste this, then your filled card.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X) and round any figures. The review does not depend on them.
Below is a one-question launch card I filled in for a customer-facing chatbot. Act as a skeptical reviewer at the decision meeting and argue the case for Hold.
For each row:
- Say whether the answer is specific enough to act on (a number, a named role, a date) or too vague.
- Point out where answers contradict each other, for example the bar the answers must meet against the tuning that has actually been done, or the scope of the first release against the questions the bot is really answering.
- Say what evidence you would ask for before accepting it.
Do not rewrite my answers and do not invent facts about my chatbot. If a row is blank or says "Not known", treat it as not ready.
Finish with: your recommendation (Launch, Hold or Re-test), the three rows that most need work, and one question I should ask my team on Monday.
My card:
[PASTE YOUR FILLED CARD]
To paste the card into prompt 1, copy its rows from the PDF.
Prompt 3: Make it yours
Paste this and answer its four questions.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X, Supplier B).
You are helping a leader apply one lesson to their own situation. The lesson's five decisions are below. Use only them and what I tell you. Never invent facts, numbers, vendors, laws or results about my organization or my industry. Where a number is needed and I have not given one, say what to measure and who should measure it, instead of guessing.
First, ask me these four questions, one at a time, and wait for each answer:
1. My industry, and roughly how many people work in my organization.
2. The customer-facing chatbot I have in mind, in one or two sentences.
3. Where it stands today: an idea, a pilot running, or about to launch.
4. What worries me most about it.
Then give me the following. Keep it short and practical:
A. My plan. For each of the five decisions: one action written for my chatbot, the role that should own it (a role, not a name), and what should be done within 30 days. If my industry makes a decision harder, for example regulated answers, refunds, or customers writing in several languages, say which decision and why in one sentence.
B. Three questions to ask my team on Monday, specific to my chatbot.
C. Ask me which one I want: a checklist for my launch, a rubric to score the bot's answers, or a one-page brief for my boss. Then write it, and finish with a prompt I can paste later to update it.
Mark anything that rests on an assumption with "(check)".
The five decisions:
1. Launch on one question: the first release answers only the question customers ask most, and answers it correctly.
2. Measure before you widen it: track the number the launch was meant to move, and add the next question only after it moves.
3. Every reply is yours: you answer for what your chatbot tells a customer, including the wrong answers.
4. Hold it to the bar customers bring: people who use general chatbots expect yours to understand them and follow a conversation.
5. Tune the answers before launch: work through the key questions with the people who know them, one by one.
Your company's assistant may answer differently, so read what it writes before you use it.
Check yourself
The team wants the new chatbot live with a dozen questions, saying one question looks thin. What is your counteroffer?
Show the answer
One question, the one customers ask most, answered well, with the measurement agreed before launch.
The bot is live on question one and question two is ready to ship. What must you know first?
Show the answer
Whether the number the launch was meant to move actually moved: the drop in call volume, against the baseline.
The team wants the bot answering more questions at once. What is your case against?
Show the answer
You answer for every reply. Most companies cannot absorb a viral, absurd AI answer, and bots that answer everything at once invite these failures.
The vendor says the narrow scope means launch testing can be lighter. What do you say?
Show the answer
No. The old tolerance for a bot that misunderstands most of the time is gone, and a one-question release does not bring it back.
Who do you sit with before the bot goes live?
Show the answer
The people who know the answers behind the key questions, tuning the bot's answers one by one before launch.
Sources
The advice draws on interviews with someone who leads AI at a large professional association, the head of a data and AI observability company, someone who has spent decades building systems that talk with people, and the technology chief of a travel media company.
Your team is choosing the chatbot's first question. Password resets are easiest to automate, but order tracking is what the contact center gets most. Which question ships first?
The chatbot is about to ship on its top question, but no one has said what success will look like. What must be settled before launch?
A customer acted on a wrong answer from your live chatbot and wants compensation. The team points out no employee wrote the reply. Who answers for it?
In testing, the chatbot answers its one question correctly, but only when customers phrase it exactly as scripted. The team wants to launch on time. What do you say?