Sort your users into three groups. The disappointed, the convinced and the over-trusting each need a different message before the next purchase.
Reject the one-stop shop. One purchase that answers support tickets and writes the campaigns is hype. Vendors chased growth and skipped making customers successful.
Keep a person in the loop. Personal success with a model says nothing about unattended use. Before approving, ask who catches the first wrong answer.
Skip the training-data pile. Pre-trained models already reason and have common sense. Start from the job you are hiring one to do.
Manage it like a new hire. Give the model a job description, responsibilities and KPIs. The hard part is the cultural habit of treating AI as code.
Three in five people say they can use AI effectively, according to the University of Melbourne and KPMG International's 2025 global study. The report's own reading: that confidence likely reflects how easy many AI interfaces are to use. It also warns of complacency and overreliance if that confidence is not accompanied by meaningful levels of understanding and literacy. Before the next purchase, find out what your people wrongly believe AI can do, and correct it.
Three groups, three messages
Generative AI users fall into three groups.
Who they are
What they believe
What they need to hear
A vocal minority
They find it useless
Dig deeper before writing it off
The convinced majority
AI is useful and worth using
How the systems work and where they stop, in plain language
The risky group
Their own success means it can run unattended
Production keeps a person in the loop
Do not wait for familiarity to do the teaching. Compared with its own 2022 survey of 17 countries, the study found no discernible change in the public's self-reported understanding of AI. Use rose anyway, and trust fell. A lack of understanding may still be the biggest problem for companies, even at a large bank that has deployed roughly 200 AI use cases. Put education for your board, executives and employees ahead of the next purchase.
Reject the one-stop shop
One purchase that answers support tickets and writes the campaigns: that is the one-stop-shop belief, and it is hype. The vendors behind this wave chased growth and skipped the work of making customers successful. That feeds the familiar hype-then-disillusionment cycle. When a pitch leans on impressive words like "orchestration", do not let it stand. Make the vendor say, in plain words, what the system will not do.
Personal success is not production readiness
Watch the risky group closest. Someone drafts memos with an assistant all week and concludes the same model can answer customer email unreviewed. Personal success proves the model can help one person. Production is a different test, and you are the one funding it. Before approval, ask who catches the first wrong answer.
Skip the training-data pile
The belief that every task needs its own pile of training data no longer holds. Pre-trained models already reason and have common sense. The work is to tell the model what you need. The next project starts with one question: what job is this model being hired to do?
Manage it like a new hire
Two habits block the shift. One treats AI as a piece of code. The other treats it as a source of insight, with people making every decision. Both are cultural rather than technical. Manage the next system the way you manage a new hire: a job description, responsibilities, KPIs.
Exercise (15 minutes): fill in the before-you-buy card
Take one AI purchase your company is about to approve, a system a vendor has pitched, or a renewal you are being asked to sign. Fill in the card for it. You won't know every line, and the lines you can't fill are the finding: every one of them gets an owner and a date. The card download includes a filled-in example that is not ready in two places, and why.
Before-you-buy understanding check (PDF)
A blank card and a filled-in example, one page each.
These three prompts go into your company's approved AI assistant. Use the first to fill in the card one row at a time; it won't guess an answer for you. Use the second when the card is done, to hear the case against buying. The third asks about your industry and your purchase and turns the lesson into a 30-day plan, with a checklist, rubric or brief for your boss if you want one. Use placeholders for company, system and people names; the help doesn't depend on them.
Prompt 1: Fill in my card with me
Paste this, then add your purchase in one or two sentences and the card below.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, vendor and people names with placeholders (Company A, System X, Vendor B) and round any figures. The help you get does not depend on them.
You are helping me fill in a before-you-buy card for one AI purchase. The card is pasted at the end.
Rules:
- Ask me about one row at a time, in order, and wait for my answer before moving on.
- Write down only what I tell you. Never suggest, estimate or invent a number, a name, a yes/no answer or a decision for me, even if I ask you to guess. If I ask, say that this row needs a real answer and ask who could give it.
- If I don't know an answer, write "Not known" and ask who should find out and by when. Put that in the Owner / by when column.
- After the last row, show the whole card as a table with three columns: Question, Answer, Owner / by when. Then list every row marked "Not known" with its owner, and remind me that any unanswered row means the understanding gap is not closed and the purchase is not ready to approve.
My purchase in one or two sentences: [DESCRIBE THE PURCHASE]
The card:
[PASTE THE CARD]
Prompt 2: Stress-test my card
Paste this, then your filled card.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, vendor and people names with placeholders (Company A, System X, Vendor B) and round any figures. The review does not depend on them.
Below is a before-you-buy card I filled in for one AI purchase. Act as a skeptical reviewer at the decision meeting and argue the case against approving this purchase now.
For each row:
- Say whether the answer is specific enough to act on (a number, a named role, a date) or too vague.
- Point out where answers contradict each other, for example a row that teaches people where the systems stop against a row that lets the system run unattended.
- Say what evidence you would ask for before accepting it.
Do not rewrite my answers and do not invent facts about my purchase. If a row is blank or says "Not known", treat it as not ready.
Finish with: your recommendation (BUY, EDUCATE FIRST or SEND BACK), the three rows that most need work, and one question I should ask my team on Monday.
My card:
[PASTE YOUR FILLED CARD]
To paste the card into prompt 1, copy its rows from the PDF.
Prompt 3: Make it yours
Paste this and answer its four questions.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, vendor and people names with placeholders (Company A, System X, Vendor B).
You are helping a leader apply one lesson to their own situation. The lesson's five decisions are below. Use only them and what I tell you. Never invent facts, numbers, vendors, laws or results about my organization or my industry. Where a number is needed and I have not given one, say what to measure and who should measure it, instead of guessing.
First, ask me these four questions, one at a time, and wait for each answer:
1. My industry, and roughly how many people work in my organization.
2. The AI purchase I have in mind, in one or two sentences.
3. Where it stands today: an idea, a pitch on the table, or about to be signed.
4. What worries me most about it.
Then give me the following. Keep it short and practical:
A. My plan. For each of the five decisions: one action written for my purchase, the role that should own it (a role, not a name), and what should be done within 30 days. If my industry makes a decision harder, for example regulated advice, customer-facing work or professional sign-off, say which decision and why in one sentence.
B. Three questions to ask my team on Monday, specific to my purchase.
C. Ask me which one I want: a checklist for my purchase, a rubric to score how ready my people are, or a one-page brief for my boss. Then write it, and finish with a prompt I can paste later to update it.
Mark anything that rests on an assumption with "(check)".
The five decisions:
1. Sort your users into three groups (the disappointed, the convinced and the over-trusting) and give each a different message, with education for the board, executives and employees ahead of the next purchase.
2. Reject the one-stop shop: when a pitch promises one system for many jobs, make the vendor say, in plain words, what it will not do.
3. Keep a person in the loop: before approving any unattended use, name who catches the first wrong answer.
4. Skip the training-data pile: pre-trained models already reason and have common sense, so start from the job the model is being hired to do.
5. Manage the system like a new hire: give it a job description, responsibilities and KPIs, because the hard part is the cultural habit of treating AI as code.
Your company's assistant may answer differently, so read what it writes before you use it.
Check yourself
A department head wants a customer-facing assistant answering email unreviewed. What do you ask first?
Show the answer
Who catches the first wrong answer. Unreviewed use is fire-and-forget.
Your board wants another AI purchase approved this quarter. What comes first on your plan?
Show the answer
Education for the board, executives and employees. Familiarity alone has not moved understanding.
A vendor pitch says one system runs sales and support. What do you do with it?
Show the answer
Send it back until the vendor says, in plain words, what it will not do.
A team will not start a project until the training data is ready. What do you tell them?
Show the answer
Skip the pile. Pre-trained models already reason and have common sense; define the job the model is hired to do.
Sources
The advice draws on interviews with the CEO of an open source database company, an AI chief technology officer at a large IT services company, the co-founder of an AI engineering consultancy and the founder of an AI advisory firm.
University of Melbourne and KPMG International (Gillespie, Lockey, Ward, Macdade, Hassed), "Trust, attitudes and use of artificial intelligence: A global study 2025," released April 2025. https://assets.kpmg.com/content/dam/kpmgsites/xx/pdf/2025/05/trust-attitudes-and-use-of-ai-global-report.pdf
One thing to hold on the numbers: every figure is self-reported, so "use AI effectively" and "understand" are the respondents' own ratings. The gap sits between two self-assessments, not a test score. The headline figures are global averages from more than 48,000 people in 47 countries surveyed November 2024 to January 2025, and they hide a large advanced-versus-emerging-economy split; for a Western audience, the advanced-economy figures are the honest ones. The University of Melbourne designed and ran the study, with funding from KPMG International, KPMG Australia and the university; KPMG sells AI advisory.