In Informatica's CDO Insights 2025 survey, 43% of data leaders named data issues (quality, completeness, readiness) a top obstacle to moving GenAI pilots into production, tied at the top with lack of technical maturity. When a pilot that impressed in a demo starts to disappoint, ask one question first. It costs nothing: what did the model actually get to see?
Models improve on their own. Your data doesn't.
Drop the belief that the next model will fix it. Models improve on their own; your groundwork does not. Fixating on having the best model distracts from that groundwork: making your data, your APIs and your industry context ready for AI.
A software company building an internal support bot found the challenge was pairing the model with clean enterprise data. Get that pairing done before you shop for an upgrade.
What an upgrade changes, and what it still can't see
Take the disappointing outputs and sort each one into a column. A failure on the left sits with the model, and an upgrade might fix it. A failure on the right sits with the data, and no release changes that. If most of your list lands on the right, move the upgrade budget to the data work.
What a model upgrade changes
What it still can't see
How well the model works, release after release
Your data, APIs and industry context, until someone readies them for AI
General-purpose capability, which matters most for consumers
Systems tailored to your business, built on your own data
Model sophistication and compute
Data accuracy, completeness and trust
Feeding it is the easy part
Feeding enterprise data to a model is easy. What defeats projects is the condition of that data: fragmented, poorly understood and inconsistently governed. Projects fail mostly on data accuracy, completeness and trust rather than on model sophistication or compute. Ask the data team which systems hold the missing answers.
General-purpose AI is the consumer prize
The most capable general-purpose AI looks like the prize worth chasing. General-purpose capability matters most for consumers. It is what makes the assistant on a phone useful. For you, the value lies in systems built on your own data, tailored to your business. Making your data usable is one project with two payoffs. It repairs the disappointing outputs. It is also where the value sits.
Build the test before you buy the upgrade
Sorting failures into columns takes evidence of your own. Tuning a model or improving its retrieval requires private evaluations built on a gold-standard dataset. Treat that dataset as a managed asset. Before anyone prices an upgrade, have the current model scored on it. Score any replacement on the same set before you sign.
Exercise: trace what the model saw
Pick one AI project of yours whose answers disappoint: an internal assistant, a customer chatbot, anything already running. Fill in the card for that project: trace a sample of the disappointing answers back to the data the model actually received, then sort each failure into the model column or the data column. You won't know every line yourself. Every line you can't fill gets an owner and a date before you close the card. The card download includes a filled-in example of a project that isn't ready to decide yet, and why.
Model upgrade or data work (PDF)
A blank card and a filled-in example, one page each.
These three prompts go into your company's approved AI assistant. Use the first to fill in the card one row at a time; it won't guess a number for you. Use the second when the card is done, to hear the case that you are not ready to decide yet. The third asks about your industry and your project and turns the lesson into a 30-day plan, with a checklist, rubric or brief for your boss if you want one. Use placeholders for company, system and people names; the help doesn't depend on them.
Prompt 1: Fill in my card with me
Paste this, then add your project in one or two sentences and the card below.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X) and round any figures. The help you get does not depend on them.
You are helping me fill in a card that decides, for one AI project with disappointing answers, whether the failures sit with the model or with the data, and whether we spend on a model upgrade or on data work. The card is pasted at the end.
Rules:
- Ask me about one row at a time, in order, and wait for my answer before moving on.
- Write down only what I tell you. Never suggest, estimate or invent a number, a name, a yes/no answer or a decision for me, even if I ask you to guess. If I ask, say that this row needs a real answer and ask who could give it.
- If I don't know an answer, write "Not known" and ask who should find out and by when. Put that in the Owner / by when column.
- After the last row, show the whole card as a table with three columns: Question, Answer, Owner / by when. Then list every row marked "Not known" with its owner, and remind me that any unanswered row means the card is not ready and no upgrade gets priced yet.
My project in one or two sentences: [DESCRIBE YOUR PROJECT]
The card:
[PASTE THE CARD]
Prompt 2: Stress-test my card
Paste this, then your filled card.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X) and round any figures. The review does not depend on them.
Below is a card I filled in that decides, for one AI project with disappointing answers, whether the failures sit with the model or with the data, and whether we spend on a model upgrade or on data work. Act as a sceptical reviewer at the decision meeting and argue the case for NO: that the card is not ready to support a decision.
For each row:
- Say whether the answer is specific enough to act on (a number, a named role, a date) or too vague.
- Point out where answers contradict each other, for example failures blamed on the model against what the sample showed the model actually received.
- Say what evidence you would ask for before accepting it, for example the question log the sample came from, the list of systems the model is connected to, or the eval set and its scores.
Do not rewrite my answers and do not invent facts about my project. If a row is blank or says "Not known", treat it as not ready.
Finish with: your recommendation (DATA WORK FIRST, UPGRADE TOO or HOLD), the three rows that most need work, and one question I should ask my team on Monday.
My card:
[PASTE YOUR FILLED CARD]
To paste the card into prompt 1, copy its rows from the PDF.
Prompt 3: Make it yours
Paste this and answer its four questions.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X, Supplier B).
You are helping a leader apply one lesson to their own situation. The lesson's five decisions are below. Use only them and what I tell you. Never invent facts, numbers, vendors, laws or results about my organization or my industry. Where a number is needed and I have not given one, say what to measure and who should measure it, instead of guessing.
First, ask me these four questions, one at a time, and wait for each answer:
1. My industry, and roughly how many people work in my organization.
2. The AI project of mine whose answers disappoint, in one or two sentences.
3. Where it stands today: a pilot running, in daily use, or being rebuilt.
4. What worries me most about it.
Then give me the following. Keep it short and practical:
A. My plan. For each of the five decisions: one action written for my project, the role that should own it (a role, not a name), and what should be done within 30 days. If my industry makes a decision harder, for example regulated data, fast-changing rules or knowledge scattered across many sites, say which decision and why in one sentence.
B. Three questions to ask my team on Monday, specific to my project.
C. Ask me which one I want: a checklist for tracing what the model actually receives, a rubric for sorting failures into model versus data, or a one-page brief for my boss. Then write it, and finish with a prompt I can paste later to update it.
Mark anything that rests on an assumption with "(check)".
The five decisions:
1. Check what the model saw: when answers disappoint, trace a sample of them back to the data the model actually received before blaming the model.
2. Upgrade the model last: models improve on their own, and only your groundwork (data, APIs and industry context made ready for AI) improves because you did it.
3. Treat feeding as the easy part: getting enterprise data into a model is easy; projects fail mostly on data accuracy, completeness and trust, so ask which systems hold the missing answers.
4. Prize your own data: general-purpose AI matters most for consumers; your value lies in systems built on your own data, tailored to your business.
5. Build your own test: before anyone prices an upgrade, score the current model on a gold-standard eval set managed like an asset, and score any replacement on the same set before you sign.
Your company's assistant may answer differently, so read what it writes before you use it.
Check yourself
A team asks to switch to a frontier model because the answers are wrong. What do you approve first?
Show the answer
A review of the wrong answers against the data the model actually saw; the fix follows from the review.
The missing answers live in systems the model was never connected to. Wait for the next release, or connect the systems?
Show the answer
Connect them. No release can reason over data it never receives.
The current model fails on questions whose answers you already feed it. Upgrade, or keep working the data?
Show the answer
Check what the model actually received. Fed-in data can still be fragmented, poorly understood and inconsistently governed, so the failure may sit with the data.
Who owns the gold-standard dataset once the pilot ends?
Show the answer
A named owner, with the set managed like any asset the next project reuses.
A vendor demo impresses on general-knowledge questions. What does that prove about your use case?
Show the answer
Nothing on its own. Have the vendor's model scored on your own eval set.
Sources
The advice draws on interviews with the CTO of an enterprise AI company, a senior executive at a data-backup and data-resilience software company, an engineering VP at a data and AI platform company, and the chief AI officer of an enterprise software company.
Informatica (survey fielded by Wakefield Research), "CDO Insights 2025: Racing Ahead on GenAI and Data Investments While Navigating Potential Speed Bumps," press release, 28 January 2025. https://www.informatica.com/campaigns/cdo-insights-2025/assets/resources/cdo_insights_2025_PDF.pdf: Data issues tied at the top with lack of technical maturity, at 43% each. The obstacle question was asked of leaders who had moved some, but not all, of their GenAI pilots to production. The sample was 600 CDOs, CDAOs and CAOs at companies with $500M+ revenue across the US, UK/EU and APAC. The survey is self-reported, and Informatica sells data-quality and data-management software, so hold the number with that in mind.
Your team wants a vendor to price a model upgrade. What must you have in place first?
The board wants to chase the most capable general-purpose AI as the main prize. Where does the value sit for your company?
A review of your disappointing AI answers finds most failures sit with the data, not the model. What do you do with the upgrade budget?
Your AI pilot's answers disappoint. The team already feeds enterprise data into the model, and asks for a more sophisticated model with more compute. What most likely defeats the project?