Skip to content

Lesson 1 of 19 in Agents in production

Ask what a so-called agent actually does

Before you compare agents, ask what one actually does: whether it calls tools, who sets the order of steps, and what it does alone.

3-minute read, 20-minute exercise. Updated . Free lesson, no account needed.

In 60 seconds

  • Define "agent" before you compare. The word covers a single prompt behind an API and fully autonomous coding agents; make someone say which one the deck is selling.
  • Ask whether it calls tools. A model can explain how to build a table and leave you without one; tools turn that knowledge into action.
  • Ask who sets the order of steps. A fixed sequence is automation with a new name, and for rule-following work often the better buy.
  • Ask what it does without a person. Get the actions it takes on its own authority before anything goes live: approval, payment, sending.
  • Budget for several agents. Most companies gain more automation from several agents collaborating on a problem over time than from a single reasoning model searching for an answer in seconds.

A free lesson from Agents in production, one of its 19 lessons. Read another free lesson

One word, stretched across everything

On 25 June 2025, Gartner estimated that only about 130 of the thousands of agentic AI vendors are real. Many vendors, it says, are "agent washing": rebranding AI assistants, chatbots and robotic process automation as agents without substantial agentic capabilities. The label covers everything from a single prompt wrapped in an API to fully autonomous coding agents. A deck that leads with the word has not told you where on that range the product sits. Make your standard question, to vendors and internal teams alike: what does this product do when it runs?

More software, same question

On 26 August 2025, Gartner forecast that up to 40% of enterprise applications would include integrated task-specific agents by 2026. In 2025 the share was less than 5%. Every application that carries one by then still gets the same first question.

What turns a model into an agent

A language model on its own is passive. Show one a table and it will explain in detail how to build it. You still have no table. Before you credit a fluent demo, ask what the product connects the model to.

Tools are the difference: a tool protocol works like a hub between the model and the things that act. Agency plus tool use is what separates an agent from a plain language model. A demand forecast or a document search your teams have already built can each become a tool an agent calls. Retrieval is one kind of tool, alongside calculators, code execution and internal data APIs.

Run each claim through three questions

Take the next claim and write the answers beside the product name; two products called "agent" can differ on all three.

Question If yes If no
Does it call tools? It can reach your systems (the order database, the ticket queue) and act on what it reads. A model behind a chat window: it answers, and it changes nothing.
Does it choose the order of steps? Given a goal, it reasons about what to do next and picks each tool. The sequence is fixed in advance: automation wearing the label.
Does it act without a person? Name every action it takes on its own authority before go-live: approval, payment, sending. A person stays in the loop.

A "no" on the middle question is often the right answer. Fixed automation covers any process you can write as if-this-then-that. You need an agent, meaning a model that reasons about what to do next, only where the process has "more exceptions than rules." The two can share one system: a simple classifier sends routine requests to the fixed automation, the rest to the agent or a person.

Ask for the parts list

The "agent for finance" promise, one purchase covering a whole function, runs into how these systems ship. What works is an ecosystem: several AI services, each tuned for one purpose. Together they work like the best assistant you ever had, finding information, checking it and handing back an answer you can use. A single reasoning model hunting an answer in seconds is one kind. Several agents collaborating on a problem over time is another, and most companies gain more automation from the second. When a deck promises one agent for a whole function, ask for the parts list. What services are inside, what each is tuned for, what passes work between them.

Exercise: pin down one agent claim

Take one claim you have on your desk now (a vendor deck, an internal proposal, or a product already running under the agent label) and fill in the card for it. Fill in what you know; you won't know every line yourself. Every line you can't fill gets an owner and a date before you close the card. The card download includes a filled-in example of a claim that isn't ready yet, and why.

What this agent actually does (PDF)

A blank card and a filled-in example, one page each.

Open the card (PDF)
Show the three AI prompts

Use AI to fill in and test your card

These three prompts go into your company's approved AI assistant. Use the first to fill in the card one row at a time; it won't guess an answer for you. Use the second when the card is done, to hear the case against this claim at the decision meeting. The third asks about your industry and your own situation and turns the lesson into a 30-day plan, with a checklist, rubric or brief for your boss if you want one. Use placeholders for company, system and people names; the help doesn't depend on them.

Prompt 1: Fill in my card with me

Paste this, then add your claim in one or two sentences and the card below.

Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn

Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X) and round any figures. The help you get does not depend on them.

You are helping me fill in a review card for one so-called agent: a vendor deck, an internal proposal or a product already running. The card is pasted at the end.

Rules:
- Ask me about one row at a time, in order, and wait for my answer before moving on.
- Write down only what I tell you. Never suggest, estimate or invent a number, a name, a yes/no answer or a decision for me, even if I ask you to guess. If I ask, say that this row needs a real answer and ask who could give it.
- If I don't know an answer, write "Not known" and ask who should find out and by when. Put that in the Owner / by when column.
- After the last row, show the whole card as a table with three columns: Question, Answer, Owner / by when. Then list every row marked "Not known" with its owner, and remind me that any unanswered row means this claim is not ready to compare or buy.

My claim in one or two sentences: [DESCRIBE THE DECK, PROPOSAL OR PRODUCT]

The card:
[PASTE THE CARD]

Prompt 2: Stress-test my card

Paste this, then your filled card.

Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn

Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X) and round any figures. The review does not depend on them.

Below is a review card I filled in for one so-called agent. Act as a skeptical reviewer at the decision meeting and argue the case for WALK AWAY.

For each row:
- Say whether the answer is specific enough to act on (a named action, a named system, a date) or too vague: in particular, whether it says what the product does when it runs, or only repeats the deck's words.
- Point out where answers contradict each other, for example what it may do without a person against who sets the order of steps.
- Say what evidence you would ask for before accepting it.

Do not rewrite my answers and do not invent facts about the product. If a row is blank or says "Not known", treat it as not ready.

Finish with: your recommendation (AGENT, FIXED AUTOMATION, SEND BACK THE OPEN QUESTIONS or WALK AWAY), the three rows that most need work, and one question I should ask my team or the vendor on Monday.

My card:
[PASTE YOUR FILLED CARD]

To paste the card into prompt 1, copy its rows from the PDF.

Prompt 3: Make it yours

Paste this and answer its four questions.

Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn

Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X, Supplier B).

You are helping a leader apply one lesson to their own situation. The lesson's five decisions are below. Use only them and what I tell you. Never invent facts, numbers, vendors, laws or results about my organization or my industry. Where a number is needed and I have not given one, say what to measure and who should measure it, instead of guessing.

First, ask me these four questions, one at a time, and wait for each answer:
1. My industry, and roughly how many people work in my organization.
2. The claim I have in mind, in one or two sentences: a vendor deck, an internal proposal, or a product already running.
3. Where it stands today: a deck I have been sent, a purchase being decided, or something already in production.
4. What worries me most about it.

Then give me the following. Keep it short and practical:
A. My plan. For each of the five decisions: one action written for my situation, the role that should own it (a role, not a name), and what should be done within 30 days. If my industry makes a decision harder, for example regulated data, customer-facing work or professional sign-off, say which decision and why in one sentence.
B. Three questions to ask my team or the vendor on Monday, specific to my situation.
C. Ask me which one I want: a checklist for vetting the next agent claim, a rubric to score a claim against the five decisions, or a one-page brief for my boss. Then write it, and finish with a prompt I can paste later to update it.
Mark anything that rests on an assumption with "(check)".

The five decisions:
1. Define what the product does when it runs, and where it sits between a single prompt behind an API and a fully autonomous agent, before you compare claims.
2. Ask whether it calls tools: which of your systems it can reach and act on, and which things your teams have already built it could call as tools.
3. Ask who sets the order of steps: if the sequence is fixed in advance it is automation, often the better buy for rule-following work, and routine requests can go to it while the rest go to an agent or a person.
4. Ask what it does without a person: name every action it takes on its own authority (approval, payment, sending) before anything goes live.
5. Budget for several agents rather than one: when a deck promises one agent for a whole function, ask for the parts list, meaning what services are inside, what each is tuned for, and what passes work between them.

Your company's assistant may answer differently, so read what it writes before you use it.

Check yourself

A deck promises an autonomous agent for customer support. What do you ask first?

Show the answer

The actions it takes without a person: approval, refund, sending.

Your process is mostly rule-following; a bidder offers fixed automation under the agent label. Buy it?

Show the answer

Often, yes. Fixed steps are what rule-following work needs.

The new agent would replace your demand-forecasting model. What happens to it?

Show the answer

It becomes a tool the agent calls.

A proposal promises one agent for the whole finance function. What do you want to see?

Show the answer

The parts list: which services, what each is tuned for, what passes work between them.

A process has more exceptions than rules. Fixed automation or an agent?

Show the answer

An agent. When exceptions outnumber rules, choosing the next step is the job.

Sources

The advice draws on interviews with the chief technology officer of an enterprise AI company, an AI strategist at a software company, a VP of research in machine learning at an AI startup, a drug-discovery scientist at a deep-tech company, an AI research lead at a large consulting firm and an engineering vice president at a data platform company.

Quiz

Join to take the quiz and see your score.

  1. A proposal describes a product only as an agent, with nothing more specific anywhere in the deck. What do you ask first?

  2. A demo agent explains in detail how it would update your order database, but nothing changes. What do you ask before crediting the demo?

  3. A product sold as an agent always follows one sequence of steps, fixed in advance. What have you really been shown?

  4. Your team must choose between a single reasoning model that answers in seconds and several agents collaborating on one problem over time. Where do most companies gain more automation?

This is one lesson from The AI Decision Course.

$299 a year (founding price)

Try it for 14 days. If it isn't useful, the full $299 comes back.

Paying with a company card? Ask your manager. Your receipt comes from Link, the checkout run by Stripe, and names the membership.

Made by Dr. Tony Hoang, host of The Artificial Intelligence Podcast.

Ask about this lesson

Ask what a so-called agent actually does

    Free preview: 3 questions a day, answered from this lesson. Don't paste confidential details.

    Ask about this lesson