Fund the baseline before promising speed. Pull throughput, stability and review load for your last four quarters while the "before" still exists.
Watch the whole delivery cycle. If writing code takes 30 of 100 units of an engineer's time, making it instant still leaves 70.
Report stability beside throughput. The larger estimated effect of AI adoption on delivery so far has been on stability.
Treat review as the next bottleneck. Generate ten times more code and the review still has to happen; track the queue and its reviewers.
Run the measurement as a study. Quantify time across design, coding, testing and review, so the net effect shows in your own quarterly numbers.
A free lesson from AI for software and technical teams, one of its 11 lessons. Read another free lesson
Before the promise, the baseline
In October 2024, Sundar Pichai, Google's chief executive, said over a quarter of Google's new code was AI-generated and then reviewed and accepted by engineers. Numbers like that one count code written, a single step of delivery. Decide now which delivery numbers engineering reports each quarter. Fund that baseline before the tools land.
Writing is one step of a longer cycle
Suppose writing code takes 30 of every 100 units of an engineer's time. Make the writing instant and 70 units remain: understanding, design, review, testing, debugging, deployment. That arithmetic is Amdahl's law: the numbers you watch must span the whole cycle. In a 2019 survey with 5,971 responses, Microsoft researchers found reading and writing code together took 15% of the workday. However fast the writing gets, fund numbers for the slower steps too.
Three numbers, four quarters
Pull each from your delivery systems, for the four quarters before a pilot starts.
Throughput: how much finished work reached production, and how long one change took from idea to deploy. DORA's 2024 survey found a small, likely negative effect of AI adoption on delivery throughput. The 2025 report found the relationship turned positive. Your own quarters show which direction yours moved.
Stability: the share of changes that broke something in production. Here the surveys carry the bigger warning. DORA's 2024 report estimates a 7.2% reduction in delivery stability for every 25% increase in AI adoption. The 2025 report still associates adoption with increasing instability. Throughput bought with more breakage is a trade-off; show the board both sides.
Review load: how long a change waits for review, and how much review each engineer carries. If developers generate ten times more code, the review still has to happen. As of mid-2024, fully autonomous generate-merge-deploy was far off; plan review capacity as the next bottleneck.
One measured trial shows the gap. On PurePerformance in 2026, a German home-energy company's platform engineer described his company's trial of an AI coding tool. Time from ticket in progress to a pull request ready for merge fell by roughly 70%, by his recollection. Yet overall change lead time improved only slightly. Put lead time, merge time and review wait on the pilot dashboard beside output, or the fast step will read as the whole cycle.
The baseline card
Fund the pull from the rollout budget: engineering time to gather the three numbers. Run it as a small study rather than an assumption. Quantify time spent across the engineering life cycle, design, coding, testing and review. The net effect of the tools then shows in numbers you already report. Fill one card per team:
Number
What it counts
What it hides
Q1
Q2
Q3
Q4
Throughput
Changes that reached production, and how long one took end to end
The size of each change, and whether it was the work the business wanted
Stability
The share of changes that broke something in production
How bad each break was, and how long recovery took
Review load
How long a change waits for review, and how much each reviewer carries
Who carries the queue, and how large each change is
Four quarters, oldest to most recent, before the pilot. Ask engineering for the same three numbers every quarter after the tools land. Answer any claim about how much code the tools wrote with the throughput row.
Exercise (20 minutes): fill in the delivery baseline card
Pick one engineering team that will meet the coding tools first, and fill in the delivery baseline card for it: the three numbers for each of the last four quarters, who pulls them, and what goes to the board once the tools land. Use the team's real delivery systems and its real incident history. You will not know every line; every line you can't fill gets an owner and a date before you put the card away. The card download includes a filled-in example of a baseline that is not ready yet, and why.
Delivery baseline card (PDF)
A blank card and a filled-in example, one page each.
These three prompts go into your company's approved AI assistant. Use the first to fill in the card one row at a time; it won't guess a number for you. Use the second when the card is done, to hear the case against funding the baseline yet. The third asks about your industry and your team and turns the lesson into a 30-day plan, with a checklist, rubric or brief for your boss if you want one. Use placeholders for company, system and people names; the help doesn't depend on them.
Prompt 1: Fill in my card with me
Paste this, then add your team and the planned tools in one or two sentences and the card below.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X) and round any figures. The help you get does not depend on them.
You are helping me fill in a delivery baseline card for one engineering team, before AI coding tools land. The card is pasted at the end.
Rules:
- Ask me about one row at a time, in order, and wait for my answer before moving on.
- Write down only what I tell you. Never suggest, estimate or invent a number, a name, a yes/no answer or a decision for me, even if I ask you to guess. If I ask, say that this row needs a real answer and ask who could give it.
- If I don't know an answer, write "Not known" and ask who should find out and by when. Put that in the Owner / by when column.
- After the last row, show the whole card as a table with three columns: Question, Answer, Owner / by when. Then list every row marked "Not known" with its owner, and remind me that any unanswered row means the baseline is not ready, because the "before" numbers may be gone once the tools land.
My team and the planned tools in one or two sentences: [DESCRIBE YOUR TEAM AND THE PLANNED TOOLS]
The card:
[PASTE THE CARD]
Prompt 2: Stress-test my card
Paste this, then your filled card.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X) and round any figures. The review does not depend on them.
Below is a delivery baseline card I filled in for one engineering team, before AI coding tools land. Act as a skeptical reviewer at the decision meeting and argue the case for NOT READY.
For each row:
- Say whether the answer is specific enough to act on (a number, a named role, a date) or too vague.
- Point out where answers contradict each other, for example a baseline that spans fewer than four quarters, a stability number nobody reports upward, or review load with no named reviewers.
- Say what evidence you would ask for before accepting it.
Do not rewrite my answers and do not invent facts about my team. If a row is blank or says "Not known", treat it as not ready.
Finish with: your recommendation (FUND THE BASELINE or NOT READY, PULL AGAIN), the three rows that most need work, and one question I should ask my team on Monday.
My card:
[PASTE YOUR FILLED CARD]
To paste the card into prompt 1, copy its rows from the PDF.
Prompt 3: Make it yours
Paste this and answer its four questions.
Prompt
Prompt from The AI Decision Course by Dr. Tony Hoang, host of The Artificial Intelligence Podcast (Spotify, Apple Podcasts): tonyphoang.com/learn
Before you start: use your company's approved AI assistant, not a personal account. Replace company, system, supplier and people names with placeholders (Company A, System X, Supplier B) and round any figures. The help you get does not depend on them.
You are helping a leader apply one lesson to their own situation. The lesson's five decisions are below. Use only them and what I tell you. Never invent facts, numbers, vendors, laws or results about my organization or my industry. Where a number is needed and I have not given one, say what to measure and who should measure it, instead of guessing.
First, ask me these four questions, one at a time, and wait for each answer:
1. My industry, and roughly how many people work in my organization.
2. The engineering team and coding tools I have in mind, in one or two sentences.
3. Where the rollout stands today: tools not chosen yet, a pilot about to start, or tools already landing.
4. What worries me most about it.
Then give me the following. Keep it short and practical:
A. My plan. For each of the five decisions: one action written for my situation, the role that should own it (a role, not a name), and what should be done within 30 days. If my industry makes a decision harder, for example regulated release cycles, audit requirements on changes, or heavy use of contract reviewers, say which decision and why in one sentence.
B. Three questions to ask my team on Monday, specific to my rollout.
C. Ask me which one I want: a checklist for my baseline pull, a rubric to score the three numbers each quarter, or a one-page brief for my boss. Then write it, and finish with a prompt I can paste later to update it.
Mark anything that rests on an assumption with "(check)".
The five decisions:
1. Fund the baseline before promising speed: Pull throughput, stability and review load for your last four quarters while the "before" still exists.
2. Watch the whole delivery cycle: If writing code takes 30 of 100 units of an engineer's time, making it instant still leaves 70.
3. Report stability beside throughput: The larger estimated effect of AI adoption on delivery so far has been on stability.
4. Treat review as the next bottleneck: Generate ten times more code and the review still has to happen; track the queue and its reviewers.
5. Run the measurement as a study: Quantify time across design, coding, testing and review, so the net effect shows in your own quarterly numbers.
Your company's assistant may answer differently, so read what it writes before you use it.
Check yourself
A pilot report says pull requests are ready for review far faster than before. What do you compare it with?
Show the answer
Your own baseline: change lead time and review wait, not the step the pilot measured.
Throughput is up since the tools landed, and so is the change failure rate. What goes to the board?
Show the answer
Both numbers together, with what you will do about stability.
Pull requests are getting bigger and review wait is climbing. What do you fund?
Show the answer
Review capacity; plan it as the next bottleneck.
A dashboard shows code written per developer up sharply. Do you report it upward?
Show the answer
Pair it with throughput and stability against the baseline before it goes up.
Sources
The advice draws on interviews with the co-founder of a company that sells AI coding assistants and the chief AI officer of a large cloud-infrastructure software company.
Sundar Pichai, chief executive of Alphabet, remarks on the Q3 2024 earnings call, 29 October 2024, published by Google as "Q3 earnings call: CEO's remarks": https://blog.google/company-news/inside-google/message-ceo/alphabet-earnings-q3-2024/ . A chief executive's spoken claim to investors, with no published method or definition of "generated"; on the April 2025 call Pichai said the share was "well over 30%", worded differently, so the two statements are not one trend line.
Google Cloud DORA, "Accelerate State of DevOps 2024" (report v. 2024.3), 22 October 2024: https://dora.dev/research/2024/dora-report/2024-dora-accelerate-state-of-devops-report.pdf . The same model estimates a 1.5% reduction in delivery throughput per 25% increase in AI adoption. Survey-based and self-reported; estimated effects rather than measured deployments, and the larger-batch-size explanation is the authors' hypothesis, not a tested result.
PurePerformance, episode 265, "80% AI-Written Code: How 1KOMMA5 Measured the Real Impact of AI on Engineering", 3 August 2026, speaker Michael Reichenbach, platform engineer at 1KOMMA5°: https://exa.ai/library/podcast/y7kysn8yy5m/episode/jlj2lxrlcy6 . One company, his own account from memory, unaudited; the lagging delivery metrics, change failure rate and deployment frequency, told them little.