Completion Was Never the Metric. It Was the Legible One.
AI made the old signal worthless. Here's the four-question artifact that replaces it.
☕ 8-minute read
We measured the wrong thing for forty years, and AI just made it obvious.
That sentence will land poorly with many L&D leaders, and I want to sit with it before I make the argument. Most of us didn’t choose completion because we thought it was a valuable signal. We chose it because it was the signal that would roll up into a dashboard our CEO or CFO would accept. We took the deal because it was easy to measure and fit on a PowerPoint slide. I took the deal because that’s what they wanted. For a long stretch, it mostly worked because the cost of completion was high enough that the number carried weight. A person had to sit through a module. Their time was real. The activity, even if it wasn’t understood, meant something. It checked a box.
That floor is gone now. When the floor drops, the metric we built on top of it drops too.
TL;DR
Completion was never measuring understanding. It was measuring seat-time presence, and the profession agreed to be judged on something easy to count because measuring comprehension was too slow.
AI made the old signal worthless. An AI can click through a module faster than any human, and a well-prompted model can generate a module that passes its own knowledge check. The credential is cheap now.
Replace completion with a four-question Comprehension Artifact that the learner owns. Decision. Trade-off. Failure mode. Two-week test. It takes ten minutes, and it is nearly impossible to fake cheaply.
🔄 So What Actually Changed
Here’s the specific shift, because “AI changed everything” is the kind of sentence that lets us keep not doing the hard thing.
In April 2026, a motivated learner with Claude Opus 4.7 or GPT-5.4 Thinking can have a model read the module for them and summarize it in 90 seconds. They can have it answer the knowledge check, and the hit rate is north of 95% without the learner reading a word of the content. If the assessment is open-response, they can have the model write something in their claimed voice that will pass an instructor’s review if the instructor isn’t looking closely. None of that requires clever prompting. That’s the default behavior for a current-generation model, and it wasn't the default six months ago.
So when we report “92% completion” to a business leader in Q2 2026, we are reporting a number that used to roughly correlate with “they sat through it and probably absorbed some of it,” and now correlates with “they or their assistant moved the content from one column to another.” That’s a real shift in what the metric means, and the metric hasn’t been updated to reflect it.
This is where I keep seeing L&D teams flinch, and I get it. The honest version of the conversation is this. Our dashboard is currently a liability rather than an asset. Producing the thing used to be the proof. Now the production is virtually free, so the proof has to be comprehensible. We have to find better ways to do this.
🎭 The Number Was Always a Dodge
Let me say something I don’t think a lot of us in our profession have been willing to say out loud. The people in L&D who pushed hardest for completion knew it was thin. It was the quick win that would be accepted. We all knew, and managers and HR teams love it. We’d been in enough rooms with enough managers to know that the person who “completed” the compliance module wasn’t always the person who understood it. But hey, they took it, so we don’t have to worry about them. We kept counting what was legible because the alternative was a conversation we couldn’t win in 30 seconds with a finance partner.
AI didn’t create this problem. AI removed the last bit of cover the problem had.
📖 The Story I Keep Coming Back To
When we scaled customer training at my last role, we took the customer training modules library from 50 to 637 in a year. That’s the kind of number that makes a great slide. The number isn’t the story.
The story is what the number hid and what we built to replace it.
The customers using those 637 modules weren’t employees doing compliance. They were dealership teams preparing to go live on a new platform where “going live” meant real money moved. Completion, by itself, was not a gate anyone would trust. So we built the gate around comprehension. Role-based paths, so a service writer wasn’t measured against a fixed operations director. While they share some modules, each has its own job tasks to be exposed to and trained on. An 80% demonstrated-understanding threshold on the role-specific path before go-live could proceed. Training is the gating control on revenue recognition, not a downstream deliverable.
When I describe that setup to L&D leaders, the most common response is “that’s not realistic in my org.” Maybe. The only reason it was realistic in ours was that the business couldn’t afford to continue “shipping” customers who didn’t understand the product. The stakes forced the design.
The stakes are now forcing the design everywhere. We haven’t caught up to it yet.
🧠 What Comprehension Evidence Looks Like
The replacement isn’t complicated. Four questions, and they’re the ones that completion never answered.
The first one is the one I come back to most. “What decision are you approaching differently now because of this learning? Name the before, the after, and what changed.” If a learner can’t name it, the learning didn’t attach to anything they did on Tuesday afternoon.
The second question is “what trade-off did this surface that you weren’t seeing?” This one is harder to fake because it requires the learner to admit what they were over-weighting before. I’ve seen more honest moments come out of that question than out of three years of post-module surveys.
The third question is where this gets uncomfortable for many L&D teams. “Where would this go wrong if someone applied it without judgment?” The learner names a failure mode they’d watch for, in themselves or on their team. That’s a very different artifact from a knowledge check.
And the last one does the heaviest lifting. “How will you know, two weeks from now, whether this learning changed your work?” The answer has to be observable. It has to name a situation, a behavior, and a date. “I’ll feel more confident” is not an answer. “On the next renewal call with an at-risk account, I’ll open with the churn-risk question instead of the value recap.” That’s the kind of specificity a manager can actually check.
Put the four together, and you get what I’ve started to call a Comprehension Artifact. I call things Artifacts more and more after being in the software world for 13.5 years now. It’s a two-page document that the learner produces at the end of a module, workshop, coaching session, or stretch assignment. It takes about ten minutes with a model as the coach. The learner owns it. It attaches to their development record, not to our LMS dashboard. This is where LMSs will either have to change significantly, or we need to look at our record-keeping very differently.
Here’s what I like about it, and this is the part that took me a while to get to. It’s specific enough to be useful in a 1:1. It’s qualitative enough that a model can’t fake a credible version without the learner’s lived context to pull from. It’s portable, so the learner carries it into their next role. It forces the moment of completion, letting us dodge, whether the learning is attached to a decision anyone can name. Having to name things and explain them pushes us to think more critically. We know it helps instructors know where someone is and how they got there.
💼 The Business Frame
If a CFO asks what changed, completion gives you a percentage. Comprehension gives you a specific behavior, a specific learner committed to change by a specific date, with a manager check-in scheduled to verify it. That’s a different kind of conversation. It’s also, honestly, the kind of conversation L&D has been quietly avoiding for most of my career, because the percentage let us exit the room before anyone asked a second question.
Completion dashboards don’t compound, either. Each completion is a one-time event with no downstream signal. Comprehension artifacts compound, because each one is a documented behavior change that either held or didn’t. Over a quarter, you get a portfolio. Over a year, you get a reliable leading signal on whether L&D spend is producing capability and true skills or just activity.
That’s the move I’d want to be making walking into 2027 budget conversations. Not because it’s trendy. Because the leaders who still only have completion to show when their budgets are questioned will struggle to answer the question, the profession has never been good at answering it. I think all of us are tired of the “why didn’t things get better?” conversation.
💡 What This All Means
Most L&D dashboards, looked at squarely, are a liability pretending to be an asset right now. The metric we built them on is broken, the tooling that broke it is getting cheaper and better every quarter, and the honest conversation with the business is the one where we name that out loud instead of waiting for them to notice.
The leaders who replace completion with comprehension in 2026 are not being brave. They’re being accurate. Those who don’t will find themselves defending a number whose signal has already decayed, and the defense will get harder every month.
🛠️ This Prompt This Week
Pick one learning event your team is running this week. A workshop, a module, a coaching conversation, a stretch assignment. At the end, run this prompt with a learner and budget 10 minutes. The output belongs to them.
You are an L&D coach. I just completed a learning experience. Your job is to walk me through a four-question comprehension artifact I will attach to my own development record. The output belongs to me.
CONTEXT
Ask me:
- What was the learning experience? (One sentence.)
- What specific work situation did you expect this learning to help with?
THE FOUR QUESTIONS
Ask these one at a time. Press for named situations, roles, and observable details. Do not let me off the hook with generalities.
Q1 DECISION
Describe one decision you now approach differently because of this learning. What was the before-state decision process? What is the after-state? What changed specifically?
Q2 TRADE-OFF
Name one trade-off this learning made visible that you did not see before. What were you over-weighting? What are you willing to weight differently now, and under what conditions?
Q3 FAILURE MODE
If someone applied this learning without judgment, where would it go wrong? Give a concrete example of a failure mode to watch for.
Q4 TEST
How will you know, two weeks from now, whether this learning actually changed your work? Name the observable evidence on a named situation by a named date.
OUTPUT Comprehension Artifact
Produce a single markdown artifact the learner can save and attach to their development record, with these sections:
# Comprehension Artifact [Learning Name]
Learner, Date, Work Situation in Scope
## Decision I Now Approach Differently
## Trade-Off This Surfaced
## Failure Mode I'll Watch For
## My Two-Week Test
## What I'd Ask Of My Manager
Close with: "This artifact is owned by the learner. Revisit in two weeks."
Begin with Context question 1.Run it once. Compare what you learn to what a completion record would have told you. That gap is the argument.
✅ The One Thing to Do This Week
Stop reporting completion as your primary metric in your next business review. This is going to be rough, but it’s worth it. Replace it, or at a minimum pair it, with three to five Comprehension Artifacts from learners in the cohort you’re reporting on. Redacted if needed. Real.
—Eian