When AI Leads to Skill Decay

[This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of the Tuck Knowledge in Practice Podcast is the audio record.]

Kirk Kardashian:

Hey, this is Kirk Kardashian, and you’re tuned in to Knowledge in Practice, a podcast from the Tuck School of Business at Dartmouth. In this podcast, I talk with Tuck faculty about their research, their teaching, and the story behind their curiosity.

Today on the show: will artificial intelligence cause our skills to decay?

AI is getting better and better every day, and in most conversations, that’s the headline—higher productivity, fewer errors, smarter systems. But what if the real risk isn’t that AI makes mistakes? What if it’s that humans slowly stop paying attention?

Tuck professors James Siderius, Rob Shumsky, and Alva Taylor have been studying what happens when humans work alongside AI, and their conclusion is subtle—and a little unsettling.

They argue that the most dangerous moment for organizations isn’t when AI is unreliable, and it’s not when it’s nearly perfect. It’s somewhere in between—when the system works just well enough that people begin to disengage. And over time, that disengagement doesn’t just affect output; it reshapes expertise itself.

So today, we’re talking about AI, productivity, and how people work with AI in their jobs.

Before we get into the topic itself, I’d love to hear how you all got into this area—why you decided to study it, and what angle you’re approaching it from.

Alva Taylor:

I’ll jump in. This conversation with Rob is very comfortable because it’s essentially the same conversation that sparked this paper—and the kind of conversation we have all the time walking around the halls at Tuck. Actually, I’m usually the one walking around bothering Rob.

My early background—I’m a strategy professor now—but I started in operations research, both as an undergraduate and in my master’s. So before I went to the “dark side” of strategy, I was an operations person.

I had this interest in misinformation—how, in digital environments, you determine what’s true and what’s not, and what the impact of that is. In operations, there’s a concept called the bullwhip effect, where small errors or distortions get amplified as they move through a system.

I was talking to Rob about this, and we thought it might make a great paper. But as we started thinking about misinformation—especially as AI use increased—we began asking: how do you design a system where, when humans and AI interact, the outputs remain valid and accurate? Where does that interaction happen, and how do you manage it?

That sparked the project. Then James joined us and really helped develop the modeling and produce the results.

Rob Shumsky:

Yeah, and as Alva said, we’re very comfortable working together—we speak the same operations language.

I came at it more from the operations side. I’ve always been interested in workflows in complex organizations—how work gets done within and between organizations—and how that interacts with economic incentives.

This topic was really spurred by my observations from my healthcare students. I teach and help run some of the healthcare programs, and they were coming in talking about AI all the time.

The question was: to what extent is outsourcing their work to AI going to diminish their skills over time? At first, it’s transcription. Then it’s ordering tests. Then diagnosing disease and coming up with treatment plans. Eventually, it might even be interviewing patients.

At what point does the physician lose their expertise? And does that matter in the long run?

We’re also observing AI’s effect on our own students. To what extent does reliance on AI diminish critical thinking and attention to detail?

For all of these reasons, we thought this was an important topic to study.

Kirk Kardashian:

Yeah, it certainly is. I think everyone can relate to what’s going on—AI is helping us, but it can also be a bit sneaky. It can hallucinate; it can lead us down the wrong path.

We tend to think of AI as something that improves productivity. But you’re looking at it as more of a double-edged sword. What’s the core organizational problem you’re studying?

Rob Shumsky:

The easy answer would be: AI is great—let’s use it and let it do our work for us. And as you said, there’s the obvious issue of hallucination—AI producing things that just aren’t true.

More broadly, generative AI tools like ChatGPT or Claude are very good at some things and not so good at others. Predicting which is which is still somewhat of an art. Some scholars call this the “jagged frontier”—the boundary between where AI performs well and where it performs poorly is uneven and hard to predict.

So if people in organizations are using these tools, it’s really important that they pay attention. Otherwise, they may pass along low-quality or incorrect outputs.

We’re interested in how organizations ensure that, in the short run, people stay engaged. And in the long run, if they stop paying attention, they may lose their skills—their ability to evaluate and intervene.

That dynamic process of learning and forgetting—what we call skill decay—is what we’re really focused on. How do you manage and control it?

Alva Taylor:

And one thing to remember is that people often say, “AI will get better, so this problem will go away.” But that’s not quite right.

The jagged frontier keeps moving. As AI improves, we apply it to harder and more complex problems. So there will always be tasks where AI performs well and others where it struggles.

That uneven boundary doesn’t disappear—it just shifts.

Kirk Kardashian:

That’s interesting. So is the jagged frontier basically describing that volatility—AI being sometimes right, sometimes wrong?

Rob Shumsky:

To some extent, yes. I had a personal example: I used AI to help with a math proof for another paper. For one problem, it gave me a perfect answer. A few minutes later, I gave it a different problem, and it produced complete garbage.

If I hadn’t been paying attention, I might have trusted the second answer because the first one was so good.

That unpredictability is exactly the challenge.

Kirk Kardashian:

Right—and that leads to one of your key findings: AI is most destabilizing when it’s in that middle zone—sometimes right, sometimes wrong. Can you talk about that?

Rob Shumsky:

Sure—but Alva, you’ve got a good perspective on why AI is different from past technologies.

Alva Taylor:

Right. We’ve always known that tools can weaken certain skills—calculators reduce mental math ability, for example.

But AI is different because it doesn’t just assist—it generates knowledge. It produces answers.

It’s also ubiquitous. You can apply it across many different types of problems. And increasingly, it can act with autonomy.

So while there are lessons from past technologies, AI introduces something fundamentally new that we need to pay close attention to.

Rob Shumsky:

Exactly. And in our model, we look at how AI reliability interacts with human behavior.

If AI is perfectly reliable, there’s little risk—people can disengage and still get good results.

If AI is predictably bad, people stay engaged because they know they need to check it.

But in the middle—when it’s sometimes right, sometimes wrong—that’s where problems arise. People may choose to pass along outputs without checking because it’s easier, and they don’t bear the full cost of errors.

So organizations have to increase incentives to keep people engaged. This is a classic principal-agent problem: aligning the incentives of workers with those of the organization.

And interestingly, in this middle zone, improving AI quality can actually reduce profits—because workers become even more tempted to disengage, forcing firms to spend more on incentives.

Alva Taylor:

You can see this in your own experience. Think about how quickly you’ve gotten used to saying, “I’ll have AI draft that email,” and just assuming it’s right.

That’s exactly how these dynamics start to play out.

Kirk Kardashian:

That resonates. I was thinking about my own experience—if I publish something inaccurate, that reflects badly on me. So I feel a strong incentive to check AI outputs.

How does that fit into your model?

Rob Shumsky:

In your case, you’re both the principal and the agent—you bear the full cost of mistakes.

But in larger organizations, those roles are separated. Workers may receive bonuses for good output but don’t fully bear the downside of errors. That creates misalignment.

That’s where the challenge lies.

Kirk Kardashian:

Right—that makes sense.

Another piece of your model is skill atrophy. How do you account for that?

Alva Taylor:

There are a few important dimensions.

First, task complexity. For simple tasks, effort can substitute for expertise—you can get good quickly. For complex tasks, expertise takes years to develop and can’t be replaced easily.

Second, the nature of the skill. Some skills—like riding a bike—stick with you. Others—like doing mathematical proofs—decay quickly if you don’t practice.

We incorporate these differences into the model.

Rob Shumsky:

Exactly. For low-complexity tasks, skill decay is slow. For high-complexity tasks, it’s much faster.

And when we model this, we find very different outcomes depending on task complexity and value.

For example, in low-value, low-complexity settings—like call centers—firms may rely heavily on AI with minimal oversight.

For high-value tasks—like healthcare—there’s strong incentive to maintain expertise.

For high-complexity tasks, we often see a sharp divide: either mostly non-experts relying on AI, or highly skilled experts staying fully engaged.

Alva Taylor:

One thing I like about the model is that it helps you see real-world examples differently.

Take a recent marketing misstep—McDonald’s CEO posted a video introducing a new burger, but the execution felt off.

An AI might recommend all the right elements—personalization, product demonstration, trending formats—but miss the nuance of how it should actually be done.

Without human expertise, those subtle execution details get lost—especially in high-stakes situations.

Kirk Kardashian:

So for leaders rolling out AI tools, what are the biggest mistakes to avoid?

Alva Taylor:

A few things.

First, assuming that once AI is “good enough,” you can relax. That’s dangerous.

Second, underestimating the cost of maintaining quality. In that middle zone, keeping people vigilant can be expensive.

Third, focusing only on output quality. You also need to test whether people can detect errors—whether their skills are being maintained.

Rob Shumsky:

And firms need infrastructure for skill maintenance—training, testing, oversight.

In our model, we simplify this to financial incentives, but in reality, there are many levers.

Interestingly, in some cases, making it easier to relearn skills can actually reduce vigilance—because people think, “I can always catch up later.”

So there are some counterintuitive trade-offs.

Kirk Kardashian:

That’s fascinating. For individuals, though—what can people do to avoid losing their skills?

Rob Shumsky:

It’s still early, but research suggests that how you use AI matters.

For example, software engineers who ask AI to explain its reasoning—and engage in a back-and-forth—tend to learn more than those who just copy outputs.

Kirk Kardashian:

Alva, are business leaders actively grappling with this?

Alva Taylor:

Yes. Three big questions come up again and again:

First, intellectual property.
Second, ensuring output quality and avoiding hallucinations.
Third, how to train people and prevent skill atrophy.

Those are the core concerns executives raise.

Kirk Kardashian:

Looking ahead—what will distinguish organizations that get this right?

Rob Shumsky:

Organizations that understand learning dynamics and keep people engaged with AI will do well.

Those that treat AI as a simple plug-and-play productivity tool will struggle.

Alva Taylor:

I agree—and I’d add that the winners will rethink organizational design entirely.

They’ll redesign workflows, incentives, training, and roles around AI. They’ll actively manage risks and rethink how work gets done.

That’s what will set them apart.

Kirk Kardashian:

Great. This has been a really interesting conversation. Thanks so much for your time.

Rob Shumsky:

Thanks, Kirk—this was fun.